Recently, several optimization approaches for portfolio selection have been proposed in order to alleviate the estimation error in the optimal portfolio. Among such are the norm-constrained variance minimization and the robust portfolio models. In this paper, we examine the role of the norm constraint in the portfolio optimization from several directions. First, it is shown that the norm constraint can be regarded as a robust constraint associated with the return vector. Secondly, the norm constraint is combined with the value-at-risk (VaR) and conditional value-at-risk (CVaR) minimizations. For the combinations, a nonparametric theoretical validation is posed based on the generalization error bound for the $\nu$-support vector machine. Thirdly, the proposed approach is applied to the tracking portfolio problem and computational experiments are conducted. Through the experiments, we see that the norm-constrained minimization of the CVaR-based deviation with a parameter tuning strategy outperforms the traditional models in terms of the out-of-sample tracking error.