It is well known that the classical Markowitz model for portfolio optimization is extremely sensitive to estimation errors on the expected asset returns. Robust optimization mitigates this issue. We focus on ellipsoidal uncertainty sets around the point estimates of the expected asset returns. We investigate the performance of diagonal estimation-error matrices in the description of the uncertainty set. We show that the class of diagonal estimation-error matrices can achieve an arbitrarily small loss in the expected portfolio return as compared to the optimum. We then formulate the problem of finding the best estimation error matrix as a bilevel program. The bilevel model allows us to numerically analyze the error when there are multiple estimates for the expected return and/or when there are additional restrictions on the structure of the estimation-error matrix. We extend our analysis to show that the diagonal estimation-error matrices can achieve an arbitrarily small loss even when there are multiple estimates for the expected return. We finally focus on analyzing the use of identity matrix as an estimation-error matrix. The results of our simulation show that robust portfolio models featuring an identity matrix as an estimation-error matrix outperform the classical Markowitz model when the size of the uncertainty set is chosen properly.