We derive computationally tractable formulations of the robust counterparts of convex quadratic and conic quadratic constraints that are concave in matrix-valued uncertain parameters. We do this for a broad range of uncertainty sets. In particular, we show how to reformulate the support functions of uncertainty sets represented in terms of matrix norms and cones. Our results provide extensions to known results from the literature. We also consider hard quadratic constraints; those that are convex in uncertain matrix-valued parameters. For the robust counterpart of such constraints we derive inner and outer tractable approximations. As application, we show how to construct a natural uncertainty set based on a statistical confidence set around a sample mean vector and covariance matrix and use this to provide a tractable reformulation of the robust counterpart of an uncertain portfolio optimization problem. We also apply the results of this paper to a norm approximation and a regression line problem.