Robust control design for constrained uncertain systems is a well-studied topic. Given a known uncertainty set, the objective is to find a control policy that minimizes a given cost and satisfies the system's constraints for all possible uncertainty realizations. In this paper, we extend the classical robust control setup by treating the uncertainty sets as additional decision variables. We develop a unified framework for studying such problems, which we refer to as constrained robust optimal control problems with adjustable uncertainty sets. In particular, given a metric for adjusting the uncertainty sets, we address the question of determining the optimal size and shape of the uncertainty sets, while simultaneously ensuring the existence of a control policy that will keep the system within its constraints for all possible disturbance realizations inside the adjusted uncertainty set. Since our problem subsumes the classical constrained robust optimal control design, it is computationally intractable in general. We demonstrate in this paper that by restricting the families of admissible uncertainty sets and control policies, the problem can be formulated as a tractable convex optimization problem. We show that our framework captures several families of (convex) uncertainty sets of practical interest, and illustrate our approach on a demand response problem of providing control reserves for a power system.