This paper proposes a two-phase optimization framework for users that are involved in demand response programs. In a first phase, responsive users optimize their own household consumption, characterizing not only their appliances and equipment but also their comfort preferences. Subsequently, the aggregator exploits in a second phase this preliminary noncoordinated solution by implementing a coordination strategy for the aggregated loads while preserving users’ privacy. The second phase relies on the solution of a bilevel program in which the aggregator’s profit is maximized in the upper level while ensuring that the aggregated residential users do not incur any economic or comfort losses by participating in the demand response program. The lower level models the users’ reaction to the aggregator’s requests. As major complicating aspects, the resulting bilevel problem features nonlinear terms and lowerlevel binary variables. This challenging problem is addressed by a mixed-integer linear single-level reformulation and the application of an exact solution technique based on Dantzig-Wolfe decomposition. Simulations with up to 10,000 residential users illustrate the advantages of the proposed two-phase framework in terms of users’ privacy, computational efficiency, and scalability.