The risk parity optimization problem aims to find such portfolios for which the contributions of risk from all assets are equally weighted. Portfolios constructed using risk parity approach are a compromise between two well-known diversification techniques: minimum variance optimization approach and the equal weighting approach. In this paper, we discuss the problem of finding portfolios that satisfy risk parity of either individual assets or groups of assets. We describe the set of all risk parity solutions by using convex optimization techniques over orthants and we show that this set may contain exponential number of solutions. We then propose an alternative nonconvex least-square model whose set of optimal solutions includes all risk parity solutions, and propose a modified formulation which aims at selecting the most desirable risk parity solution (according to some criteria). When general bounds are considered, a risk parity solution may not exist. The nonconvex least-square model then seeks a feasible portfolio which is as close to risk parity as possible. Furthermore, we propose an alternating linearization framework to solve this nonconvex model. Numerical experiments indicate the effectiveness of our technique in terms of both speed and accuracy.