We consider a multi-objective risk-averse two-stage stochastic programming problem with a multivariate convex risk measure. We suggest a convex vector optimization formulation with set-valued constraints and propose an extended version of Benson's algorithm to solve this problem. Using Lagrangian duality, we develop scenario-wise decomposition methods to solve the two scalarization problems appearing in Benson's algorithm. Then, we propose a procedure to recover the primal solutions of these scalarization problems from the solutions of their Lagrangian dual problems. Finally, we test our algorithms on a multi-asset portfolio optimization problem under transaction costs.