Expected-value-constrained programming (ECP) formulations are a broad class of stochastic programming problems including integrated chance constraints, risk models, and stochastic dominance formulations. Given the wide availability of data, it is common in applications to have independent contextual information associated with the target or dependent random variables of the problem. We show how to incorporate such information to efficiently approximate ECPs, and prove that the solution set of the approximate problem approaches the true solution set exponentially fast. We illustrate our approach with a portfolio optimization problem that exemplifies the importance of taking contextual information into account in problems with expected-value constraints.