Long-term planning for electric power systems, or capacity expansion, has traditionally been modeled using simplified models or heuristics to approximate the short-term dynamics. However, current trends such as increasing penetration of intermittent renewable generation and increased demand response requires a coupling of both the long and short term dynamics. We present an efficient method for coupling multiple temporal scales using the framework of singular perturbation theory for the control of Markov processes in continuous time. We show that the uncertainties that exist in many energy planning problems, in particular load demand uncertainty and uncertainties in generation availability, can be captured with a multiscale model. We then use a dimensionality reduction technique, which is valid if the scale separation present in the model is large enough, to derive a computationally tractable model. We show that both wind data and electricity demand data do exhibit sufficient scale separation. A numerical example using real data and a finite difference approximation of the Hamilton-Jacobi-Bellman equation is used to illustrate the proposed method. We compare the results of our approximate model with those of the exact model. We also show that the proposed approximation outperforms a commonly used heuristic used in capacity expansion models.