The Progressive Hedging (PH) algorithm is one of the cornerstones in large-scale stochastic programming. However, its traditional development requires that all scenario subproblems are solved per iteration, and a probability distribution with finitely many outcomes. This paper introduces a stochastic dual PH algorithm (SDPH) to overcome these challenges. We introduce an adaptive sampling process and utilize a stochastic conjugate subgradient method for direction finding, while applying an inexact line-search to update the dual variables. We present two alternative approaches to study its convergence: one based on a fixed-point theorem as in the original PH paper, whereas the other utilizes a stochastic stopping-time framework. Moreover, our computational experiments demonstrate that the new algorithm not only addresses the bottlenecks of the original deterministic PH, but also potentially overcomes issues pertaining to scalability of traditional PH for very large data sets.