This survey proposes a unifying conceptual framework and taxonomy that systematically integrates Machine Learning (ML) and Reinforcement Learning (RL) with classical paradigms for Constraint Satisfaction and Boolean Satisfiability solving. Unlike prior reviews that focus on individual applications, we organize the literature around solver architecture, linking each major phase—constraint propagation, heuristic decision-making, conflict analysis, and meta-level structural learning—to its corresponding learning paradigm. We review the evolution from symbolic constraint propagation to modern neuro-symbolic optimization, highlighting the methodological convergence between Operations Research and Artificial Intelligence. Building upon the Lazy Clause Generation and Conflict-Driven Clause Learning architectures, we introduce ML/RL-enhanced algorithmic modules that demonstrate how data-driven inference can augment logical reasoning. Our proposed taxonomy connects solver components to specific learning approaches, including Graph Neural Networks, Transformer encoders, and policy-gradient algorithms such as Proximal Policy Optimization. We identify key research challenges—particularly the preservation of logical soundness, generalization across problem distributions, and interpretability of learned heuristics—and outline a roadmap toward scalable hybrid optimization frameworks that unify symbolic reasoning with data-driven learning.