Inverse optimization (IO) has emerged as a powerful framework for analyzing prescriptive model parameters that rationalize observed or prescribed decisions. Despite the prevalence of discrete decision-making models, existing work has primarily focused on continuous and convex models, for which the corresponding IO problems are far easier to solve. This paper makes three contributions that broaden the foundations and applications of inverse optimization in discrete decision-making. First, we propose a new approach for approximating the inverse-feasible region of solutions to discrete optimization problems by modifying their linear relaxations. Second, we integrate these modified relaxations with existing methods to develop a unified framework for solving IO problems. Finally, through the lens of inverse optimization, we provide a fresh perspective on the sensitivity analysis of prescriptive solutions to discrete optimization problems, and demonstrate the computational advantages of our models and algorithms.