A systematic study of the proximity properties of Bregman distances is carried out. This investigation leads to the introduction of a new type of proximity operator which complements the usual Bregman proximity operator. We establish key properties of these operators and utilize them to devise a new alternating procedure for solving a broad class of joint minimization problems. We provide a comprehensive convergence analysis of this algorithm. Our framework is shown to capture and extend various optimization methods.