Given an affine space of matrices L and a matrix \theta in L, consider the problem of finding the closest rank deficient matrix to \theta on L with respect to the Frobenius norm. This is a nonconvex problem with several applications in estimation problems. We introduce a novel semidefinite programming (SDP) relaxation, and we show that the SDP solves the problem exactly in the low noise regime, i.e., when \theta is close to be rank deficient. We evaluate the performance of the SDP relaxation in applications from control theory, computer algebra, and computer vision. Our relaxation reliably obtains the global minimizer in all cases for non-adversarial noise.