Quasi-Newton methods form the basis of many effective methods for unconstrained and constrained optimization. As the iterations proceed, a quasi-Newton method incorporates new curvature information by performing a low-rank update to a matrix that serves as an approximation to a Hessian matrix of second derivatives. In the years following the publication of the Davidon-Fletcher-Powell (DFP) method in 1963 the Broyden-Fletcher-Goldfarb-Shanno (BFGS) update emerged as the best update formula for use in unconstrained minimization. More recently, a number of quasi-Newton methods have been proposed that are intended to improve on the efficiency and reliability of the BFGS method. An extensive survey and evaluation of the most prominent of these modifications was given by Gill and Runnoe [11]. This paper focuses on two of the most effective approaches considered in the survey and give extensive results from implementations evaluated on a set of unconstrained problems from the CUTEst test collection.
It is shown that a combination of self-scaling and a factored Hessian shows significant and consistent improvement over the conventional BFGS method. The results appear to contradict the widely-held view that self-scaled updates and stable factorization methods provide no additional benefits over a conventional implementation.