We propose data profiles as a tool for analyzing the performance of derivative-free optimization solvers when there are constraints on the computational budget. We use performance and data profiles, together with a convergence test that measures the decrease in function value, to analyze the performance of three solvers on sets of smooth, noisy, and piecewise-smooth problems. Our results provide estimates for the performance difference between these solvers, and show that on these problems, a model-based solver performs better than geometry-based solvers, even for noisy and piecewise-smooth problems.
Citation
Preprint ANL/MCS-P1471-1207, Mathematics and Computer Science Division, Argonne National Laboratory, April 2008