Structural optimization of non-conservative systems with respect to stability criteria is a research area with important applications in fluid-structure interactions, friction-induced instabilities, and civil engineering. In contrast to optimization of conservative systems where rigorously proven optimal solutions in buckling problems have been found, for non-conservative optimization problems only numerically optimized designs were reported. The proof of optimality in the non-conservative optimization problems is a mathematical challenge related to multiple eigenvalues, singularities on the stability domain, and non-convexity of the merit functional. We present a study of the optimal mass distribution in a classical Ziegler's pendulum where local and global extrema can be found explicitly. In particular, for the undamped case, the two maxima of the critical flutter load correspond to a vanishing mass either in a joint or at the free end of the pendulum; in the minimum, the ratio of the masses is equal to the ratio of the stiffness coefficients. The role of the singularities on the stability boundary in the optimization is highlighted and extension to the damped case as well as to the case of higher degrees of freedom is discussed.