The Legendre-Fenchel Conjugate of the Product of Two positive definite Quadratic Forms

It is well-known that the Legendre-Fenchel conjugate of a positive definite quadratic form can be explicitly expressed as another positive definite quadratic form, and that the conjugate of the sum of several positive definite quadratic forms can be expressed via inf-convolution. However, the Legendre-Fenchel conjugate of the product of two positive definite quadratic forms is not clear at present. Jean-Baptiste Hiriart-Urruty posted it as an open question in the field of nonlinear analysis and optimization [`Question 11' in \emph{SIAM Review} 49 (2007), 255-273]. From convex analysis point of view, it is interesting and important to address such a question. The purpose of this paper is to answer this question and to provide a formula for the conjugate of the product of two positive definite quadratic forms. We prove that the computation of the conjugate can be implemented via finding a root to certain univariate polynomial equation, and we also identify the situations in which the conjugate can be explicitly expressed as a single function without involving any parameter. Some other issues, including the convexity condition for the product function, are also investigated as well. Our analysis shows that the relationship between the matrices of quadratic forms plays a vital role in determining whether the conjugate can be explicitly expressed or not.