This paper considers the robust phase retrieval, which can be cast as a nonsmooth and nonconvex optimization problem. We propose two first-order algorithms with adaptive step sizes: the subgradient algorithm (AdaSubGrad) and the inexact proximal linear algorithm (AdaIPL). Our contribution lies in the novel design of adaptive step sizes based on quantiles of the absolute residuals. Local linear convergences of both algorithms are analyzed under different regimes for the hyper-parameters. Numerical experiments on synthetic datasets and image recovery also demonstrate that our methods are competitive against the existing methods in the literature utilizing predetermined (possibly impractical) step sizes, such as the subgradient methods and the inexact proximal linear method.