This paper develops a robust optimization based decision-making framework using a nonparametric perturbation of a reference utility function. The perturbation preserves the risk-aversion property but solves the problem of ambiguity and inconsistency in eliciting the reference utility function. We study the topology of the perturbation, and show that in the decision-making framework the price of perturbation is increasing and concave. When the reference utility is given at discrete points, we reformulate this optimization problem as a second-order cone program. The Monte Carlo sampling method is used to solve the general case that a reference utility is a continuous function, and the convergence of this method is proved. The usefulness of the robust utility optimization framework is illustrated with the help of a portfolio investment decision problem.