We use the lexicographic order to define a hierarchy of primal and dual bounds on the optimum of a bounded integer program. These bounds are constructed using lex maximal and minimal feasible points taken under different permutations. Their strength is analyzed and it is shown that a family of primal bounds is tight for any 0\1 program with nonnegative linear objective, and a different family of dual bounds is tight for any packing- or covering-type 0\1 program with an arbitrary linear objective. The former result yields a structural characterization for the optimum of 0\1 programs, with connections to matroid optimization, and a heuristic for general integer programs. The latter result implies a stronger polyhedral representation for the integer feasible points and a new approach for deriving strong valid inequalities to the integer hull. Since the construction of our bounds depends on the computation of lex optima, we derive explicit formulae for lex optima of some special polytopes, such as polytopes that are monotone with respect to each variable, and integral polymatroids and their base polytopes. We also classify P and NP-hard cases of computing lex bounds and lex optima.