Symmetry handling is a key technique for reducing the running time of branch-and-bound methods for solving mixed-integer linear programs. In this paper, we generalize the notion of (permutation) symmetries to mixed-integer semidefinite programs (MISDPs). We first discuss how symmetries of MISDPs can be automatically detected by finding automorphisms of a suitably colored auxiliary graph. Then known symmetry handling techniques can be applied. We demonstrate the effect of symmetry handling on different types of MISDPs. To this end, our symmetry detection routine is implemented in the state-of-the-art MISDP solver SCIP-SDP. We obtain speed-ups similar to the mixed-integer linear case.