Designing policies for a network of agents is typically done by formulating an optimization problem where each agent has access to state measurements of all the other agents in the network. Such policy designs with centralized information exchange results in optimization problems that are typically hard to solve, require to establish substantial communication links, and do not promote privacy since all information is shared among the agents. Designing policies based on arbitrary communication structures can lead to non-convex optimization problems which are typically NP-hard. In this work, we propose an optimization framework for decentralized policy designs. In contrast to the centralized information exchange, our approach requires only \emph{local communication exchange} among the neighboring agents matching the physical coupling of the network. Thus, each agent only requires information from its direct neighbors, minimizing the need for excessive communication and promoting privacy amongst the agents. Using robust optimization techniques, we formulate a convex optimization problem with a loosely coupled structure that can be solved efficiently. We demonstrate numerically the efficacy of the proposed approach in a case study from contract design in supply chains and a classic control application. We show that the proposed approach leads to solutions that closely approximate those obtained by the centralized formulation only at a fraction of the computational effort.