Multi-robot systems (MRS) have shown collective behaviors and enhanced capabilities in the literature. Real-time control of MRS is challenging due to the exponentially growing state space. This scalability issue becomes more difficult with a highly constrained system. In this paper, we present a Model Predictive Control (MPC) with a decentralized state space and parallelized computations. Our MPC framework effectively models the dense constraints of reconfigurable multi-robot systems and enables scalable real-time control of the system. We show that the proposed MPC enables significantly faster computation compared with an MPC with a centralized state space. We tested our algorithm on up to eight robots in simulation and three robots on the hardware platform. Detailed implementation can be found here: url{https://github.com/allisonjseo/decentralized-mpc}.