For robots to understand the environment they interact with, a combination of geometric information and semantic information is crucial. In this thesis, we propose a fast and scalable Simultaneous Localization and Mapping (SLAM) system that represents indoor scenes as a graph of semantic objects. Leveraging the observation that artificial environments are structured and occupied by recognizable objects, we show that a combination of compositional rendering and sparse volumetric object graph as the map results in a SLAM system suitable for drift-free large-scale indoor reconstruction. While object-based SLAM has been proposed in the past, we improve on both object reconstruction quality, trajectory accuracy, and online performance. We also propose a semantically assisted data association method that results in unambiguous and persistent object landmarks. We deliver an online implementation that can run at about 4-5Hz on a single commodity graphics card, and provide a comprehensive evaluation against state-of-the-art baselines.