We present a method for autonomous exploration in complex three-dimensional (3D) environments. Our method demonstrates exploration faster than the current state-of-the-art using a hierarchical framework-one level maintains data densely and computes a detailed path within a local planning horizon, while another level maintains data sparsely and computes a coarse path at the global scale. Such a framework shares the insight that detailed processing is most effective close to the robot, and gains computational speed by trading-off details far away from the robot. The method optimizes an overall exploration path with respect to the length of the path and produces a kinodynamically feasible local path. In experiments, our systems autonomously explore indoor and outdoor environments at a high degree of complexity, with ground and aerial robots. The method produces 80% more exploration efficiency, defined as the average explored volume per second through a run, and consumes less than 50% of computation compared to the state-of-the-art.