May
2025
Towards Efficient and Accurate Neural Geometry and Appearance Representations
Authors:
Abstract:
The challenge of developing scene representations for world modeling has persisted for a long time. The advent of neural scene representations, which leverage neural networks and volume rendering, has unlocked new possibilities for constructing highly realistic world models. However, volume rendering—the key mechanism behind these advances—remains computationally expensive due to the large number of MLP evaluations required for each sampled point along a ray. Prior research has tackled this inefficiency by introducing novel neural networks or data structures. In this thesis, we take a different approach by revisiting the mathematical formulation of volume rendering and introducing GL-NeRF, a new perspective that employs Gauss-Laguerre quadrature. GL-NeRF enables efficient querying of learned geometric representations without incurring additional computational overhead. Specifically, it dramatically reduces the number of MLP evaluations needed for volume rendering without relying on extra data structures or additional neural networks. Its straightforward formulation allows seamless integration into any NeRF-based model. We first establish the theoretical foundation for using Gauss-Laguerre quadrature and then demonstrate its versatility by incorporating it into three different NeRF frameworks. This approach results in a more efficient geometric representation, advancing towards a more generalized model of the world.
Another essential aspect of world modeling is appearance modeling, typically achieved through inverse rendering within the framework of neural scene representations. This process seeks to disentangle geometry, materials, and lighting from image data—a long-standing challenge in computer vision and graphics. Recent breakthroughs in neural rendering have enabled highly realistic and physically plausible inverse rendering results by integrating neural scene representations with inverse rendering techniques. The emergence of 3D Gaussian Splatting as a novel scene representation has further propelled this progress, demonstrating real-time rendering capabilities. However, these models often treat opacity and material properties as independent parameters, leading to theoretical inaccuracies. Drawing inspiration from radiative transfer theory, we enhance the opacity term by introducing a neural network that takes material properties as input, thereby enabling the modeling of cross-section—a material-dependent term in physics—along with a physically sound activation function. This approach allows the gradients of material properties to be influenced not only by color but also by opacity, enforcing an additional constraint that improves optimization. By integrating our method into three Gaussian Splatting-based inverse rendering models, we achieve substantial improvements in novel view synthesis and material modeling. This formulation underscores the importance of leveraging correct physical priors to develop more accurate and generalizable scene representations across diverse scenarios.
This thesis marks an initial step toward enhancing neural scene representations in both geometry and appearance, with the ultimate goal of developing generalizable neural models that comprehensively capture both aspects.
Another essential aspect of world modeling is appearance modeling, typically achieved through inverse rendering within the framework of neural scene representations. This process seeks to disentangle geometry, materials, and lighting from image data—a long-standing challenge in computer vision and graphics. Recent breakthroughs in neural rendering have enabled highly realistic and physically plausible inverse rendering results by integrating neural scene representations with inverse rendering techniques. The emergence of 3D Gaussian Splatting as a novel scene representation has further propelled this progress, demonstrating real-time rendering capabilities. However, these models often treat opacity and material properties as independent parameters, leading to theoretical inaccuracies. Drawing inspiration from radiative transfer theory, we enhance the opacity term by introducing a neural network that takes material properties as input, thereby enabling the modeling of cross-section—a material-dependent term in physics—along with a physically sound activation function. This approach allows the gradients of material properties to be influenced not only by color but also by opacity, enforcing an additional constraint that improves optimization. By integrating our method into three Gaussian Splatting-based inverse rendering models, we achieve substantial improvements in novel view synthesis and material modeling. This formulation underscores the importance of leveraging correct physical priors to develop more accurate and generalizable scene representations across diverse scenarios.
This thesis marks an initial step toward enhancing neural scene representations in both geometry and appearance, with the ultimate goal of developing generalizable neural models that comprehensively capture both aspects.
copied = false, 2000);
">
@mastersthesis{Yong-2025-146399,
author = {Silong Yong},
title = {Towards Efficient and Accurate Neural Geometry and Appearance Representations},
year = {2025},
month = {May},
school = {Carnegie Mellon University},
address = {Pittsburgh, PA},
number = {CMU-RI-TR-25-36},
}
author = {Silong Yong},
title = {Towards Efficient and Accurate Neural Geometry and Appearance Representations},
year = {2025},
month = {May},
school = {Carnegie Mellon University},
address = {Pittsburgh, PA},
number = {CMU-RI-TR-25-36},
}