Abstract
Ray tracing enables 3D Gaussian fields to serve as a representation for physically based light transport. Faithful inverse rendering requires forward rendering and backward optimization to be defined within a consistent light-transport pipeline. Existing Gaussian inverse-rendering methods typically rely on splatting-derived G-buffers and screen-space optimization. The local affine approximation of perspective projection can introduce inconsistencies with path tracing, while simplified rendering formulations often neglect or approximate indirect illumination and visibility. Therefore, we propose PTIR-GS, a splatting-free path-traced inverse-rendering framework for 3D Gaussian fields that unifies forward rendering and backward optimization within the same recursive multi-bounce light-transport pipeline. We formulate light transport over overlapping Gaussian primitives directly in path space, enabling Monte Carlo path tracing and pathwise gradient replay over the same recursively sampled transport paths. The framework jointly optimizes materials and environment light under the full rendering equation, with ray-traced visibility and global illumination explicitly evaluated. Extensive experiments demonstrate competitive material estimation and improved path-traced rendering quality, producing more plausible shadows, reflections, and relighting under global illumination.
Framework
Our framework jointly defines forward rendering and backward optimization for 3D Gaussian fields within a unified ray-tracing pipeline. Given a 3D Gaussian scene, we construct ray-traced equivalent surface interactions, evaluate visibility and multi-bounce path-traced light transport under the full rendering equation, and support optimization of environment illumination using a compact Spherical-Gaussian representation. Multiple importance sampling reduces the variance of incident radiance estimation. Gradients are propagated through path replay backpropagation via equivalent interactions, ensuring consistency between backward optimization and the forward ray-traced transport paths.
Results
Arbitrary Light Relighting
- PTIR-GS is fully based on path tracing and supports relighting with arbitrary light sources, such as point lights, area lights, and environment maps.
Benchmark Qualitative Comparisons
- We show NVS, material decomposition, and relighting results on the TensoIR and Synthetic4Relight datasets. PTIR-GS achieves consistent qualitative results. Click the buttons above to switch scenes.
Indirect Illumination Modeling Comparison
- We show scenes with obvious indirect illumination in the TensoIR and Synthetic4Relight datasets. PTIR-GS provides more realistic and detailed indirect illumination than the others. Click the buttons above to switch scenes.
Light Only Display
- We show light-only rendering results to demonstrate that PTIR-GS recovers plausible illumination while preserving scene geometry and object details. Drag the sliders to compare the novel view synthesis results with the light-only components.
Lego Normal Qualitative Comparisons
- PTIR-GS usually obtains better geometric structures, although we do not deeply explore geometry in this work. Below, we compare the Lego normal maps of different methods (w/o any prior).
- A possible reason is that, under the ray-tracing framework, the GS depth is determined by its peak response along the ray, rather than by the Gaussian center depth as in splatting, leading to more accurate depth estimation.
Citation
@misc{ptir-gs,
title={Path-Traced Inverse Rendering with Global Illumination in 3D Gaussian Fields},
author={Junke Zhu and Hao Zhang and Yutian Zhu and Ang Li and Chenxiao Hu and Meng Gai and Fei Zhu and Zhangjin Huang and Sheng Li},
year={2026},
eprint={2606.09606},
archivePrefix={arXiv},
primaryClass={cs.GR},
url={https://arxiv.org/abs/2606.09606},
}