neural-raytracing

Neural Ray Tracing — Radiance Cache

The light-transport analog of the neural physics engine thesis: keep visibility and direct lighting analytic, learn only the indirect transport with a tiny tied MLP. One analytic ray + next-event estimation + a network lookup replaces the many-bounce random walk after the first hit.

How it was done

Render decomposition: L = emitted + direct(analytic) + indirect(learned).

1. Ground truth: a small PyTorch path tracer (engine3d/raytrace.py) renders high-spp references and splits each pixel into emitted / direct / indirect components.
2. Training data: first-hit surface points + normals paired with the path-traced indirect radiance at those points.
3. The cache (engine3d/neural_rt.py): one tiny MLP shared across the whole scene (the tied-embedding structure used everywhere in this project) maps (hit point, normal) → indirect RGB.
4. Composition: at render time, trace one analytic primary ray, add analytic direct lighting (NEE), and look up the cache for the rest.
5. Evaluation (experiments/w9_neural_radiance_cache.py): PSNR on a held-out camera view, compared against an equal-cost few-spp path trace, plus the spp the baseline needs to match the neural render.

experiments/w11_world_lighting.py applies the same recipe to bake a world ambient/GI field (world_light.pt) over the voxel world — this is the "neural GI" used live in the Neural World demo Space, parsed in-browser by pt_loader.js.

Checkpoints

Validation

Reproduce

python experiments/w9_neural_radiance_cache.py   # trains + renders comparison
python experiments/w11_world_lighting.py         # bakes world_light.pt