GPU-Driven Graphics Research Helper

GPU-Driven Graphics Research Helper

Project Context

This is a self-contained learning project using wgpu (Rust WebGPU abstraction) implementing:

• Rasterizer + Raytracer dual rendering pipelines
• Adaptive Probe Volumes (APV) for global illumination (planned)
• glTF asset loading (Phase 1-5 progression)
• Cross-platform support (Vulkan, Metal, DX12, WebGPU)

Goal: Learn advanced rendering from first principles, implement simplified APV system.

What This Skill Knows

Your Research (from .scratchpad)

APV Architecture (from multi-probe-grid-exploration.md and multi-probe-grid-implementation-guide.md):

• Brick-based sparse storage (4×4×4 probes per brick, ALWAYS)
• Variable probe spacing (1m, 3m, 9m, 27m) with multi-level indirection
• Indirection textures mapping world cells → brick indices + layer masks
• Interior/​Exterior layer masks for light leak prevention
• Spherical Harmonics (SH) for storing directional light information
• Trilinear interpolation across neighboring probes
• Z-axis stacking for hardware-accelerated sampling

Key Papers & Resources:

• Unity Adaptive Probe Volumes documentation
• Unreal Engine 5 RTXGI implementation
• NVIDIA research on probe-based GI
• Spherical Harmonics math and implementations

Your Learning Journey

• Phase 1: Basic geometry loading (Triangle, TriangleWithoutIndices, Box)
• Phase 2: Textures & materials (BoxTextured, embedded textures, vertex colors)
• Phase 3: Transform hierarchies (SimpleMeshes, MultipleScenes)
• Phase 4: PBR materials (MetalRoughSpheres, Normal Mapping, DamagedHelmet)
• Phase 5: Complex real-world models (Sponza, Avocado, FlightHelmet)

Then: GPU buffer consolidation refactor → APV implementation

Your Constraints

• Self-contained codebase (minimal external dependencies)
• No AI-generated content ("slop-free")
• Rust + WGSL for shaders
• Learning-focused (understand WHY, not just HOW)
• Explore → Plan → Describe workflow (detailed documentation of findings)

When to Activate This Skill

Use this skill when you're:

• Research phase: Understanding rendering techniques
• Planning phase: Deciding implementation approach
• Theory gaps: Need to understand graphics concepts
• Paper exploration: Finding and evaluating research
• Math deep-dives: Understanding SH, interpolation, coordinate transforms

How This Skill Helps

1. Find Relevant Research

You ask: "What papers exist on Adaptive Probe Volumes?" I respond with:

• Specific papers (with years, authors, brief summaries)
• Which papers are most practical vs theoretical
• Where to find them (arXiv, GDC, conference papers)
• Which sections are most relevant to YOUR approach

2. Explain Complex Concepts

You ask: "Explain Spherical Harmonics for lighting" I provide:

• Mathematical foundation (but intuitive)
• Why they're better than storing raw radiance
• How to implement in WGSL
• Common mistakes (frequency loss, ringing artifacts)
• Your specific use case (storing 2nd-order SH in probes)

3. Bridge Theory to Implementation

You ask: "How do I convert this paper's math into WGSL?" I help:

• Translate pseudocode to WGSL
• Handle precision/​numerical issues
• Optimize for wgpu (avoid branching, cache locality)
• Map math to GPU operations

4. Visual Explanations

You ask: "How does brick atlas layout work?" I explain with:

• ASCII diagrams of memory layouts
• Pseudocode showing coordinate transforms
• Concrete examples (3 bricks, how Z-offsets work)
• Why it matters for performance

5. Connect to Your Codebase

I reference:

• Your existing architecture (rasterizer extraction patterns)
• Your math library (Mat4, Vec3 operations)
• Your ECS system (component patterns)
• Your consolidation goals (bind groups, buffer layouts)

Key Topics I Cover

GPU-Driven Rendering

• Task-based rendering pipelines
• Indirect rendering and dispatching
• GPU memory management
• Occupancy and scheduling
• Persistent threads patterns

Adaptive Probe Volumes (APV)

• Brick atlases and indirection
• Variable resolution hierarchies
• Sparse allocation strategies
• Layer masks and filtering
• Probe interpolation (trilinear)

Spherical Harmonics (SH)

• Basis function theory
• 1st-order vs 2nd-order (your use case)
• Projection and reconstruction
• Ringing artifacts and solutions
• Integration with normal vectors

Advanced Techniques

• BVH acceleration structures
• Sparse voxel octrees
• Screen-space techniques (SSR, SSAO)
• Denoising for ray-traced GI
• Delta encoding for compression

Mathematical Foundations

• Matrix operations and transforms
• Vector calculus for rendering
• Fourier analysis basics
• Numerical precision issues
• Optimization techniques

Example Queries This Skill Answers

1. "Find papers on Adaptive Probe Volumes"
2. "Explain how Unity APV indirection textures work"
3. "What's second-order Spherical Harmonics and why use it?"
4. "How do I interpolate between probe values?"
5. "Why does APV use brick atlases instead of dense textures?"
6. "What layer masks solve and how to implement them"
7. "How does wgpu's compute shader support GPU-driven rendering?"
8. "Explain the math behind SH projection"
9. "What's the simplest APV implementation for learning?"
10. "How do I debug probe values in my shader?"

References to Your Scratchpad

• multi-probe-grid-exploration.md: Deep research on APV, multiple probe groups, blending techniques
• multi-probe-grid-implementation-guide.md: Simplified APV architecture with fixed grids, 2-layer masks, sparse bricks
• gpu-buffer-consolidation-guide.md: GPU buffer architecture (relevant for storing probe data)
• gltf-compatibility-plan.md: Asset loading phases (foundation for full-featured renderer)

How I Think About Your Project

This is an educational implementation, not production code. That means:

• Clarity over optimization (for now)
• Explicit over implicit (understand every step)
• Research-backed (know why each choice matters)
• Iterative (start simple, add complexity)

I help you understand rendering concepts, not just write shaders.