Neural Rendering for VaM VR — Project Progress
I’ve been developing an experimental integration of NVIDIA Neural Rendering into VaM’s VR rendering pipeline. It is already working locally, and I’d like to share the progress and current capabilities.
The project supports:
- One or two NR passes, with separate temporal histories for each eye and pass.
- Scene depth and motion data, including additional skin motion tracking for multiple characters. Characters are automatically included or excluded based on whether they fall within the NR processing area.
- An adjustable processing area per eye. You can set its width and height instead of processing the entire VR image. A smaller area reduces the GPU workload of NR, while VaM continues rendering the full scene normally.
- Three visual styles, with controls for tone, structure, skin detail, color mode, and white point.
- Optional history and scene depth, with workload reductions where those calculations are no longer needed.
- Menu protection, keeping VaM’s interface readable within the NR processing area.
All settings are available directly through Session Plugins → NRSceneProbe → Open Custom UI. There is no separate overlay or F10 panel, and settings are saved between sessions.
The current implementation is VR-only. Installation includes native components alongside the session plugin; adding the script alone is not sufficient. Once installed, VaM can still be launched through its normal executable.
The visual improvements can be substantial, especially with two passes. However, NR adds GPU workload, and performance depends on the scene, processing area, and pass count. Development and testing so far have been carried out on an RTX 5090, so broader hardware compatibility and performance are not yet established.
There are still limitations. Long hair can show temporal artifacts, and certain styles can produce different shading between the two eyes. Protecting menu pixels also does not completely prevent an open menu from influencing NR’s treatment of the surrounding scene.
This is an experimental community project, not an official NVIDIA or VaM integration. For now, I’m sharing progress and discussing possibilities. No download is being published yet.
Screenshot: NR settings integrated into VaM’s Session Plugins UI, with two-pass rendering selected and a 1920 × 1080 processing area per eye.