Plugins + Scripts VaM ControlNet Capture

[harry kant]

harry kant submitted a new resource:

VaM ControlNet Capture - Session plugin for exporting matching Beauty, Depth, and OpenPose ControlNet images from

VaM ControlNet Exporter

VaM ControlNet Exporter is a Virt-A-Mate 1.x session plugin for AI image generation workflows, especially with ComfyUI and ControlNet.

It exports matching reference images from a selected WindowCamera: Beauty render, Depth map, and OpenPose / COCO-18 pose image.

All exports use the same camera and resolution, so the generated Beauty, Depth, and Pose images line up correctly and can be used together in ControlNet-based workflows...

Read more about this resource...

 

[rabbit1qaz]

im not into AI but intrested in start to using it, what could you achieve in the end with that? could you post some end images/movies with how was it in vam and endresult

 

[harry kant]

im not into AI but intrested in start to using it, what could you achieve in the end with that? could you post some end images/movies with how was it in vam and endresult

Hey, good question!

What is ControlNet?

In Stable Diffusion (A1111 / ComfyUI) you normally only have a text prompt. ControlNet adds an extra image to control how the image is generated.

This plugin exports some crutial control images directly from VaM:

• Beauty - the normal VaM render... you can make edge pictures in Comfy UI via Preprocessor very easy
• Depth - a grayscale image where white = close, black = far. This keeps the exact body shape, pose and camera distance from VaM in the final AI image.
  I had to build a custom render pass for this, because raw depthTexture in VaM misses hair (transparent queue, ZWrite Off) and clothing (cutout queue) - they come from different meshes.
• Pose / OpenPose - the skeleton of your Person atoms. Keeps arms, legs, hands and face position.

What can you achieve with it?

VaM is perfect for posing, but you want a different style in the end. You pose everything in VaM, export Depth + Pose with this plugin, then load those into ControlNet in ComfyUI and generate the final image with any model you like
- same pose as in VaM, but with a completely different look, cartoony or photorealistic.

I will post some end results (VaM viewport vs. final SD result) when I am home, and also include example workflows for ComfyUI.

How and if the ControlNets work depends a lot on the model you use

More details here: ControlNet on GitHub
Good intro for ComfyUI: ComfyUI ControlNet Examples

 

[harry kant]

harry kant updated VaM ControlNet Capture with a new update entry:

Depth Capture and Preview Fixes

Improved the Depth capture pipeline after version .4.var: active Person bodies, clothing, and hair are now rebuilt from VaM’s current GPU buffers so they better match the Beauty render. Preview timing was also adjusted to reduce duplicate/old-pose Person artifacts, UI layers are excluded from ControlNet passes, and Beauty preview/export now use the same selected active WindowCamera more consistently.

Read the rest of this update entry...

 

[harry kant]

NOTE: Update uploaded and pending moderator approval - not live yet.
Improved Depth capture pipeline:
- Active Person bodies, clothing, hair now rebuilt from VaM's current GPU buffers (matches Beauty)
- Fixed preview timing to reduce duplicate/old-pose artifacts
- Excluded UI layers from ControlNet passes
- Beauty preview/export now use same active WindowCamera consistently

 

[harry kant]

harry kant updated VaM ControlNet Capture with a new update entry:

Depth Ghost Figure Fix and Developer Documentation

Fixed ghost/duplicate figure artifacts in the Depth pass by improving the capture timing and VaM-specific proxy handling, while also adding clearer C# XML documentation for developers.

Read the rest of this update entry...