Plugins + Scripts Video Renderer for 3D VR180, VR360 and Flat 2D & Audio + BVH Animation Recorder

[Bob Nothing]

Huh, deleting the .var and redownloading it worked. Must have been cached somehow??? Anyhow, gonna try streaming now

 

[Bob Nothing]

Just in case you didn't see it and might be interested: With my modified version of the renderer sending frames directly to FFmpeg for video encoding (linked and described earlier in this thread), you can save even more time and especially disk space without any loss in quality. If you use the streaming feature, it sends uncompressed frame data to FFmpeg (the JPEG/PNG setting is ignored). The only quality loss you get is in the final video encoding itself, so it will produce the same results as the original version with PNG, but faster and using less disk space: Skipping the JPEG/PNG encoding makes it faster, and it only takes as much disk space as the final video file (as opposed to hundreds of gigabytes) since it doesn't save individual frame images to disk.

Ok, when I try to stream to ffpmeg it crashes when I start the stream from the plugin with this error - tcp://127.0.0.1:54341?listen: Invalid data found when processing input - Any idea what's causing that?

 

[Bob Nothing]

Ok, got it figured out with the help of AI This is my updated fmmpeg command - ffmpeg -y -f rawvideo -pix_fmt rgb24 -s 3840x2160 -r 60 -i tcp://127.0.0.1:54341?listen -vf vflip -c:v libx264 -preset slow -pix_fmt yuv420p -crf 20 -loglevel debug angeldoubledildo.mp4 - Apparently the 'high' isn't valid, it is now set to 'slow' and that seemed to have resolved it. My first realtime render is going now

 

[Bob Nothing]

Just an update, it worked I am now realtime streaming my renders Thank you @yunidatsu!

 

[nibisiwuyila]

Hello
Can this be used for PoV view and how?

 

[Bob Nothing]

Hello
Can this be used for PoV view and how?

You would want to attach the atom with the render plugin to the head, record your PoV animation, then play it back to render.

 

[nibisiwuyila]

You would want to attach the atom with the render plugin to the head, record your PoV animation, then play it back to render.

Thank you, so this is not something like Embody right? I need create an empty atom, link and snyc with head on postion and rotation, then add this plugin to this empty atom, finally I can start the play and recording, right? But here it will cause that the hair and eye will be displayed, how to solve that?

 

[Bob Nothing]

Thank you, so this is not something like Embody right? I need create an empty atom, link and snyc with head on postion and rotation, then add this plugin to this empty atom, finally I can start the play and recording, right? But here it will cause that the hair and eye will be displayed, how to solve that?

You are 100% correct. So, I would start by using VaM to move it to the exact head object, then with the plugin active (so that you can see what it sees) adjust the Render atom forward until it is just past the hair/eyes, then parent it to the head atom. That way you won't have the issue of the hair/eyes getting rendered but you'll get the PoV you are wanting. Also, you can adjust the field of view (FoV) in the plugin to get that just how you want it as well.

 

[polishjelly]

I've had good success with this but would like to record a scene with video plating in the background. 1-2 times the video would play super fast...then it stopped playing at all and just has a frozen tv in the background. any help or thoughts?

im using a videocontroller plugin- not the native video support in this plugin

 

[Bob Nothing]

I've had good success with this but would like to record a scene with video plating in the background. 1-2 times the video would play super fast...then it stopped playing at all and just has a frozen tv in the background. any help or thoughts?

im using a videocontroller plugin- not the native video support in this plugin

I'm not entirely sure I understand your question, but my guess is that you're using Timeline in your scene. ANY instances of timeline need to be set to 'game time' instead of 'real time' in the options or the animations will play super fast inside the render. If you set them to gametime your render will come out perfect.

 

[RonBurgundy]

Ok, got it figured out with the help of AI This is my updated fmmpeg command - ffmpeg -y -f rawvideo -pix_fmt rgb24 -s 3840x2160 -r 60 -i tcp://127.0.0.1:54341?listen -vf vflip -c:v libx264 -preset slow -pix_fmt yuv420p -crf 20 -loglevel debug angeldoubledildo.mp4 - Apparently the 'high' isn't valid, it is now set to 'slow' and that seemed to have resolved it. My first realtime render is going now

I wanted to try encoding into AV1 because it's higher quality than 265 and supported by all browsers but VAM freezes up for some reason shortly after the stream successfully starts.
This is the command i'm using

ffmpeg -y -f rawvideo -pix_fmt rgb24 -s 3840x2160 -r 60 -i tcp://10.10.10.2:54342?listen -vf vflip -c:v libaom-av1 -crf 23 -pix_fmt yuv420p -b:v 0 video.mp4

I've tried yuv420p, yuv422p, yuv444p and the 10bits with no success. Ai chat wasn't able to help me either. Basically it looks like it sucessfully connects the stream and then after a couple secconds packets stop coming into ffmpeg

 

[RonBurgundy]

I wanted to try encoding into AV1 because it's higher quality than 265 and supported by all browsers but VAM freezes up for some reason shortly after the stream successfully starts.
This is the command i'm using

ffmpeg -y -f rawvideo -pix_fmt rgb24 -s 3840x2160 -r 60 -i tcp://10.10.10.2:54342?listen -vf vflip -c:v libaom-av1 -crf 23 -pix_fmt yuv420p -b:v 0 video.mp4

I've tried yuv420p, yuv422p, yuv444p and the 10bits with no success. Ai chat wasn't able to help me either. Basically it looks like it sucessfully connects the stream and then after a couple secconds packets stop coming into ffmpeg

ok i realized that the LIBAOM encoder is just flawed, instead i switched to LIBSVTAV1 (much more efficient and I can actually LOAD my CPU by choosing how many threads to use)
BIG advantages of the AV1 codec?

1. Smaller file size for the same quality compared to H265
2. NOT proprietary and supported by ALL browsers for streaming/playing

ffmpeg -y -f rawvideo -pix_fmt rgb24 -s 3840x2160 -r 60 -i tcp://127.0.0.1:54341?listen -vf vflip -c:v libsvtav1 -preset 4 -crf 20 -pix_fmt yuv422p10le -svtav1-params tune=1:film-grain=0:lp=16:fast-decode=1 -b:v 0 -g 300 -threads 16 video.mp4

The quality is noticeably better using yuv4:2:2p-10bit, which i think is a good compromise. I especially notice the difference when the overall scene is darker (not a lot of light/contrast) with 4:2:0 (8bit) you LOSE a lot of detail in the darker areas and there's a lot of banding.

-g 300 I extended the keyframes to 1 in every 5sec (60x5), which reduces file size but increases seek times especially when streaming. For better seeking, you might be better off doing 2 seconds which would be -g 120

Here's what the Tune means

• tune=0 (PSNR):
  • Command: -svtav1-params tune=0:film-grain=0:lp=16
  • Result: Sharper details, potentially more visible noise or artifacts in complex scenes. Slightly higher bitrate (e.g., 5–10% larger file size) because it preserves pixel fidelity even where it’s less noticeable.
  • Quality: Technically accurate, but not always "pleasing" to the eye.
• tune=1 (SSIM):
  • Command: -svtav1-params tune=1:film-grain=0:lp=16
  • Result: Smoother, more visually appealing output. May soften fine details slightly but reduces perceptible artifacts. Slightly smaller file size (e.g., 5–10% less) due to perceptual optimization.
  • Quality: Better subjective experience, especially for streaming or casual viewing.

Spoiler: Some Nerdy explanations between the different color formats

I asked Grok "Taking the source input into account (rgb24), is there a quality difference between yuv420p, yuv422p or the 10 bit yuv420p101e, yuv422p101e or yuv444p101e? Let us say for example I encoded a 4k/60 video using yuv420p and it was 100mb, approximately what would the file sizes of the other color options?"
Yes, there’s a quality difference between yuv420p, yuv422p, yuv420p10le, and yuv422p10le when encoding your RGB24 source, primarily due to differences in chroma subsampling and bit depth. Since your input is rgb24 (8-bit RGB), let’s break down how these pixel formats affect quality, especially in the context of your AV1 encoding with libsvtav1.

Key Concepts​

1. Chroma Subsampling:
   • RGB24 stores full color information (4:4:4) with separate red, green, and blue channels at 8 bits each, no subsampling.
   • YUV formats split the signal into luma (Y, brightness) and chroma (U/V, color), often subsampling chroma to reduce data while preserving perceived quality.
   • 4:2:0 (e.g., yuv420p, yuv420p10le): Chroma is subsampled to 1/4 the resolution of luma (half horizontally and vertically). Common for video compression, but loses some color detail.
   • 4:2:2 (e.g., yuv422p, yuv422p10le): Chroma is subsampled to half the horizontal resolution of luma, but full vertically. Retains more color detail than 4:2:0, at the cost of larger file sizes.
2. Bit Depth:
   • 8-bit (e.g., yuv420p, yuv422p): 256 levels per channel. Matches your RGB24 input’s bit depth but may introduce banding in gradients during conversion.
   • 10-bit (e.g., yuv420p10le, yuv422p10le): 1024 levels per channel. Offers smoother gradients and better color precision, even if the source is 8-bit, by reducing quantization errors during encoding.

Quality Comparison from RGB24 Source​

Here’s how each format stacks up when converting from your 8-bit RGB24 (4:4:4) input:

1. yuv420p (8-bit, 4:2:0)​

• Chroma: Subsampled to 4:2:0, losing 75% of the color resolution compared to RGB24. Fine for most content, but sharp color edges (e.g., text, graphics) may blur slightly.
• Bit Depth: Matches RGB24’s 8-bit depth, so no additional precision is gained. Gradients may show banding if heavily compressed.
• Quality: Lowest of the four due to aggressive chroma subsampling and no bit-depth advantage. Best for smallest file sizes, but sacrifices detail.

2. yuv422p (8-bit, 4:2:2)​

• Chroma: Subsampled to 4:2:2, retaining twice the horizontal color resolution of 4:2:0. Better for content with horizontal color transitions (e.g., video with text or fine patterns).
• Bit Depth: Still 8-bit, so no improvement over RGB24 in dynamic range or gradient smoothness.
• Quality: Better than yuv420p due to less chroma loss, but still limited by 8-bit depth. Noticeable improvement in color fidelity over 4:2:0, especially for 4K.

3. yuv420p10le (10-bit, 4:2:0)​

• Chroma: Same 4:2:0 subsampling as yuv420p, so color resolution is reduced compared to RGB24.
• Bit Depth: 10-bit encoding provides more precision, reducing banding in gradients and preserving more detail during compression, even from an 8-bit source.
• Quality: Better than yuv420p due to 10-bit depth, despite identical chroma subsampling. The quality boost comes from improved luma and chroma quantization, not color resolution.

4. yuv422p10le (10-bit, 4:2:2)​

• Chroma: 4:2:2 subsampling retains more color detail than 4:2:0, closer to RGB24’s full 4:4:4.
• Bit Depth: 10-bit depth enhances gradient smoothness and compression resilience.
• Quality: Highest of the four. Combines better color resolution (4:2:2) with 10-bit precision, making it the closest to RGB24’s original quality while still being compressed.

5. yuv444p (8-bit, 4:4:4):​

• Chroma: No subsampling (4:4:4), matching RGB24’s full color resolution. Every pixel retains its original color detail.
• Bit Depth: 8-bit, same as RGB24, so no additional precision beyond the source.
• Quality: Superior to yuv420p and yuv422p due to full chroma fidelity. Matches RGB24’s color resolution but may still show banding in gradients due to 8-bit limitations.

6. yuv444p10le (10-bit, 4:4:4):​

• Chroma: Full 4:4:4, preserving all color detail from RGB24.
• Bit Depth: 10-bit, offering smoother gradients and better compression resilience than RGB24’s 8-bit.
• Quality: Highest possible quality among these options. It exceeds RGB24’s precision due to 10-bit depth while retaining full chroma, making it ideal for archival or professional use.

Quality Ranking (RGB24 Source)​

From your 8-bit RGB24 input:

1. yuv444p10le: Best (full chroma + 10-bit depth).
2. yuv444p: Excellent (full chroma, but 8-bit limits gradients).
3. yuv422p10le: Very good (half chroma horizontally + 10-bit).
4. yuv420p10le: Good (quarter chroma + 10-bit).
5. yuv422p: Moderate (half chroma horizontally, 8-bit).
6. yuv420p: Lowest (quarter chroma, 8-bit).

• Chroma Impact: 4:4:4 (yuv444p, yuv444p10le) retains all color detail, making it noticeably sharper for synthetic content (e.g., text, UI) compared to 4:2:2 or 4:2:0. For natural video, the difference is subtler but still visible in color-rich scenes.
• Bit Depth Impact: 10-bit formats (yuv420p10le, yuv422p10le, yuv444p10le) reduce banding and improve quality under compression, even from an 8-bit source, due to better quantization.

Practical Impact​

• From 8-bit RGB24: Since your source is 8-bit, 10-bit formats (yuv420p10le, yuv422p10le) don’t add new information but improve encoding quality by reducing artifacts (e.g., banding, color bleed) during compression. The jump from 4:2:0 to 4:2:2 is more noticeable with high-contrast color content (e.g., gaming, UI footage) than plain video.
• 4K 60fps: At this resolution and frame rate, chroma subsampling differences (4:2:0 vs. 4:2:2) are more apparent on large screens or with detailed content. 10-bit helps maintain quality under AV1’s compression.

Recommendation for Your Use Case​

Given your priorities (quality slightly over size, streaming, 4K 60fps, max CPU usage with libsvtav1):

• yuv422p10le: Best quality option. Retains more chroma detail (4:2:2) and leverages 10-bit encoding. File size will be larger than 4:2:0, but since you prioritize quality, this aligns with your goals. Still streamable with AV1’s efficiency.
• yuv420p10le: If file size becomes a concern, this is a strong compromise. 10-bit improves quality over 8-bit, and 4:2:0 keeps sizes smaller. Most streaming platforms (e.g., YouTube) use 4:2:0, so it’s highly compatible.
• Avoid 8-bit (yuv420p, yuv422p): Since AV1 supports 10-bit natively and your CPU can handle it, 8-bit limits quality unnecessarily.

File Size Estimation​

File size in video encoding depends on:

• Chroma Subsampling: More chroma data (4:4:4 > 4:2:2 > 4:2:0) increases bitrate.
• Bit Depth: 10-bit increases data per pixel (~20–30% more than 8-bit).
• Content: Complex scenes (e.g., high motion, detail) amplify these differences.
• Encoder Efficiency: AV1 (e.g., libsvtav1) compresses better than H.265, but relative trends hold.

Let’s assume:

• Your 4K 60fps video encoded with yuv420p (8-bit, 4:2:0) at CRF 20 is 100MB.
• Same CRF, preset (e.g., 6), and duration for all formats.
• Typical content (e.g., mixed natural and synthetic video).

Here’s an approximate file size comparison:

1. yuv420p (8-bit, 4:2:0): 100MB (baseline).
   • Minimal chroma data, 8-bit efficiency.
2. yuv422p (8-bit, 4:2:2): ~130–150MB.
   • Double the horizontal chroma resolution increases bitrate by ~30–50%. Exact increase depends on color complexity.
3. yuv420p10le (10-bit, 4:2:0): ~120–140MB.
   • 10-bit depth adds ~20–40% more data over 8-bit, even with same chroma subsampling, due to larger sample size.
4. yuv422p10le (10-bit, 4:2:2): ~150–180MB.
   • Combines 4:2:2’s ~30–50% chroma increase with 10-bit’s ~20–40% overhead. Typically 50–80% larger than yuv420p.
5. yuv444p (8-bit, 4:4:4): ~180–220MB.
   • Full chroma (4x the chroma data of 4:2:0) increases size significantly, ~80–120% over yuv420p. AV1’s compression mitigates some of this.
6. yuv444p10le (10-bit, 4:4:4): ~220–260MB.
   • Full chroma plus 10-bit depth. Roughly 120–160% larger than yuv420p, depending on content and compression efficiency.

Relative Increase Summary​

Format	Approx. Size	% Increase vs. yuv420p
yuv420p	100MB	0%
yuv422p	130–150MB	30–50%
yuv420p10le	120–140MB	20–40%
yuv422p10le	150–180MB	50–80%
yuv444p	180–220MB	80–120%
yuv444p10le	220–260MB	120–160%

---

Factors Affecting Size Variation​

• CRF: A lower CRF (e.g., 15) amplifies these differences; a higher CRF (e.g., 25) compresses them closer together.
• Content: High-motion or colorful content (e.g., gaming) increases sizes more with 4:4:4 or 4:2:2 vs. 4:2:0. Simple scenes (e.g., talking head) show less variation.
• Preset: Your preset 6 balances speed and compression. A slower preset (e.g., 4) reduces sizes across all formats but takes longer.

---

Recommendation for Your Use Case​

Given your priorities (quality slightly over size, streaming, 4K 60fps, libsvtav1):

• yuv444p10le: Maximum quality, matching RGB24’s chroma and exceeding its bit depth. File size (~220–260MB) is larger, but AV1 keeps it streamable. Ideal if quality is paramount and bandwidth isn’t a tight constraint.
• yuv422p10le: Excellent balance (~150–180MB). Retains more chroma than 4:2:0, with 10-bit benefits, and is still efficient for streaming.
• yuv420p10le: Most practical for streaming (~120–140MB). Widely compatible, 10-bit quality boost, and smallest size among 10-bit options.

 

[RonBurgundy]

Just in case you didn't see it and might be interested: With my modified version of the renderer sending frames directly to FFmpeg for video encoding (linked and described earlier in this thread), you can save even more time and especially disk space without any loss in quality. If you use the streaming feature, it sends uncompressed frame data to FFmpeg (the JPEG/PNG setting is ignored). The only quality loss you get is in the final video encoding itself, so it will produce the same results as the original version with PNG, but faster and using less disk space: Skipping the JPEG/PNG encoding makes it faster, and it only takes as much disk space as the final video file (as opposed to hundreds of gigabytes) since it doesn't save individual frame images to disk.

BTW i have to tell you that your modification of the plugin is AMAZING!
I have 2 small suggestions if you are able to modify the plugin again.
1. it doesn't remember the IP address when you save the scene, but it remembers everything else (it would be nice if it was kept)
2. This is more important, would you be able to have it support vertical video resolutions and 4:3 as well? 9:16(1080x1920, 2160x3840) and 3:4(1080x1440, 2160x2880)? OR, if ti needs to match up with the host display, even if we set it turned 90/270degrees and DISPLAYS the same, have an option to be able to send the raw images rotated either 90/270/180?

that way i don't have to re-encode videos to rotate them, i'm sure i can do it on the recieving end but it seem like it would make more sense on the plugin side, plus 4:3 and 3:4 would be nice so i don't have to crop and reencode those vids.

this is what i'm currently doing, i rotate the capture plugin 90degrees, then i will re-encode the file again with rotation. seems like a bit of a waste

 

[RonBurgundy]

@Bob Nothing @yunidatsu
so i loaded on of Xacreux's scenes https://hub.virtamate.com/resources/dance-no-2.54069/
and had a problem with it constantly flickering.
I've never seen it before but there are NO session plugins, no postmagic either.

it flickers rapidly between a standard image and an X-ray

 

[Datel]

@Bob Nothing @yunidatsu
so i loaded on of Xacreux's scenes https://hub.virtamate.com/resources/dance-no-2.54069/
and had a problem with it constantly flickering.
I've never seen it before but there are NO session plugins, no postmagic either.

it flickers rapidly between a standard image and an X-ray
View attachment 474657

View attachment 474658

Looks like something in this scene interrupting rendering queue, and seems all meshes has flipped normals every second frame

 

[malek]

Perfect, but it has a bug, when adding any glass object from the reflective category makes it render things in an inverted way, idk how to explain it, but you can reproduce it easily. And thank you for your hard work

@Bob Nothing @yunidatsu
so i loaded on of Xacreux's scenes https://hub.virtamate.com/resources/dance-no-2.54069/
and had a problem with it constantly flickering.
I've never seen it before but there are NO session plugins, no postmagic either.

it flickers rapidly between a standard image and an X-ray
View attachment 474657

View attachment 474658

I'm not sure if this is totally related, because it doesn't flicker for me, its just an on/off thing once you place or remove the reflective object

 

[Link404]

Hi! Love the plugin, I've been using this with an MMD player, and I was wondering if it was possible to capture while using the player cam (the default camera) and not the external camera that appears on the object that the plugin is added onto. The MMDplayer has it's the ability to control the camera and it just takes over the main player camera. Setting up the external camera that this plugin comes with can be an absolute pain to align the external cam perfectly, compared to the just using the default camera.

If anyone knows if it's possible to just record the default camera instead. I would love to know.

 

[RonBurgundy]

Hi! Love the plugin, I've been using this with an MMD player, and I was wondering if it was possible to capture while using the player cam (the default camera) and not the external camera that appears on the object that the plugin is added onto. The MMDplayer has it's the ability to control the camera and it just takes over the main player camera. Setting up the external camera that this plugin comes with can be an absolute pain to align the external cam perfectly, compared to the just using the default camera.

If anyone knows if it's possible to just record the default camera instead. I would love to know.

made a small guide on this here:
https://hub.virtamate.com/threads/t...ing-for-timeline-animations.49431/post-149899

 

[cqhkj]

Is there any way to record subtitles made by vamoverlays?

 

[Askeladd]

Can this record 2D camera viewport into a 2D video ? So basically just do offline rendering
I am not sure I understand the description

 

[RonBurgundy]

Can this record 2D camera viewport into a 2D video ? So basically just do offline rendering
I am not sure I understand the description

yes that's what i do, it also records audio, which some others don't do