Plugins Video Renderer for 3D VR180, VR360 and Flat 2D & Audio + BVH Animation Recorder

[Eosin]

Eosin submitted a new resource:

VR Video and Image Renderer for 3D VR180 and VR360 - Render VR180 and VR360 3D videos and images at ultra-high resolutions and framerates.

This plugin renders VR180 and VR360 videos and images inside Virt-A-Mate in stereoscopic 3D and monoscopic view. You can render at very high levels of resolution, quality and frame rate that would be impossible in real time, to create VR videos that play back perfectly smooth even on less powerful machines or mobile headsets, all while potentially looking better than the native game. Performance is excellent thanks to the LilyRender360 shader rendering out more than 100 frames per minute at...

Read more about this resource...

 

[EMPIR]

Need help.

Is this correctversion of ffmpeg? Also are you talking about this exe

? Where to put those command line?

Need more explanation.

 

[ragingsimian]

Do you take cash, kisses, or both? Because I've been wanting someone to release a plugin like this since the day I first learned about VAM.

You have an unnecessary feature for VR180 and VR360 images and video.
There shouldn't be any options for aspect ratio
VR180 mono is 1:1 per eye - you have 1:1 so that's fine
VR180 stereo is 2:1 SBS - you have 16:9 which is close and can be fixed in edit.
VR360 stereo top bottom of which gets you back to 1:1

This capture below for example is from the plugins 1:1 setting. It has horizontal squeeze to become SBS-HALF instead of SBS-FULL.
16:9 is better but you still have to resize (squish) the image vertically in the editing process to get to 2:1 (SBS-FULL)

Here are some cool VR360 stereo test videos to see what I mean

Can 8K Make A Difference in 360 VR Video Experiences?

Introducing a benchmark tool for comparing playback quality

medium.com

Notice all of the resolutions are 2:1 or 1:1 unless squeezed to SBS-HALF

1. visbit-8k-stereo.mp4: YUV444P, 7680 x 7680 @ 30fps
2. visbit-6k-stereo.mp4: YUV444P, 5760 x 5760 @ 30fps
3. visbit-6k-stereo-yuv420.mp4: YUV420P, 5760 x 5760 @ 30fps
4. visbit-4k-stereo.mp4: YUV444P, 3840 x 3840 @ 30fps
5. visbit-4k-stereo-yuv420.mp4: YUV420P, 3840 x 3840 @ 30fps
6. visbit-4k-squeeze-stereo.mp4: YUV444P, 3840 x 1920 @ 30fps
7. visbit-4k-squeeze-stereo-yuv420.mp4: YUV420P, 3840 x 1920 @ 30fps

Real world videos can be different depending on the camera, lenses and editing.

 

[ragingsimian]

Need help. View attachment 75469 Is this correctversion of ffmpeg? Also are you talking about this exe View attachment 75470? Where to put those command line?

Need more explanation.

You are using windows so you need to type it into a windows command prompt. Either CMD or PowerShell will work. Whichever you are used to.

Be careful if you look up FFMPEG instructions online. Those will usually be Linux or MAC users. The command lines have slightly different behavior between Windows and unix command lines.

Davinci Resolve (it's free) or Abobe Premier (not free) also import image sequences for editing. You can also do it with Blender's (free) video editor.

I would check whatever video editor you plan to use to see if it can import video sequences before encoding to video then bringing it onboard. Saves you a step and drive space.

 

[EMPIR]

What am i doing wrong here? Placed exe files in vr render folder and run powershell. go tthis error

 

[ragingsimian]

View attachment 75474
What am i doing wrong here? Placed exe files in vr render folder and run powershell. go tthis error

For windows name of the program is "ffmpeg.exe" not just "ffmpeg"

Also in windows the slash is the other direction for directories.
You need to replace "output8k/" with "output8k\" instead.
You might also end up needing to put the parameters that have "%" in them in quotes to keep windows from treating that special. Maybe, I'm not 100% sure on that.

 

[VAMguy]

Is there an option to adjust the FOV? I attached the clist to an Empty even I place it up close to the people's face. The FOV is still too wide like 5 meters away.

 

[Eosin]

Eosin updated VR Video and Image Renderer for 3D VR180 and VR360 with a new update entry:

Version 1.1 with flat video support, preview window, mute sound, hotkeys

Version 1.1

- Added flat video recording, with supersampling by MacGruber.
- Mute all sound while recording.
- Added preview window.
- Added hotkeys.
- Fixed 6K, 7K, 12K resolution
- Reduced VRAM consumption slightly.
- Improved memory estimate (maybe).
- Added warnings for texture size and VRAM consumption exceeding capabilities of GPU.

Read the rest of this update entry...

 

[Saint66]

This is just too great, easy to use. You have to get used to place the Empty right for your needs, but with the screenshot function this is easy to estimate.
Would be supercool if the plugin could do the video encoding afterwards automatically, some ppl most likely don't get the ffmpeg part right.
Also sound recording would be even more sexy

But as I said, superb plugin, tyvm!

 

[Lione]

Hej, this looks really interesting from the Overview

Can you please explain a workflow for

- Creating VR 180 Stereo Screenshots

- Creating a VR 180 Stereo Video


- Do i have to disable the Menu so it wont be captured sametime ?

I tried it today( PNG and JPG ) and i have 600 pics with all the same Screenshot now not even showing the Model which i placed correct and i even moved the Camera around her slowly stutterish.

I also experienced that my PC is on 100% load if i even click start recording.
I cannot move mouse or anything smooth on a i9 10900k / 64 GIG RAM and a RTX 3080

-What VR players ( PC and Oculus Quest ) can playback the Video Screenshots and Videos and what tagging do the files need for example in Quest 2 / Pigasus player to be detected accurate then ?

Is the Plugin meant to be used on desktop or in VR sametime ?

Also is there a Stop button once pic creation starts ?

You see so many questions and i am sure i did something wrong receiving 600 identical screenshots not even showing a face

 

[Eosin]

Eosin updated Video and Image Renderer for 3D VR180 and VR360 and Flat 2D with a new update entry:

1.2: Small change to framerate handling

- Game's physics rate now remains untouched. Instead using Unity capture time setting for frame rate control.
- Changed hotkeys

This should only affect the case of rendering faster than the physics rate, such as when using the frame skip option.

Read the rest of this update entry...

 

[Saint66]

Hej, this looks really interesting from the Overview

Can you please explain a workflow for

- Creating VR 180 Stereo Screenshots

- Creating a VR 180 Stereo Video


- Do i have to disable the Menu so it wont be captured sametime ?

I tried it today( PNG and JPG ) and i have 600 pics with all the same Screenshot now not even showing the Model which i placed correct and i even moved the Camera around her slowly stutterish.

I also experienced that my PC is on 100% load if i even click start recording.
I cannot move mouse or anything smooth on a i9 10900k / 64 GIG RAM and a RTX 3080

-What VR players ( PC and Oculus Quest ) can playback the Video Screenshots and Videos and what tagging do the files need for example in Quest 2 / Pigasus player to be detected accurate then ?

Is the Plugin meant to be used on desktop or in VR sametime ?

Also is there a Stop button once pic creation starts ?

You see so many questions and i am sure i did something wrong receiving 600 identical screenshots not even showing a face

As already described, add i.e. an Empty atom to your scene and place it facing the content you want to record. Add the VR Recorder as plugin to the Empty atom and open custom settings.

Taking a single screenshot or a video should be self-explaining, there are buttons for it

- you don't have to disable menu, it wont be recorded
- the output is only pictures, even if you choose to take video. You have to merge them as described with i.e. FFMPEG. Examples for a command line to start this: (replace the strings in bold which wahtever name you specified):
ffmpeg -framerate 30 -i output_%06d.jpg -c:v libx265 -level:v 6.2 -crf 25 output8k.mp4
ffmpeg -framerate 30 -f image2 -i output_%06d.jpg -vcodec libx264 -crf 25 out.mp4
- VAM runs like 2 frames/sec during the rendering, you should just leave it alone until its finished
- I have tried it only in desktop, works flawless. Dunno about VR, but I can guess the additional performance would explode my PC
- Stop button is "Esc", like explained.
- I didnt tried animating the Empty yet for camera movement, this could be interesting
- if you dont see your model or desired part of the scene: add Passenger as plugin for the empty and posess it, this will give you a clue what will render out. I position the Empty, do a single screenshot and adjust it that way.
- I tried Skybox VR player on Quest2, works like a charme. Any other VR player which supports 180/360 degrees should be fine as well

 

[Eosin]

Hallo, thanks for your comments.

Do you take cash, kisses, or both? Because I've been wanting someone to release a plugin like this since the day I first learned about VAM.

You have an unnecessary feature for VR180 and VR360 images and video.
There shouldn't be any options for aspect ratio
VR180 mono is 1:1 per eye - you have 1:1 so that's fine
VR180 stereo is 2:1 SBS - you have 16:9 which is close and can be fixed in edit.
VR360 stereo top bottom of which gets you back to 1:1

This capture below for example is from the plugins 1:1 setting. It has horizontal squeeze to become SBS-HALF instead of SBS-FULL.
16:9 is better but you still have to resize (squish) the image vertically in the editing process to get to 2:1 (SBS-FULL)

Thank you, this is a great suggestion, I added a 2:1 setting now and more resolution options for 1:1 output. However since everything gets projected back onto a sphere, rendering at a wider aspect ratio than 2:1 should really only mean that horizontal quality will be slightly better than vertical quality, no? The videos still look correct in VR even if you render 180° to 64:9, but at very low resolutions it can become apparent that there is less data vertically. This is virtually imperceptible at high resolutions on 16:9 though.

What am i doing wrong here? Placed exe files in vr render folder and run powershell. go tthis error

When you use PowerShell you have to prefix commands like so:

.\ffmpeg -i xxx output.mp4

However you can open a traditional command line by using Start -> Run -> "cmd".
You can also create a .bat file to execute command line commands. Note that for batch files you have to replace % with %% because % is a special character in .bat.
There are also different GUI tools for ffmpeg you can find on Google.

Is there an option to adjust the FOV? I attached the clist to an Empty even I place it up close to the people's face. The FOV is still too wide like 5 meters away.

For VR content it is nearly unheard of to use an FOV other than 180 or 360, since they will be viewed inside a headset where everything gets undistorted, and all VR video players and websites expect 180 or 360 degree content. For the 2D video recording I added today you can choose a lower FOV to create a traditional video.

Hej, this looks really interesting from the Overview

Can you please explain a workflow for

- Creating VR 180 Stereo Screenshots

- Creating a VR 180 Stereo Video


- Do i have to disable the Menu so it wont be captured sametime ?

I tried it today( PNG and JPG ) and i have 600 pics with all the same Screenshot now not even showing the Model which i placed correct and i even moved the Camera around her slowly stutterish.

I also experienced that my PC is on 100% load if i even click start recording.
I cannot move mouse or anything smooth on a i9 10900k / 64 GIG RAM and a RTX 3080

-What VR players ( PC and Oculus Quest ) can playback the Video Screenshots and Videos and what tagging do the files need for example in Quest 2 / Pigasus player to be detected accurate then ?

Is the Plugin meant to be used on desktop or in VR sametime ?

Also is there a Stop button once pic creation starts ?

You see so many questions and i am sure i did something wrong receiving 600 identical screenshots not even showing a face

Most of your questions should already have been answered kindly by Saint66 above. If you receive only identical images, it just means that your animation isn't playing. So you can start it manually or use the combined button to start playback and record (assuming you have an animation). If you have something else that causes movement (like a plugin) you can just press record video or use the hotkey (F11).
I haven't experienced any system slowdown, indeed your CPU should actually experience reduced load while rendering because it has to wait for the GPU to render out the frame before it can advance the animation/simulation.
Can you confirm that this problem still occurs with the current version?
You can try setting Process Priority to Normal in the VaM user setting and see if that changes anything.
If you want to move the camera during the video, you would animate that in advance on the Atom holding the plugin, you're not really intended to use VaM or VR while a video is being recorded, although you could probably take single screenshots from inside VR.

 

[Eosin]

Eosin updated Video and Image Renderer for 3D VR180 and VR360 and Flat 2D with a new update entry:

1.3: Added 2:1 aspect ratio and more options for 1:1 resolutions

- Added 2:1 aspect ratio and more options for 1:1 resolutions

Read the rest of this update entry...

 

[FredricRed]

So trying to get this straight, is that I use this plugin to create a "Slideshow" of sorts to then place in another program to create a 180-360 VR Video? One I can watch in VR?

 

[Eosin]

Eosin updated Video and Image Renderer for 3D VR180 and VR360 and Flat 2D with a new update entry:

v5: fixes for wrong FOV & wrong preview color

- Fixed incorrect field of view calculation for preview and flat renders
- Fixed preview window in wrong color space
- Preview window remains open during render to verify correct orientation

Read the rest of this update entry...

 

[ragingsimian]

Hallo, thanks for your comments.


Thank you, this is a great suggestion, I added a 2:1 setting now and more resolution options for 1:1 output. However since everything gets projected back onto a sphere, rendering at a wider aspect ratio than 2:1 should really only mean that horizontal quality will be slightly better than vertical quality, no? The videos still look correct in VR even if you render 180° to 64:9, but at very low resolutions it can become apparent that there is less data vertically. This is virtually imperceptible at high resolutions on 16:9 though.

Overlap isn't used to deal with aliasing by a player unless it has anti-aliasing code. It will just toss the extra pixels from the overlap. It will do it either by cropping or "squishing" by tossing extra. It's a waste of rendering if you send the wrong aspect ratio to a player.
If you put 16:9 into a video editor you can pick an algorithm like bicubic, lancsoz, that tries to use the extra pixel data for more anti-aliasing. (It's basically super sampling) But that can also create artifacts.

For VR content it is nearly unheard of to use an FOV other than 180 or 360, since they will be viewed inside a headset where everything gets undistorted, and all VR video players and websites expect 180 or 360 degree content. For the 2D video recording I added today you can choose a lower FOV to create a traditional video.

Sex-Like-Real uses 200 degrees not 180 for the FOV for the films they make directly.

When I was doing cube map projects with blender I would render at 270 (not render the rear face) but then turn that into a VR360 equirect.

You get to look fully left and right and if you are sitting in a chair - you don't look backwards.

Saves rendering time. So 270 should be easy. Don't render the rear face.

 

[ragingsimian]

Actually ... Instead of giving an FOV option it would be easier to let people pick sides that won't be rendered and instead use a solid RGB color for that side.
ASSUMING the library or API or whatever that's providing the cube map camera(s) give you access to decide the fate of the faces before sending it to the cool seamless stitcher.

 

[morkork]

this is amazing, thank you!

 

[VAMguy]

For an "easier" way to encode those hundreds of pic to movie using ffmpeg command. I use avidemux (free). Just drag the first pic (_000000.jpg) to the avidemux window. It will import the squent pic then choose your encoder and config to get your movie. Before encode you also can edit the movie, resize, add filter, sound, BGM etc.

Avidemux - Downloads

 

[Eosin]

Sex-Like-Real uses 200 degrees not 180 for the FOV for the films they make directly.
When I was doing cube map projects with blender I would render at 270 (not render the rear face) but then turn that into a VR360 equirect.
You get to look fully left and right and if you are sitting in a chair - you don't look backwards.
Saves rendering time. So 270 should be easy. Don't render the rear face.

Very interesting, I didn't even know this. But now that you mention it, I remember that DeoVR has those 200° and 220° choices (but I never once used them). However I assume that these are fisheye projections and not equirectangular as I tried rendering out something in 200° and 220° and unfortunately the results were distorted in DeoVR with those lens profiles. I think since we are not constrained by the limits of real world lenses, in most cases users would just render out a full 360° view to give the viewer more peripheral vision, instead of something in between 180 and 360. At 8K resolution currently about 70-90% of time is spent encoding images to JPG/PNG, 10-25% of time is spent converting cubemap to equirectangular, and <5% of time is spent on physics and rendering 14 cameras, so unfortunately rendering 2 fewer cameras doesn't make a dent.

For an "easier" way to encode those hundreds of pic to movie using ffmpeg command. I use avidemux (free). Just drag the first pic (_000000.jpg) to the avidemux window. It will import the squent pic then choose your encoder and config to get your movie. Before encode you also can edit the movie, resize, add filter, sound, BGM etc.

Avidemux - Downloads

I just tried this out, indeed I heard about it before but forgot about it. This is actually much better than ffmpeg for the average user like me, and you still have access to all those advanced options. Thank you, I will add this to the description.

 

[Jackaroo]

Awesome work on this! Im wonderig if it will ever be possible to allow it to record Reshade effects as well?

 

[ragingsimian]

... 10-25% of time is spent converting cubemap to equirectangular, and <5% of time is spent on physics and rendering 14 cameras, so unfortunately rendering 2 fewer cameras doesn't makes a dent.

I was going to ask you about that. PNG is brutally slow but you get full color to manipulate in a video or image editor.

Since I do LUTS and other color items with Davinci Resolve I don't record with Post Magic turned on. It's useful get the light right that will get the effect you want in post but then I turn it off to render. Another speed boost. BUT to do that I have to use PNG to get as much raw color data to process with.

TIFF would be disk destroying huge but can you do OpenEXR? That is much faster to compress and decompress for quick editing pipeline use.
" Zip (16 scanline blocks) deflate compression applied to blocks of 16 scanlines at time. This tends to be the most effective style of compression to use with rendered images that do not have film grain applied. "

OpenEXR - Wikipedia

en.wikipedia.org

https://rainboxlab.org/downloads/documents/EXR_Data_Compression.pdf

 

[Eosin]

Eosin updated Video and Image Renderer for 3D VR180 and VR360 and Flat 2D with a new update entry:

v6: Fix video recording not stopping, add preview crosshair and resizing + smooth stitching support

- fixed video recording not actually stopping automatically for non-integer recording times (now integer only)
- added preview resizing and preview crosshair
- preview window is now always square when rendering VR
- added support for LilyRender360 smooth stitching (improves results when rendering some forms of post-processing, but increases VRAM consumption)
- increased default preview FOV to 140 to better estimate VR view
- default MSAA to off, filename empty
- remove infobox due to lack of...

Read the rest of this update entry...

 

[PixelPorts]

Anyone help me! i cant seem to record at all! i click on all the buttons and nothing happens only the menu goes away and doesnt even record @Eosin please help.