Plugins VAM-VHT VAM Virtual Home Theatre

Hi all,
I managed to separate the audio channels present in a video file using the Unity built-in videoplayer and to assign each to a different audiosource (the spheres you see in the pictures), mantainig the spatial audio feature.
The result is you have "virtual real" audio speakers you can position in the virtual space at will, with each audio channel coming out from each speaker indipendently. You can position the speakers literally with your virtual hands if you like.

This allows to create a "virtual real" home theatre environment inside VAM (intended to be used in VR, not too much sense to use it in desktop mode even if it works).

It supports 2 (stereo), 6 (5.1) and 8 (7.1) channels, allowing to enjoy Dolby Digital/DTS and Dolby TrueHD/DTS-HD Master -like audio environments.

Audio is enhanced with the use of HRTF functions for better and realistc 3D spatial audio and reacher audio stage (more info here: https://en.m.wikipedia.org/wiki/Head-related_transfer_function)

The plugin comes with an in-world UI with the following features:

- Open a video file from everywhere in your VAM folder (I didn't put any file extention filter, anyway I'll talk about file formats later). A reset button can clear the opened file
- Play/Pause/Stop the videoplayer
- A time scrub / seeker slider with a timer set on the lenght of the movie being played
- A screen size slider to resize the screen dimensions, with reset button
- A screen curvature slider to curve the screen, with reset button
- An audio offset slider to offset audio and video should they not be synchronized, with reset button
- An audio speed slider to adjust audio speed should it go faster or slower than video (you could experience this on the long run, like 10 or 20 minutes of playback, depending on your PC performance), with reset button
- A toggle to activate the screen emitted light according to the content being played on screen
- A toggle to activate a rear black panel to keep the 16/9 aspect ratio with black bars in case of videos with aspect ratio different from 16/9
- A toggle to hide or show the spheres/speakers (you don't want them being visible while watching a movie)
- A toggle to activate video looping
- Separate audio volume sliders for each audio channel
- A matrix to manually switch (swap) the channel assigned to each speaker should they not be correctly mapped in the video file (i.e. the Central channel could be wrongly assigned to the Front Left speaker and so on)
- A (yellow) button to drag & drop the UI in the virtual space

The UI auto hides: just look at the yellow button (or at its immediate right) to pop it up.

The Virtual Home Theatre auto checks the channels number in the audio track and sets the number and position of the spheres/speakers consequently. It sets the UI consequently too.

The Virtual Home Theater comes as a SubScene you can load in your preferred scene/environment (in Edit Mode add a SubScene Atom and in the SubScene tab click on "Load Existing..."). The SubScene includes the screen, the UI and the 8 audiosources/spheres.
Two demo Scenes are also included: a basic environment and a movie theater environment, both featuring the default VAM mate. I used the MacGruber Life plug-in too in the Scenes. Thanks a lot to MacGruber for the Life plug-in and to Romolas for the Cinema Environment.

For those not familiar with Home Theater environments, the speaker labels are as follows:
- FL: Front Left Speaker
- FR: Front Right Speaker
- C: Central Speaker
- LFE: Low Frequency Element Speaker (a.k.a. SubWoofer)
- BSL: Back Surround Left Speaker
- BSR: Back Surround Right Speaker
- SL: Surround Left Speaker
- SR: Surround Right Speaker

A nineth little white sphere is present, both in the SubScene and in the demo Scene, with the "Listening Spot" label, which represents the spot where you should position your head for ideal listening. The speaker spheres are indeed positioned according to the position of the little white one.
In particular objects parenting is as follows:
- BSL, BSR, SL, SR, and the screen (EmptyScreen Atom) are directly parented to the Listening Spot sphere: this means that if you move the Listening Spot sphere, the other mentioned objects will move consequently
- FL, FR, C, and LFE are parented to the screen: this means that if you move the screen (moving the EmptyScreen Atom), the other mentioned objects will move consequently

The Virtual Home Theater autoadjusts volume attenuation so that the volume never become too low even if you place the speakers far away from the listening spot (like in a cinema environment for exemple), though mantainig attenuation if you move around the listening spot

About file format and conversion:

You'll mostly have MKV file and AC3 (5.1) audio. Due to the built-in videoplayer and Windows Media Foundation limitations, those files are not directly compatible with the player.
You'll have the best results with "MP4" (suggested for stereo and 5.1 channels) and "webm" (suggested for 7.1 channels) formats. Both MP4 and webm are just containers for the video and audio tracks. Long story short, here are the best formats (and the formats you should convert your file into) for each case, with the (free) tools and parameters to use for conversion:

- Stereo: use MP4 container with x264 video encoder and AAC (2 channels) audio encoder. Use "AnyVideoConverter" (https://www.anvsoft.com/any-video-converter-free.html) with the following parameters:
- Be sure to select the audiotrack you want to convert (should your file have multiple audio tracks) from the specific dropdown box in the movie panel
- Profile: Customized MP4 movie (*.mp4)
- Basic Settings:
Video Size: Original
Quality: High
- Video Options:
Video Codec: x264
Video Bitrate: original bitrate
Constant Quality: n/a
Frame Rate: auto
Video Aspect: auto
Encode Pass: 1
- Audio Options:
Audio Codec: AAC
Audio Bitrate: 320
Sample Rate: 48000
Audio Channels: 2

- 5.1: same as stereo but with "Audio Channels: 6" in the Audio Options

- 7.1: Use webm container with VP8 video encoder and Vorbis audio encoder. Use "Shutter encoder" (https://www.shutterencoder.com/en/) with the following parameters:
- Choose function:
Output codecs: VP8
- Bitrates adjustment:
Video bitrate: 4000 (write it by hand)
Audio bitrate: 320
- Audio settings:
Convert: OGG and switch "Stero" to "Multi" (and be sure 48K is written besides "Multi")
- Click on "Start function" to start conversion

For both 5.1 and 7.1 files, channels should be correctly assigned to speakers in the Virtual Home Theatre and no further action should be required. Anyway the correct mapping for 7.1 file should be as follows (you can always remap them using the in-world UI should they be wrongly mapped):

- FL --> 1
- FR --> 3
- C --> 2
- LFE --> 8
- BSL --> 6
- BSR --> 7
- SL --> 4
- SR --> 5

The plugin comes with a short 7.1 .webm demo video file (a video from Dolby to demo the Dolby TrueHD potential). Use it to test the pugin features (you'll have to remap channels as earlier indicated). UPDATE: it seems to me that probably you have to extract the .webm video file from the .var archive for the videoplayer correctly play it (open the .var archive as if it was a .zip file)

The plugin has been created and tested using Quest2, but I didn't use any VR specific code, so it should work with all compatible VR headsets too.

One last thing: did I mention it supports 3D movies (both Side-by-Side and Over-Under formats)?

Tired of watching movies on a giant screen in your headset all alone? From now on you can do it with your preferred mate! One different each time if you want, without jealousy problems One?... Why just one?...

Comments are welcome, and do not forget to review and rate if you like it

Enjoy!

Credits:
Thanks to Acid Bubbles for his dev tools, very useful.
Thanks to MacGruber for the Life plugin I used in the Demo Scenes.
Thanks to Romolas for the Cinema Envronment I used in one of the Demo Scenes.
Thanks to VamSander: I used his code as a starting point, and for inspiration.