VAM-VHT VAM Virtual Home Theatre

Plugins VAM-VHT VAM Virtual Home Theatre

Sinful_Rose

Member
Messages
32
Reactions
61
Points
18
Sinful_Rose submitted a new resource:

VAM-VHT VAM Virtual Home Theatre - A virtual home theater environment for VAM with support for 2.0, 5.1 and 7.1 discrete audiosources

View attachment 196517
View attachment 196518
View attachment 196519
View attachment 196520

Hi all,
I managed to separate the audio channels present in a video file using the Unity built-in videoplayer and to assign each to a different audiosource (the spheres you see in the pictures), mantainig the spatial audio feature.
The result is you have "virtual real" audio speakers you can position in the virtual space at will, with each audio...

Read more about this resource...
 
@Bob Nothing I downloaded an Atmos demo video and did some research. Atmos introduces the concept of Objects instead of "classical" channels. For home environments typically Objects are encoded in 7.1 channels with additional metadata (Joint Object Coding, JOC ) with each element’s Object Audio Metadata (OAMD) and JOC payload included in the bitstream. At play time, an Object Audio Renderer (OAR) uses the OAMD to render the Atmos audio appropriately to the listening environment.

So, I could add as many audiosources as needed, but I would lack the OAR component which would "translate" the 7.1 channels in a Dolby Atmos rendering. So implementing Atmos simply can't be done.

Hope it is sufficently clear. Read here for more info: https://tutorials.hybrik.com/dolby_atmos/ (the "Dolby Atmos for the Home" section in particular).
 
@Bob Nothing I downloaded an Atmos demo video and did some research. Atmos introduces the concept of Objects instead of "classical" channels. For home environments typically Objects are encoded in 7.1 channels with additional metadata (Joint Object Coding, JOC ) with each element’s Object Audio Metadata (OAMD) and JOC payload included in the bitstream. At play time, an Object Audio Renderer (OAR) uses the OAMD to render the Atmos audio appropriately to the listening environment.

So, I could add as many audiosources as needed, but I would lack the OAR component which would "translate" the 7.1 channels in a Dolby Atmos rendering. So implementing Atmos simply can't be done.

Hope it is sufficently clear. Read here for more info: https://tutorials.hybrik.com/dolby_atmos/ (the "Dolby Atmos for the Home" section in particular).
I greatly appreciate you taking the time to research it. Thanks for looking into it!
 
Finally it has been made. We needed something like this for a long time. Thank you!

2 suggestions: one allow us to change aspect ratio of the movies. I had one movie that it seems had no black bars encoded but on the screen in vam everything was squeezed making everyone look thin and tall. Maybe a drop down list with standard aspect ratios to choose along with manual sliders for those odd encoded movies. Second if possible implement HRTF profiles. A vr video player on steam does this well VHT. It has a demo so you can try it out to see what I mean. Your plugin the audio is separated but I get little sound stage and depth from audio. VHT on steam with the right HRTF profile makes the sounds very 3D/realistic.

Thank you again you rock pal! Off to start encoding some movies now.
 
@Sam Gold Glad you appreciate ?
As for aspect ratios I chose to have fixed aspect ratio to 16:9 as most the videos are encoded with that ratio. Anyway you probably still have to re-encode your files for format support, just re-encode the video to 16:9 too at the same time. Let me know if this can be a solution for your videos.
As for HRTF I know what you're talking about and I know VHT from steam. A piece of software which costs 30 euros developed since 2019. Great audio implementation but poor attention to environments. Anyway I'll have a look to how HRTF are implemented and if they can be applied to VAM and Unity built-in videoplayer. Not very confident about it though. In the meanwhile, to improve sound stage, try playing with the audiosources moving them around (play with the distance/height from the listening spot). I didn't too much test in that sense, more interested in making it working ?
Thanks a lot for your suggestions. Cheers?
 
Last edited:
@Sam Gold Glad you appreciate ?
As for aspect ratios I chose to have fixed aspect ratio to 16:9 as most the videos are encoded with that ratio. Anyway you probably still have to re-encode your files for format support, just re-encode the video to 16:9 too at the same time. Let me know if this can be a solution for your videos.
As for HRTF I know what you're talking about and I know VHT from steam. A piece of software which costs 30 euros developed since 2019. Great audio implementation but poor attention to environments. Anyway I'll have a look to how HRTF are implemented and if they can be applied to VAM and Unity built-in videoplayer. Not very confident about it though. In the meanwhile, to improve sound stage, try playing with the audiosources moving them around (play with the distance/height from the listening spot). I didn't too much test in that sense, more interested in making it working ?
Thanks a lot for your suggestions. Cheers?
Yeah but, not but, yeah but... Just tried forcing 16:9 on Vampire Hunter D from 1985 and it looks squashed. I would have to put video in a video editor to add bars to fit 16:9. Anyways its your plugin you do what you want.

Still happy for the modern videos I will be able to watch them with my girls in a cool setting. Like you said, VHT on steam lacks environment which really is needed to make it more believable. I even tried making green screens of my vam girls lying in bed with me as 3D png file for steam VHT but lack of movement from girls killed motivation to make more scenes for it.

I will try your suggestion moving speakers around for better sound presence and depth. Cheers!
 
@Sam Gold Good news... I implemented HRTFs... Really great improvement in sound stage!? We've been lucky, as VAM implements a very old Unity version from 2018... They'll be available with next release. I wanna add some other features before releasing, aspect ratios included (but no promises). Stay tuned! ?
 
@Sam Gold Good news... I implemented HRTFs... Really great improvement in sound stage!? We've been lucky, as VAM implements a very old Unity version from 2018... They'll be available with next release. I wanna add some other features before releasing, aspect ratios included (but no promises). Stay tuned! ?

:D Outstanding!

Can't wait but it will give me more time to encode more movies. Now I have to see how to deal with those cpu lags. Might be temps or a vam thing. I saw your plugin handle them fine when they happened no desync on video. Yes it is amazing what has be done with vam using that old Unity with so many limitations. Sex is a powerful motivator at any time period.:unsure:
 
:D Outstanding!

Can't wait but it will give me more time to encode more movies. Now I have to see how to deal with those cpu lags. Might be temps or a vam thing. I saw your plugin handle them fine when they happened no desync on video. Yes it is amazing what has be done with vam using that old Unity with so many limitations. Sex is a powerful motivator at any time period.:unsure:
The lags could be performance issues. Check your VAM frame rate in the Performance tab. Try lowering the graphics and see how it goes. VAM in VR alone is very demanding in terms of computing resources, you are adding on top of it a videoplayer with audio computing and now HTRFs... All this doesn't come for free... I do my best too keep the code simple and performant as I can, but I code on a high end PC with an rtx4090 and sometimes it can't keep the max settings...
 
Sinful_Rose updated VAM-VHT VAM Virtual Home Theatre with a new update entry:

Version 3

Changelog

Major changes:
- Use of HRTF functions for better and realistc 3D spatial audio and reacher audio stage (more info here: https://en.m.wikipedia.org/wiki/Head-related_transfer_function)
IMPORTANT NOTE: HRTFs work properly only with audio encoded at 48KHz (follow the instructions in the plugin overview for a correct encoding)
- Automatic support for all possible video formats: the screen auto-resize according to the video aspect ratio. A rear black panel can be...

Read the rest of this update entry...
 
@Sam Gold version 3 is out now! Let me know if it solves your issues ?
Fantastic! You nailed the 3D positional audio! I can really hear sound moving around me proper. Even heard a few subtle sounds from some clips I made for testing I didn't hear with my Binaural USB DAC. Heard a chair sliding on the far left and I could hear the bare feet from the girl running away from the shooter as if she was running away from me. And you managed to fix the aspect ratios as well, thank you. What a difference from the previous update. Well done!

Suggestions:

1. Have a toggle switch to swap L/R stereoscopic for 3D vids. I have Ant Man and it is playing inverse the 3D and had no way to switch them around.
2. Maybe include a Loop switch for vids. Sometimes there are vids like music videos that would be great to have on loop like having scene mocap dancing with video for backdrop like a concert. Just a suggestion.

Errors I encountered:
1. Whenever cpu spike lag, stereo audio would play from more speakers that I would get a reverb effect. I had to adjust audio sync then reset it and it would fix it. I think it is on my end. No spikes=no audio swapping or duplicating channels.
2. I think your plugin might have problems playing videos with higher frames per second. The standard frame time would be fine just my 48 and 60 fps vids would cause a clipping/pop noise every 5-10 sec. Almost like its is running out of a buffer cache or something. Here is a link with 2 video samples that cause the problems. Test it out on your end and see if it my end. I used the same encoding settings for the lower frame rate vids and they played with no pop/click. https://gofile.io/d/6Z1W10 One is a 5.1 surround demo for testing interpolated to 59 fps and the other Ant Man in 3D at 48 fps.

Oh and the Lighting Effect is the TOPS!! I can't believe how amazing it is seeing the light reflecting on my girls skin while watching the video. VHT on steam needs that effect. So immersive that little detail of light flickering. It looks better than Big Screen on steam with its version of lighting effect. It was too strong felt unreal really.

Now that you got this working how about a separate plugin the make all sounds from vam audio source HRTF ;) I never like that spatial effect. It sounds too much like it's coming from a sewer environment. But it might cause more overhead than needed. Just a thought.

Anyways I will wait to see what you find with those high frame rate vids. Holding off encoding my library but super excited nonetheless.

You Rock!!!:D
 
Fantastic! You nailed the 3D positional audio! I can really hear sound moving around me proper. Even heard a few subtle sounds from some clips I made for testing I didn't hear with my Binaural USB DAC. Heard a chair sliding on the far left and I could hear the bare feet from the girl running away from the shooter as if she was running away from me. And you managed to fix the aspect ratios as well, thank you. What a difference from the previous update. Well done!

Suggestions:

1. Have a toggle switch to swap L/R stereoscopic for 3D vids. I have Ant Man and it is playing inverse the 3D and had no way to switch them around.
2. Maybe include a Loop switch for vids. Sometimes there are vids like music videos that would be great to have on loop like having scene mocap dancing with video for backdrop like a concert. Just a suggestion.

Errors I encountered:
1. Whenever cpu spike lag, stereo audio would play from more speakers that I would get a reverb effect. I had to adjust audio sync then reset it and it would fix it. I think it is on my end. No spikes=no audio swapping or duplicating channels.
2. I think your plugin might have problems playing videos with higher frames per second. The standard frame time would be fine just my 48 and 60 fps vids would cause a clipping/pop noise every 5-10 sec. Almost like its is running out of a buffer cache or something. Here is a link with 2 video samples that cause the problems. Test it out on your end and see if it my end. I used the same encoding settings for the lower frame rate vids and they played with no pop/click. https://gofile.io/d/6Z1W10 One is a 5.1 surround demo for testing interpolated to 59 fps and the other Ant Man in 3D at 48 fps.

Oh and the Lighting Effect is the TOPS!! I can't believe how amazing it is seeing the light reflecting on my girls skin while watching the video. VHT on steam needs that effect. So immersive that little detail of light flickering. It looks better than Big Screen on steam with its version of lighting effect. It was too strong felt unreal really.

Now that you got this working how about a separate plugin the make all sounds from vam audio source HRTF ;) I never like that spatial effect. It sounds too much like it's coming from a sewer environment. But it might cause more overhead than needed. Just a thought.

Anyways I will wait to see what you find with those high frame rate vids. Holding off encoding my library but super excited nonetheless.

You Rock!!!:D
@Sam Gold Hi Sam, I'm glad you appreciate?
Just to say that I downloaded and tested your files: with max settings they both are actually choppy. Then I try to lower all settings (just to extreme the test conditions) and they sound perfect (at least with my actual version which is slightly different form the last distribuited one).
So I can definitely say it's a performance issue. Test them with lowest settings and let me know how they go. Oh, and with the Ant Man one, the SBS 3D works perfectly... Is the one you said has the reverse 3D problem?
 
@Sam Gold Hi Sam, I'm glad you appreciate?
Just to say that I downloaded and tested your files: with max settings they both are actually choppy. Then I try to lower all settings (just to extreme the test conditions) and they sound perfect (at least with my actual version which is slightly different form the last distribuited one).
So I can definitely say it's a performance issue. Test them with lowest settings and let me know how they go. Oh, and with the Ant Man one, the SBS 3D works perfectly... Is the one you said has the reverse 3D problem?
Yeah that movie was reversed. Weird it was fine for you. Anyways check this out...I just found out I have been using an old version of vam 2 updates old ? And to just kick the dog once again, I am now under-volting my gpu. Random crashes in vam every session. Had to drop power by 20% to keep it stable. About to test VR soon let you know how it goes. Ran some tough scenes and gpu doesn't sweat it. Had some cpu lags but 10 min apart. Before it was every 1-2 min.
 
Sinful_Rose updated VAM-VHT VAM Virtual Home Theatre with a new update entry:

Version 4

Changelog:

- Code ha been completely reviewed for improved performance, cleaner audio, even at higher frame rate videos and ambient light turned on
- The audiosources desynch drift bug has been fixed (in previous version you could hear desynched multiple voices coming from the different audiosources)
- The in-world UI now auto hides: just point your controller toward the yellow button to pop it up. You can drag the UI with the yellow button as usual
- Speakers and audio channels are...

Read the rest of this update entry...
 
@Sam Gold I made a new version, better performance and much cleaner sound, but I need at least a beta tester as I don't know if the new version just fit my pc or is good for everyone (audio speed problem: audio goes faster or slower than video, not just a "regular"offset). I fixed it but I don't know if the fix is good just for my pc. Would you be so kind to test it before I release?
Thanks!
 
Last edited:
I love all your work here, I do have a quick question. Do you know of a way with your work on this to record spatial audio out of VAM? So, for example, I create a scene in VAM that I want to record to make a movie, all I've ever been able to get is stereo sound. Is there a way to get the same spatial audio in at least a 5.1 format recording out of VAM? Spatial sound exists in VAM/VR obviously, I just can't figure out how to get spatial audio in the recording.
 
I love all your work here, I do have a quick question. Do you know of a way with your work on this to record spatial audio out of VAM? So, for example, I create a scene in VAM that I want to record to make a movie, all I've ever been able to get is stereo sound. Is there a way to get the same spatial audio in at least a 5.1 format recording out of VAM? Spatial sound exists in VAM/VR obviously, I just can't figure out how to get spatial audio in the recording.
Hi Bob. Don't know. What do you use for recording? An external software? VAM is basically Unity. I would start from Unity docs if I were you. To record 6 discrete audio channels I think for sure you have to place 6 discrete Unity Listeners in the Scene, and then find a software which can interface to Unity and record the 6 Listeners as discrete audio channels. Hope it helps ?
 
Back
Top Bottom