This is the Head Tracking plugin. It moves camera according to your head moves.
This plugin only manages camera: you turn your head to the left, VaM's camera turns to the left and image on your display runs to the right (so you see what was behind the left side of your monitor). Same with other directions. Eyes are not tracked, just head.
To have a quick impression, take a look on how similar headtracking is used in flight simulators: the first minute of this video
gives a good idea how does such tracking works. Notice how game image moves to the sides when the guy turns his head. Note the guy uses different software and hardware. (that is a random video from youtube I found searching for "flight simulator head tracking". I am not affiliated with that person or channel)
I have to warn you it is shaky [october 2020]. I have some ideas how to fix this, but those are not implemented yet.
First, you set up a webcam either on top or under your monitor, then you run HeadTracker6DoF.exe , which is a head pose sensor (sources are available on my github). Next, you add this plugin.
downloads page at github: https://github.com/Eugene-E0a80fd8080ff8e/HeadTracker6DoF/releases
project sources and docs: https://github.com/Eugene-E0a80fd8080ff8e/HeadTracker6DoF
You may use F2 button to change middle point. Move camera with your head to something what should become a new middle point, hold F2 while returnng your head to a normal position. Camera stays still while you holding F2.
You should add HeadTracker.cs as a session plugin.
To run HeadTracker6DoF.exe , you need to download either from VaM-hub or from my github. The file would be HeadTracker6DoF.zip . Unzip it and run the exe file.
Note it works nicely with Passenger plugin from acidbubbles.
HeadTracker6DoF is based on dlib 68-point head landmarks detector. It also uses OpenCV.
version v0.3 adds support of opentrack -- the software which enables similar functionality in flight simulators.
This plugin only manages camera: you turn your head to the left, VaM's camera turns to the left and image on your display runs to the right (so you see what was behind the left side of your monitor). Same with other directions. Eyes are not tracked, just head.
To have a quick impression, take a look on how similar headtracking is used in flight simulators: the first minute of this video
I have to warn you it is shaky [october 2020]. I have some ideas how to fix this, but those are not implemented yet.
First, you set up a webcam either on top or under your monitor, then you run HeadTracker6DoF.exe , which is a head pose sensor (sources are available on my github). Next, you add this plugin.
downloads page at github: https://github.com/Eugene-E0a80fd8080ff8e/HeadTracker6DoF/releases
project sources and docs: https://github.com/Eugene-E0a80fd8080ff8e/HeadTracker6DoF
You may use F2 button to change middle point. Move camera with your head to something what should become a new middle point, hold F2 while returnng your head to a normal position. Camera stays still while you holding F2.
You should add HeadTracker.cs as a session plugin.
To run HeadTracker6DoF.exe , you need to download either from VaM-hub or from my github. The file would be HeadTracker6DoF.zip . Unzip it and run the exe file.
Note it works nicely with Passenger plugin from acidbubbles.
HeadTracker6DoF is based on dlib 68-point head landmarks detector. It also uses OpenCV.
version v0.3 adds support of opentrack -- the software which enables similar functionality in flight simulators.
I have tried to do similar in VaM and it does not look any immersive without that mesh forming a well. You need a specially designed scene to have that effect :(
As of now, Z-axis detection is very imprecise. Relying on it causes very unpleasant in-and-out shaking. In my experiments I had to override with z=0.8 meters.
The HeadTracking6DoF.exe sensor actually detects and transmits both head rotation and position relative to camera.