Resource icon

Other Green screen Passthrough ALXR client/ALVR APK for Pico 4

It is a version of Korejan i modified to have green passthrough instead of black.
 
I just got a Pico 4 yesterday and have been having some issues with Passthrough and overal performance with ALXR. Mind if I ask you a few questions about this subject and the shared file?
 
I just got a Pico 4 yesterday and have been having some issues with Passthrough and overal performance with ALXR. Mind if I ask you a few questions about this subject and the shared file?

So does color passthrough works with vam? how does it look? how is it overall?
 
Well, I only have it for less than 2 days and have some issues with ghost-like passthrough, but yeah, color passthrough works with ALVR+XR.
I'm hoping to ask some questions to someone with more experience as I'm getting too much transparency in the person and terrible latency compared to other streaming options, not sure if that's normal.
 
ALXR works but it is not a good option compared to something like virtual desktop which is the best in my opinion. Unfortunately virtual desktop does not handle passthrough at the moment but i think it is in the roadmap. Meanwhile ALXR does the work with some stutterings i'm afraid. Better than nothing.
As for pico 4 color passthrough, it is working but it is not wonderful and i'd suggest to wait for a better solution like quest 3 for exemple.
I dont have any latency with pico 4 and alxr nor do i have some "ghost-like passthrough" as you say (not sure what you mean)
 
I've just updated with a version with more recents commits from Korejan's fork (not latest) that i use. Since then i also had a update of pico OS and i don't have stutterings anymore like i had before. I don't know if it comes form the OS update or from latest commits on ALXR. Try it to see if it resolve your issues.
 
I was writing this as you pushed an update, will try that soon.

Thanks for your review and willing to answer my questions. I only got the device a few days ago and wanted to compare what I'm getting to others.

Oh, yes, VaM in Virtual Desktop is very smooth, hoping there will be passthrough in it at some point.
Alas when I tried ALVR/XR the image quality is much inferior, FPS, etc. This could be also the server configuration on my side, I'm not sure, that's why I was looking for other people's experiences to figure out if it's normal for now.

The passthrough works, using left menu + A/B depending on the mode. My feeling is that it's reversed somehow, I don't think it's supposed to be as I'm getting.
For example in a empty scene using virtual hands:
  • in one mode I get semi-transparent hands (ghost-like), but the passthrough is great as I see my room well
  • in the other mode, my hands are passthrough and green everywhere
I see the background color changer plugin colour graph showing passthrough on the bottom half of the square, while I've seen other people's screenshots from a Quest on the top half. Not sure if that's what supposed to be like.
 
I think you have this ghost-like effect when you're in blend mode. The other mode (mask) is the one you should use. If you see green instead of your room, perhaps you have not selected the "good green". In the color changing plugin, if you have selected greenish you should see some of your room in the top right corner of the color picker. If that's the case it's a good thing, it means that it is working. You then have to click in the part where passthrough seems to work.
From what you say, the passthrough part seems to appear on the bottom half, i have no idea what is causing that... Try to ajust the slider, perhaps with some trial and error you can obtain the good result.
 
Hi, Tim.
Had a chance to try out the new version you shared and compared it to the latest Korejan's nightly. Your version shows the green passthrough area in the top right quadrant, which is the one I needed. Korejan's nightly shows the passthrough area on the bottom left and right quadrants, which gives that unfortunate reversal in mask mode that is not what one wants for VaM. So, super thank you 🙇‍♂️ for making your version, I can now have more than a ghost in my room 😊
It's a huge step forward, and now the goal is to diminish that thick green line outlining the model. I'll take some screenshots shortly to show what I see, could you also share one to compare?

I've also change the ALVR server to this version:
This helped a lot with the shitty latency and framerate I was getting, still a little blurry but at least it's not crazy choppy like before. Still need to play a little more with it.
Also, I misunderstood how to use the CameraBackground plugin. I read it as if it needed to be loaded as a Session plugin and Scene plugin, but only one instance needed 🤦‍♂️ .
 
Last edited:
Here's a screenshot with passthrough:

alxr_passthrough.jpeg


The scene is the default template in VaM. I get those green lines, much more noticeable using the headset, and trying different MSAA values in the user preferences didn't make much difference.
It's pretty amazing how it is, and I know the lines won't ever disappear completely, but any tips you might have to reduce them do share.

Regarding the ALVR server, I now edited the advanced settings a little and it's now much sharper and performant. Little by little it's getting there.
 
Hi, unfortunately i don't know how to make the green outline disapear. At first i've configured the mask to match only one RBG value corresponding to straight green [0,255,0] but it does'nt work as the green hue is not the same everywhere. Perhaps it has to do with the way the image is extracted in the shader and i think the outline problem is linked to that issue. I don't have any solution as it largely surpass my area of competence.
So i just choose the darker green possible as it is less noticeable.
 
Great, I've reached what is expected. As development goes hopefully it will get better.

I saw someone using a male atom with a dummy skin to make it passthrough, but only briefly tried it. Did you try that also or any other tips to share?
 
Here's a screenshot with passthrough:

View attachment 223844

The scene is the default template in VaM. I get those green lines, much more noticeable using the headset, and trying different MSAA values in the user preferences didn't make much difference.
It's pretty amazing how it is, and I know the lines won't ever disappear completely, but any tips you might have to reduce them do share.

Regarding the ALVR server, I now edited the advanced settings a little and it's now much sharper and performant. Little by little it's getting there.


that backgroundd looks so blurry and messy. Does it looks exactly like this in headset?
 
that backgroundd looks so blurry and messy. Does it looks exactly like this in headset?

No, in the headset it looks a lot better. I took the screenshot at night and my room is not well lit, that brings a lot of noise to the picture. Then in the headset you see it less warped and much more vibrant, but that also means the green outline is also more noticeable.
Anyway, once you get a lady dancing in front of you, you also pay less attention to the background :LOL:
 
No, in the headset it looks a lot better. I took the screenshot at night and my room is not well lit, that brings a lot of noise to the picture. Then in the headset you see it less warped and much more vibrant, but that also means the green outline is also more noticeable.
Anyway, once you get a lady dancing in front of you, you also pay less attention to the background :LOL:

so how immersive it is?
 
My opinion : As for passthrough, it is immersive but not that great. Pico 4 has no added value compared to quest 2 exept for color passthrough which is cool but not that great.
As for the headset generally speaking : On paper pico 4 has better definition but i don't see any differences and i think contrast is not handled as well as the quest. I have astigmatism and don't wear glasses, the world is always a little blurry for me so perhaps i'm not very objective on the matter.
The pico is confortable but the quest too if you add some accessories and the battery lifespan is similar but cannot be enhanced like the quest could.
Pancake lenses are supposed to get rid of sunrays effects but if you have a thin head like i have, the pico headset let some light pass on the sides which kill immersion and add reflections on the lenses. So to benefit of passthrough i have to find the right room illumination so not much light enter the headset but sufficiently to have a image not too noisy.
 
Here's my thoughts too. I have a Reverb G2 and a Pico 4, never used a Quest 2, so my impressions are between those two devices.

Passthrough
The Reverb has very limited passthrough options, so my Pico is the only reasonable passthrough experience I have. The single camera with no depth correction is ok to use for either a brief moment to situate yourself in the room without having to take off the headset, or for longer times if you have something else in front taking your attention. It's not like there's a real person in front you, but it's still immersive enough to have part of that feeling. Passthrough works well when it's not the center of attention, unsurprising I think, and I suppose that a grey (Q2) or coloured passthrough (Pico) are not that much of a difference when you use it in VaM, you are paying attention to something else ;)

Immersion
This is very subjective naturally. The green outline in the passthrough border reduces immersiveness, like when you see it in old or bad VFX in movies, and it's not a clean passthrough video like a good camera, but to compensate the sight of the place where you are gives you a boost in immersion, so you gain and lose things, and how you value some over others is what adds or takes your immersion experience.
Say you have a scene with an environment or just a black background, is that a more immersive experience than the room you're in, together with all the passthrough cons like green outline, no depth correction, etc? If you want to believe it's believable enough I think.

Improving the passthrough experience
  • Better lighting in your room and VaM scene
  • Use suitable scenes where passthrough can be an asset
  • Chill and be creative on how you use it, it's for fun, not a dedicated AR headset
Just some quick notes on comparing a G2 to a Pico 4 overall. I'm very surprised with the lenses in the Pico 4, they're bright and the FOV that is on focus is quite good. The G2 for example, a v1, has a narrower area that you can see focused without having to move your head. However, the G2's colour and contrast is noticeably better, something I immediately noticed, and the sound too with the built-in headphones.
 
Here's my thoughts too. I have a Reverb G2 and a Pico 4, never used a Quest 2, so my impressions are between those two devices.

Passthrough
The Reverb has very limited passthrough options, so my Pico is the only reasonable passthrough experience I have. The single camera with no depth correction is ok to use for either a brief moment to situate yourself in the room without having to take off the headset, or for longer times if you have something else in front taking your attention. It's not like there's a real person in front you, but it's still immersive enough to have part of that feeling. Passthrough works well when it's not the center of attention, unsurprising I think, and I suppose that a grey (Q2) or coloured passthrough (Pico) are not that much of a difference when you use it in VaM, you are paying attention to something else ;)

Immersion
This is very subjective naturally. The green outline in the passthrough border reduces immersiveness, like when you see it in old or bad VFX in movies, and it's not a clean passthrough video like a good camera, but to compensate the sight of the place where you are gives you a boost in immersion, so you gain and lose things, and how you value some over others is what adds or takes your immersion experience.
Say you have a scene with an environment or just a black background, is that a more immersive experience than the room you're in, together with all the passthrough cons like green outline, no depth correction, etc? If you want to believe it's believable enough I think.

Improving the passthrough experience
  • Better lighting in your room and VaM scene
  • Use suitable scenes where passthrough can be an asset
  • Chill and be creative on how you use it, it's for fun, not a dedicated AR headset
Just some quick notes on comparing a G2 to a Pico 4 overall. I'm very surprised with the lenses in the Pico 4, they're bright and the FOV that is on focus is quite good. The G2 for example, a v1, has a narrower area that you can see focused without having to move your head. However, the G2's colour and contrast is noticeably better, something I immediately noticed, and the sound too with the built-in headphones.

That's about what I'm seeing on my Quest Pro. It can be pretty immersive at times and you do start to ignore the outline after a while. Scene selection is key.

Actually found a scene by accident where the model leaves the "room" to change and comes back. Randomly my scene lined up with the doorway in my actual room and the effect was quite entertaining. IMO this seems better for voyeur type scenes vs those where there is a male model interacting.

This is a window into the future though.
 
Back
Top Bottom