- Messages
- 77
- Reactions
- Points
- 18
just 18.5. the actual alxr dev is working on v20 versions but it will be a ways off from what I can tell
-
Hi Guest!
We have recently posted a VaM2 2024 Progress Update on Patreon!
Check out the details here: https://www.patreon.com/posts/vam2-august-2024-111164439 -
Hi Guest!
We are excited to announce a new feature on the Hub: Favorites!
You can now add resources to your favorites, and organize your favorites into collections! You can check out the details in our official announcement!
How to Customize ALVR-Client for Quest 2 Passthrough on Windows with Any Mask Color
- Thread starter krch
- Start date
just 18.5. the actual alxr dev is working on v20 versions but it will be a ways off from what I can tell
- Messages
- 77
- Reactions
- Points
- 18
I've noticed some sparkly artifacts when I use the passthrough now. Do you know what might be causing that? Didn't see those before.
- Messages
- 77
- Reactions
- Points
- 18
uh which one do you have? latest one on my post? dont know what wouldve caused it but I have noticed it too in some builds. might be too sesitive with the chroma key
Sorry I haven't been responsive here, just either busy or dealing with headaches & lethargy, about the recent changes, there isn't a new vertex buffer, there isn't one at all now 
basically before I was creating a vertex & index buffers for a (screen-space) quad the new version removes them and replaced with a single over-sized triange, the positions & uvs are constants in the vertex shader i also removed some redudant matrix transforms plus some other minor tweaks. So the changes are mainly removing buffers, removing pipeline setup and modified/simplified vertex shader.
Regarding delta-e formula that you guys having been looking at and/or testing, delta-e 2000 while the most accurate way to perceptually compare colours it is the most costly, surprised it was even usuable but i was considering it as an non-default option. I don't think you need to use this to deal with the outline issues, from what I've briefly read here it comes off like chroma spill, if that is the case you should be able to handle this seperately while using a cheaper, simpler versions of delta-e. There's like 3 or 4 standards each with more accuracy but more computationally costly.
Just search for chroma key spill, that is what I will be looking at soon.
basically before I was creating a vertex & index buffers for a (screen-space) quad the new version removes them and replaced with a single over-sized triange, the positions & uvs are constants in the vertex shader i also removed some redudant matrix transforms plus some other minor tweaks. So the changes are mainly removing buffers, removing pipeline setup and modified/simplified vertex shader.Regarding delta-e formula that you guys having been looking at and/or testing, delta-e 2000 while the most accurate way to perceptually compare colours it is the most costly, surprised it was even usuable but i was considering it as an non-default option. I don't think you need to use this to deal with the outline issues, from what I've briefly read here it comes off like chroma spill, if that is the case you should be able to handle this seperately while using a cheaper, simpler versions of delta-e. There's like 3 or 4 standards each with more accuracy but more computationally costly.
Just search for chroma key spill, that is what I will be looking at soon.
Last edited:
- Messages
- 77
- Reactions
- Points
- 18
Sorry I haven't been responsive here, just either busy or dealing with headaches & lethargy, about the recent changes, there isn't a new vertex buffer, there isn't one at all now
Regarding delta-e formula that you guys having been looking at and/or testing, delta-e 2000 while the most accurate way to perceptually compare colours it is the most costly, surprised it was even usuable but i was considering it as an non-default option. I don't think you need to use this to deal with the outline issues, from what I've briefly read here it comes off like chroma spill, if that is the case you should be able to handle this seperately while using a cheaper, simpler versions of delta-e. There's like 3 or 4 standards each with more accuracy but more computationally costly.
Just search for chroma key spill, that is what I will be looking at soon.
yeah no worries I had also planned to message you and never did, just been trying to learn vulkan from zero so I can ask more useful questions
as for the computational cost of deltaE I wouldn't be too concerned
I did look into chroma key spill and mostly saw people suggesting to feather the pixels on the outside which I guess we could do by taking the coordinates in the frag shader but I'm not sure how costly that could be. I've also seen suggestions about having another layer that matches alpha to color but that could be costly.
Anyway I have some stuff I want to try but I'd love to hear your ideas. Let me know if you're feeling well enough to chat on discord or whatever
Last edited:
I'm no expert in this area but the typical colour spaces used for chroma-keying are ycbcr formats, HSV, LAB. Basically anything that seperates out the luma and chroma is going to be way better than my initial version for supporting any colour key, definately don't want to use rgb.
I forgot to mention a few things, in OpenXR there are two ways a runtime can support passthrough (a runtime could support both), there is the core spec way which is environment blend modes or explicit passthrough extensions. The code paths are different and the latter typically have more/higher level functionality.
With the core spec way, there are two non-opaque env blend modes, additive blend where pure black pixels are treated as transparent and an alpha blend mode where there's alpha value to blend passthrough & render layer.
In alxr I've add support for a both methods but currently the core-spec way I'm not using the passthrough shaders, and picking additive blend if both non-opaque modes are available as that already handles a kind of chroma-keying of black for free. To better support runtimes that do this method of passthrough, i'll be re-using these shaders but I maybe slightly modifying the masking shader to either convert matching pixels to pure black for additive mode or set an alpha value (as is now).
With the non-core spec way of passthrough extensions, there is typically settings to configure how composition is done in code that you wont see in the shader, you may want to consider reading this, it's for Oculus's vendor specific passthrough extension but htc has a simillar (though less feature rich) version.
I forgot to mention a few things, in OpenXR there are two ways a runtime can support passthrough (a runtime could support both), there is the core spec way which is environment blend modes or explicit passthrough extensions. The code paths are different and the latter typically have more/higher level functionality.
With the core spec way, there are two non-opaque env blend modes, additive blend where pure black pixels are treated as transparent and an alpha blend mode where there's alpha value to blend passthrough & render layer.
In alxr I've add support for a both methods but currently the core-spec way I'm not using the passthrough shaders, and picking additive blend if both non-opaque modes are available as that already handles a kind of chroma-keying of black for free. To better support runtimes that do this method of passthrough, i'll be re-using these shaders but I maybe slightly modifying the masking shader to either convert matching pixels to pure black for additive mode or set an alpha value (as is now).
With the non-core spec way of passthrough extensions, there is typically settings to configure how composition is done in code that you wont see in the shader, you may want to consider reading this, it's for Oculus's vendor specific passthrough extension but htc has a simillar (though less feature rich) version.
I'm not familliar with the xyz colour space but I have seen rgb to lab conversions using that as an intermediate step. Yes there's no reason to convert the key for every pixel shader invocation, precompute that once, I've seen that repository but I don't recall all the available functions. Note the function that samples the video texture outputs a pixel in linear rgb space (check baseVideoFrag.glsl to see `sRGBToLinearRGB`), that needs to be done in the shaders as these textures typically don't have a vulkan format sRGB type for automatic conversion to take place (out of my control, AHardwareBuffer imports).
I'm using meta's passthrough extensions (+ HTC, and Pico), what i mean is I support using either way, some runtimes like Meta's OpenXR runtimes do not support passthrough using the core spec blends mode, only with vendor extensions. All I meant is for the case of other runtimes that only support the core blend modes way. I'll just be adding some missing logic in C++ to switch those modes (like with quest) and re-using the mask shaders (with the improvements i/u/we do in the end). The only thing you need to note that i might be tweaking mask shader a little. it's not a big deal really.
Not sure which constants are you referring too?
- Messages
- 77
- Reactions
- Points
- 18
- Messages
- 77
- Reactions
- Points
- 18
builds for all android versions updated to 0.13.0 on my other page. If you missed the update on that page definitely go check it out. I actually think the mask mode is as good as it can get, at least for a game like vam with configurable colors. You can even adjust the background within the entire green spectrum (i.e. changing only hue) which improves compatibility a bit
- Messages
- 77
- Reactions
- Points
- 18
@krch @animetiddyenthusiast
Hey with v0.15 I've upgraded the NDK version to the newest (LTS) that came out a couple of days ago, v26.0.10792818, it's easy to switch back the to the last version but very soon this will be the min version because of some upcomming changes I have.
Later on tonight I'll be pushing a bug fix for Lynx R-1, it will be v0.15.1 once that is available @krch you can add this headset to the list of supported if you like but until I've done the client side UI (I've started to work on it) and Lynx doesn't have physical controllers the only way to switch passthrough modes atm is with adb, e.g:
adb shell setprop debug.alxr.passthrough_mode <-mode-> where <-mode-> is one of None, BlendLayer, MaskLayer
This system property can be used for any android based headset.
If you guys have been wondering why there are multiple apks, without going into much details the main reason is because some android openxr runtimes for standalone headsets are not compatible with the OpenXR cross-vendor loader for android and provide vendor specific ones, Lynx's runtime is one of those temporarily so it has it's own apk but eventually it will be so the generic apk (alxr-client.apk) will be the one. In the longer term there will only be a single apk, regardless of vendor specific loaders but that's a bit more tricky to deal with right now.
Hey with v0.15 I've upgraded the NDK version to the newest (LTS) that came out a couple of days ago, v26.0.10792818, it's easy to switch back the to the last version but very soon this will be the min version because of some upcomming changes I have.
Later on tonight I'll be pushing a bug fix for Lynx R-1, it will be v0.15.1 once that is available @krch you can add this headset to the list of supported if you like but until I've done the client side UI (I've started to work on it) and Lynx doesn't have physical controllers the only way to switch passthrough modes atm is with adb, e.g:
adb shell setprop debug.alxr.passthrough_mode <-mode-> where <-mode-> is one of None, BlendLayer, MaskLayer
This system property can be used for any android based headset.
If you guys have been wondering why there are multiple apks, without going into much details the main reason is because some android openxr runtimes for standalone headsets are not compatible with the OpenXR cross-vendor loader for android and provide vendor specific ones, Lynx's runtime is one of those temporarily so it has it's own apk but eventually it will be so the generic apk (alxr-client.apk) will be the one. In the longer term there will only be a single apk, regardless of vendor specific loaders but that's a bit more tricky to deal with right now.
- Messages
- 8
- Reactions
- Points
- 3
Excellent! and Thanks! I'll update the tutorial this weekend..
- Messages
- 77
- Reactions
- Points
- 18
havent been around but I did see the ndk upgrade. Will update my version soon, appreciate the ping
Overwrite default.json with whatever scene you want to be your default scene.
