Guides Customized global illumination with SkyMagic

[MacGruber]

MacGruber submitted a new resource:

Customized global illumination with SkyMagic - Create customized global illumination and ReflectionProbes for your scene using SkyMagic.

SkyMagic is a plugin included in the MacGruber Essentials package. It allows you to bake and import custom global illumination into VaM. It can be used in a number of different ways:

• Bake custom reflections within VaM to have global illumination match your light setup perfectly.
• Use the baked ReflectionProbe of a scene/prefab imported from Unity as global illumination.
• Use some nice a...

Read more about this resource...

 

[hazmhox]

Hey MacGruber! oeshii was asking for more content for skyboxes, so I was -first- trying to make Spaceboxes compatible with your plugin. And since i've never really used it... just a few things :

When you enable global on your script, you lose control of the ambiant light in the lighting settings... which is logical from a Unity standpoint, since using the skybox as a reference makes you lose the "color control".

One thing tho, you loose "camera exposure", which also makes sense to me, because this is some kind of "fake" exposure in reality ( since camera exposure only arrived with HDRP )... am I right ?

This is just for clarification, because the GI / Lighting difference between unity assets and person atoms always kinda disturbs me.

( I really hope that @meshedvr can unify it properly for all assets in 2.x )

 

[MacGruber]

Hey MacGruber! oeshii was asking for more content for skyboxes, so I was -first- trying to make Spaceboxes compatible with your plugin. And since i've never really used it... just a few things :

When you enable global on your script, you lose control of the ambiant light in the lighting settings... which is logical from a Unity standpoint, since using the skybox as a reference makes you lose the "color control".

One thing tho, you loose "camera exposure", which also makes sense to me, because this is some kind of "fake" exposure in reality ( since camera exposure only arrived with HDRP )... am I right ?

This is just for clarification, because the GI / Lighting difference between unity assets and person atoms always kinda disturbs me.

( I really hope that @meshedvr can unify it properly for all assets in 2.x )

Please make sure to use the newest version (Essentials.6), there was an issue fixed with those sliders not working anymore, likely since some recent VaM update.

 

[hazmhox]

Ho ok !

So the lighting is obviously (kinda) completely separated from Person and Assets.

 

[MacGruber]

So the lighting is obviously (kinda) completely separated from Person and Assets.

Lighting is done not by the light (as one might naively suspect) but by the shader used by the asset. Assets imported from Unity usually use the "Standard" shader, while VaM does it's own weird custom stuff, I believe it uses a library called Marmoset. Both methods have their pros and cons.

 

[hazmhox]

Yeap indeed, you always have to make choices for shaders

Another thing. I was trying to fiddle around with realtime probes. You wouldn't happen to know why the probe is unable to Render the cubemap in realtime ?

Discussing with oeshii, he was saying that the limitations of not being able to rotate the skybox, or in reality that the cubemap wasn't rotating with the skybox is a bummer when you try to light properly your scene.

But with a single probe, I find strange that it isn't able to render in realtime. I've checked with a test plugin on the CUA, and the probe seems good, enabled, good settings and everything.

 

[MacGruber]

The limitation of rotating the skybox is not with VaM, it's with Unity ReflectionProbes. If you rotate your Sky, the ReflectionProbe won't do the same. The Unity shader simply does not support probe rotation for performance reasons. If you want to "light properly your scene", you won't do it like this anyway. You would load a sky sphere via CUA (without SkyMagic, just as background), rotate it however you want and setup your scene. THEN use SkyMagic Exporter to take a cubemap shot, which you reimport through Unity and use SkyMagic Loader to make it the VaM sky.

I don't know why realtime probes don't work either, I tried for weeks, but couldn't get them to work. Doesn't mean there isn't a way, but I didn't find it, it might be something that would need to be set when building VaM in the Unity Editor, so it can't be enabled by a plugin. This is why I wrote SkyMagic Exporter. Technically you could extend SkyMagic to generate cubemaps in realtime, though. However, you will need to apply a spherical convolution shader to make glossy reflections work correctly, essentially a fancy blur through mipmap levels. I went the easy way and had Unity Editor do that step for me, this is why you need to reimport through Unity at the moment.

 

[hazmhox]

Yup. I do know that this is a Unity limitation
I was more or less telling you what oeshii was saying. That's why I tried to investigate on the realtime probes.
In my case I have no issue whatsoever to make my own assets and create my Reflection Probes. I'm even baking my own content in Unity since I'm used to it.

Thank you for your answer and feedback. I'm gonna try to hit meshed on Discord and see if he can do something about it or tell me what is the problem. That would really improve custom skyboxes for users that are not used to Unity and want an update in realtime in VAM when they tweak the skybox.
The script is über simple, you can even just make an update every second, that's enough... at least for virtual photography artists. I would use a realtime reflection probe for RT / VR.

 

[hazmhox]

@meshedvr answered on Discord this :

It looks like I have realtime reflection probes turned off in the quality settings. I never used them, so I never enabled that option. I can turn it on for next release. You could also try turning on in a plugin using QualitySettings.realtimeReflectionProbes = true;

I've tried to enforce it through code, but it does not change anything. The probe is set to realtime, with Time slicing to "all faces" and update method to script.
I have something really basic on the update to test it out :

            // If the next update is reached
            if (Time.time >= nextUpdate)
            {
                logDebug(Time.time + ">=" + nextUpdate + " Render probe: " + probeComponent.isActiveAndEnabled + " QSets: " + QualitySettings.realtimeReflectionProbes);
                nextUpdate = Mathf.FloorToInt(Time.time) + 1;
                probeComponent.RenderProbe();
            }

I've said to meshed that I was going to @ him on the discussion so that he could give potential leads, or eventually discuss about enabling the feature in the build. I'm not sure I remember exactly how the player works when you build it... I don't know if you disable some features (like realtime probes) if the final player cannot enforce the settings afterwards if you did not include the setting in at least one of the qualities.

Or maybe there is something to do on the skybox side that would allow it to work...
I've spotted this in a thread recently :

Go to Project Settings > Graphics > Always Containing Shaders and add these three shaders:
- Hidden/CubeBlur
- Hidden/CubeCopy
- Hidden/CubeBlend

This is more or less the results of my investigation, hope it can help.

EDIT : With a short test, Shader.find cannot access the Cubemap shaders, so I guess that's the culprit

 

[meshedvr]

QualitySettings by nature are all things that can be set an runtime. So turning it on should have worked, or there is some other issue. If you can send me a sample VAR file I can debug in the Unity editor to see if I can figure out why they are not working

 

[hazmhox]

I guess so yeap. But since Shader.find cannot access these shaders and a few posts here and there are talking about having these to be able to update in realtime, maybe that's the problem.

This is something I could try on a temporary build on my side tho ^^

Here you go

You have inside : test-reflectionprobe which is a baked probe. And test-reflectionprobe-rt which is the realtime one.

 

[meshedvr]

I was able to get your sample working with 2 fixes:

1. I turned on realtime reflection probes in Quality Settings. Note this will be on by default in 1.20.1.5
2. I fixed your Reflection Probe settings. You had Refresh Mode set to "Via scripting". I changed it to "Every Frame"

 

[meshedvr]

Ignore the gif quality (256 color limit), but you can see it is working here:

 

[meshedvr]

One last note - the shaders I use in VaM are based on the Marmoset shaders which came out before Unity added the standard shaders that supported realtime reflection probes, so don't expect native VaM objects to work with this. This is something to look forward to in 2.X though!

 

[hazmhox]

Ho yeah !
Thank you very much

And yes, more or less it was for the clothes (which are really nice with SkyMagic, so I guess with a realtime probe it'll work)... and my custom content.

My probe was configured "per script" because I was thinking of controlling it through script... I'm not sure that GI needs a per frame basis update... especially in this case were it is subtle and not a lot of content has huge reflections like the example character I made. I'm also not sure if the cost grows up depending on the scene complexity.

I don't know what @MacGruber thinks about that ? Do you know if you'd make a per frame update or a manual update with a configuration in SkyMagic if the realtime probe is working ?

 

[MacGruber]

The problem with real time updates, beside obvious performance concerns, is that VaM doesn't have a way of sorting objects into layers. You might not want all objects in the scene present in the reflection. E.g. if you place a reflection probe for a character/clothing, you don't want that character in its own reflection. Same for other close by objects. Its especially an issue if you only use a single probe for the whole scene (when its supposed to work with Marmoset via SkyMagic) So when taking the shots by hand, you can disable atoms by hand.

The other unsolved issue I already mentioned in an earlier post is that you need to apply spherical convolution across the mipmaps to the reflection to make gloss work correctly. So far that's only working when going via UnityEditor. Making the shaders available mentioned by @hazmhox (Cube_Blue/Copy/Blend) might do the trick, but you would still have to figure out how to use them correctly.

 

[AlexV]

Hello MacGruber,
I made a lot of tests trying to set the global illumination using your plugin but didn't succeed.
I've tried to use a simple big green wall as the light source. Specular works great:

but Global Diffuse light is defiantly not related with my wall:

First I don't get what is the source of the Global Illum in this scene? Second how to make my green wall the source of the Global Illum?
Could you enlighten me, please?

 

[AlexV]

Hello MacGruber,

Files: https://mega.nz/folder/SMYVwQja#D-c5rUQ-KBt8oLYA2wsieg

 

[MacGruber]

Hello MacGruber,

Files: https://mega.nz/folder/SMYVwQja#D-c5rUQ-KBt8oLYA2wsieg

Hm, seems this broke again in some recent VaM version. Will have to dig in, might take a few weeks until I get to it.

 

[AlexV]

Great! No rush. I am looking forward to get the right lighting.
Definitely the plugin is underestimated, maybe because it cannot be setup in one click.

 

[FCG]

Is this still being worked on?

 

[MacGruber]

Is this still being worked on?

Hm, seems I forgot about it Too much going on. I was planning on using SkyMagic for my Benchmark scenes, so I probably have to fix it then. But no promises.

 

[AlexV]

I still have a hope you will fix it. For me this is the most important plugin. I am sure 99% of VAM users just don't recognize the plugin can simulate amazing GI with almost zero performance cost.

 

[MacGruber]

@AlexV @FCG Burned some 5-6 hours between yesterday and today on this. Sadly, no dice. I think it has something to do with the spherical harmonics setup just being copied from the Museum sky. So the diffuse you get is from that sky. I tried to compute my own values from the cubemap, but for some reason that just results in some uniform yellow'ish diffuse lighting that seems unrelated to the actual cubemap. Not sure what's wrong with it. If anyone wants to continue tinkering with this, I added the following in the UpdateSky method of SkyMagicLoader:

Cubemap cube = mySkyProbe.customBakedTexture as Cubemap;
int miplevel = 0;
int mipsize = cube.width;
for (; miplevel<cube.mipmapCount && mipsize > 8; ++miplevel)
    mipsize /= 2;                    
Cubemap copy = new Cubemap(mipsize, cube.format, false);
for (int i=0; i<6; ++i)
    Graphics.CopyTexture(cube, i, miplevel, copy, i, 0);
Destroy(copy);
mySky.CustomSH = null;
mySky.SH = new SHEncoding();
SHUtil.projectCube(ref mySky.SH, copy, 0, false);
mySky.SH.copyToBuffer();

This may be something rather stupid, but I don't see it.

 

[Dragontales]

Forgive me if I'm missing something obvious, but I'm trying to view the instructions on using this and it only opens up really small. Zooming in only scales up the low res.