Use of static images as background for image capturing?

[braxo]

Hi,

I'm new to VAM trying it out, with the goal that I'm thinking about making a Visual Novel using RenPy (will be my first, so a smaller game while learning the ropes).
So my use would mostly be capturing images, but seing how great animations everyone are able to do I will probably also try some shorter videos of animations.

I'm just in the beginning with VAM, so sorry if I'm missing something essential.
One thing I'm wondering about is if I need to use 'real' environments for every scene. For a story I would potentially need many different environments for the different scenes. In bigger/important scenes using an environment in 3d will of course be very good - interact with the objects, like sitting on a sofa. Or use different angles for the images. But I guess it might be difficult to find many different environments I can use that fits the story (of course, I understand I could create them myself with Unity (?) but that would mean another tool to learn..)

I thought for some scenes it could be enough for the characters to stand in front of a static background. So I tried to use an ImagePanel atom, on which I added an image, and put as a background. (There were a few different version of the atom, not sure what the differences are..)
And it kind of works. Though it didn't feel as if the feet would stand naturally on the ground in the image, which of course could be solved simply by not showing the entire characters in the images. But also, the characters cast shadows on the image panel Could maybe work for indoor scenes, but looked very strange with an outdoor image.
(I havn't tried to actually caputure any images yet, but I assume it will look the same as what I am viewing)

Would that be an approach that could work, using static images as backround? Or are there any other suggestions what I can try?

 

[MacGruber]

You are looking for the Emissive version of the ImagePanel. That just renders the plain image, without light/shadow applied. It still casts shadows, though.

You could also give this plugin a try. It's meant for VR video rendering, but can also do regular video and screenshots. I think it has an option to put a background image in:

Video Renderer for 3D VR180, VR360 and Flat 2D & Audio + BVH Animation Recorder - Plugins + Scripts -

This plugin renders VR180, VR360 and traditional 2D videos with audio and VR/panorama images inside Virt-A-Mate in stereoscopic 3D and monoscopic view. You can render at very high levels of resolution, quality and frame rate that would be...

hub.virtamate.com

 

[braxo]

Thanks for the reply!
That the panel itself would cast a shadow shouldn't be an issue, it would be behind what I want to capture.
I tried the Emissive ImagePanel, but think the result was the same. So not sure if I do it correctly. I took a screenshot to show a simple example. Though after I took the screenshot I did play around with properties called 'Specular*' and then I could get rid of the shadow from the character - so maybe that's what I need to do?

But I realized another thing, that I guess is part of why it doesn't feel completely "correct" (disregarding the odd angles of the background in the example). In a real environment atom, the character would cast a shadow on the ground. Even if I could tweak the angles and sizes so that the characters fits better together with the background, without the shadows it probably would feel like the character somehow was floating above the ground.
Or maybe there are better ways to try and make such captures?

I'll give the plugin a try as well. I did view a tutorial you have made, about capturing video. That looked pretty good, but this plugin would give the advantage of adding the background image directly in the plugin?

 

[hazmhox]

I think you're overthinking this.
There are a lot of "ready to use" enviros (or HDRIs)... you should find something that fits your story without much trouble.

If you don't have the skills right now, think your story around available content on the hub and not the other way around up until you can create the assets yourself (if it ever happens).
With backgrounds, you will somehow always be disappointed of the end result compared to an actual location/enviro.

 

[MacGruber]

I tried the Emissive ImagePanel, but think the result was the same. So not sure if I do it correctly. I took a screenshot to show a simple example. Though after I took the screenshot I did play around with properties called 'Specular*' and then I could get rid of the shadow from the character - so maybe that's what I need to do?

You did something wrong. Use an ImagePanelEmissive at default settings. Just set the background image. There is no shadow on the image, just on the floor (which isn't emissive here)

That looked pretty good, but this plugin would give the advantage of adding the background image directly in the plugin?

Main reason for the plugin is that you can render video frame by frame, rather than needing a beefy PC to render in realtime AND crunch the video. It allows you to render at 8K resolution and then downscale to FullHD. It's an insane raw-hammer method, but improves image quality a lot. Supposedly the plugin can even do 16K, although I think Unity engine doesn't support that.

Eosin's plugin is actually originally based upon my own SuperShot plugin for screenshots. Hair improves a LOT, textures are sharper and shadows have less noise.

MacGruber Essentials - Plugins + Scripts -

Your most essential MacGruber tools in a convenient single VAR package. Note that you can download older versions of this plugin for free from here. AudioReverb Adds an Reverb filter effect to the audio source you place the plugin on. It's...

hub.virtamate.com

With backgrounds, you will somehow always be disappointed of the end result compared to an actual location/enviro.

I agree. It is always difficult to get the light look right when you have to match up to a photo to make the character look as part of the scene. Especially if the character should throw shadows on objects on your background or cause ambient occlusion. You already noticed the issue with feet appearing to float.

If possible, use some real 3D objects for the close range around the character and then fill the background with a HDRI. SkyMagic plugin can help to get the light right better in such situations.

 

[Origin69]

I've done this a few times just to quickly add a little flair when showing off a model/clothing.

The lighting is the most important part I think. If you pick background images that you feel you can emulate with lighting shade and intensity in vam, your character will seem more "in" the scene.
It depends on the story you are telling, bit I have found outdoor scenes are a little harder to light well than an indoor scene with different looking light sources.

Here's an example of an image as the background. I only added a few lights, but It kind of works. (Looking at it now I should probably added more lights lower down to cover the bounce/edge light on her thigh, but you get the idea.)

Welcome to Vam, and Good luck with your novel.

 

[MacGruber]

@Origin69 As said above, I would recommend using SkyMagic plugin to create a global illumination source ("sky") that is at least a vague match for your environment. That is especially important for shiny materials such as the leather. Right now your global illumination is just plain grey-blue.

Workflow is a bit convoluted, but worth it. Like this:

1. Prepare your VaM environment, just the background stuff. In your case you probably want to duplicate that background image also BEHIND the camera....because that's what you like to see reflected on shiny materials.
2. Use "SkyMagic Exporter" to create a 360deg screenshot at the location where you like to place your character.
3. Use "SkyMagic Importer" to import that 360deg screenshot into Unity Editor. Automatically does some processing.
4. Export the Prefab produced by the Importer as an Unity AssetBundle.
5. Load the AssetBundle in VaM using a CustomUnityAtom (CUA).
6. Put "SkyMagic Loader" onto that CUA. It will grab the Cubemap and apply its as a custom sky for VaM.
7. Now shiny materials (wet skin, leather, latex, metal, ...) will use a specially blurred version of the 360deg screenshot as reflection source.

There is also a tutorial for SkyMagic around if you need more info.

Comparison when materials actually (somewhat) reflect what is around them, rather than just "something":

 

[braxo]

If you don't have the skills right now, think your story around available content on the hub and not the other way around up until you can create the assets yourself (if it ever happens).
With backgrounds, you will somehow always be disappointed of the end result compared to an actual location/enviro.

Yes, I think I'll do that.. I can put the story I already have fleshed out on hold, and try to come up with a new one that know I can find assets for.

You did something wrong. Use an ImagePanelEmissive at default settings. Just set the background image. There is no shadow on the image, just on the floor (which isn't emissive here)

I still couldn't get it to work. I simply started up a new default scene from the template, added the panel and did nothing more than resizing it and moved it behind my charachter.. Can I have changed some system settings by mistake?
But, I will leave that for now and simply start by using environments.

Main reason for the plugin is that you can render video frame by frame, rather than needing a beefy PC to render in realtime AND crunch the video. It allows you to render at 8K resolution and then downscale to FullHD. It's an insane raw-hammer method, but improves image quality a lot. Supposedly the plugin can even do 16K, although I think Unity engine doesn't support that.

Ah that sounds very nice! I don't have a beefy GPU, so sounds it will make it easier. Plus now I can skip my schemes about how I can motivate why I should bring home a 4K screen from the office

I've done this a few times just to quickly add a little flair when showing off a model/clothing.

Nice to hear that my idea wasn't totally of the charts
I still think it could be useful when making a novel, but maybe for smaller / less important scenes. Should be easier to find images that fits the needs, than 3d environments. But yes then lightning would become a different issue I guess.

Welcome to Vam, and Good luck with your novel.

Thanks