Guides How to use InvokeAI to create your own custom face textures

[Treee6]

Treee6 submitted a new resource:

How to use InvokeAI to create your own custom face textures - How to use InvokeAI to create your own custom face textures

Hey everyone!

I was wondering if there is a way to use InvokeAI to create face textures for VAM. Unfortunately, I haven’t found a properly trained model to create textures out of nothing, but with a little guidance, we can already create pretty good looking textures with AI.

First, you’ll need to install InvokeAI and download a preferred AI model. A good resource is Civit.ai. For this tutorial, I am using ‘aZovyaPhotral_v2’, but...

Read more about this resource...

 

[csrt88]

Could you make a video tutorial showing the process?

 

[mandazi39]

This is really cool! :O
I like how you implemented this and the results are pretty interesting! We gotta train an ai model on high rez face textures
Would be awesome to do what you did and it pump out for the limbs and gens as well!

I've also added ai to my work flow a while back and saw how well it does, not only with eyes and lips, but with the symmetry that looks better on VAM models.

Have you tried using this techinque for normals maybe? I haven't been able to wrap my head around normals but see it can have some pretty striking results, especially with the Mature women models and their details.

 

[Treee6]

Could you make a video tutorial showing the process?

It is not planned. Which steps are hard to follow?

 

[Treee6]

This is really cool! :O
I like how you implemented this and the results are pretty interesting! We gotta train an ai model on high rez face textures
Would be awesome to do what you did and it pump out for the limbs and gens as well!

I've also added ai to my work flow a while back and saw how well it does, not only with eyes and lips, but with the symmetry that looks better on VAM models.

Have you tried using this techinque for normals maybe? I haven't been able to wrap my head around normals but see it can have some pretty striking results, especially with the Mature women models and their details.

I have not tried to generate normals, and my guess is, that it won't work very well. Only if there are trained models for this.
But there are allready impressive tools for generating normals from pictures. I can not remember the name of the tool, but there was a node editor for texture manipulation some years ago which allowed for generating spec- and normalmaps from a single picture and the output was mind blowing. Then Unity did what Unity does so often: they bought the software, and I've never seen it again.

 

[Sally Whitemane]

A while ago I did generate many faces with early Invoke 3.x release just to see how far I can push this.
The inpainting or img2img approach is what I tried first too.
But it's limited to low variation with the denoising.
The new node workflow is better in my opinion.
The key for more variation is to use Control Nets and generate the full image.
Probably have to blend the outer parts in photoshop later to match the skin tone near seams.

However my method did eat VRAM. Inpainting maybe the only way for low VRAM cards due to the smaller generated part of the image.
With no image upscaling trickery a 24 GB card can generate (slightly above) 2048x2048 before running into VRAM limit.
Not sure how far a 16 GB card can be pushed before out of memory. Doubt it's enough for 'native' 2K.

If I had more time the I'd would create and share a custom node-workflow (in InvokeAI loadable json file) to show this.
I can only suggest to try it out with 1 strong or 2 Control Nets. It's absolutely critical that InvokeAI does stick to the 'shape' of the UV. You will need to fight some things it likes to generate with a negative prompt - like ears on the sides.

 

[csrt88]

It is not planned. Which steps are hard to follow?

It's not that its hard to follow I just like video tutorials better versus just pic explanation

 

[Treee6]

A while ago I did generate many faces with early Invoke 3.x release just to see how far I can push this.
The inpainting or img2img approach is what I tried first too.
But it's limited to low variation with the denoising.
The new node workflow is better in my opinion.
The key for more variation is to use Control Nets and generate the full image.
Probably have to blend the outer parts in photoshop later to match the skin tone near seams.

However my method did eat VRAM. Inpainting maybe the only way for low VRAM cards due to the smaller generated part of the image.
With no image upscaling trickery a 24 GB card can generate (slightly above) 2048x2048 before running into VRAM limit.
Not sure how far a 16 GB card can be pushed before out of memory. Doubt it's enough for 'native' 2K.

If I had more time the I'd would create and share a custom node-workflow (in InvokeAI loadable json file) to show this.
I can only suggest to try it out with 1 strong or 2 Control Nets. It's absolutely critical that InvokeAI does stick to the 'shape' of the UV. You will need to fight some things it likes to generate with a negative prompt - like ears on the sides.

Sounds very interesting! I have not used the node based workflow, but I have to look into this. But I've experimented with controlnet in automatic1111 with various success.

Right now, I am experimenting with another approach, by simply generating a portrait and then create the texture and morph out of it in the classical way. It gives for sure more variance but is a lot of manual work.

But I'd love to generate textures and morphs out of the blue sometimes in the future with a simple prompt.