Has anyone used ControlNet or open pose editor to generate poses from images?

[VRmatt]

I've used ControlNet and Open Pose Editor to read an image, and use the pose it generated with Stable diffusion. But I'm wondering if there's a similar way to create vap poses from images?

 

[AlexV]

Stable Diffusion or ComfyUI?

 

[VRmatt]

Either, but I natively work in SD. (I have both)

 

[AlexV]

Oups, just noticed you want to transfer a pose from SD to VAM, don't you ? I think it is impossible - the pose generated in SD is just s set of 2D lines, no way it can be transferred to 3D accurately.

I am looking for the opposite direction - transfer a pose from VAM to SD/ComfyUI.

 

[hazmhox]

I am looking for the opposite direction - transfer a pose from VAM to SD/ComfyUI.

You want the line drawing that ComfyUI can use?
I could make a mode for Studio Maestro for that, that's not very complicated. (I can't promise the delay tho).

I've used ControlNet and Open Pose Editor to read an image, and use the pose it generated with Stable diffusion. But I'm wondering if there's a similar way to create vap poses from images?

There is none at the moment.
Assuming a pose through stable diffusion might be easy. But it's a bit more complicated in 3D.

I don't think it's impossible, it's just probably a complicated process for an app that has a community which has shared hundreds of poses already... which you can work with as a good starting point.

 

[AlexV]

I could make a mode for Studio Maestro for that, that's not very complicated. (I can't promise the delay tho).

Recently I started to work with video. Free AI tools can create fantastic result from any video, including CGI. However, for better control it would be nice to have depth and pose video. Do you you think you can create the script? At least could you provide an idea how to create it? I have the basic knowledge in scripting.
Below is screenshot from original VAM video record and processed in AI: