Has anyone used ControlNet or open pose editor to generate poses from images?

[VRmatt]

I've used ControlNet and Open Pose Editor to read an image, and use the pose it generated with Stable diffusion. But I'm wondering if there's a similar way to create vap poses from images?

 

[AlexV]

Stable Diffusion or ComfyUI?

 

[VRmatt]

Either, but I natively work in SD. (I have both)

 

[AlexV]

Oups, just noticed you want to transfer a pose from SD to VAM, don't you ? I think it is impossible - the pose generated in SD is just s set of 2D lines, no way it can be transferred to 3D accurately.

I am looking for the opposite direction - transfer a pose from VAM to SD/ComfyUI.

 

[hazmhox]

I am looking for the opposite direction - transfer a pose from VAM to SD/ComfyUI.

You want the line drawing that ComfyUI can use?
I could make a mode for Studio Maestro for that, that's not very complicated. (I can't promise the delay tho).

I've used ControlNet and Open Pose Editor to read an image, and use the pose it generated with Stable diffusion. But I'm wondering if there's a similar way to create vap poses from images?

There is none at the moment.
Assuming a pose through stable diffusion might be easy. But it's a bit more complicated in 3D.

I don't think it's impossible, it's just probably a complicated process for an app that has a community which has shared hundreds of poses already... which you can work with as a good starting point.

 

[dollarsmocap]

With our webcam mocap software, Dollars MONO, you can capture poses from images and apply them to VAM characters.

Here are some examples of the process,

Screenshot Breakdown,
Top left - The images are displayed in the browser.
Bottom left - OBS captures the browser feed and inputs it into MONO using the OBS virtual camera.
Right - VAM receives the pose data from MONO and applies it to the characters.

For details, please check out our plugin page,

Dollars Motion & Facial Capture - Plugins With Paid Service -

This plugin can integrate the following Dollars MoCap products with VAM characters, Dollars MONO: Real-time full-body and facial motion capture with finger tracking using a webcam or video files. Dollars DEEP: Enhanced motion capture...

hub.virtamate.com

 

[hazmhox]

For details, please check out our plugin page,

Dollars Motion & Facial Capture - Plugins With Paid Service -

This plugin can integrate the following Dollars MoCap products with VAM characters, Dollars MONO: Real-time full-body and facial motion capture with finger tracking using a webcam or video files. Dollars DEEP: Enhanced motion capture...

hub.virtamate.com

This does help with the original request, but for our community members that might not pay attention or read a lot:

• This is a paid plugin
  • Not cheap if you get a fulltime license (compared to an average VAM patreon subscription)
  • Without refund policy.

You are free to advertise however you want on the internet, but please, on VAM's hub.... be more upfront and transparent when you advertise products.

 

[dollarsmocap]

Thank you for you comment.

• Not cheap if you get a fulltime license (compared to an average VAM patreon subscription)

Sorry, we don't think it's appropriate to consider the price as an issue just because it's 'Not cheap'.

I understand that you believe we should anchor our price to other VAM Patreon subscriptions, but we feel that if the development of the core features is not related to the price of VAM itself, it would not be reasonable to link our pricing to VAM's price.

From a motion capture solution perspective, I believe that an independent pricing model is essential for VAM to get more serious motion capture options like Rokoko and so on, similar to iClone.

• Without refund policy.

Regarding our refund policy, we offer a full-featured, permanent free trial, so we choose not to provide a refund policy to reduce administrative costs. We also remind users on multiple pages to ensure they are satisfied with the trial before paying. Especially since we are not using a subscription model, we have opted not to offer refunds.

By the way, the feature mentioned in this thread can be fully achieved for free.

You are free to advertise however you want on the internet, but please, on VAM's hub.... be more upfront and transparent when you advertise products.

Thank you for your feedback. We understand that you feel this way, and we’re always working to improve how we communicate about our products to ensure clarity.

 

[AlexV]

I could make a mode for Studio Maestro for that, that's not very complicated. (I can't promise the delay tho).

Recently I started to work with video. Free AI tools can create fantastic result from any video, including CGI. However, for better control it would be nice to have depth and pose video. Do you you think you can create the script? At least could you provide an idea how to create it? I have the basic knowledge in scripting.
Below is screenshot from original VAM video record and processed in AI: