VaM 1.x This might be an unrealistic idea... using ComfyUI to parse the movements from 2D videos and apply them to VAM.

[vvs123]

First, I have no programming knowledge at all. I just chanced upon a video introducing the motion transfer workflow of SCAIL Dancer, which mentioned that the motions from 2D videos could be exported in JSON format. While this isn’t actually feasible, I still want to bring it up since the method is completely free—who knows, it might work...
It developed by Kijai, this workflow extracts video motions into VitPose, then converts them to DWPose, and finally renders NLFpose into a motion reference with 3D information for SCAIL Dancer. I’ve tried to figure out if the data output by the "Pose Detection VitPose to DWPose" node is in JSON format, but it seems to output a dict instead. Could this possibly be converted into pose information compatible with VAM?

I asked an AI if it’s possible to convert this dict into JSON data, and it replied that dict is a Python-exclusive data type, and conversion to JSON requires additional processing—for instance, handling NumPy arrays. I have no idea if this is actually achievable in practice.

kijai-ComfyUI-SCAIL-Pose
SCAIL-Pose

 

[Clownmug]

I looked up a little about SCAIL and I don't see anything to suggest that it can generate animation data. It seems like it's just a controlnet for image to video generation.

However, there are other AI solutions specifically for generating 3D animation like QuickMagic and DeepMotion. If you're looking for a local model to use with ComfyUI I've heard HY Motion can generate animation with text prompts at least, though I've never tried it myself.

In VaM there are plugins like BVH Player and mmd2timeline you should be able to use to bring in whatever is generated.