Brief Introduction
MikuMikuDance (commonly abbreviated to MMD) is a freeware animation program that lets users animate and create 3D animated films, originally produced for the Vocaloid character Hatsune Miku. The MikuMikuDance program itself was programmed by Yu Higuchi (HiguchiM) and has gone through significant upgrades since its creation. Its production was made as part of the VOCALOID Promotion Video Project (VPVP)
This is the introduction of MMD from Wikipedia. However, in the following context when we mention MMD, we might more likely be referring to the various dance motion files that are popular online, mainly in the styles of K-POP and J-POP. Thanks to those MMD creators who might not be well-known in the VAM community. They continuously turn the latest and most popular dances of Korean idol groups into MMD animations. Many of them not only have high animation quality but, more importantly, are very sexy.
Here is an example,
Everyone loves music, and who doesn't enjoy watching sexy dance moves? MMD actually has a large user base within the VAM community. Over the years, our creators have continuously developed and updated various free or paid MMD playback plugins for VAM. Beyond the musical dance scenes created by other genius creators in our community, MMD also offers players another option to enjoy music and dance.
While using the MMD plugin to watch dances is both easy and straightforward, doing secondary creations based on it in VAM becomes a bit more complicated. The process of transferring animations, like those from MMD, to another platform like VAM can be complex. It typically involves converting the motion files to a compatible format, ensuring that the animations map correctly to the character's skeleton, and optimizing them for the best performance in the new environment.
You might have already seen many dance motions that have been processed and converted in VaM Produce 69. After going through this guide, you can create similar scenes as well.
So that's what this tutorial is all about.
Preparation
Though in the past, this seemed like a hard thing to accomplish, but fortunately, we have a powerful animation editing tool called Timeline prepared for us by Acid Bubbles. Not long ago, another creator, sfishere, developed a plugin for Timeline called mmd2timeline. This allows MMD enthusiasts like me to not only enjoy thousands of MMD dances in VAM but also to build upon them for my own creations.
In this tutorial, we will be using the following plugins:
Required plugins:
Timeline v6.3.1 by Acid Bubbles (You'd best use version v6.3.1 or later, and I'll explain why later on.)
mmd2timeline v0.12 by sfishere
Optional:
Embody v1.6.0 by Acid Bubbles
Utilities v1.4.0 by Acid Bubbles
VUML 1.9 by JayJayWon
ActionGrouper 1.9 by JayJayWon
MacGruber Essentials 16 (free) by MacGruber
If you aim to process camera motion (CamRide) more efficiently in terms of FPS, without using the window camera but still maintaining the dynamic adjustment effect of FOV, this tutorial will also introduce how to achieve this using the aforementioned optional plugins.
This guide comes with an accompanying scene as courseware, which includes all the plugins required for the guide. It's designed to help everyone understand by providing a point of reference. I hope it does so.
Part One - Conversion
Step 01, We have created a new scene and added character atoms. The first thing to do is to open the hand control panel and change the control methods of both hands to JSONParams. In the future, if you import the converted Timeline motion file into other scenes, remember to make such changes, otherwise, the Timeline will not be able to control the ten fingers.
Step 02, We loaded the Timeline and mmd2timeline plugins separately on the character atom. In actual use, you don't have to keep them on the character atom at the same time. Once the Timeline motion is imported, the mmd2timeline plugin can be removed.
Step 03, The panel of the mmd2timeline plugin is very intuitive. First, you need to decide whether to enable the high heel effect. Please note that it will be more troublesome to change it after the conversion is completed, so make a choice based on your needs. Then click "InitAtom". After initializing the character atom, click "Import Vmd".
A side note about the high heel effect is that for motions converted by mmd2timeline, if you check the high heel effect, you won't need to rely on other plugins to assist with high heels in future use (a very convenient option). You only need to put on the default high heels that come with VAM to achieve the high heel effect and the physical collision of the shoe heel. If you want to put on other high heels for her, just wear two pairs of shoes and adjust the transparency of the default high heels to hide them.
Step 04, Navigate to the motion file you need to convert, which is typically with a .vmd extension, and click to complete the loading.
Step 05, Before you start the conversion, you can adjust the conversion options. Of course, you can also make adjustments midway. The Sample speed does not have any impact on the final result and the quality of motion conversion, so you can keep these values at their default settings. As for the bone adjustments, in most cases, it's not necessary unless you find obvious anomalies in the motion after the conversion. Regarding this point, we will elaborate and explain in detail in the optimization chapter later on.
Now, let's directly click on "Start Sample," and then go make a cup of coffee and wait patiently.
Step 06, After mmd2timeline has finished sampling the MMD motion, click "Export Person Animation", give it any name you like and save it.
Step 07, Next, we'll open the Timeline panel. In the "More" tab, select "Import/Export animations", and then follow the guidance of Timeline to import the motion file you just converted into Timeline.
Step 08, What we need to note here is that after importing into Timeline, it usually plays in a loop by default. We don't want this (especially when you plan to use the CamRide introduced later). Therefore, in the "Edit" section, deselect the "Loop" option. After turning off the Loop, the action of your last frame is the same as your initial frame, so we need to make some adjustments next. First, in Timeline, click to select all control targets, making them highlighted.
Step 09, Drag the Timeline's timeline to the last frame (you'll notice it's the same as the initial frame), then use the "<f" key to go back one frame, select "Copy", and then return to the last frame and choose "Paste".
Step 10, Now, open "Scene Audio" and load the music file into the scene. Then return to the Timeline, click "+" next to the Targets Tab, and select "Add audio track." You'll notice that an additional trigger control entry has been added to your timeline.
Step 11, Now, let's add an AudioSource to the scene.
Step 12, Open the control panel of AudioSource and change the default settings to what is shown in the picture. I strongly recommend you do this, especially if you plan to use CamRide.
Step 13, Let's go back to the Timeline, and add such a trigger at the initial frame of the "Audio 1" timeline you added earlier.
In the past, we required additional steps to ensure the motion and music synchronized perfectly. However, in the latest version of Timeline, Acidbubble successfully addressed the longstanding issue of real-time synchronization that had been a concern for many. We owe him our gratitude. This is why I recommend using the latest version of Timeline. As of now, your conversion work is essentially done.
Step 14, You can go to the Plugin Presets panel and save the entire setup as a plugin preset. This way, you can call upon it as an animation preset in any other scene. Now, the basic MMD motion conversion is essentially complete.
Part Two - CamRide
Typically, the vast majority of MMD files come with a camera motion file, which, after being converted through mmd2timeline, can be directly used to produce CamRide for use in VAM. Let's now delve into how to achieve this.
Step 15, Open the WindowCamera control panel in the scene and load the Timeline plugin as well as the plugin named mmd2timeline.CameraAtom.cslist. Again, after completing the conversion, you can also remove the mmd2timeline plugin.
Step 16, Just as we did before, click on "Import Camera Vmd" and select the camera motion we want to convert.
Step 17, Camera motion doesn't need to be sampled like the previous character motion processing. So, after loading the file, just click "Export" and save it with any name you prefer.
Step 18, Just like how we processed the character's motion earlier, import the camera's motion into the camera atom's Timeline. After unchecking Loop, copy the second-to-last frame and overwrite the last frame. Additionally, you'll need to go to the Options tab and uncheck "Sync with other atoms".
Step 19, Then, return to the trigger point we added in Step 13 on the character atom's Timeline to start playing the music. Add two triggers: the first trigger stops and resets the Timeline on the camera atom, and the second trigger synchronizes the camera's motion with the music and character motion to start simultaneously.
In fact, by this step, the regular conversion operations are already done. But if you want to achieve the best results, there are still many details regarding the camera that we need to address.
First, there's the issue of camera height. When creators in the MMD community produce MMDs, they often base them on cartoonish character models. The height, proportions, and even the bones themselves of these characters differ significantly from the models we typically use, which are closer to real human proportions. This not only leads to potential challenges when adapting MMD character motions to VAM (which we will delve into and address in subsequent chapters), but it also often makes the camera height seem off. In many cases, you might feel that the default camera leaves too much empty space at the top of the frame.
Fortunately, this problem is easy to solve. Here's a simple solution:
Step 20, Let's create a sub-scene and name it "CamPan." Its function is just as its name suggests, used for adjusting the height of the camera.
Step 21, Place our WindowCamera into the CamPan sub-scene and set it as its child atom.
Step 22, In this way, you can adjust the height of CamPan to appropriately modify the frame ratio of the entire camera motion without changing the camera motion itself. Based on my experience, in most cases, you can try values between -0.1 and -0.2. You might have noticed the difference in the camera framing in the screenshot compared to before, and the position of the character in the frame now looks more reasonable.
Up to this point, all the steps seem to have been completed. However, from the perspective of scene performance, I do not recommend you to process CamRide using the WindowCamera. Continue to operate as per the following instructions to implement CamRide without using the WindowCamera, which can save you at least dozens of frames in frame rate loss.
So, let's continue:
Step 23, First, we uncheck these two options and turn off the WindowCamera.
Step 24, Create a new empty atom, name it "CamRide", and move it to a position that completely overlaps with the location of the WindowCamera. Ensure its position coordinates and angles align perfectly with the WindowCamera. The simplest and most effective way to do this is to use the "Select Align To From Scene" function.
Step 25, On the CamRide empty atom, load the following plugins: ParentHoldLink from MacGruber Essentials, FieldOfViewController from Utilities, VUML, ActionGrouper, and Embody.
Step 26, With ParentHoldLink activated, modify the control settings of CamRide as shown in the screenshot above.
Step 27, Open the VUML plugin panel, click on "Add VFLOAT", and set it according to the screenshot above.
Step 28, After opening the control panel for ActionGrouper, adjust the settings to match what's shown in the screenshot. The purpose of this is to pass the FOV value obtained from VUML in the previous step to FieldOfViewController, in order to change the desktop FOV. You might have noticed that the desktop FOV has changed.
You're all done.
Now, even without using WindowCamera, we can still achieve the functionality of CamRide. Moreover, by changing the desktop FOV, we can realize the variation in FOV. This method of handling CamRide might not seem much different from directly using WindowCamera when the scene is simple, like in this tutorial example which is almost an empty scene. However, when you are creating richer and more complex scenes, you'll be able to notice the difference in their performance.
Part Three - Use the dance you converted in VaM Produce 69 or create your own scene
Although VaM Produce 69 provides many pre-made dances, you can also load your own converted motions directly into this scene for use. You just need to perform a few simple steps:
Step 29, Just as we did in Step 10 earlier, load the corresponding music file into the scene.
Step 30, Open the character atom's "Plugin Presets" panel and load the plugin preset we saved earlier, replacing the existing preset.
Step 31, Open the Timeline, and under the "Edit" tab, change the Animation name to "Dance".
Step 32, Open the Preset panel of WindowCamera and replace the current preset with the one you saved earlier. There's no need to make any modifications to the Timeline. If you want to adjust the camera height, there's an atom called "CamPan" in VaM Produce 69. Simply locate and adjust its height.
Then click play.
When you want to create your own scene or import dance motions into other scenes, the steps are the same.
Part Four - Further optimization of Motion
Well, this is a big topic with many aspects to consider and numerous methods to handle. You could practically write a book on this chapter alone. So, for today, we'll leave it at that. In the previous sections, we've discussed how to convert MMD motion for use in VAM. In the next update, we'll delve deeper into optimizing them further to ensure they perform even better within VAM.
If you have any questions or suggestions, please leave a comment in the discussion. I'll do my best to respond to everyone.
