Troubleshooting ActorX for Maya

The last post I made, I showed off one of the new rigs I had completed. The bad part about that rig, however, is that the mesh was bad. When exporting out of Maya, using ActorX, as an PSK, Maya either crashed, or after a while, output a message in the Maya output window, and did nothing. The message simply read:

processing mesh meshNameShape

It's my understanding that the modeler was using a new feature available to Zbrush that allowed seamless work-flow from Zbrush to Maya. Apparently, this feature has some bugs and could have possibly messed with the vertex order, or left bad history on the mesh. So if this happens to you, Here are a few things to try:

1. The mesh may contain bad history, or history unrelated to skin clusters. If this is the case, select your mesh and open your hypergraph. Click the button, Input and Output Connections. Here, you can see the nodes attached to your mesh. Find any node that is not related to skin clusters, or bones, and delete it. Typically they are along the lines of mesh tweaks, or tools that were used while modeling.

To prevent having to do this, always clean up your given mesh before binding it to your rig, which entails: deleting history, moving pivots to the world origin, freezing transformations, etc. Do not try these steps after binding because it will mess with your skin cluster data. If this issue comes up after you have finished your rig, you will more than likely have to re-import your mesh, and bind it to the skeleton again. Which brings us to our next method.

2. If you have tried everything in the first method, then try reimporting your geometry. Do not simply drag and drop the Maya file into your scene, it will just bring in the bad mesh again. Try exporting the geometry as different file types. The one that worked for me was an FBX file. The model was made of several pieces, so I exported each one in a separate file. I brought those into my scene, did a really fast dirty skin bind, and it still didn't work. I didn't give up!

My next step, was to detach the model pieces, combine each piece into one mesh, and cleaned up the model (delete history, etc.). This is highly not recommended. In my case, the model was not textured, or mapped properly in anyway, which meant that the model the artist had to work on, was the one I rigged. In a team environment, this is just plain bad. Luckily, not much was done to the model after I had received it, so all was not lost.

Having combined all the geometry, means the vertex order was rearranged, so any skin cluster history would be rendered useless. If you do this, know it will mean you have to bind the model to your skeleton again, and adjust any point weights again. That is what happened in my case, but the model exported just fine afterward, with the skeleton attached.

So here are some steps to follow before working any further on your rig, or skin weights:

1. Bring in your model and clean it up! Delete history, move all the pivots to the world origin, freeze transformations, the works. Do not leave any unnecessary nodes laying about.
2. Build your skeleton and do a quick skin bind to your mesh. Try exporting the mesh with the skeleton before working any further. If it exports just fine, great!. If not, try the next step.
3. Talk with the modeler about how they worked on the model, and if they exported it from another program, look at their process and see where things might go wrong. If fixing their process doesn't work, let them know that you are taking the next step in order to get it to work, and see if it might hinder any of their work.
4. Take the original mesh file, and export as a different file type. These can be anything from FBX to OBJ (recommended) but you may choose what you wish, so long as it fixes the problem. Bring in the new objects, do a dirty bind, and try exporting using ActorX. If that doesn't work, try step 5.
5. with all your new geometry imported, combine all the pieces into one mesh. Renamed it appropriately, and attempt exporting with ActorX using a dirty smooth bind. If it works, continue working. If not, make a post on the forums for UDK.

I hope someone finds this tutorial useful. Please feel free to comment or send me an email. Just a note, this is my first time using ActorX. I have been using Maya for well over six years, and have worked on several other game engines, and these steps typically work on the Maya end of things.

Happy rigging!