Techie Tech Tech

Ah yes, the tech.  What interesting and cutting edge technology elements will I be using to make this project totally enthralling from the interactive perspective? Well, let me tell you the production pipeline as I see it… a few items TBD until I can nail down the time commitments from some generous co-workers of mine but I’ve got a good picture thus far.

The company I work for, Mixamo, is a tech company that hosts an online end-to-end 3D character animation solution to anyone who lacks the skills or resources to make these assets from scratch.  Basically, we make it super easy for anyone to get a custom 3D (not stereoscopic, that’s different… confusing, I know!) character moving around in a game or film project in very little time.

So… I’m using the whole Mixamo pipeline for most of the major elements of my project. There are a few sundries mixed in there, but for the most part its software and hardware that I have access to by nature of working for a great tech company that develops pretty awesome IP.  If I had to make this stuff myself, let alone try to outsource it, I would be looking at months of work or paying a freelance 3D artist around $15,000.

Character Creation

With Fuse I can create a character myself and customize it to my heart's content!

To get a character that looks pretty much like me, I’ll use Fuse, Mixamo’s character creation tool.  The best part about using Fuse is that I can make the character myself with tons of customization options in case I choose a monologue featuring a character that is older or slightly more alien-species than myself.  No, seriously, I can make ME except as an alien.  I’ll show you, I promise!

Getting the Character Ready to Animate

I’ll have to cheat a little here since Mixamo is still working on a direct way to make Fuse character faces rigged (just think, ready to animate).  Rigging is an intricate process of tying the 3D model’s skin (also called mesh) to underlying bones so that animators can move the character around without things looks really wonky.  Its very time-consuming and takes a person with a specialized skill set to get it right.  Luckily, I’ve got two things working for me:

The Auto-Rigger. 

Mixamo’s online answer to automatically getting a body skeleton into any uploaded 3D humanoid mesh.  This will get me a rig that is ready to go in just a couple minutes – the alternative is about 5 days of work and close to $3500 at least to pay someone to do this.

The magic touch of a very awesome TD (Technical Director) that I work with.  

He shall remain nameless until I check with him that he is able to share his rigging magic with me and that he has extra time on his hands to do so… he’ll be using Autodesk’s Maya to do this magic.  Now… what to bribe him with…

Animating the Face

And finally, the very tricky part.  Again, facial animation is something so subtle and takes long, grueling hours to get right by professional animators.  If you can imagine it takes film quality animators a whole day just to complete 1 second of animation… you do the math, it takes a long ass time.  So I’m going to skip all that.
In this project one of my main objectives is to test some facial capture technology (motion capture, except just for the face) that Mixamo has developed.  Its a way for me to transfer my performance to a 3D mesh without incurring a huge production cost, and getting an immediate result.

Face Plus real-time facial capture

That’s what I’ll be using.  And the only hardware I need is my Macbook with an OSX that is 10.9.0 or higher and the existing webcam in the laptop.  Here’s how I would describe using Face Plus: like video-taping my own face onto any character of my choosing.  Here is a video that can help to explain.
The game engine I’ll use is Unity 4.3
As of this writing, Face Plus is only a plugin to be used inside Unity (or MotionBuilder), a powerful game engine that brings all these 3D elements together to make games, apps or even film content.
I’ll export the recorded animated character from Unity once I have the take I want.  This will require some tutorial guidance…

Putting It All Together

I’ll be filming my face as I use Face Plus to get the facial capture data from my performance.  In fact, to get it right I’ll likely try to get a couple of cameras from different angles set up so that I can easily show that all the data was taken in one sitting using one application.
Since my video editing skills are, let’s say, under development, I’ll need to take some classes on how to cut all this together.  Likely I’ll use Final Cut, just because that’s the software I have available on my laptop and the one I know the most.
And that’s that.  Some moving parts but not too bad.  I’ve listed out the elements, just for fun:
Fuse — Auto-Rigger / Maya — Unity / Face Plus — Final Cut — Up onto the Blog!
Macbook, Cameras
Tags :