September 26, 2018
Instructor designs Motion Capture Classroom using Unreal Engine 4
During this course, students can learn mocap by immersing themselves in a real-time learning environment. Using a combination of hardware (motion-capture suits and facial capture techniques) and software (Unreal Engine, MotionBuilder), students are able to write, direct, virtually shoot, and edit a film within Unreal Engine.
We spoke with Deepak at University of Texas at Austin to learn more about this incredible class.
Please tell us about your background.
My name is Deepak Chetty. I’m a filmmaker, educator, digital artist and developer based out of Austin, Texas. I received my BFA from Pratt Institute in 2007 where I studied film and the foundations of fine art. I worked as a post-production professional and freelance camera operator in the film and television industry for a few years before moving to Austin to get my MFA in film production from UT’s renowned film program.
During my time at UT, I focused on directing and cinematography. I decided I would like to learn as much as I could about emerging technology to fill my filmmaker’s toolkit with as many options as possible and was led to Unreal Engine as a content creation tool. I was hooked. I have since been using UE4 in the Intro to Immersive Media class at my alma mater, which I’ve taught since 2015.
The popularity of that course, as well as the advancement and relative democratization of this technology to filmmakers, has led to the creation of the Motion Capture Studio class at UT, available to upper division undergraduates and graduate students.
The Motion Capture Studio sounds amazing. Would you mind telling us a little more about it?
The Motion Capture Studio class will utilize UT’s Vicon optical motion-tracking system to allow students to generate virtual performances that will be laid out, lit, filmed and edited in UE4. It will allow students to explore the idea of fully computer generated filmmaking using real performances that can be modified, edited, and iterated in real-time. It was created to empower the next generation of filmmakers and creators to explore the veritable universe of freedom they will experience while working in a completely real-time environment using this technology. Students will use UE4’s Sequencer tool to do the majority of these tasks and will be building their virtual sets in engine as well.
What inspired you to choose UE4?
UE4 was always going to be the software we’d use in this class, or any other I teach for that matter, in which it would be applicable. The first thing most of my students tell me is that unlike many other 3D or development applications that are out there, UE4 is unparalleled when it comes to user-friendly design. If there was a triangle consisting of ease of use, image fidelity (very important for film people!), and feature sets, normally you’d be sacrificing an option. UE4 does not. This makes it a no-brainer for us at the film school because we can’t expect the students to come into these classes with experience in programming or advanced 3D applications.
What is great about this program is it’s a mile wide, and also a mile deep. The students themselves can decide how far into it they want to go once I’ve got them skimming the surface. On the pure virtual filmmaking side in regards to an educational process, they could go an inch deep, barely scratching the surface of what you can actually do within the engine and still create stunning pieces that would rival anything we were doing with past workflows. The amount of time saved by a non-traditional render workflow and the real-time playback allows students to spend so much more time being creative and iterating on their designs and ideas.
How did you previously teach this topic?
Previously we were using programs that featured render pipelines, lower quality viewports and a combination of complicated workflows, interfaces, and non-real time interactions that led most projects to be more simplified in their execution. We were not expecting students to tell stories, it was more about the shot. Singular. Now we expect a narrative, and with the toolset provided coupled with the ease of use and the robustness of the entire system, we know that isn’t asking too much of our students, even if they are relatively new to motion capture and virtual filmmaking.
Please describe your workflow using real-time technology.
At UT we have a dedicated motion-capture stage in one of our studios. Using the Vicon system, we capture performance into Motionbuilder, where we assign character rigs. Then we import into UE4 and begin the fun stuff. We are extremely excited to integrate MoBu streaming directly to UE4 this fall. We are also working on a real-time facial capture solution so we can capture that in tandem with our body motion. Right now we are recording facial animations separately during the traditional ADR process.
Could you talk a little bit about your experience of using UE4 as an educational tool?
Using UE4 in an educational setting has proved to be one of the more rewarding experiences of my career as an educator. This is because, at the end of every semester, I am so amazed by the work I am seeing from students, many of whom have never used a game engine in their life.
Within 15 weeks of our class (one 2.5 hour session a week) they are going from perhaps never creating anything in a game engine to being fully capable of producing interactive content and becoming Blueprint pros. Many students are coming from other departments in the university as well, not just on the film side. We’ve had journalism students, biology students, and fine arts students take the classes with the idea and goal of creative immersive and interactive content in their own respective fields. I've also taught UE4 to high schoolers who have come away from a two-week session with all the skills they need to dive in and get started on larger projects which gives me full confidence that UE4 could be integrated into curriculums for much younger students who are interested in interactive development and virtual cinema.
The freedom and empowerment students get when learning a piece of software like UE4 is astounding; it opens them up to creative possibilities they did not think were possible or perhaps thought were out of their reach. The Motion Capture Studio class in particular, I hope, will open up many avenues for creativity that would be limited or frankly impossible with live-action production at a student level. The idea that they could create an action-packed short without safety and cost issues, or visually elaborate art pieces without material costs or logistics and space needs makes UE4 invaluable as a creative tool of storytelling, expression, and artistic discovery at an educational level and beyond.
With the technological evolution of filmmaking, what advice would you give a student entering into the film and television industry?
I would encourage students who want to be content creators to familiarize themselves with current and evolving tech as much as they can. Even diving into the tech and software just to learn the vocabulary of a given part of the industry is important, seeing as that it is the job of the director or content creator to be able to successfully communicate their vision to their collaborators so that the vision can be brought to life.
Students who understand at a base level how programs like UE4 work will be much better equipped to tackle projects in a future that involves motion-capture integration, real-time VFX, and interactive content. This will allow for a greater sense of authorship over their works, a larger understanding of the endless possibilities and, an overall sense of empowerment when it comes to creative vision.
Campus Tests
Check out these campus tests created in the Motion Capture Class at University of Texas at Austin:
Sci Fi Shootout and King Chris and Pug
Created using Marketplace prop and character model assets by Synty Studios.
Abstract Dance
Joint project by Deepak and the Department of Theatre and Dance. Created with a rigged model and starter content shapes. Derived from raw mocap takes, no filters or smoothing applied. The turnaround for this project took one day.
To learn more about this course and other emergent and digital media area courses visit the University of Texas at Austin.
About Deepak
Deepak Chetty is an award-winning filmmaker, visual artist, and educator based out of Austin, TX. He teaches full-time at UT Austin in the RTF (Radio/Television/Film) Department where his classes are taken by upper division undergraduates and graduate students. He is a partner in Digital Quilt, an emergent media imaging and consulting company, and directs commercial work through Revelator, a full service creative production studio. He is currently developing two feature films and one short film he plans to create in Unreal Engine.