Hybrid Workflows: The Key to Creative Control

300 has widened the scope of hybrid workflows. © Warner Bros. Pictures. 

It is indeed a wonderful era for filmmakers. Everyday we seem to break new ground and leave what had been uncharted territory far behind in our wake. The industry is on the verge of a new era created by the digital revolution, the era of the hybrid workflow. No longer do filmmakers find it necessary to wait upon set builders and actors to fulfill their visions. Now image-based 3D-performance capture and the various pre/post vis pipelines often supplant live action. As Sebastian Sylwan, Autodesk's senior film & TV industry manager, puts it, "It is a brand new world and an exciting one to be part of. You never know what will come up next. We've broken yet another barrier, and we're getting closer to allowing filmmakers to express their vision without technical limits. Our goal is to enable that vision every step of the way."

Sylwan sees the hybrid workflow as the convergence of CG movies, live action and games into a single digital production environment that serves a hybrid purpose. "We are part of a new frontier and we are committed in developing the tools necessary to support present and future workflows."

What does Sylwan see as the catalyst driving this new paradigm? "An all-digital pipeline from acquisition to finishing is the key and every project pushes the boundary a bit further in one or more areas of the pipeline. Projects coming out of Weta and ILM such as the Lord of the Rings films and Star Wars (respectively) got the ball rolling, but the film 300 really took this workflow to a whole new level. A Canadian company Hybride did an incredible job of integrating greenscreen live-action elements with computer generated characters and environments for 300. Other films that have set the mark for animated digital humans, such as the Spider-Man series, Superman Returns, Beowulf and Avatar (currently in production) and these will certainly become milestone films. Also, fully CG animated films like Monster House are helping shape the process in a convergence of pipelines and tools."

As for Autodesk, how have they helped shape this convergence to a hybrid workflow? "Autodesk is in an excellent position to help realize these new types of workflows since our integrated solutions are key in all of the most critical parts of the pipeline. Most performance capture pipelines use Autodesk MotionBuilder to aid in realtime pre-visualization of captured performances. Then the captured data gets transferred down the pipeline via Autodesk FBX, which is the universal format that allows most major 3D packages to accept motion-captured data. Since these are 3D pipelines, Maya and 3ds Max are at the core, for anything from previs, layout, camera, modeling, animation, all the way to rendering. Autodesk Flame is also an ideal tool for these types of projects where the key is interactivity in managing a large number of filmed and CG elements since it gives artists greater creative control. Last but not least, color grading workflows are often based around Autodesk Lustre."

More and more, Sylwan finds that speed is their clients' greatest need regardless of the medium. "Our clients are coming across an increasing need to deliver more, better and faster. This is across the board in films, television and games. Although each project generally provides our clients with its own unique set of challenges--the need to create greater pipeline efficiencies and more cost effective workflows is a constant. Across the board our tools for previs, 3-D, vfx and post are striving to be more tightly integrated, interoperable and deliver end-to-end solutions so that our clients can streamline pipelines and increase efficiencies while expanding their ability to create and innovate. This is their differentiator in the market."

Autodesk's Sebastian Sylwan is ready for the brand new world of visual effects.

And what advantages do Autodesk clients incur using this hybrid paradigm in terms of efficiency, time and cost? "Directors appreciate the creative control that hybrid workflows represent. Without being burdened by the constraints of weather conditions, shooting locales and location budgets, they can focus on creating the environments, digital crowds and, most importantly, actors' performances that meet their vision without compromise. When you are not limited by physical restrictions you have total creative control. This opens a new universe of possibilities for filmmakers. They can spend their money and time on what affects the most impact to the final picture."

With these new pipelines on the cutting edge of technology and production workflow, digital acquisition and generation are changing the way films are produced. Providing filmmakers with greater speed and efficiency in realizing their vision is the name of the game. What is key to accomplishing that? "Autodesk tools, being integrated and interoperable, enable that change," explains Sylwan. "The key is maintaining creative control at every step of the way."

Speaking of new paradigms, there are also new distribution platforms benefiting as well. Ron Martin, a SOFTIMAGE|XSI client, is the director of visual effects for Sanctuary, the Web-delivered and HD episodic, that has been touted as "Buffy meets X-Files." Each bi-weekly episode from Stage 3 Media, Vancouver, B.C., is 15 minutes in length.

What particular client need inspired the use of the new hybrid paradigm for Sanctuary? "We had massive demand for previs in our work, due to the lead in time and the amount of CGI design that had to be incorporated," Martin explains. "We used SOFTIMAGE|XSI 3D software for many of the previs layouts and concept designs."

The two things that precipitated Martin's decision to let image-based 3D performance capture supersede live action in his previs pipeline were the amount of time needed with actors prior to shooting and the lack of studio accessibility. "Those two things are very expensive, so we took it as far as we could before we needed to spend 'real movie money'."

What difficulties were they able to overcome? "The lack of access to actors and the director was tough. We made a lot of decisions, which we knew we could handle and in the form of small camera animations received feedback via email or vpn. We saved a lot of time on set with set-ups and blocking. The pacing of the show was comfortable as well, before we hit set."

Surf's Up had a workflow that combined fully animated 3D with live-action camera moves. © Sony Pictures Animation. 

As for the advantages of incorporating this hybrid paradigm into his workflow with SOFTIMAGE|XSI 3D, Martin gained a great deal in terms of efficiency and cost when it came to the actual production. "We knew where the camera had to go and what areas we could dress to make the shots interesting. We could also push back a bit on the hopes to scale back budget where necessary. We saved a lot of money on props which were not going to be interacted with and the amount of time required to dress the set."

Tracking was another important cost-saving aspect that Martin pointed out. "Camera tracking on set made this opportunity feasible. Walking away with as much camera data as possible saved our budget."

Getting back to feature films -- specifically animated features -- I wrote about Surf's Up from Sony Pictures Animation in my May article on CG water effects. As it turns out, Surf's Up was also a rather unique hybrid. James Williams, the head of Layout on Surf's Up, explained why the feature wound up blending a fully animated 3D production with live-action camera moves. "The documentary nature of the production dictated the use of a live-action camera. We wanted to capture the spontaneity of a hand-held, live-action camera but with the subtlety and nuance of characters animated by conventional means."

Of course, whenever one is dealing with a hybrid that involves motion, one has to be aware of eyelines. "One of our key challenges we faced was how to make sure that the animators knew where the camera was going to be to capture their performance and to make sure eyelines were accurate with respect to the camera's position. This was done by placing rough cameras into the shot before the animators began their work. Then, once the initial animation of the character's body was complete, the final camera was motion captured. Then the animator could complete his shot with the final camera. This enabled us to hone the animation very precisely."

Williams points to motion capture as one of main advantages of the hybrid paradigm in terms of efficiency. "The wonderful part of motion capture is that it is a realtime performance. Directors Ash Brannon and Chris Buck and our director of animation were able to review and comment on the camera work as the camera operator was working. It is very rare in conventional 3D animation for any part of the process to be done, reviewed and approved simultaneously."

In Lustre, color key and shape geometry combined to create a matte in the secondary grading mode. 

Meanwhile, one of the long-standing challenges to working with digital images and is handling color accurately across the visual effects and post-production workflow. As Sylwan previously mentioned, Autodesk's Lustre is one product that addresses color grading across workflows. Another product that meets this challenge is Color Symmetry, a suite of plug-ins developed by Duiker Research Corp. "As images come into and are tweaked within animation, compositing and painting packages, Color Symmetry let artists and facilities immediately preview how the results will look on film," explains Haarm-Pieter Duiker, the head of Duiker Research. "They can accurately and consistently check color from within and among the software they are using, saving time and reducing guesswork. They can also build a full color pipeline right out of the box to match their shots to any real or created film look."

Not having a solution for the color matching what is being produced in animation and effects packages for a motion picture can be a major source of lost time and extra work in the digital post-production process. On productions that derive largely from digital data or have created digital looks, this is all the more true because you do not have shot film plates to commonly reference as the artists and facilities contributing to the project create, iterate and integrate their work. So how exactly does Color Symmetry help its clients resolve these issues?

"First, Color Symmetry lets them bridge the gap between the images as seen in popular visual effects packages and the images as they would appear on film," Duiker points out. "Color Symmetry emulates film looks and recreates them faithfully as artists create and manipulate images, removing the need to constantly print to film to check results. Color Symmetry's Film Profiles can represent customary film stocks as well as those generated daily via a Digital Intermediate (DI) process. Second, Color Symmetry eliminates the disparities between the ways animation, compositing, rendering and image manipulation packages handle color, providing color conversion tools even between 2D and 3D toolsets with support for Cineon/DPX Log color image data as well as floating-point HDR Linear color image data. Third, Color Symmetry supports widely available formats so that data can be interchanged between multiple facilities. And, finally, it lets a company of any size or scope easily build a color pipeline that is ready for feature film."

Duiker adds, "With some of the exciting recent innovations in filmmaking, much of the work takes place without the common denominator of traditional shot film plates. So clients need tools that allow them to connect to a medium and work consistently to it across applications, artists and even facilities. Color Symmetry is addressing these needs on the color side by providing ways of communicating and integrating shots with consistency even on complex shows."

In this era of digital expansion, it is quite obvious that filmmakers are enjoying both consistency and greater creative control like never before with a convergence of workflows throughout the pre and post visualization process. Expanding technology has not only opened this new frontier but has also brought filmmakers additional benefits in terms of greater efficiency and cost savings. So, as filmmakers break new ground and sail into uncharted territory, one has to agree with Autodesk's Sylwan when he says, "These are exciting times. Stay tuned. This is the new frontier of digital production."

J. Paul Peszko is a freelance writer and screenwriter living in Los Angeles. He writes various features and reviews as well as short fiction. He has a feature comedy in development and has just completed his second novel. When he isn't writing, he teaches communications courses.