Sony Imageworks' Tim Sarnoff Shares His Three-Pronged Approach to 3D

Sony Pictures Imageworks was five years old when Warner Bros. executive Tim Sarnoff took the helm in 1997. The company had already gotten on the map by luring five-time Oscar winner Ken Ralston away from ILM, and his presence had prompted perennial Ralston collaborator Robert Zemeckis to tap Imageworks for visual effects work. Signing Ralston was a move, Sarnoff observes, that said Sony was serious.

Seven years later, Ralston and Imageworks have just completed the visual effects on Zemeckis latest film, The Polar Express (a Warner Bros. release on Nov. 10). But the nature of the all-CG film, and Sony Imageworks contributions to it, speak volumes about how the studio has evolved. Not only have Ralston and company brought their proven visual effects skills to bear on the films fanciful worlds, but this project also marks the debut of the studios ImageMotion performance capture technology. Harnessing the latest Vicon capture hardware and applying proprietary software, Sonys ImageMotion team gave Zemeckis a way to use more expressive MoCap performances by Tom Hanks as the basis for five animated characters in the film. With another Zemeckis-produced project, Monster House, already booked on the ImageMotion stage, it looks as though Sarnoff can confidently add a performance capture business to the Imageworks portfolio.

But perhaps the most dramatic development of all is Imageworks role as a CG animation provider for sister company Sony Pictures Animation. Imageworks 3D animators are already committed to providing animation for Open Season, with characters voiced by Martin Lawrence, Debra Messing and Ashton Kutcher and directed by The Lion King co-director Roger Allers. Also on the horizon is Surfs Up, co-directed by former Pixar artist Ash Brannon and Chris Buck of Tarzan fame.

Regardless of the technical differences between the performance-captured stars of Polar Express, the animated critters of Open Season or the photoreal digital actors of the blockbuster Spider-Man franchise, Sarnoff sees a unifying 3D trend character creation. What ties all three of these businesses together is character animation. Developing that ability, he believes, is the greatest part of our success.

A Matter of Characters

That success was by no means assured. When I arrived at Imageworks, Sarnoff recalls, they were finishing Starship Troopers and Contact. We analyzed what was going well and what was going less well, and the character animation department was the least utilized part of the company. Even though the character animation group at that time had unbelievable artists, we hadnt had the work to prove that.

Sarnoff credits the 1999 animated effects film Stuart Little with changing perceptions. When we focused our efforts on doing digital characters, we were not necessarily worrying about whether it was a live- action film or an animated film. The first Stuart Little was clearly a live-action film and Stuart Little 2 (2003) was clearly an animated film. Seventy-five percent or more was animated. The Polar Express is a film with motion-captured people that comes out looking like an animated movie. Despite the fact that it looks like an animated film, there were a lot more live-action people involved on The Polar Express than there were on Stuart Little 2. When Id meet with The Polar Express team, much of what wed talk about were the environmental effects. This is, without question, the largest visual effects movie weve ever made it also happens to be the largest animated movie weve ever made, too!

Like Pixar and PDI before it, Imageworks had gotten its feet wet doing character animated shorts the Academy Award-winning The ChubbChubbs!, followed by the Oscar contender Early Bloomer. So when Sony Pictures Digital president Yair Landau announced in February that the Sony Pictures Animation division would debut with Open Season, it came as no surprise that Imageworks had been chosen to produce the CG. Sarnoff explains the relationship by saying, Sony Pictures Animation is a separate division that is run by Penny Finkelman Cox and Sandra Rabins. They report to Yair Landau, which Imageworks also does. We are partners in production. Sony Animation utilizes Imageworks as its production arm.

Sarnoff sees this part of Imageworks portfolio as offering distinct business opportunities, even beyond whats on tap with Open Season and Surfs Up. Fortunately, I have a great setup, with Open Season starting now and Surfs Up probably starting six months or a year from now, for delivery in 2007. But were not prohibited, frankly, from doing an animation project for another studio.

A Flexible Model

That flexible attitude isnt surprising for anyone whos toiled in what has traditionally been a service-oriented business like visual effects. Sarnoff notes, Sixty percent of what Imageworks does is still from other studios. I treat every studio exactly the same, including Sony. We are very clear about being even-handed, and that was something that was very hard to prove.

Sarnoff knows the problem well, having once been svp at Warner Digital, the short-lived visual effects company that Warner Bros. established in 1995. Despite a roster of experienced people, the company failed to convince directors such as Zemeckis to let them handle enough of Warner Bros. effects projects (like Contact).

The irony of this history isnt lost on Sarnoff. He believes, Warner Digital would have succeeded had Warners placed their bet on it. It had the right structure and a strong crew. Warner Digital would have been a great way for the studio to do a CG film. Sarnoff, who previously had worked at Warner Bros. Animation, thinks the studios storied history in 2D animation would have positioned it to capitalize on the rising tide of 3D-animated features. Their timing was perfect, but Sony Imageworks timing was even more perfect, and it lasted long enough to survive.

Having parlayed its digital effects expertise into doing CG-animated features, could Imageworks be creating a model that other studios will follow? Sarnoff thinks so. If they dont say it directly, theyre certainly going to do it indirectly. People need to find what makes the most amount of sense to maintain the highest utilization of their facilities. I dont think that anybody is going to do exactly the same things that were doing. Sony Pictures needs an animation business because we need franchise characters and the animation business is going to provide that. But it is also a very useful utilization of Imageworks that we are doing animation projects. To that end, I scratch my head wondering why other studios didnt try to do that.

I think part of the reason that other companies failed to do this well was because when the animation business came in, it over-ran by a large proportion the amount of work that the rest of the company had. It became too dominant in that business. Sarnoff doesnt expect this to happen at Imageworks, where he hopes to keep a flexible balance between CG animation work, live-action digital effects and the ImageMotion business. We have enough momentum in our visual effects business that were not going to leave that business because were suddenly doing an animation project. Animation is a minority percentage of what this company does as a whole.

Blurring the Lines

Of course, what qualifies as animation vs. visual effects is becoming harder to tease apart, especially when similar computer technologies are used for both. Certainly The Polar Express demonstrates the hybrid nature of the state of the art. Sarnoff asserts, Were looking at how we can best utilize our development efforts without necessarily distinguishing whether its an animated movie or a live-action movie. We use technology regardless of whether or not the end result is going to wind up being an animation project. No show is an island.

One of the great advantages is that I dont necessarily need to start from scratch from one show to the next; I can actually utilize some of the technology from one show to enhance our abilities on another show. One of the things that has happened with other studios in their effort to create multiple lines of digital effects companies where one group was only responsible for animation and another group was only responsible for visual effects was that you have multiple companies doing the same things. So you fail to amortize your investment.

While there traditionally have been noticeable differences between fanciful animation and the photoreal work that earned Imageworks its initial reputation, Sarnoff believes that as the dividing lines continue to blur, new perspectives will come into play. That was really very evident when we started hiring 2D animators and putting them on computers to learn how to animate Stuart Little. We suddenly realized what they brought to the show that we didnt have when we were just utilizing people from the 3D world. There was a whole new perspective. And when we started hiring camera people and model makers and putting them in our 3D match-move department, we got a whole new group of perceptions that we didnt have before. Im seeing the same thing now with the Sony Animation projects. When they are looking at the visual effects group and vice versa theyre taking advantage of each others point of view.

A Performance Paradigm Shift

No development at Imageworks better demonstrates this mind-meld better than the companys new ImageMotion business. While developed for the all-CG Polar Express, Sarnoff stresses, ImageMotion doesnt necessarily have to create a CG look. It could also have a live action look, he notes, pointing to its potential for doing photoreal digital characters for films such as Spider-Man. ImageMotion is a way that we can categorize these projects, in terms of the type of crew that we would put on it, he explains.

In terms of how he sees the ImageMotion business evolving, Sarnoff says, Im not actually looking to have just the process itself. ImageMotion is a performance capture process that is a little more detailed than the motion capture process that other companies have. If someone wants to come and apply the technology and acquire data, we certainly will be able to do that for them. Were not going to exclude others from utilizing it. But currently were using it for ourselves in-house.

Just as the demands of The Polar Express spurred Imageworks to develop the ImageMotion technology, Sarnoff sees Monster House challenging them to take it further. Weve been able to upgrade our ImageMotion system so that there is a dramatic improvement now for Monster House beyond what we had for Polar Express. When we did that film, it felt like everyone was on edge, just making sure that this thing actually worked. Now, walking into a Monster House capture session, it feels like youre walking onto any set in the studio. Its remarkable how blasé everybody is about a technology that didnt exist a year ago!

Without divulging too much, Sarnoff says the Monster House actors (including Mitchel Musso, Sam Lerner, Spencer Locke, Steve Buscemi, Maggie Gyllenhaal and Kevin James) will inspire the look and/or feel of the characters. But its stylized in a different way than The Polar Express. Unlike Tom Hanks acting juggernaut on that film, Sarnoff expects Monster House may have actors playing multiple roles, but as secondary options. It wasnt planned so that one actor could play many roles though thats a great way to amortize an actors cost.

The studios performance capture process, which allows multiple actors to be captured while performing together, is notable, says Sarnoff, because a director can have a one-on-one experience with an actor. And you have actors who are not particularly worried about `the camera. Theyre worried about the performer that theyre performing with. This is literally a closer relationship between director and actor than a normal filmmaking process, where you have a camera in between them. Now you move that camera away. Its very much like a theatrical rehearsal, and the director afterward can `record the performance from the angle that he or she chooses. Its a truly unique way of making a film.

When we started on The Polar Express, we could literally only capture one actor and his gross motor movements once, and capture his facial movements later. We expanded to literally three people captured in a 10 x 12 stage and we are now, on Monster House, on a 30 x 40 stage. And we are probably going 20 to 30% faster than before. Pretty soon well be able to get rid of the cameras altogether and just `remember it, he laughs.

Fluid Dynamics

Whats interesting about Imageworks hybrid approach is that its allowing artists to both specialize or to cross disciplines on certain projects. The specialists in the ImageMotion process are going to stay with Imageworks, says Sarnoff. We actually have lined up a third project after Monster House so that theres longevity, and we can keep the `brain trust active here in a single place.

At the same time, Monster House has attracted veteran visual effects talent such as supervisor Jay Redd (Stuart Little, Haunted Mansion). Sarnoff observes, What is unique about Imageworks is that we are able to offer to our artists and producers and software engineers an opportunity to not necessarily spend their entire lives doing one kind of film. The issue for me is efficient utilization.

The secret behind all of these types of projects is that they are all very different from each other in terms of the amount of time each one takes to be accomplished. Surfs Up and Open Season are three or four-year projects from beginning to end. If you look at Spider-Man 3 or movies with invisible effects, those are often nine to 10- month projects. People dont necessarily come off those films at the same time. You might find one person doing an animated project for two years and then moving on to a motion capture project for a year. They dont get bored. Many of the Stuart Little group are now on Open Season, or The Lion, the Witch & the Wardrobe (an effects film that will be Shrek director Andrew Adamsons take on the C.S. Lewis story from the Chronicles of Narnia). Theyre very different projects. Ultimately what Im looking for in this company is to keep people excited about the next thing they want to work on.

Sarnoff adds, For me, as the manager of a company that has a lot of artists, how do we keep them engaged? I am never embarrassed about offering an artist an opportunity to work on a specific project here. If youre not a `superhero person, maybe youd like to work on a Martin Scorsese project like The Aviator (currently being supervised for Imageworks by Oscar-winning supervisor Rob Legato).

But for superhero visual effects aficionados among the Imageworks ranks, Sarnoff promises, Were absolutely sure there will be a Spider-Man 3 and that helps us to do our long term planning. Id be willing to bet that there will be a Spider-Man 4 also, which really helps me in very long-term planning. There is very little reason why we would not be engaged in every one of the Spider-Man projects, considering the success of the past two.

With a dozen experienced visual effects supervisors on deck, including Ralston, Legato, Jerome Chen (The Polar Express) and Rich Hoover (Seabiscuit), Sarnoff expects there will be no shortage of assignments for live-action visual effects films. Sarnoff acknowledges that its actually a comfort factor to know that the growing business for invisible digital effects will keep the companys talent busy. Visual effects assignments for Bewitched, (starring Nicole Kidman) and Wes Cravens Cursed suggest that Imageworks original business continues to thrive.

The Long View

Im thrilled that this idea has actually worked, admits Sarnoff. For now, this is a plan thats not necessarily bulletproof but it certainly works for this company. Its given us a great deal of opportunity to utilize people over a long period of time. I have a number of people I would say probably more than 500 people who have been here for more than five years. Longevity is important. You have to have people who will communicate efficiently, and it takes a certain amount of time for people to figure that out. We have a rule here called `the 15 minute rule. That means if you dont have an answer for how to do something within 15 minutes, you stop what youre doing and you ask someone. When you know which person will have the right answer, thats when the 15 minute rule works.

Everybody here knows what the vision for the company is. The plan and the products that are within each one of the separate categories actually exist pretty clearly This company is never going to be, in my mind, 100% live action or animation. It doesnt make sense for us to go back to being a company that does no animation or no live action, and that seems to be pretty well understood by everybody. They all agree this is working for them because were talking to them not only about the project theyre working on today but the project thats coming next month or next year. This morning we were running through projects for 2006 and who we were going to book for them. Theres still bumpy water, but the river is wider and it runs just as fast.

Ellen Wolff is a Southern California-based writer whose articles have appeared in publications such as Daily Variety, Millimeter, Animation Magazine, Video Systems and the website CreativePlanet.com. Her areas of special interest are computer animation and digital visual effects.