The Digital Eye: Sony's Future with 3D Environments and Acting

In this month's "The Digital Eye" column, Sony Pictures Imageworks' Jenny Fulle describes how a perceptive combination of creativity, talent and technology is advancing the art.

Image courtesy of Deron Yamada. © 2004 DYA367.

Image courtesy of Deron Yamada. © 2004 DYA367.

Our goal for everything we create here at Imageworks is that the end result be believable. You might think that's a little strange, since at the moment we're working on things like surfing penguins and web-slinging superheroes that don't exist in real life. Being believable is something deeper and more challenging than just being realistic, however. Good 3D digital productions also require artists who can convey emotional truth.

3D effects and animation, which really are two-dimensional representations of depth, enable convincing integration between live-action and digital effects. Obviously that's essential for characters, but it's important for environments too. That's why we try to set up productions in a way that makes it easy for the different disciplines to work with each other. On next summer's Surf's Up, for example, the character animators and designers and directors had a definite impact on the people who handle water and other liquids. The water specialists then created waves that look and feel amazingly real. And now they've taken that pipeline and customized it for the entire Surf's Up production. I can't think of a film we've done that doesn't leverage our environment expertise.

There have been many other times when that collaborative spirit has pushed us to accomplish great things. Picture Doc Ock in Spider-Man 2: each of his tentacles has its own distinct personality. That extra emotional touch added depth to Alfred Molina's character, and it was possible only because we brought our animators and programming people together to work on him.

The same goes for the flames in the upcoming Ghost Rider. The main character and his motorcycle are engulfed in flames, but going 300 miles-per-hour would extinguish the fire in the real world. These are hellfire flames, though, so our animators determined how to make them believable even though they don't conform to the laws of physics. That's not anything Maya can handle natively! A lot of what we've done in the past 15 years had never been done before.

We're really doing something that no one else can do. We're not just doing shots. It's an exciting challenge every day, and we are driven to new heights with each project. Partly, that's because audiences are becoming more sophisticated, which makes filmmakers want to come up with something fresh in each movie. But it's also because once we determine how to do something, we get really good at it and we can put what we've learned into our system. The ideas are the same, but we can now do much more and do it more efficiently.

On the first Spider-Man, for instance, the buildings were a huge creative and logistical nightmare. We spent weeks and weeks getting them exactly right -- they were indistinguishable from real buildings and they fit with director Sam Raimi's vision for what he wanted. Now we can do them like they're easy. We like to think of it as a virtual backlot -- the largest virtual backlot in the industry.

Every new project is still a new challenge, though. One 3D character used to be a huge deal -- and with Spider-Man 3 we have five (not the least of which is Sandman)! I admit our plans didn't always work out exactly as we expected, but there's no doubt it will set a new bar when we've accomplished it.

We spend a lot of time learning and developing and improving our skills and tools, but we make sure we don't reinvent the wheel every time. We document and save everything, so that next time we can start from there and build something even better.

By following that procedure, we can devote more of our capabilities to creating convincing 3D characters without compromising on visual or emotional authenticity. You obviously intellectually know that Brandon Routh can't actually fly, but when you saw Superman Returns you totally believed in the reality of Superman. There's not a moment that you don't think you're looking at a real person. The perfection of what we do is on display when you look up at that giant screen.

In the first Spider-Man, buildings were a huge creative and logistical nightmare that took weeks to get right. Now Sony Imageworks can create them easily. © 2002 Columbia Pictures Industries Inc. All rights reserved.

In the first Spider-Man, buildings were a huge creative and logistical nightmare that took weeks to get right. Now Sony Imageworks can create them easily. © 2002 Columbia Pictures Industries Inc. All rights reserved.

With our huge library of assets at our fingertips, we're freed up to work on the next level of creativity. More efficiency leaves more time and flexibility available to the filmmakers. Cuts change, directors change -- using 3D CGI, now the performance can change as the shot need changes.

A lot of directors appreciate being able to change something after they shot it. There's one particular building in Ghost Rider that Mark Steven Johnson thought would look better if it were quite a few stories taller, so we changed it to fit his vision.

Another good example of that is in Superman. There's a shot build that I think is on the DVD that shows how they made Brandon fly, and it goes from greenscreen to finished shot. Well, he's not smiling in the original greenscreen. Our artists went so far as to animate his face to make the scene more believable.

Because we've been pushed to do more authentic environments and characters, we've also been able to come into animation at the top rung of the quality level. It's our heritage. Stuart Little, Polar Express, Open Season, Monster House, Surf's Up, which we're working on now -- we cover quite a wide spectrum, but our core philosophies are consistent no matter what. It doesn't matter what we take on -- visual style, tone, whatever -- we can hit it.

Jenny Fulle

Jenny Fulle

It starts with putting your team together. If you put all of the best people in the world together on one project it doesn't necessarily mean you'll get the best result. The first step is to take into consideration personality matches and who would be good at what, until you achieve a balance of personalities, abilities and skill sets.

Then the first decision to make is, are your characters creatures or human? Everyone from the supervisor on down knows what they're best at. It doesn't matter whether it's a big team or a smaller team, they know. Imageworks encourages everyone to explore and develop their capabilities instead of getting pigeonholed, so doing this is more complicated that it might seem at first.

The second decision is whether it's photorealistic or stylized. We've done both and we like doing both. The animation is a style choice, how you want to bring that character to life, while the environments extend across any style.

Once your team is assembled and those decisions have been made, you're ready to apply your artistry, talents, tools and technology toward making the filmmaker's ideas a reality. We've got the best technology in the world, but it wouldn't be any good if we didn't have the artistry and experience needed to bring imagination to life.

Jenny Fulle, evp of Sony Pictures Imageworks, is currently overseeing visual effects and animation production on the feature films Ghost Rider and Spider-Man 3. She recently oversaw production on Open Season, the first fully animated CG feature film from Sony Pictures Animation, and currently is serving the same role for SPA's upcoming CG feature Surf´s Up.

In her role as executive producer at Imageworks, Fulle's credits include Academy Award nominated The Chronicles of Narnia: The Lion, The Witch and The Wardrobe, Academy Award-winning film Spider-Man 2, Academy Award and BAFTA nominated Spider-Man, and The ChubbChubbs, Imageworks' first animated short film and Oscar winner for Best Animated Short Film in 2002. Her other credits include Zathura, The Aviator, The Matrix Revolutions, The Matrix Reloaded, Bad Boys II and Stuart Little 2.

Fulle joined Imageworks in 1997 as svp of production and executive producer.