Summer Animation: Attaining Hyper-Reality

The big breakthrough in Shrek the Third is global illumination. With GI, every detail in the architecture connects things together and provides richness. All Shrek the Third images © & DreamWorks Animation Llc.

Sonny and Cher once sang that "the cars keep on going faster all the time," but were their duo still around today they might be talking about advances in CG software. The computer animation studios are playing "can you top this" with proprietary stand-alone and plug-in programs that keep narrowing the gap between pixels and real life. Even with (or because of) ever more powerful computers, the need for software that can cut corners and ride herd on huge amounts of data is greater than ever before.

None of the summer's big three animated features is attempting to create a hyper-real world for itself; the goal of their technological window-pushing is to make their imaginary internal worlds as real as possible on their own terms.

DreamWorks Animation: Shrek the Third

"We try to make each film look better than the last," says Philippe Gluckman, visual effects supervisor on DreamWorks' Shrek the Third. "Even though the Shrek world isn't hyper-realistic, there's a certain degree of realism."

For Gluckman, the big breakthrough in Shrek's latest adventure is in global illumination. In real life, light bounces around from one object to another. Photographers use this to their advantage and put bounce cards around the model to illuminate it.

"The computer lighting models didn't know how to do this -- it's very complex behavior. We began prototyping an affordable solution on Shrek 2 so we could get the nice behavior without spending 24 hours on every frame. There have been global illumination algorithms in the past, which looked great but were completely unaffordable. It's very complex to shoot all these rays of light and calculate how they bounce around. The initial attempts were completely impractical, so we developed our own solution and used it with great success -- we all fell in love with the results. It's possible to do it by hand, but it takes a long time for lighter to emulate all the subtleties that you get in that kind of behavior; they'd have to put in 50 lights to get the same complexity."

Gluckman points to a scene in Shrek the Third where Fiona and her princess pals are running through the castle. "One shot I remember is when they run under those archways. The whole environment is basically lit with global illumination. Architecture is difficult in the sense there's a lot of flat panes and they'll have a tendency to go a little dead if there isn't enough detail in there. With GI, every detail in the architecture connects things together: the objects illuminate one another and provide a lot of richness.

"Another place that comes to mind is towards the end of the movie when Shrek is in chains and fights the prince. That was lit way more simply than we used to, just using the one light and getting all the bounces out of it. In that particular case, it gave us very dramatic lighting: it definitely supported the mood we were trying to establish. What's interesting about it is even though it's a physical behavior and you get something pretty realistic, we still have tools on top of that for all the usual art directing we like to do. We often make decisions on purpose to get away from reality. In GI, when a character is next to a red wall, you're going to get a lot of red light bouncing onto that character; you still want the lighter to be able to manipulate the image in case the red concentration is too high."

In fact, the system-wide use of GI also enabled artists to animate overcast skies for the very first time at DreamWorks.

Realistic hair -- once the holy grail of CGI -- looks even more realistic in Shrek the Third, thanks to "Rigid Rod," DreamWorks' in-house strand simulator. "We now have a number of long flowing hairstyles capable of all kinds of fancy motion. In the past we had to be more selective where we used those things."

Once upon a time detailed hair called for extensive rendering time or any number of extra artists to achieve the desired look. "The way hair moves is very complex. We had a few great hair shots in Shrek 2, like when the prince did his slo-mo moment. It looks great but it's embarrassing how long it took -- literally months. It was a conscious decision to go all-out on that one shot, but we wanted to do stuff like this all over the new movie and not have it be such a problem.

"We developed our own tools to make this more automatic. When you look at Rapunzel's braid or Merlin's long beard, Rigid Rod figures out where the hair is supposed to be based on whatever the character is doing. Of course there's still hand-tweaking, but now we have an out-of-the-box solution that works."

The other advance Gluckman points to is in the characters' clothing: "In the first Shrek, we were only able to create free-flowing cloth on a single design, like Princess Fiona's skirt, while the rest of her wardrobe was skin-tight. Now we're able to have flowing cloth on all parts of the characters, so cuffs, sleeves, jackets and long dresses can drape and behave realistically. Not only does this give us a wide variety of silhouettes for our crowd scenes, it also helps us better define characters like the princesses. There are a lot of costume changes in the movie, some of which -- like when Shrek and Fiona are dressed in their `royal outfits' -- include several layers of clothing that are animated together."

Sony Pictures Animation: Surf's Up

With a title like Surf's Up, Sony Picture Animation's second feature (opening June 8) is awash in H2O. Wrangling all that moisture fell to that film's visual effects supervisor Rob Bredow. "We were really driven by the desire to give the audience the experience you were surfing on these waves, or you were the cameraman, in the water, shooting it yourself."

Bredow and company started by watching "every surfing documentary ever made," including John-Paul Beeghly's Step into Liquid. Beeghly himself advised the production to the point of taking Bredow out to a spot in the ocean where 50-foot waves form and crash thanks to an enormous underwater mountain.

Bredow's goal wasn't absolute photoreal realism either, but realism in service to the movie's world. "We wanted to stylize certain things -- the foam on the water is more stylized than photoreal, a little more designed. All our colors are pushed slightly; we wanted a contrast between the water in `Shiverpool,' which is basically Antarctica where Cody starts out, and Tahiti or as we call it, `Pen Gu Island.' There's already a pretty strong contrast between the two places, but we pushed the design and the colors on it."

Bredow had a huge assortment of tools to work with in the course of the film's three-year production schedule and used them to create some 20 minutes of surfing footage. "We wrote a bunch of custom tools to control the white water explosions when wave crashes down on itself. Those explosions are made of literally billions of particles that all had to be simulated and we had to figure out a way to render them.

"We used a combination of different tools, but at its core was a point instancer system that plugged into RenderMan to generate those millions and millions of points just as the renderer needed them. The big trick in those systems is not running out of memory. We `sliced' the whitewater into layers and `diced' it into areas of the screen so we wouldn't have to make more than a few million points at a time even though there might be a billion in the final frame. And that was just for the whitewater explosions."

replace_caption_hyper05_SurfsUp-PK21-sol12_.jpg

Controlling wave motion was an entirely different challenge. Previous methods were based on simulating changes in wind speed or the slope of the ocean's floor, and Bredow admits to using some simulation tools; one in particular measured the wave's falling speed "so it moves in natural way and your mind doesn't wonder whether it's syrup or water."

Overall, though, Bredow decided on a more-hands on approach. "The wave is actually a character in our movie, and just like our other characters they're rigged with controls that we used to direct their evolution. We have some demos where you can actually see the control rings around the waves. As you grab the ring and rotate it, the wave actually evolves and crashes over. It's really fun and it's completely interactive. The system was built with a lot of custom software inside Maya and gave us ability to choreograph our waves according to the needs of each shot. It's based on work we did on Cast Away for a stormy water sequence during the plane crash. We used a similar wave control system, but this film took it to a higher level. Just for fun we took one of our Surf's Up wave shots and rendered it photoreal just to show that we can; the different was mostly in color choices, textures and the foam, and the disparity wasn't very high."

For one of Bredow's staff, in particular, working on Surf's Up was a dream come true. "John Clark went from working in our layout department to wave animation lead and did 90% of the movie's waves. He'd been surfing all his life and knew how waves felt and looked from all different angles and was a fan of the genre. He was able to nail the right look."

Water wasn't Bredow's only challenge. "When you do waves there's usually a beach, and every time the penguins are walking on the beach the sand has to interact with them.

"Dan Kramer, one of our CG supervisors wrote a `height field' simulation called Sandbox. When a character steps down, the displaced sand is pushed to the edge to create the right footprint look. After the foot leaves, the sand fills back in up to a point, particle by particle. We had controls for how damp or sticky the sand is, or the angle of how it fills back in. We have a lot of experience in doing this kind of footprint system efficiently, like in The Polar Express where we had characters walking in the snow.

Controlling wave motion was an entirely different challenge in Surf's Up. Some simulation tools were used and one in particular measured the wave's falling speed so it moves naturally.

"One of the reasons there's a lot of detail in Surf's Up is that you needed a lot of detail in the water to make it feel like you were really riding those waves, which drove the amount of detail everywhere. It meant the sand had to have a certain amount of detail and the jungle had to have detail down to the leaves. We didn't want the jungle to feel like a different environment, so all the leaves move. That was another simulation system: we had dials for how much wind we wanted to blow through them. We used a variation of the crowd system we developed for The Polar Express and for multiple movies since then. You can't have all these trees on the beach and have them not blowing in the wind, it just doesn't look right.

"The level of detail that goes into these films to make them feel like they have high production values is higher than it was 10 years ago -- the bar is continually raised. You have to do it in a way that isn't distracting. If people are watching footsteps on sand instead of feeling for the characters, you haven't done your job right. The real key is to make sure all the effects are playing a supporting role."

Surf's Up's premise is that the film is a "documentary" produced by penguins, for penguins, about up-and-coming surfing penguin Cody Maverick. Bredow employed a variety of tricks "to continually remind the audience this was being photographed by a variety of crews. We simulated the look of various film stocks -- 200, 400 ASA speed for different times of the day when our `documentary crew' would be shooting. SPEN -- the Sports Penguin Entertainment Network -- shot on HD video, which had its own look.

A "height field" simulation called Sandbox was written so that when a character steps down, the displaced sand is pushed to the edge to create the right footprint look.

"We had a great time. We have `footage' from the 1920s, 1950s and `70s all through this movie. We had a lot of fun digging up old footage and matching our work as precisely as possible, down to the scratches and dirt. We tried a lot of things to get that aged look. We dragged the film around the parking lot behind our cars, scratched it in the projector on purpose and then scanned the footage into the computer. Ironically, all that stuff ended up looking still too clean. We ended up finding old stock footage and extracted its scratches to use in different shots."

At one point, Bredow emulated a classic Tex Avery gag: "Because penguins made this movie, we used penguin feathers instead of hair getting caught in the gate. If you watch the movie in slo-mo or pause on certain frames you might appreciate that." That Avery gag had a hair bouncing in the projector's gate until one of the cartoon's characters plucked it free. It supposedly drove projectionists around the country to distraction and forced MGM to place a warning sticker on its film can. A similar moment happened during an early Surf's Up's screening:

"A number of shots came from a video camera on Cody's surfboard in a waterproof housing. The camera got pretty beat up out there, that's our story behind it, so there are drop-outs on the `tape.' We simulated the video dropout look, it's a subtle effect. We showed it at a big, big venue on a 120-foot screen with all the latest Sony projectors and 15 projection experts making sure the equipment is running properly. When that shot came up, all 15 of them stood up and started running around trying to figure out what's going on. `Uh guys,' we had to tell them, `it's in the footage'

One of the challenges on Ratatouille was animating fur for the crowds of furry rats. PRMan was used to get the rendering time down to two hours. All Ratatouile images © Disney/Pixar.

"We came up with other artifacts too. The underwater cameras were usually old 16mm cameras, and when they get towards the end of the reel there's usually a light leak that comes in from the right side of the picture when the cameraman changes reels on the beach. We simulated that, did a lot of those fun kinds of looks. The audience isn't going to know all this, but they're going to feel it. A number of people who watched the film wanted to know if any of it was actually photographed, so it has that authentic feel in there. If penguins could surf and make movies about themselves, this is what it would look like."

Pixar Animation Studios: Ratatouille

Michael Fong is a 12-year Pixar vet and supervising td on Ratatouille (which opens June 29 through Disney). One of his challenges was animating fur -- like water, originally one of the most difficult textures for CGI to capture. That was a long time ago; nowadays, detailed fur is a standard part of CG animation. (Monsters, Inc.'s Sulley Sullivan comes to mind): but there's an awful, awful lot of it in Ratatouille.

"We had crowds of furry rats and it was important to us to render all their fur. We were pushing our crowds team to render thousands of them at the same cost as a normal character. If it takes too long to render, our lighters can't make the picture as beautiful as they want it to be. If it takes 40 hours at the very beginning to render one frame, you're not really getting any feedback on what you're doing -- you're kind of lighting blind.

"They really whacked at the problem, and finally used PRMan to get the rendering time down to two hours. The solution was to use less fur on the rats as they got farther away from the camera, but make each strand thicker. The foreground rats might have a few million hairs, but the far away rats only a few hundred. Those background rats would look stupid in close-up, but in the distance you can't tell the difference.

Pixar came up with "Scatter," an improved subsurface scattering system that gives the artist control over where light energy travels once it penetrates a surface. This gives characters' skin a nice rosy glow.

"The RenderMan group just delivered release number 13 of PRMan and we're the first movie to use it inside Pixar. This is their new multi-threaded magical super fast renderer with built in ray tracing and it's a big change from our previous versions."

Pixar also optimized rendering time by creating "brickmaps," simplified 3D representations of the environment of each shot. Ray tracing is done from this stand-in instead of the shot's "real" environment, using an in-house technology called Trace Radiance. "It's significantly faster," says Fong, "because it's not real geometry. The wine glasses in your final image are refracting the image behind it and reflecting the image around it from the brickmap."

Pixar also came up with "Scatter," an improved subsurface scattering system that gives the artist control over where light energy travels once it penetrates a surface and how its color changes in the process. "We previously used subsurface scattering for skin to give it a nice rosy glow. We did this on The Incredibles and Sharon Callahan, our director of photography for lighting, liked it but had reservations.

"She thought subsurface scattering as we previously implemented it -- in a straight, out of the book way -- gave the characters a waxy, not quite human look. She asked for a new system: subsurface scattering with artistic control that ignores the physics completely. We use subsurface scattering to describe where the light energy is traveling, then we allow her to control what that light energy is doing color-wise."

Fong describes the film's visual style as "realism with a painterly look. This is a handmade world." In fact, director Brad Bird says their fantasy version of Paris "is a lush-look[that] has a very different visual identity from anything else that has come out of the studio." And, in particular, they used subsurface scattering to help convey food in a very evocative way. "We've used it with food here, particularly fruit. There's a certain amount of light that penetrates the surface of the food and hits things under the food that then glow back through the food. We had consultants who were gourmet cooks give an overview of not only how food looks but also how things are set up in the kitchen. It's not remotely real but gives the fantasy a footing. There was a lot of effort put into even how food looks when you're preparing it. What causes sauce to curl around when people are stirring it? A lot of this is CG math and bending ones and zeros to emulate something that's organic."

It's a description that could apply to all three films: while Shrek the Third takes place in a fairy tale land called Far Far Away, and Surf's Up in a penguin-populated universe, their imaginary, real-looking worlds were indeed constructed by artists' hands -- using 21st century virtual tools, of course.

Joe Strike is a regular contributor to AWN. His animation articles also appear in the NY Daily News and the New York Press.

Joe Strike has written about animation for numerous publications. He is the author of Furry Nation: The True Story of America's Most Misunderstood Subculture.