Diving Into the Digital Deep End With 'Shark Tale’s' Rafferty

Party time in Oscars penthouse apartment. All images courtesy of DreamWorks.

In the opening shot of Shark Tale, the artists got a chance to play with the companys famed logo. As CG supervisor Kevin Rafferty observes, The little DreamWorks boy is fishing, remember?

In what Rafferty calls a comedic entrance into the film, the camera dives beneath the surface into an underwater city where mobster sharks keep a tight fin on the fish population. Explaining the reveal of this world, Rafferty notes, We have a couple of our `star sharks swimming over what looks to be open seas above a kelp floor. But as the camera cranes down, we realize that were not in open water the sharks are swimming above the rooftops of a brownstone neighborhood. We crane down further, and the kelp becomes coral and the coral becomes building columns and then you start to see a fire escape a fire escape under water! That opening is exemplary of our wanting viewers to do a double-take, and realize theyre seeing a `brownstone made of coral!

Shark Tale was the first foray into an all-CG feature for both Rafferty (above) and DreamWorks Glendale animation team.

Animating an all-CG feature is a first for Rafferty as well as for DreamWorks Glendale team. Which is saying something, given Raffertys extensive career on the leading edge of CG effects filmmaking. His credits include The Last Starfighter, Digital Prods. pioneering effort that brought CG models into live-action filmmaking. Then at PDIs L.A. branch, Rafferty worked on Batman Returns and Cliffhanger before moving up to ILM. There his many credits include the digitally diverse movies Casper, Dragonheart, The Mummy Returns, The Perfect Storm and Star Wars: Episode One The Phantom Menace.

Building the Beast

Before he could tackle the lead CG supervisor role on Shark Tale, Rafferty had to help build the CG team. I came to DreamWorks at the end of 2001, he recalls. My last day at ILM was a Friday and my first day at DreamWorks was the following Monday. We had what we called a brain trust all of the leadership for Shark Tale had been drawn from ILM, PDI, Centropolis, Sony Imageworks, Digital Domain, Tippett Studio, Pixar and Blue Sky. Having leadership from several different studios was great because everybody had their own subset of experiences, and varied ideas about how to mold a new pipeline. My first couple of months were spent sussing things out on a whiteboard.

In the back of their minds was DreamWorks ultimate goal of uniting this new CG division with the companys Shrek 2 while wed be training our hirees. Since Rafferty had worked previously with PDIs proprietary software, he knew it was inherently more complex than an off-the-shelf system. We were ready to try to embrace PDIs pipeline, but we couldnt impose on the PDI folks to entertain our questions while we tried to get everything done on the schedule that Shark Tale had.

The fish work themselves into a lather at the carwash.

So we ran the numbers again and again and realized that the only way we could go was with a hybrid pipeline, Rafferty recalls. The L.A. talent base that would make up the bulk of the crew was likely to be familiar with Alias Maya software, Pixars RenderMan and to some degree mental images mental ray so those three components were chosen as the core of the Shark Tale toolkit. They were also the foundation on which in-house tools would be built.

All Hands On Deck

It took a year and a half to get through the design, hiring and training phases, a process that Rafferty refers to as a melting pot. We wanted to help re-train anybody already employed at DreamWorks who had the mettle to undergo training in 3D. We knew that DreamWorks 2D character animators had been working all their lives to hone a skill thats really unmatched, and we wanted to take that know-how and apply it to 3D animation. We tried as much as we could with some success to transition 2D animators into the 3D world. Some of the animation tools that we wrote here even emulated 2D animation tools to help them transition to 3D.

Another area where we had a big transition was in CG effects 2D effects artists transitioned into 3D effects quite well. Some of DreamWorks 2D artists were already using Maya to create effects. We combined all of these efforts with a massive interview schedule of people from the outside world as well.

As we were looking, Squares film production unit shut down, so we hopped on a plane to Honolulu and got some great folks there for lighting and surfacing. It was an unfortunate demise for the industry itself, but we were fortunate enough to be hiring when The Secret Lab also was shuttered. Since both of those former studios had people who were trained in Maya and RenderMan software, notes Rafferty, We lucked out.

Taking the Plunge

The look they set out to achieve for Shark Tale was highly stylized. And the fish stars of the film voiced by Will Smith, Robert De Niro, Renee Zellweger, Jack Black, Angelina Jolie and Martin Scorsese played things for irreverent laughs as well. Given this tone, suggests Rafferty, We took the anthropomorphism of the fish to another level. They walk as well as swim. And they built a city with streets and streetlights which of course fish dont need. The one thing that we did not want to do was to hit people over the head with the fact that were underwater in every single shot. There were establishing shots that had underwater caustics, flowing kelp and particulate matter suspended in the water, with `God-rays coming through. We used the bells and whistles that tell you youre underwater where it counted in wide shots. But usually in the close ups we just used particulate matter to hint that were underwater.

All of which makes the Shark Tale milieu distinctly different from Pixars CG blockbuster, Finding Nemo. Rafferty observes, We have an underwater city and Times Square, built by our fish population. We have some icons of downtown in Times Square, and your eye immediately recognizes, `Oh, its Times Square. But wait thats not the Virgin Building, its The Urchin Building. Although our characters dont wear clothes, you can see what they do. We have taxi fish out in the middle of Times Square theyre yellow angelfish with black checks on them. Our mail fish are colored much like mail trucks, with the color integrated into the scales of their skin.

We got more generic with our crowds. We basically used just a handful of different pieces of geometry, but they all had five or six texture styles that we could randomly access. That gave us the variety that we needed. We could specify a school of fish looking a certain way, and have both variety and sameness at the same time.

replace_caption_wolff07_Shrek2-S2MS002.jpg

Modular Modeling

Building the fish city itself was a gigantic undertaking, admits Rafferty. The complex geometry of the city set presented huge memory issues. There were blocks and blocks where the rooftop of each building is alive with blowing kelp and moving coral. We had a lot of things to get through the pipeline in an economic way. It was enough to bring any healthy computer to its knees!

DreamWorks has a strategic alliance with Hewlett-Packard, so they were able to tackle this challenge by harnessing an army of HP boxes running Linux. On the software side, explains Rafferty, We used Maya Paint Effects to get a lot of those coral and kelp effects working. We knew that as light as we could make the geometry using Paint Effects, we would have render those things in layers in order for them to be actually rendered, and look believable.

To create the buildings themselves, the DreamWorks technical team developed Toolbox Town, which offered the capability of selecting assets from an online menu. For example, says Rafferty, You could pick a building shell, or a doorway stoop, a window style or a cornice it was basically a digital backlot.

Cooking Up Scampi

We had a development team from production technology that wrote what we called Maya Assets. It gave us a way to keep track of all the assets within Maya. The team also took advantage of Maya instancing, and developed in-house software that Rafferty says took to the next level the economy of Maya instancing.

He offers an example: For any particular building, wed have one shell that had 50 windows, but we would actually be able to use one window to read into memory and instance it 49 more times. Once we put together a building shell roof, windows, doors with Toolbox pieces, they would have all of their inherent texture styles that we had picked. But we could also overlay a projection map over the whole thing to make it look more like one piece. This helped us out immensely.

A key capability developed during the production of Shark Tale was the instancing tool built by DreamWorks animation technology group. Called SCAMPI for SCatter A Million Parts Interactively this scattering tool allowed Raffertys team to put together the complex cityscape fairly quickly. Each of our `instance-able assets had three or four different texture styles to them. So we could make it look slightly different or even very different but it would actually be the same geometry.

Early sketch of the dancing carwash fish.

Swimming With the Fishes

In animating the main characters in Shark Tale, Raffertys team took their cues from lipstick-cam footage taken of the voice actors reading their lines. You `get who each character is right away. Its not only how they look, but their mannerisms. The challenge of capturing Smiths easy charm, De Niros famed gruffness and Scorseses hyperactivity were things that Rafferty asserts, We got tenfold.

A key challenge faced by Rafferty was where to apply subsurface scattering to this cast of slippery sea life. Our main characters had the whole nine yards. They had multi-leveled surfacing, sub-surface scattering all the bells and whistles. Some of our ancillary characters even had that too.

But for the secondary crowds, whose movements were developed using Softimages Behavior software, the subtleties of subsurface scattering werent feasible. We had to almost traditionally light them with a few more spotlights, reports Rafferty. Since these characters tended to be in the background, he jokes, Depth of field was our friend.

Illuminating the Deep

One of the many things that we wanted to do to fulfill the visions of the production designer (Daniel St. Petrie) and the art directors (Samuel Michlap and Seth Engstrom) was to use the tools that live-action effects movies use to achieve photorealism. To create this underwater fantasy yet believable world, we used global illumination to create `painterly photorealism.

This was achieved through a combination of mental ray and version 11 of RenderMan software. The latter capability was not available when DreamWorks had begun building its pipeline, notes Rafferty, Which is why we still did want to rely on mental ray. Once RenderMan 11 came along, we used it when we could, but in the back of our minds we knew that RenderMan 11 only had `Version 1 of its ray tracer.

However, Rafferty did use RenderMan for the jellyfish characters Ernie and Bernie. We didnt use it all the time, but for close ups it did pay off. Most of this show I would say over 90% of the time we used a hybrid of RenderMan and mental ray.

Enter LUIGI

Despite using RenderMan, the Shark Tale team did not use Pixars Maya-to-RenderMan interface MTOR. DreamWorks engineers Yun-Chen Sung and Motohisa Adachi instead developed an in-house tool called LUIGI. LUIGI stands for Lighting User Interface for Global Illumination, explains Rafferty. Everybody knows that the first thing you have to find out about a tool is its name. Since this a film spoofing the Mafia, LUIGI seemed right.

We didnt have a lot of time, so rather than create a whole new set of tools with its own viewport and its own curve editor, we created something that would hang alongside of Maya Live. Basically, we used the Maya viewport and Maya curves and talked back and forth between LUIGI and Maya. When we wanted to see a render, wed go over to LUIGI and say `render.

The Alias folks were so helpful with this. We actually had a team of custom engineers from Alias that met with us once a week for as long as we wanted to see what we needed. I cant praise their efforts enough.

Live Action Sensibilities

Given Raffertys two decades of experience with CG visual effects, he naturally brought live-action sensibilities to bear on Shark Tale. The difference that I embraced was that we didnt have plates we created the environments, and that allowed us to integrate the characters that much more. That was the big challenge that drew me to this project. Instead of DPs and stages and locations, we had layout. I wanted to bring cinematography into layout.

That was the one area where Rafferty, who is a member of the cinematographers guild, knew he actually could help out the most. While there is no official DP credit on Shark Tale (unlike Pixars films), Rafferty suggests, I spoke quite a bit with the head of layout about the different looks you can get with various crane and dolly moves. Weve implemented ideas from camera match-moving in live-action filmmaking things like a little camera bobble and little bit of delay. All those things put the human element into CG camera movement. Its a virtual camera, but its still a camera and should be used as such.

We also wanted to bring kind of a live-action sensibility to the art of compositing in a CG feature. We wanted to use compositing to give the characters that little edge wrap that really sets them into a shot. Its like saying, `What would this shot look like if it had a Pro-Mist filter on it? Weve used compositing tools to do that, adds Rafferty, who notes that DreamWorks used Apples Shake software to composite Shark Tale. Especially in the areas of camerawork and compositing, he observes, we can instill live-action sensibilities into a 3D feature world.

On the Horizon

Like DreamWorks, Lucasfilm and Sony are also venturing into all-CG feature films, and Rafferty thinks well see more digital effects artists bringing their live-action skills bear on the genre. I can only hope that will influence it.

As for the team that worked for 18 months making Shark Tale about 72 animators, 60 lighters, 10 compositors and 10 effects artists many have already begun training on the PDI system. Over the next three or four films, we will become a unified studio as far as the pipeline is concerned. Well have the same team, but different tools.

What role LUIGI and SCAMPI may play in the future is uncertain. Rafferty suggests, The theory and practice of the tools will be useful. Im not sure how much further the tools themselves are going to go. Maya will be a part of the PDI pipeline, as it has been for a while for modeling and effects. But it will take a lesser role in the lighting arena.

As for Rafferty, the all-CG production taught him that Im very schizophrenic Id love to do a CG feature, then two live-action features, then another CG feature to keep the edge honed on both sides. Now that would be a perfect world.

Ellen Wolff is a Southern California-based writer whose articles have appeared in publications such as Daily Variety, Millimeter, Animation Magazine, Video Systems and the Website CreativePlanet.com. Her areas of special interest are computer animation and digital visual effects.