Another Summer of CG Water Effects

There are four major features released this summer with notable CG water effects: Disney's Pirates of the Caribbean: At World's End, PDI/DreamWorks' Shrek the Third, Sony Picture Animation's Surf's Up (opening June 8) and Universal's Evan Almighty (opening June 22). So, with this deluge upon us, we decided to revisit our CG water effects article that we did a year ago.

At that time, Industrial Light & Magic had just completed work on Poseidon and Pirates of the Caribbean: Dead Man's Chest using a fluid engine, which Ron Fedkiw, a professor of computer science at Stanford University, devised called PhysBAM, which stands for Physics Based Modeling. It is a core math engine that is part of a three-fold system along with ILM's Zeno interface, which is like an equivalent to Maya that their artists use, and a fluid engine that was created by the R&D group at ILM. So, I called on ILM to see if anything had changed now that they were concluding work on At World's End and Evan Almighty. I spoke with Joakim Arnesson, the CG supervisor on the Maelstrom. The Maelstrom, you ask, what's that? That's the water sequence in At World's End that is by far the franchise's most complex.

Arnesson, who also worked on Poseidon and Dead Man's Chest, says the main difference between the Maelstrom sequence and prior water effects created by ILM is that the Maelstrom required a total CG environment. "There's a battle going on between the two ships, and that battle is staged in a huge whirlpool. We couldn't shoot that or make it in a studio. We had to generate it. At first we tried an easier approach with geometry. We, of course, had animatics that Gore [Verbinsky] had already approved. So, we had something to follow in regards to shape and size and speed and how big this Maelstrom was going to be. We started out by forming geometry (in the shape of the Maelstrom) just to get a feel for it and then apply our shaders and detailers. But with the formed geometry, we quickly saw that (the waves) didn't behave or didn't look like the real stuff. Also, we wanted the center to spin faster. So, we tried it with the fluid engine we have."

But like the choppy sea they were creating, the work did not go smoothly. "The problem with this was it's such a large Maelstrom: 2,000 units across, to simulate that with a fluid engine with high enough detail to actually see some waves moving around was going to be our toughest thing."

replace_caption_peszko02_Pirates3-Pearl_Dut.jpg

Indeed, Arnesson says that they had to push their fluid engine much further this time than they did on Poseidon especially with one of the fluid dynamics engineers from Stanford, Frank Losasso-Petterson, on board. Originally, Losasso-Petterson came to ILM to work on Evan Almighty, the contemporary Noah comedy, which, no doubt, contained its own interactive water effects challenges that relied heavily on the same fluid engine system as At World's End. "He [Losasso-Peterson] came on in October or November to run the fluid sims and increase the detail and simming on higher and higher resolutions," adds Arnesson. A tight time frame also pressured the effects team at ILM. "The hard part with this additionally was that we didn't have a lot of time to do R&D. The idea was that we would leverage what we had done on Poseidon as much as possible, but we still had to come up with how to solve a stormy sea. For Poseidon, there was just a ship sinking. But to have a stormy sea with breaking waves, we had to do something that was pushing the technology further along. Also, I'd like to mention that just because you have a fluid engine doesn't mean that you get everything for free. You have a base resolution to play with and from there you have to add more detail even in the shader and rendering phase. The fluid engine couldn't even handle it because we upped the resolution so much that there's not enough memory in the world."

Basically, you can think of a fluid simulation as a 3D grid sent to the computer, and the computer then calculates its fluid dynamics. "Now, when that doesn't fit in one machine with memory," explains Arnesson, "then you have to start splitting that into smaller cubes, and all those small cubes are communicating with each other. And so in the end, with our fluid supervisor here, I think he ran on 40 processors in parallel."

To be sure, ILM has raised the bar with this latest development, "This is a completely 3D environment: computer-generated with fluid dynamics and secondarily to have particle sims on top of that. On the first one [Curse of the Black Pearl], it was more plate photography of the water. I worked a little on the second one [Deadman's Chest]. There were a few shots when the ships were breaching that we used a fluid engine on. But apart from that, this is on a much, much bigger scale. I think the Maelstrom sequence has about 370 shots. A lot of that is mostly action and live-action going on in the foreground, with the Maelstrom as a generic render and composite and then just plugging it in for maybe 80% of the shots. [There's also] custom rendered shots, where the camera movement is very extreme, often with live- action in the foreground and the CG maelstrom in the background. Then there are the hero shots, roughly 30 or so. That's where we actually see the ship, which is a computer-rendered, generated ship, interact with the fluid dynamics' water surface, and we see the water line."

With about 370 Maelstrom shots, At Worlds End is on a much bigger scale than the previous Pirates features.

Arnesson credits ILM's success to an outstanding team effort. "To create these effects, it requires a lot of people, of course, at all levels, everything from layout people to animators who animated the characters on the CG ships. Then, of course, we have effects tds that did some of the particle simulation, and we had expert compositors. The fluid engine, that just gives you mostly the water surface. From there you need to make splash, then spray, and then mist." So, according to Arnesson, from a wave that is just breaking, you start out with the data from the fluid engine simulation. Then, you run a secondary simulation to get the heavier splash, the white foam. Then, from that simulation, you run another simulation on top of those particles to get the spray and then still another to get the mist. "So, you go from large detail to smaller to smaller to smaller detail. It's very involved. I can't stress that enough," states Arnesson, who has worked on many other films that required water effects, including Perfect Storm and Master and Commander. "It never seems to be easier even though we have newer technology. It just that it looks better. We keep adding more detail and more detail. Even if the computers get faster, we add more detail which makes them go slower."

On Shrek the Third, the effects team at PDI/DreamWorks tackled the task of having to create an ocean by using modules from their previous work and adding new techniques to raise the bar. Matt Baer, head of effects on Shrek the Third, most recently served as the supervising effects lead on the hit jungle comedy Madagascar as well as on the animated mini-movie, The Madagascar Penguins in a Christmas Caper. He also worked as an effects developer on Shrek 2 and as an effects animator on the original Shrek. "Some of the advancements are directly related to our fx and lighting teams getting craftier with these water effects. Almost all of our films have some sort of water effect, which have given many of our team experiences with different techniques for both geometry creation and render treatments."

Using smaller modules is the key at the animation studio. "Other advances come from new and improved modules in our effects/lighting toolset. We avoid building a black box that solves everything and instead try to create smaller modules that can be plugged together to achieve a specific look. This makes it easier to share techniques from one show to another. In some cases we can pull different tools from previous shows. Often times we will use the tools with little to no tweaks -- and in other cases we will need to replace or create new modules to meet the artistic demands of the show."

On Shrek the Third, they used modules from previous films -- including particle clustering tools, geometry deformation, heightfield tools and texture mapping techniques -- and also some new innovations. "We developed new techniques for some of the foam and water interaction shots. We used fluid dynamics occasionally for hero splashes -- and the wake around the ship, where we did a 2D fluid sim, but for the most part the Shrek the Third ocean relied on older techniques such as displacement maps and particle simulations for splashes."

About 90% of the water effects on Shrek the Third were created using their proprietary tools. "The main ocean surface is a complex network of displacement maps. Many of the splash and foam effects begin in Maya, but are enhanced greatly before rendering through our suite of proprietary particle tools."

replace_caption_peszko05_MattBaer.jpg

Baer believes that one of the biggest challenges in raising the bar is communication. "We spend a lot of time on our water effects so there is a considerable amount of discussion into where we will get the most bang for our buck. Creating an ocean requires senior level artists that really understand all the tools they have at their disposal. They need to decide when to use existing tools and when to create something new."

As to the overall look of Shrek the Third, Baer adds, "We worked really hard to create an ocean that both fits into the Shrek world and gives a good sense of scale to the boat sequences. I think some of the ocean lighting is really beautiful. There are a few hero foam shots where we spent a lot of time -- we wanted to make impressive images without distracting the overall story."

One can imagine by the title Surf's Up that water plays a major role in the new animated feature about surfing penguins. While the whole believability of the water is, of course, based on the collaboration of a team of animators at Sony, John Clark, the wave animation lead, is basically the one who gave the water's surface its overall shape. "I used a character animation rig that was designed to reshape a nurbs plane into the shape of a moving wave, and that nurbs plane would then displace a surface in Houdini, which would then have the water displacement and the water effects applied to it. It was my job to take all of my knowledge and information and research and create this wave shape that would break like a realistic wave in the ocean."

And Clark's knowledge of breaking waves is vast, indeed. He began surfing at age four in Florida and has been doing it most of his life. In college he was in the National Scholastic Surfing Assoc.'s surf contests. He has also judged contests and has always been involved in the surf culture in one form or another. This includes shooting surfing home movies. So, it is no wonder that he was quite involved in designing all the surfing in the movie as well as the breaking waves. As he puts it, "Surfing has been one of my callings along with art and animation."

The idea was to import the wave rig into a shot and then set up the general location of where that wave would break in relation to the island. But before that could be done, Clark needed to find references. "So, I spent a great deal of time before we even got into shots researching waves, studying lots of references, doing a lot of surfing." Having a passion for surfing, Clark was familiar with a lot of great surfing cinematography. "We were actually able to take their techniques and emulate them with our water. So, you'll see that a lot of shots ramp up into high speed. They go from real time to high speed and back into real time again. And you're going to see that water slow down all of a sudden. These are references drawn from the best surfing cinematographers out there."

Because of all the real-life surfing references, the film is shot in a way that is unique for animation. "It's more of a reality show/documentary style. Our department, which is headed by James Williams, had a motion capture camera that we used for about 90% of the movie. We have this really great hand-held feel to the movie because of that. So, you really get the feeling that there's someone on a jet ski or on another surfboard filming the surfers. It grounds you in the reality that this is like a surfing video. Everybody who's seen it is just blown away by how good it feels to be in the water and to feel like you're really surfing."

replace_caption_peszko08_SurfsUp-chl-ShotBu.gif

After doing all the research, Clark took the wave rig and began creating generic wave animation that he could then use as a starting point on each shot. "In order to create this generic animation, I would use this wave rig, scale it up to whatever type of wave we were using. Our goal was to use the same wave rig for all the different types of waves in the movie."

In the movie, there are three different types of waves. "The basic one we call the Pipeline wave, which we tried to make look as much like the waves that break at the Pipeline in Hawaii. The other wave type is the Mavericks wave, and there's a surf spot in Northern California with big waves that I call Mavericks. The Mavericks wave was going to be used on the competition beach. It's supposed to be gigantic and intimidating. So, I modeled the Mavericks off of the actual wave that breaks at Mavericks. Then the last type is called the Spilling Breaker. It's a term we came up with to best describe a slow-moving, mushy wave that doesn't have a tubing curl to it. It's all foam and white water. There's no open tube on it. It's more like a long-boarding kind of wave like a First Point, Malibu wave."

The three different wave types are based on the story, itself. "Different sides of the island and different places where Cody surfs have different kinds of waves," Clark explains, "For example, Pen Gu Island was going to be all small Spilling Breaker waves. It's where Cody learns to surf. Z's Beach, named after the deceased surfing champion Big Z, is discovered later in the movie. That's all the Pipeline style of wave. Then the other one is Competition Beach. That's on the side of the island that catches the biggest waves, and that's where we use the Mavericks wave."

Clark credits the genius of the rigging team for being able to use the same rig for all three types of waves. "The way the rig works is imagine a very large nurbs plane with a set of concentric rings through the center of it. Let's say I rotate the ring in the center. Then the a few of spans of the isoparms of that nurbs plane will raise up in the shape of a wave then roll over and crash and die out. So, the ring basically is driving a series of blend shape curves through a cycle. If I take one ring and animate it and then take another ring and animate it, but I offset the time a little bit, and then the ring beside it, I animate in the same way and offset its time a little bit, I can create a wave that rises up and breaks down the line and then dies out. I was able to use the wave rig to create the three different styles of waves and about 25 generic starting points for each wave. So, it's basically a rig that drove blend shapes, and those blend shapes were creating the shape of the wave." Clark was able to switch between different styles of blend shapes for the three types of waves -- Mavericks, Pipeline, and Spilling Breaker -- on the fly. The difference in scale ranges from the Mavericks wave that was the height of a two-story building down to the tiny Spilling Breaker waves that were only knee-high.

Wave animation lead John Clark was the perfect choice for Surfs Up as he began surfing at age four and is involved in the surfing culture. On the right is a render of a typical pipeline wave with all the elements composited.

And does Clark believe that Surf's Up raises the bar on CG water effects? "I don't think anyone has ever seen anything like this. We're able to create a realistic breaking wave that's surfable. So, not only does it look realistic from the beach, but put the camera in the middle of it on the tip of a surfboard and ride through the tube then whip that camera around and look behind the surfer and see the white water spitting out of center of the vortex of the wave. And the whole time it feels real; it feels like you're really there. And I know because I've been there many times. You can check it against any CG water that's ever been created and it's more thrilling, more realistic and more enjoyable. At the same time it's stylized, and the waves also play a character role in the movie. The waves are really friendly or they're very intimidating or they're stormy and wild. They're huge and towering or they're fun and fantastic. So, throughout the movie you're going to see water playing an actual story role... You're really going to fall in love with the waves in the movie, and that's a huge accomplishment."

Clark was also excited by the fact that they combined water effects with surfing and had the surfers do their tricks while getting tubed. "You're seeing the sunlight reflected through the roof of the wave and you can see the transparency of the bottom of the ocean. It's real eye candy."

J. Paul Peszko is a freelance writer and screenwriter living in Los Angeles. He writes various features and reviews as well as short fiction. He has a feature comedy in development and has just completed his second novel. When he isn't writing, he teaches communications courses.