'Order of the Phoenix': Escalating 'Potter' VFX -- Part 2

Read part one of VFXWorld's in-depth coverage of one of the summer's top blockbusters.

Although Double Negative did the bulk of the work on Harry Potter and the Order of the Phoenix (which opened July 7 from Warner Bros. Pictures), overall visual effects supervisor Tim Burke spread more than 500 shots among seven different vendors. Leading the pack was The Moving Picture Co., a key contributor to all the Harry Potter movies. Greg Butler supervised the effort, with vfx producer Jay Heapy, and supervisors Charley Henley, Clwyd Edwards and Ferran Domenech. The 230-shot assignment included a series of nose replacement on actor Ralph Fiennes, who plays the evil sorcerer Voldemort, an eerie effect that MPC had pioneered on the previous film. "We were able to re-use the original assets, including the facial rigging set-up and the textures, with some updates," Butler notes. "We used our new sub-surface scatter model that allowed us to obtain more realistic results much faster. On set, Ralph had tracking markers on the face that we used to track on the CG nose. The digital appliance went from his eyes down to the upper lip and was about three inches wide. We always tried to keep as much of the real skin as possible."

Further to the nose-less effect, MPC was asked to tackle the final confrontation between Voldemort and Dumbledore. When the evil sorcerer cast a fire snake, Dumbledore retaliates with a water bubble. Both manifestations made large use of Flowline, a fluid simulation engine developed by Munich-based Scanline. "We used Flowline on Poseidon, and actually, the fire simulation in Order of the Phoenix was very similar to the oil fire effect that we had done for that film," Butler observes. "We had great support from the Scanline team who worked a lot to adapt the software to our needs. For the fire creature, we started by doing keyframe animation on a snake geometry using Maya. Once we got approval for the performance, we took it into Flowline to generate the fire. With this in mind, we had included lots of cracks in the surface in order to justify the emission of hot gases that would all ignite. Since the fuel was constantly emitting out of the body, the fire reacted organically to any movement that the creature did."

On Dumbledore's side, the giant water globe was created as a displaced sphere rendered with many Flowline fluid and particle simulations. For the shot of the bubble bursting, MPC used a filmed element of a big dump tank on set. On all the shots, the company employed RenderMan for rendering, and Shake for compositing.

Parallel to their "creature" work, MPC tackled major set extensions for the Ministry of Magic, a huge location in which the final battle takes place. The geometry was modeled based on blueprints, with finer details provided by a LIDAR scan. Then, MPC did an extensive photo survey of the set, using multiple exposures and a flat lighting. These images were then employed as color maps for projection onto the geometries. Then, a shader applied all the highlights and reflections. "The set ended up being so huge that we had great difficulties to render it," Butler notes. "There was so much geometry and textures, so many lights, so much ray-tracing, that it took us months to figure out an efficient way to render it all procedurally... " Later on, in-house rigid body dynamics system PAPI was used to simulate hundreds of windows explosions on the digital set. In another sequence, MPC did a complex set extension on the seemingly endless marble staircase -- a CG environment except for the area surrounding the actors.

For MPC, one of the most enjoyable aspects of the project turned out to be the fireworks effects. In a comedic sequence, Ron's twin brothers Fred and George launch a series of magical fireworks at their new Headmistress. "The fireworks were created with Maya's particle system and RenderMan's firework shader," Butler says. "We also used lots of proprietary tools, including a shader that allowed each particle to act as an individual light source. For the 'fireworks dragon', we animated a 3D geometry that was used to emit and launch firework particles. This sequence really was a lot of fun to work on, especially the part when we had to go and shoot real fireworks for reference!"

Creepy Creatures

Having created, with great success, the bull dragon in the previous Harry Potter movie, Industrial Light & Magic was a logical choice to handle the Thestrals, scary but benevolent creatures that look like a winged reptilian horse. At ILM, the 100-shot assignment was overseen by vfx supervisor Tim Alexander and vfx producer Stephanie Hornisch, with CG supervisor John Walker, animation supervisor Steve Rawlins, digital production supervisor Robert Weaver and compositing supervisor Sean McKenzie. "The Thestral was an interesting creature to work on, due to its unique design," Alexander says. "We first scanned a full size maquette that production had sent to us, and used the point cloud to model the geometry. We actually had to build two models, which was pretty unusual. Since the creature is basically skin and bones, it was very important for the skin to slide against the bones. So, using Maya, we built a standard model that represented the outer surfaces, and a high-resolution bone structure for the surfaces underneath. We utilized ZBrush to add a lot of details on that geometry."

The Thestrals were brought to life using keyframe animation for the body, and a simulation for the skin. To this purpose, character rigging supervisor Eric Wong developed within Zeno a methodology that connected the skin layer to the bone structure. The system attracted the skin to the surface, but was loose enough to allow it to slide over the structure. It proved tricky for concave spaces where the skin had to be suctioned in while still sliding over it. "Sometimes, delicate movements in the keyframe animation would be lost through the simulation," Alexander says. "Depending on the shot, we would then either increase the animation, so that it read more through the simulation, or dial the simulation out of the area."

For the aerial shots, ILM started with the background plates, and animated the CG Thestrals flight. Once approved, the data was sent to the stage in London where it drove the motion rig on which the actors were filmed. After live action had been combined with the CG creatures, the animation was finalized. The composites were created in Shake.

Innovative Simulations

ILM also handled the return of the Dementors, sinister creatures that they had first tackled on Prisoner of Azkaban. The team did not use the assets that had been created for that movie, as ILM's pipeline had changed in the meantime -- the main difference being a switch from B-spline modeling to subdivision modeling. CG artists were still able to use the original Dementor skeleton, but the cloak was completely redesigned from scratch. Animation for the characters was created in two layers: first, keyframe animation for the body, then cloth simulation for the cloak.

"Just like the Thestrals, the Dementors were built and animated in Maya," Alexander explains. "The Dementors' body was textured using displacement maps and paintwork, whereas the cape was created using many different displacement and opacity maps. We also added geometry to the edges of the cloth, so that rather than painting a frayed edge, we actually made a frayed edge. We hung little pieces of string off of the edge of the cloth, and ran a secondary simulation on those. It gave us an edge that was really tattered, while on Azkaban, we simply had opacity maps that we couldn't use in a simulation."

The CG team implemented another major change to the cloak design and rig. On Order of the Phoenix, the cloth was made out of long strips, whereas on Azkaban, it was one large piece. "The director wanted the strips to be able to reach up and interact with Harry, as if it was an octopus," Alexander notes. "In order to do that, we added what we called an 'attractor' on each strip. Animators could guide the strips by telling attractors where they should go. When we ran the cloth simulation, it gave us an organic movement in which the strips would reach out to pre-designated points. It took a lot of fine-tuning to make it look natural, and not like the cloth was being pulled by the attractors. In the end, the director decided to go back to the original Dementor look, and we didn't use this octopus effect. Still, it was a really interesting R&D project that we'll hopefully be able to use on upcoming assignments."

Half-Man, Half-Horse, All-CG

Back in London, another key contributor to the Harry Potter saga was busy crafting its own load of strange creatures. Responsible for more than 100 shots, Framestore CFC produced a herd of centaurs, a sinister house-elf and the title sequence. The team was led by vfx supervisor Craig Lyn and vfx producer Amy Beresford, with CG supervisor Ben White, 2D supervisor Alex Payman and animation supervisor Max Solomon.

FCFC employed a variety of tools on the project. Modeling was a combination of Maya, XSI and Mudbox. Rigging and animation were carried out in Maya, whereas effects work was done in Houdini. Rendering was handled with MTOR/PRman and compositing in Shake.

Initially, the centaur sequences were much more developed, including an extensive stampede and long dialogue scenes. However, the shot count was dramatically cut down during production, due to editing reasons. "The stampede sequence was reduced down to three or four shots in the final cut," White says. "As for the dialog, the centaurs are now almost silent. This did affect us, mostly in the animation phase. To create the centaurs, we scanned a maquette that we re-modeled to our needs. In terms of animation, we had all the challenges of portraying both animal and human characteristics, plus the problem of combining them in a convincing way. This raised interesting problems with the perceived center of mass, for example. The plates had been shot with stand-in actors, both for performance reference as well as to provide something for the other actors to work against. The animation team also shot reference material of horses, which was helpful for particular actions. When you add in facial animation and lip sync, there was lot of work involved in a single centaur, let alone a group of 10 to 12!"

Updating Skin Simulation

FCFC took this opportunity to develop a new technology. The skin and muscle behavior was initially prototyped in Houdini, and then taken into a proprietary Maya plug-in. Rather than using blend shapes or other tricks to simulate the action, a full dynamic sliding skin simulation was applied over underlying bone and muscles. Dynamic hair simulation was achieved with a combination of various set-ups using Maya tools for the tail, and Syflex cloth for more demanding hair simulation around the faces.

The new skin simulation system was also applied to Kreacher, a creepy house-elf that had to make a strong impression on screen, although it appeared in less than a dozen shots. "Production provided us with cyberscans and concept drawings of the character," White says. "The cyberscan data was remodeled into subdivisions in Maya and XSI. Then, displacement maps were extracted, and additional details were added with Mudbox. For the animation, we worked in two stages. First, we animated it to deliver a perfect performance. But we realized that the result lacked subtlety. It was too 'obvious' in a way. So, working against our instincts, we dialed down the animation until we reached a point where Kreacher was not moving much at all -- and all of the sudden, the character felt more real. This was a character that didn't need to gesticulate to have presence. It was a really interesting approach."

Hogwarts Magic

At Cinesite, visual effects supervisor Michael Illingworth handled a completely different category of visual effects, encompassing 120 shots. Once again, the company created the multitude of candles hovering in the Great Hall. Five basic candle models were created in Maya and replicated. CG supervisor Jon Neil and his team wrote automated systems that randomly positioned the candles in spirals at varying heights above the tables, and looped the flame flicker animation. Depth passes and light directional passes were created to get the right depth of focus. Depending on the shots, compositors used either Inferno or Shake.

As on all previous movies, Cinesite also tackled the creation of the "living" paintings on the walls inside Hogwarts. Established in the first movie, the technique involved a motion analysis system that tracked movement from the live action elements, and used this information to warp corresponding areas of the paintings' canvass. The system then added painterly effects to the subject's movement.

The Prophet Newspaper animated photographs were also part of the usual Cinesite assignments. One shot, in particular, started with a massive pullback -- created by ILM -- on the prison of Azkaban, and continued into a front-page photograph on the newspaper.

Parallel to the digital work, large scale miniatures have always played a key role in Cinesite's contributions to the saga. The company has typically been responsible for most of the establishing shots on Hogwarts. For Order of the Phoenix, the huge original 1:24 scale model was once again taken out of storage, and amended for the new production. In addition to that, model unit supervisor Jose Granell and his crew built a 1:16 scale model of Hogsmead, the local town.

The Secret Address

Also returning as a vendor was Rising Sun Pictures in Australia. Led by vfx supervisor Kat Szuminska and CG supervisor Dan Bethell, the team produced 82 shots covering many different sequences. One of RSP's key assignments was the creation of a secret house that magically appears from between two regular buildings. To create the effect, six houses were built in Maya from reference photographs and a LIDAR scan of the set. The main components of the secret house were hand animated, while dynamics bought the curtains and plant to life. Multiple layers of particle effects, combined with 2D elements, generated dust and debris to augment the shots. The sequence was rendered using 3Delight renderer, together with custom shaders and a proprietary render-time geometry injector called Venom. The shots were composited together in Shake.

Even more complex was a series of shots introducing Hagrid's hut and its environment. "The hut was built as a low resolution hand-modeled mesh based on a LIDAR scan," Bethell explains. "Extra geometry detail was added through displacement maps, and by turning the mesh into a 3Delight subdivision surface with a high crease value on every edge. This trick rounded corners just enough to sell the reality of an object, and provided extra detail for the specular lighting component."

Some of the larger hut shots required the creation of full 3D environments. The grass was a simple geometry created using Maya Paint Effects. "Since there was so much of it, we exported it as a large set of delayed read archive RIBs, split into localized chunks of grass to reduce rendering overheads," Bethell says. "We rendered it, along with all the other elements with 3Delight -- the sky and Hogwarts were separate passes, and exported RIBs for rendering with a version of Liquid with Paint Effects export support.

"The grass was textured in a particularly interesting way. We created a shader that baked a point cloud of the UVs of the ground geometry, converted this into a 3D texture, with U in Red and V in Green, and then used the 3D texture call in the 3Delight shading language to apply these UVs to the grass. The result was UVs from the ground being filtered up to the grass above, allowing us to use the same texture for the grass as for the ground. Darkening the roots of the grass using the paint effects UV information sold the trick, and also made occlusion renders unnecessary, as the difference between the look of occlusion and a simple gradient from root to tip was minimal."

RSP contributed subtle effects for the tense scene in which Harry becomes possessed by Voldemort. To show Voldemort's eyes and sockets appearing on Harry's face, the team used three distinct elements within Shake: an iris/pupil treatment morphing Harry's wide pupils to match Voldemort's small pin-hole pupils; an iris color change; the darkening of Harry's eye sockets, using three different mattes that were tracked into position.

As Harry Potter's resident vfx supervisor since 2002, Burke has always made sure that the artists crafting the magic of the movies had enough time to do a first rate job. In fact, the only artist who has consistently been racing against time is himself, as he must wrap postproduction on one movie and prepare the next one at the same time, while dealing with vendors based on three different continents. Now, that's some kind of magic...

Alain Bielik is the founder and editor of renowned effects magazine S.F.X, published in France since 1991. He also contributes to various French publications, both print and online, and occasionally to Cinefex. In 2004, he organized a major special effects exhibition at the Musée International de la Miniature in Lyon, France.