'Kingdom of Heaven': MPC’s Sword and Sandal Hat Trick

With Kingdom of Heaven, MPC turns a hat trick for vfx work on sword and sandal movies. All images and © 2005 Twentieth Century Fox Film Corp. All rights reserved. Photos courtesy of Twentieth Century Fox.

Just when you thought you had seen the mother of all battles, Hollywood delivers yet another combat scene of unprecedented scale. After The Lord of the Rings trilogy, Troy and Alexander, fans of epic movies will have their eyes wide opened for the sequence of the siege of Jerusalem in Twentieth Century Foxs Kingdom of Heaven (opening today, May 6, 2005). This new sword and sandal epic is brought to us by Ridley Scott, the filmmaker who single handedly revived the genre with Gladiator. Inspired by his lost-in-development-hell Crusades project, Kingdom of Heaven tells the tale of a tormented young knight (Orlando Bloom), who embarks upon a life-changing journey to defend Jerusalem from the advancing troops of Saladin.

Incidentally, the battle scenes of Troy, Alexander and Kingdom of Heaven all have one element in common: they were realized, in part or completely, by The Moving Picture Co. (MPC) in London. For the Scott movie, the effort was supervised by Tom Wood (Harry Potter and the Chamber of Secrets), who also served as 2D supervisor: We worked with overall visual effects supervisor Wesley Sewell (visual effects editor of Pirates of the Caribbean) and produced about 440 shots, but the final cut will probably feature 380 of them only. Several effects sequences ended up on the cutting room floor when the first cut came in at three-and-a-half hours. The project was really interesting as it gave us an opportunity to take the technology that we had just developed for Troy to a new level. The project was presented to us in winter 2003, several months before the Wolfgang Petersen movie was released, but our new crowd simulation software was already starting to generate some buzz in the industry. Several other effects companies in the U.K. were bidding on the show, but it seemed that our approach was the most sophisticated. We were officially awarded the project in February 2003 with an initial slate of 250 shots. Other studios collaborating on the effort included Double Negative, which produced about 150 shots, and Framestore CFC with about 20 shots.

Vfx supervisor Tom Wood (left) and CG supervisor Gary Brozenich suggest that a main difference between KOH and Troy was the nature of the battle.

Pushing the Technology

For everyone at MPC, the initial thought was that Kingdom of Heaven would largely rely on the tool set that had been developed for Troy: instead of two armies battling in front of the walls of Troy, it would be one army attacking the other one behind the walls of Jerusalem Not much of a difference, right? Surprisingly, the two projects required very different approaches. At MPC, the general opinion was that Troy was a 3D project and that 2D was basically assembling it, recalls Wood. On the other hand, Kingdom of Heaven was a project in which 2D was actually producing the content. The other difference was the nature of the battle. On Troy, we had two armies violently interacting with each other, with random movements and a lot of individual actions. On Kingdom of Heaven, the focus of the simulation was on the Saracen army approaching, and then laying siege to the city. It meant we had to carry out a much more organized army simulation, with troops marching in formation and operating a wide variety of war machinery.

Wood and CG supervisor Gary Brozenich (Troy) started with two weeks of motion capture shoot. Stuntmen were shot on a stage equipped with a 14-camera Vicon system. Reproductions of the various war machines, with real ropes and actual weight, were used to obtain realistic manning movements from the performers. The data was then archived in groups corresponding to each machine: there was the trebuchet group with its 20 different pre-animated operators, the siege tower group with its team of 50, etc. The other major aspect of the session was the motion capture of the archers formations. These soldiers follow a very precise strategy: walk forward behind a shield, rise, shoot, duck, walk forward, etc., a complex choreography that had to be precisely reproduced by the computers. Each action was sampled five times, by two different performers, in order to provide a sufficient variety of body language. No more than two stuntmen were motion captured at a time. For the all-important cavalry (an army division absent from Troy), 24 cameras were used to motion capture the movement of horses.

The complete motion capture data was then processed and added to MPCs proprietary MLE (Motion Library Editor) system, creating a library of all the possible moves that both the crusaders and Saracens soldiers needed to perform. The next stage involved creating a database of 3D machines and props, including trebuchets (catapults), siege towers, arrows and flags. Using Maya, modelers built accurate CG replicas of the real objects from measurements and photographic textures taken on the set.

Armed with industry buzz about MPCs new crowd simulation software, KOH provided an opportunity to take the technology they had just developed for Troy to a new level.

A Different Kind of A.I.

Meanwhile, MPCs match-move and tracking unit was busy preparing the live-action plates in which the CG animation had to be integrated. When I was on set in Morocco, I did a manual survey of the terrain, explains Wood. Since we didnt have the budget for a Lidar scan, I used regular survey equipment. Later, the plates were match-moved and tracked in Boujou and Maya Live.

The animation was produced in MPCs in-house crowd simulation tool ALICE, also developed for Troy. MLEs main asset the capability of each agent to act autonomously was used in a different way than it had been on Troy. We mostly based the crowd simulation on the motion capture data, says Wood. MLE is great for simulation involving lots of random movements. It produces agents that can act pretty much alone. On Kingdom of Heaven, this ability was not really required, as we had to generate a completely organized army. There was very little hand-to-hand combat to animate. It was really more a siege than a ground battle.

Brozenich concurs: This time, we tried as much as we could to use the full motion capture clip length, as opposed to always employing the MLE system of breaking down the original clips to assemble new clips. On Kingdom of Heaven, MLE was used to create blends between the clips, to generate transitions and avoidance, or to make modular caches of A.I. driven moments, rather than as a full blown A.I. system driving the whole shot. There were a number of shots done using a full A.I. approach as well. We just used a scaling level of intelligence depending on the needs of the shot.

The agents were laid out in groups that could be anywhere from 10 to 10,000 men, depending on the shot. These groups were then broken down into smaller caches and cached out as modular caches. Those caches were combined through the ALICE system to become one large cache that became the render cache. The new modular approach of the system allowed MPC to rapidly modify the CG animation to adapt it to a different shot. As the editing process went on, many shots were moved around within the battle sequence. As a result, the animation that had been produced for, say, a siege tower attack, often needed to be adapted to a completely different type of action.

MPCs animators took great advantage of PAPI, a brand new physics API software that was added to the MLE/ALICE package for Kingdom of Heaven. Anytime we had a soldier that needed to die, we used this dynamics system to create a realistic ragged doll type animation, explains Brozenich. An agent would go from being purely motion capture driven to being a PAPI character within a single shot. Since the software is a physics engine, it was also able to handle rigid body simulation, which we did for the shot of the siege tower topping over. The CG model was quite complex as it was completely physics driven: any board could pop out at any point. MPC added 15 CG towers to the three full-size set pieces that had been built by the production. All the compositing was carried out in Shake.

Creating the Saracen Army

Depending on the shot, up to 100,000 digital soldiers appear on screen. To be honest, you cant really tell whether there are 50,000 soldiers or 100,000 soldiers in a shot, notes Wood. After youve laid out the first 50,000, the rest is just there to fill up the space. You dont even see any movement. These background troops could actually be a matte-painting. In terms of how exactly large the Saracen army was, there never was an official number that MPC had to stick to. Each shot was dressed for the camera: basically, the soldiers were laid out until the shot looked good!

Once completed, the animation was handed over to a lighting technical director or a shot director. Both had a large amount of control on the look and the type of each soldier: whether they carried a shield or a spear, how high they wore their shield, the pattern on the shield, etc. says Brozenich. We had six or seven different characters for each army and about six or seven texture and geometry variations on each. With ALICE being able to swap out objects between the characters and with RenderMan adding more variations, it was almost impossible to find two agents that were completely identical.

MPC approached Jerusalem as if it were a digital miniature. The city was broken down into several very large sections and built from a series of about 20 geometries, which could be moved around, rescaled and rotated.

One of the major animation challenges was finding a way for the A.I. of the horses to directly drive the A.I. of their riders. To this purpose, MPC developed a new tool called Cut and Shut. Horses and riders had been motion captured together, but the new software had the capability to chop the torso off of the rider and replace it with the torso of any performer who had been motion captured on foot. Automatically, the animation of the rider would inherit some of the motion of the horse and create a realistic illusion. Thanks to Cut and Shut, MPC managed to produce an enormous range of riders from a limited amount of horse motion capture.

The movie features several shots in which the entire Saracen army is computer-generated, but, in most cases, the foreground soldiers were real extras. We told the production that we didnt want our CG soldiers to be higher than 1/4 of the screen, remarks Wood. Most of the time, our crowd simulation was integrated behind the foreground action, which necessitated extensive rotoscoping that was executed in Commotion and Shake. The rotoscope script was then directly imported into compositing. We rendered the CG army in about 20 passes, including three depth passes. The challenge was that the shots were heavily backlit. It required very careful compositing as the dust had to be clearly defined in depth. This type of effects is very complex to handle, but when its well done, it looks fantastic!

The Birth of a Legendary City

All this action takes place in front of medieval Jerusalem. The city walls were built in Morocco as a full-size set by production designer Arthur Max (Gladiator). The city was basically designed at MPC, explains Wood. We had some artwork that was provided by the art department, and what we called Ridleygrams. These are tiny sketches that Ridley draws all the time to explain what he has in mind. We opted for a romantic rendition of Jerusalem, moving real buildings around to produce results that were more pleasing to the eye. Also, the landscape surrounding the city is completely different than what it actually is: the real Jerusalem is surrounded by a fertile and hilly land, while in the movie, its barren and flat. This was requested by Ridley who wanted a flat clean ground on which the battle could take place.

MPC approached Jerusalem as if it were a digital miniature. The city was broken down into several very large sections and built from a series of about 20 geometries. These could be moved around, rescaled, and rotated, in order to provide a large variety of buildings. Since the city would always be seen at a certain distance, very simple geometries were needed. These 3D models were then assembled in large chunks and set up as 2 1/2D projection objects. Hundreds of still photographs taken in Morocco or Jerusalem were converted into matte-paintings that were projected onto the geometries from many different angles. More detailed geometries were laid out in UV space to provide a higher level of details in the displacement maps, color maps and bump maps. Those geometries could be moved in and out of every shot and were used to dress the scenery.

Wood and his crew are quick to emphasize that the effects produced for Kingdom of Heaven are probably the best work ever done by MPC. From our point of view, three elements made this show an achievement, concludes Wood. First, the quality of the live-action plates was just exceptional. We never had a problem with any shot at all! Second, the beauty of the film itself made our effects shots all the more successful. Third, our toolset made the picture. Troy was sort of a test ground and it all came to fruition with Kingdom of Heaven.

Alain Bielik is the founder and special effects editor of renowned effects magazine SFX, published in France since 1991. He also contributes to various French publications and occasionally to Cinéfex. He recently organized a major special effects exhibition that opened Feb. 20 at the Musée International de la Miniature in Lyon, France. Displays include original models and creatures from 2010 Odyssey Two, Independence Day, Ghostbusters, Cliffhanger, Alien Vs. Predator, Alien 3, Pitch Black and many more. The exhibition runs through Aug. 31.