The studio revisits and revamps Hulk, Marvel’s smash-‘em-up, fan-favorite digital character they first brought to the screen back in 2003.
Even with the crazily short time interval between work on Infinity War and Endgame, ILM continued to upgrade their production technology to push for every performance gain possible. “The audience and client expectations get bigger, the tools get better and we’re always trying to push the envelope,” Earl notes. “In terms of the toolset, for a good portion of the production we were improving tools that we already had. In the case of Smart Hulk, we rebuilt our whole facial pipeline with the end goal of trying to give our animators ultimate control in creating Hulk’s performance.” The new setup, in particular a solver called Anyma developed by Disney Research in Zurich, allowed for better fidelity in translating the facial performance of Mark Ruffalo onto Smart Hulk. “We did a Medusa session with Ruffalo to get a whole set of facial shapes to drive our facial library for Smart Hulk,” Earl continues. “It was a per-frame match for the solvers that got retargeted onto Smart Hulk. The beauty of the improvement to the system was that it enabled animators to creatively go in and say, ‘His smile is too broad here. Let’s tweak it.’ You could dial out the solve and go full shapes or you could use a mixture of both. Most of the shots went through the head-mounted camera solves onto animation. There were a few shots that were pure keyframe animation.”
For Smart Hulk’s dialogue, the key was staying true to Mark Ruffalo’s performance by capturing as much of it as possible. “What we wanted to have was not just a cache of points with all of the data, but to convert those caches and mesh into a shape library that animators could control and have those curves on the shapes from the get-go,” Lum remarks. “Before, when changing something, it would have to be additive. We couldn’t subtract without losing the performance itself. This new solving of the data allowed the animators that control.”
In Endgame, Smart Hulk travels back in time and encounters Hulk from the original Avengers. “They were two distinct models and so we had them as separate assets,” Earl shares. “It was fortunate that we still had some of the animators who had worked on Avengers as they were able to guide some of the newer shots.” The Hulk asset from Avengers was restored and updated with the latest muscle simulations and technology. “It wasn’t a complete one to one, as we augmented the model in the face and the color a little bit,” the VFX supervisor adds.
In addition to their work on Smart Hulk, ILM handled many other areas of VFX production. For example, new suits of armour were needed for War Machine, Iron Patriot and Rescue. “The trickiest part was handling when their head was showing and we had to work within the plate,” Lum reveals. “Surprisingly, we didn’t have to do much repositioning work. We needed the fluidity of the superhero character, to get them into those superhero poses without the geometry crashing into each other. That was definitely a task for rigging to give us a system that allowed that flexibility but kept the armor looking solid. War Machine is bulkier [than Iron Man] and has less flexibility, which restricted the range that you can get into the physicality. And Iron Patriot is a bulky character with massive arms and shoulders. Even though that restricted the range you could show him in, we still got some cool poses and action out of him.” ILM also did suit work on Rescue, the name given to an armor-wearing Pepper Potts, played by Gwyneth Paltrow. “That was cool,” Lum recalls. “We weren’t restricted to anything that had been in previous films like we were with the other characters. It was interesting to see how an iron suit would move in a more feminine way.”
Iron Man’s look got an upgrade through a new nanotech suit of armor. “We had the bleeding edge form more into the panels that sit down and lock into place, not like in prior films when it was more of a rigid suit,” Earl says. “Rigging these things is always a challenge because if it’s moving too much it starts to look stretchy and rubbery. We’re always trying to have panels move against each other or slide into place. We looked at artwork of the prior suits, which had more breaks and gaps. However, getting the suits tighter and more bleeding edge also makes hiding that stuff harder. So, you try to find the areas where you’re able to shrink and stretch or overlap and move.”
As far as cool Avenger garb, Endgame introduced an entirely new line of digitally-created time travel suits. “The challenge here started with the idea that these suits are a combination of Stark and Pym tech,” Earl explains. “Starting back on Winter Soldier, Joe and Anthony wanted to bring a sense of reality and physicality into things that are magical or haven’t been seen before. How can you make it feel like these suits are flowing out of this time GPS and covering their entire bodies? The shots are quick but at the same time, you want to tell the story that the suits are consciously taking them to the time and place they need to go to.”
As far as environments, all the Quantum Realm time-travel shots were fully CG while plates were shot in Scotland, then highly augmented for New Asgard. “We looked through a bunch of the buildings they’d used in Asgard as well as Scandinavian buildings, like Norwegian fishing villages,” Earl states. “The trick about the Quantum Realm shots is that the Avengers are shrinking and travelling at the same time. It becomes a challenge of how do you do it? When you are building that geometry procedurally on the fly, how are you animating through it? We gave animators some targets they could hit, so they could make sure where they were driving to would coincide with the geometry that would be scaling and built on the fly at render time. It was definitely a lot of back and forth between animation, effects and compositing; we couldn’t make it too complex, because your eye needed to see that though they went off to different places, they always came back to some familiar prior Quantum Realm stuff.”
ILM shared the third act final battle sequence with Weta Digital. “Weta built the original crater and destroyed compound assets, then handed the geometry over to us,” Earl says. “Then we textured and built out around the crater. We also constructed our own set dressing like concrete, rocks, and trees. Aside from the initial handoff, we didn’t share too much back and forth. We worked on the beats where Hawkeye hands off the gauntlet to Black Panther, Black Panther does his run, Ebony Maw comes in, Black Panther throws the gauntlet to Spidey, who goes into instant kill mode and gets lifted out by Thor’s hammer. Then we did the Doctor Strange and Iron Man beat, plus when Captain Marvel flies in and destroys Thanos’ ship that crashes into the bay.”
According to Lum, in the final battle, “There was a lot going on in the background. We had to make sure that every single character had behaviours that suited them. The Ravengers from Guardians of the Galaxy are like a cavalry, while the Chitaurians are animalistic and savage. We had to use additional motion-capture shoots to get more performances.” Especially in large, chaotic fight sequences, it’s important to direct the viewer’s eye within the chaos of the frame. “We didn’t fill the background with random fighting,” Lum continues. “Black Panther has the gauntlet, so he is everyone’s focus. Everything converges on what’s important. There is another beat with Spider-Man. It was shot onstage with Tom Holland. Spider-Man is a kid, not an action hero. We added the spider legs onto his back and they do all of the fighting. Spider-man is not a combat expert like Black Panther, who is more superhuman. The fight actions we chose for him were more acrobatic. We were trying to be mindful of what a character is capable of and make sure that was always part of their actions.”
“One of the big things that we’ve done over the course of the last films is try to bring all our shot work up at the same time,” Earl describes. “It makes it more challenging, but we get everything rendered as soon as we can, even if it’s at a lower quality level, so we can focus on getting consistency between the shots and work on the beats. We can bring it all up to a level that allows Joe and Anthony, Dan and Jeff Ford [the editor] to look at it in context. It also prevents us from falling into the trap of focusing too much on one shot. Edits change so you don’t to put your eggs into one basket.”
Getting an opportunity to reunite with Marvel’s Bruce Banner alter ego that ILM first engaged back in 2003 was a highlight for both Earl and Lum. “I definitely loved seeing people’s reactions to Smart Hulk,” Earl concludes. “The end battle is epic as well. Being able to be part of that was super cool. It’s fun to keep going with the same characters who we’ve been a part of since the earlier films, like not only Captain America and Iron Man, but Black Panther and Spider-Man. It has been a great run with the Russos and Marvel on these movies. It’s fun to see it all culminate with Endgame.”
Trevor Hogg is a freelance video editor and writer best known for composing in-depth filmmaker and movie profiles for VFX Voice, Animation Magazine, and British Cinematographer.
