Zooming in to the Scale of ‘LEGO Ninjago’

Warner Bros.’ third installment in the LEGO franchise, The LEGO Ninjago Movie, directed by Charlie Bean, Paul Fisher and Bob Logan, delivers a new perspective on the LEGO universe, fusing the stop-motion style of the previous films with elements of the natural world as though a kid were creating a LEGO universe in his own backyard.

Sydney-based animation facility Animal Logic has handled all of the animation for all three films. Animation director Matt Everitt, who worked on the previous two LEGO films, as well X-Men: Days of Future Past (2014), Walking with Dinosaurs 3D (2013) and Happy Feet Two (2011), explained that he started working with director Charlie Bean early in preproduction, before the film was even greenlit.

Everitt, who has been with Animal Logic for about nine years, reported that roughly 680 artists had about 40 months to complete the film. “The scope of the project was pretty epic in terms of the kind of film we wanted to make -- taking LEGO from the LEGO world where everything is made out of bricks, out into a more natural environment and trying to figure out how that LEGO was going to look and interact,” he says.

The goal was that all the LEGO configurations seen on screen could be physically reproduced, so the film was virtually constructed brick-by-digital-brick. But first, the animators had to learn how to work within the limitations of the LEGO universe.

“Every single animator, when they first joined the movie, they were given time to experiment and play with the LEGO,” explains Everitt. “Basically, we have a giant LEGO play pen with every single LEGO piece ever made in CG, and the animators were encouraged to experiment with that and see what kind of solutions they could come up with.”

He added that it took a while to learn how to make the LEGO “minifigs” (LEGO characters) perform and emote, although many on the team had already learned the tricks of the trade on The LEGO Movie and The LEGO Batman Movie. “It just takes practice to learn what works,” he says.

The film, which just finished up its sophomore weekend in theaters, tells the story of six teenage ninja minifigs on the island city of Ninjago, who must defend the city from the evil Lord Garmadon (Justin Theroux). Meanwhile, Lloyd Garmadon (Dave Franco), the group’s leader must come to terms with the fact that Lord Garmadon is his father. The film features some epic fight scenes, which were choreographed by legendary martial artist Jackie Chan, who also stars in the movie as ninja Master Wu.

“When we were watching Jackie and his stunt team do the choreography for the fight scenes, the initial reaction from the animators was, ‘How the hell are we going to do that?’ Trying to translate that into what a minifig can do, when you first think about it, you think it’s impossible,” Everitt recounts.

-- Read it now on ANIMATIONWorld:
Director Charlie Bean Assembles an Army of Animators for ‘The LEGO Ninjago Movie’ --

“With LEGO, you can’t have a character rest his chin on his hand, or put his elbow on the table, but sometimes you might want to get the feeling of a character doing that,” Everitt explains. “A LEGO character can’t reach across and scratch his other ear. He’s just physically incapable of it. So, you have to figure out the exact situations, and figure out your performance within the limitations, and within the world of LEGO.”

Everitt confided that the more they experimented and played with LEGO, the easier it became to find solutions to even the most complex challenges. But for him, the biggest challenges were creative rather than technical.

“The fights were one of the biggest challenges,” he says. “I’m always trying to tell an emotional story and I’m a big fan of the less-is-more philosophy, especially when working with LEGO. I’m a fan of simplicity, and I’m a fan of reduction and trying to take unnecessary elements out of the shot, and out of a piece of action, and just trim it down to the core of what you’re trying to say with that sequence.”

“Because they’re so simple, when you put LEGO on screen, the slightest nuance reads very broadly,” he elaborates. “But it also means that a lot of the tricks the animators get used to when they’re animating other CG features, where you’ve got crazy bendy limbs and overlap and you can do squash and stretch, all these little tricks that people often fall back on, we don’t have them in LEGO. So, as an animator you’re very exposed. Every single choice that you make is very clear on screen.”

Because of the film’s stop-motion aesthetic, the animators couldn’t even rely on motion blur to “hide” some of the movements. “We treat it very much like it’s stop motion,” says Everitt. “Every single frame is hand-crafted. We don’t let the computer do anything…. And so the animators had to trust themselves a little bit more and learn to really hone their acting choices.”

Everitt reported that animators at Animal Logic work in Softimage XSI, but after the first two LEGO films, it’s been highly customized for the unique needs of LEGO films. The company has its own in-house renderer called Glimpse.

Natural Environments

The island city of Ninjago and its surrounding mountains are built from nearly 12.7 million virtual LEGO blocks. In the real world it would cover approximately 841 square meters -- slightly smaller than the base of the Great Pyramid -- and the city’s tallest building, the Ninjago Tower, would be 22 feet high.

FX supervisor Miles Green, who shared a VES nomination in 2014 for his work on the original LEGO Movie, was in charge of integrating the natural elements into the film.

“The cat was definitely one of the more challenging creatures I’ve had to create up until this point. I think it’s the familiarity that people have with them. It’s almost like the uncanny valley when [VFX artists] create digital humans. We’re more likely to notice a digital human if there’s something a little bit wrong or off, because we’re just so attuned to looking at people. I think the cat was almost like the creature version of that.”

“Charlie Bean was really keen to push everything into the natural domain,” explains Green. “The brief at the beginning of the show was imagine you’re a kid in the backyard and then imagine how would you recreate the effects. You know, rain would be like it’s coming out of a watering can.”

In the film, each of the ninjas has a special power to control the elements -- including earth, water, lightning, fire and ice. Of course, all of these elements behave quite differently on the small scale than on the large scale.

Green explained that, early on, Bean told them he didn’t want LEGO-block-based effects for the elements. “And we’re like, ‘Really? We’ve just gotten really good at brick-based effects,’” he quips.

As a result, the FX department, which according to Green relies on Side Effects Software’s Houdini, had to create many of its own elements and shoot lots of reference materials.

“We were dropping these LEGO prototypes into fish tanks and pulling them out to see how water sat on the edge of the LEGO set,” he explains. “We were taking the little prototypes of Lloyd’s minifig down to the beach to see how the water and foam compared in scale. We did our own internal shoots on a mini scale -- squirting them with little water pistols or plant misters.”

Green and his team also had to study things like the fine ice crystals that form in a freezer, how wet sand interacts with minifigs, and slow-mo footage of firecrackers and sparklers.

“A lot of effects artists are used to working at a huge scale,” he says. “They’re coming from [films] where they’ve got to build the biggest effects and make these huge explosions and now we’re saying, ‘Well, what does an explosion look like to a minifig?’”

For example, the Fire Ninja, Kai (voiced by Michael Peña), is able to shoot flames from his double-barreled Fire Mech. “But this isn’t a human-size flame thrower -- it’s probably more like taking a deodorant can and lighting it. That’s about the size that a small LEGO man could produce,” explains Green.

“The same goes for missile trails,” he adds. “We found this great reference [footage] of these guys that make match rockets, with a bit of tin foil and a match.” Often the team relied on such reference shots from YouTube, because, being in Australia, where wild fires are a perpetual hazard, firing match rockets or using deodorant cans as a flame thrower is a bit risky.

The team also studied the fight scenes in old Bruce Lee movies, where often the actors would put a little bit of flour or chalk dust on their hands in order to give a sense of impact when they punched each other. It was an element that Bean particularly wanted to add.

“And we’re like, ‘Flour? On a minifig scale? What does that mean?’” says Green. “But the things that became important are those little bits of dust that the camera picks up, what we call ‘air motes’ getting perturbed. It’s those things that help sell it. So we were trying to put flour on the ends of the contact points. You’re talking 50 to 100 grains of dust. That was a challenge for the FX artist to work at that macro, micro level.”

Ninjago itself is on an island. And while the VFX artists at Animal Logic were quite familiar with creating big ocean waves, the water had to behave more like a shallow duck pond.

“It was very shallow so at scale we couldn’t have waves crashing in. But then Garmadon has these huge mechs that emerge out of the water -- sort of like Pacific Rim but on a macro [zoom] level.”

Of course, as these LEGO mechs come out of the water, the water needed to drain off believably for their scale. A scene where the mechs walk across a sandy beach called for one of the largest simulations ever run at Animal Logic, with more than 100 million grains of sand appearing on the shoreline.

“LEGO does one of two things. All the water drains off quickly and then [internally], they would hold all the water, and then as they start walking down the beach, the water sort of sloshes out of them every so often,” Green says. “There were challenges with that, trying to get the water to flow off and look nice, to look attractive. Not to have this water come sloshing out of the mouth of these huge LEGO mechs.”

CG and The Unstoppable Beast

CG/VFX supervisor Greg Jowle, whose credits include X-Men: Days of Future Past (2014), The Great Gatsby (2013) and Happy Feet Two (2011), explained that originally, Ninjago was scheduled to be released before The LEGO Batman Movie, but most of the time, production of the two films went on side-by-side at the facility, with crew being shifted from one film to the other as needed.

“We wanted to make sure that we didn’t redo work from one film to another or be in a situation where we had to constantly transfer work between the projects, so we set up a global library so that if anyone found a new brick that we hadn’t built yet and needed to build it, (as a simple example), it would go into this library and that was available for all projects.”

Jowle explained that, from the outset, they knew they would have to be able to replicate plants at the minifig scale and that the filmmakers wanted plants with an “Asian influence.”

“We used things like bonsai trees, just because they look cool. Then we obviously couldn’t use real grass because it would be taller than a minifig and it wouldn’t look like grass,” explains Jowle. “But we wanted to have that sense of manicured lawns, so we studied a bunch of different moss types and decided to do moss. If we were in the city we wanted to have little plants peppering out and blending in with the LEGO. Then when we went out to the jungles you would occasionally have bits of LEGO almost like ruins scattered throughout an otherwise wholly natural environment.”

“So we developed two tools specific for Ninjago, but with an idea that they would be tools that we’d continue to update and refine,” he continues. “One was a basic sort of procedural tree builder to help distribute twigs and leaves and stuff like that onto models. We called that Spruce. Then we also developed the system for essentially just propagating instances to do forest and less covered ground. We called that system Spawn.”

He explained that the studio’s Glimpse in-house rendering system had been primarily designed as a LEGO brick renderer and completely optimized for that task, so it had to be upgraded to accommodate all of the more natural elements and new shaders that had to be developed. “Things like wanting to be able to spread out a bunch of little moss assets to make it feel like a big field or something to help us build trees so that you didn’t have to hand place leaves,” says Jowle.

The film takes a major twist when the climactic battle between Lloyd and his father inadvertently awakens “The Unstoppable Beast” -- a Godzilla-like house cat that threatens the city of Ninjago.

Jowle reported that director Charlie Bean was keen to use a live-action cat.

“He felt that it was unlikely that we’d be able to get that natural charm that a cat has,” he says. “I was trying to convince him to just go with a CG cat simply because, from experience, I knew that things change and regardless of how much we would want to make the shots work with whatever plates that we caught of the cat, it would just become a little too difficult at times. There’d be a desire to have a shot and it would just be near impossible because you never got that plate.

“Then, for a long period of time, we compromised with the idea that if we had a really hero shot of the cat, we would make that live-action. If we had an action that we knew we’d never get the cat to do, that would be a CG version,” Jowle adds.

Of course, Jowle wanted as much time as possible to build and refine the CG model of the cat, not just the rig build, but the fur and the motion as well.

“We developed two tools specific for ‘Ninjago,’ but with an idea that they would be tools that we’d continue to update and refine. One was a basic sort of procedural tree builder to help distribute twigs and leaves and stuff like that onto models. We called that Spruce. Then we also developed the system for essentially just propagating instances to do forest and less covered ground. We called that system Spawn.”

“You have to go over these things as many times as possible and really fine-tune them,” he says. “To do that you have to know exactly which cat you’re building. But we didn’t even know where the live action was going to be shot. The studio wasn’t sure initially. It might have been shot in China; it might have been filmed in L.A.; it might have been filmed in Sydney, which was just adding to my stresses.”

At a certain point, Jowle decided he couldn’t “wait forever” and started doing internal tests with a Bengal cat that belonged to one of the artists. “It was a relatively trained cat, if there’s such a thing. So we did some reference shoots with that, just to kind of see what the cat looked like in amongst the LEGO, to see how it looked from different angles.”

From that, they built a generic CG cat -- one that wasn’t overly refined in terms of detail, but still gave them a basic rig to start figuring out what was going to work, what wasn’t going to work.

“The studio then decided that we could cast the cat ourselves, so we started doing a casting, brought various cats in, and had the director pick the one that he liked,” says Jowle.

After they had done a number of shoots with one particular cat, the studio decided they wanted to go with a cat from a “cat casting agency.”

“We found a casting agency in Sydney, and we managed to find two twins, which was great. It gave us a little bit more bandwidth. As the one got a little bit too tired or was just no longer interested, we could put it back in the cage and try the other one for a while,” he says. “As anybody who’s done this stuff can tell you, animals and children are really difficult things to shoot, and cats are fairly un-trainable. You can get them to do certain things but you can’t get them to do things on command like you might be able to achieve with a dog.”

The twin cats were shot with an Alexa camera, in 3.5K resolution at 60 fps, so they could slow down the footage to catch all the nuances of the cats’ movements. In order to avoid freaking out the cats, they decided to shoot wide and crop in on the cat in post.

“In the meantime we were also developing the CG version,” Jowle says. “It became pretty clear with things like integration into the CG from a lighting standpoint, that we needed to use the CG cat, except of course for the live action book end.”

“It was definitely one of the more challenging creatures I’ve had to create up until this point,” adds Jowle. “I think it’s the familiarity that people have with them. It’s almost like the uncanny valley when [VFX artists] create digital humans. We’re more likely to notice a digital human if there’s something a little bit wrong or off, because we’re just so attuned to looking at people. I think the cat was almost like the creature version of that.”

Jowle said that he was delighted to read some of the early film reviews that commented on the integration of a “live action” cat into a CG environment. “That was exactly what I was hoping for,” he says. “I was hoping very much that people would think it was real, because that was the director’s desire. The director wanted people to watch the film and think it was real.”

Perhaps the moral of the story is that it’s easier create a believable CG cat, with all the technical challenges that that entails, than it is to get a real cat to perform on command.

But of course, any cat owner could have told you that.

Scott Lehane is a Toronto-based journalist who has covered the film and TV industry for 30 years. He recently launched VRNation.tv -- an online community for VR enthusiasts.