ILM’s Ian Comley and Charmaine Chan Talk ‘The Creator’ VFX

Fresh off a five VES Award win evening last Wednesday, Gareth Edwards’ gripping sci-fi thriller, The Creator, heads into the final few days of Oscar voting.

From 20th Century Studios, New Regency, and Entertainment One, the film stars John David Washington (Tenet); Gemma Chan (Eternals); Ken Watanabe (Inception); Sturgill Simpson (Dog); Allison Janney (I, Tonya); and newcomer Madeleine Yuna Voyles.

In the film, amidst a future war between the human race and the forces of artificial intelligence, Joshua (Washington), a hardened ex-special forces agent grieving the disappearance of his wife (Chan), is recruited to hunt down and kill the Creator, the elusive architect of advanced AI who has developed a mysterious weapon with the power to end the war… and mankind itself.  Joshua and his team of elite operatives journey across enemy lines, into the dark heart of AI-occupied territory… only to discover the world-ending weapon he’s been instructed to destroy is an AI in the form of a young child.

Humans trying to harness AI… what could go wrong?

In a far-ranging interview, ILM VFX supervisors Ian Comley and Charmaine Chan discussed the unique, innovative way director Gareth Edwards made the film, from how he enlisted John Knoll and ILM to help “pitch” the project to how he spontaneously shot the film across exotic Southeast Asian locations without previs, working from his own ideas of where, when, and how VFX would eventually best be used. And how the visual effects were eventually figured out and produced, essentially through reverse engineering involving fitting them around, over, and within the live-action shots. And how fun it was to work on.

Dan Sarto: When you came onto the film, what were your earliest discussions about VFX and how they’d be produced?

Ian Comley: Gareth did a little pitch upfront. He knew he had a rather modest budget. So, he effectively went on an incredible holiday, shooting in amazing locations like Thailand, Nepal, Cambodia, and Indonesia, and came back with this sort of beautiful cinematic holiday reel. That's when he approached ILM. On our very early tests, we actually had John Knoll involved. Those did lay some of the foundations for approach to using this beautiful cinematography, then backing into, as you say, what the effects could be, and what augmentations could we do and where. Gareth didn't necessarily know exactly how a simulant was going to ultimately look, or what was the silhouette and the design language? It was still forming, but we could start augmenting that test.

And that became the pitch that went to the studio to get it greenlit. So, when he was finally able to actually shoot the main feature, he took exactly the same approach. Beautiful stuff in camera. Amazing shots. Coming back, he knew the narrative threads that were required, of what needed to happen where and when. But in terms of exactly what we would do to each shot, certainly the assets were far from finished designs when he was shooting. So, there was a lot to untangle. He thought perhaps that we were going to be augmenting more buildings and vehicles and things than we were. Perhaps we had to do slightly more heavy lifting on 3D builds of some of the vehicles and structures. But in terms of the people, it was the same approach as well. He shot great performances in wonderful light and then said, “Well, all right, we know this person's going to be a simulant, that's the narrative thread there, but actually at least through the background, can you make them robots?”

Charmaine Chan: The way that Gareth shoots, even though he went out and did all the photography first, I think he still very much had that vision in mind of what things were going to be. Even though he didn't have exact concept art or models of things that he wanted, I think his vision was still there while he was shooting. So, when we got to the stage of actually adding a building back there or changing that person into a robot, that already existed in his mind. And, when we got to the stage of adding the visual effects, it didn't seem disconnected at all. It's like James Klein, our production designer, knew how to tune into Gareth's vision. He gave us the visuals that was needed to then create the renders that we made. But since everyone was already aware of Gareth’s vision, everything ran very smoothly from his mind to design concept to the final image that you see on the screen.

DS: Did that appreciably change the way you approached the creation of the visual effects or was it more it just changed at what point you began your creation process and how your work was integrated into the film? How much did Gareth’s approach change your overall work producing the visual effects?

IC: We had to think about it differently, for sure. If you walk into a film with absolutely locked scripts or time invested in a complete previs, then you can size it up nicely. We know we can bid on it with time and money. I mean, on this film, Gareth knew what it needed to be, but not necessarily all the specifics and how the edit would come together. I think the original cut was five hours long, which he had to cut down. There was a lot of shaping that had to follow. So, we worked in what we refer to as like the sandbox. His pot was x-size and we absolutely wanted to always give him the best visual bang for his buck on any given shot. And sometimes we would trade off within the sandbox. It's like, “Well you can have those three robots, but actually, we need to make sure we hit this thing over here.”

There was a bit of bartering, but with his intention clear, and with his visual effects background - he wasn't necessarily talking the nitty-gritty of exact techniques with us - he knew where to put something in an image to get the most value for his money. It should be these people, not those people, as the robots. And it allowed us to make concessions, the big hero character in the foreground, the sort of middle-distance big 3D or perhaps 2 1/2-D structure versus the very simple, almost like roto-shaped silhouette in the background. We would have these conversations with him about what the pieces you're trying to get in the frame. What's most important here? And then we could change our technique to try and produce that in the most efficient way, so he had more bucks to spend in the sandbox for something else in mind.

CC: This is also just a fun challenge, when you get approached with the fact that all the photography is already shot. We had minimal VFX data, whether that be LiDARs or HDRIs. How do we take something that already exists and make it work? I think as VFX artists, we embrace those challenges. We want to try new ways of approaching VFX. At the end of the day, this was not anything new. It was a lot of roto and paint was a lot of matchmoves. There were a lot of DMP extensions like we've done for years with current technology. The fundamentals were still the same. It was just a new way of thinking.

DS: Did you have anyone on set during Gareth’s travails across Southeast Asia?

CC: We had an on-set VFX supervisor, Andrew Roberts, who grabbed as much data as he could. When we had it, it was absolute gold. But the way that Gareth shoots, let's say Andrew was with the second unit shooting drone photography of some wide establishers, and Gareth decided to leave Cambodia and go to Tokyo to do a shoot somewhere else. Suddenly you've got two units running and we have no data on the other shoot. So, when we could get the information that we needed, it was great, but it wasn't always possible.

DS: So overall, what were the main visual effects you created for the film?

IC: Certainly, the augmentations, like character augmentations, be it simulant or robot. We did a lot of that, both in terms of ultra hero characters and everything afterwards. I mean, with the vehicles, we intended to take a similar approach, and, in some instances, we could leverage quite a lot. But some of the replacements ate more into the image. So those were more the classic 3D vehicle design, putting those in, perhaps using window inserts and such things from the footage where we could bring the worlds together. All the NOMAD stuff, the full 3D aspect, that's almost like feature animation.

We'd seen this amazing, rich photography of the first two thirds of the movie, the New Asian footage, beautiful and consciously kind of gritty and enticing. What we didn't want was when we then dug into all our 3D work at the end, to miss all that richness, all that imperfection, and just have this pristine feel. It needed to be a continuous, authentic look throughout. So, the sort photoreal feature animation section needed to riff off the real in order to pull that off. That was definitely a feat. I mean, the NOMAD as an asset, the design and build, it's vast and getting the design, the feeling of those shots, the characters and vehicles, lots of environments, it all took a long time.

CC: I don't have the exact stats, but I feel like a good majority of every shot in the film has VFX, from small VFX shots to big VFX shots. Even our main actor, who's not AI or a simulant, had prosthetics. It's almost like every shot of our main actors, whether it’s Joshua or Alphie, they all had had a little bit of VFX to help out, to set this future dystopian, AI robot simulant world. And to that point, every environment was slightly changed one way or another, even if it was the smallest amount of detail, like adding some LED lights, that would still be considered VFX shots. This film involved such a pleasurable variety of VFX work, from fully 3D giant NOMAD shots to doing new technology with Stagecraft or adding a little bit of headgear.

DS: Last question. ILM has worked on basically every type of visual effects-driven project there is. The Creator represents a new way of filmmaking, especially when it comes to the shooting and planning of VFX. The film looks incredible. Do you think this will lead to more films made in similar fashion, with smaller budgets, less up-front visualization, and more directorial freedom, all coming together in a movie that’s just as compelling and visually innovative as the biggest budget tentpole?

IC: That's a big one to finish with.

DS: Yeah, sorry.

IC: Yes. I think it will inspire a huge number of filmmakers. We're super proud of the result. We were thrilled to work on it, and I think there were a lot of filmmakers who will want to work in the same way. But there's the real magic. It's Gareth, it's the cinematographers, it’s the VFX, it's the absolutely deep design language and all that rich process as well. All the stuff that's not necessarily in the frame that makes it successful. It's not necessarily a done deal just to pick up a small camera.

CC: Everyone just keeps asking us, how did you do it? And it's like, we did it because as VFX people, we're generally an afterthought. It's a post-process. Someone handles it, but sometimes, you're no longer dealing with the director, showrunner, or any of the creatives who were there from the start. What is special about this project is Gareth literally included us from the start. He wouldn't have been able to get that pitch done if he didn't go up to John Knoll and be like, “How do I do this? Can we do a little test to see if this works?” He was very adamant about making sure we could do this. And part of that success was he included VFX from the start. And I think that's a key factor that a lot of studios and directors don't think about. We're as much in tune to this process as everyone else in any other department. And if you consider us an afterthought, then it's not going to work.

IC: And that's crucial because on so many projects that use visual effects, the studios can talk down the quantity that contributed to a film because of potentially negative audience reaction. That may stem from when audiences see the use of visual effects as perhaps gratuitous. It's pretty clear that we were brought in to solve this problem that needed to be thought out. More crucially than Gareth's specific approach to the run and gun, indie filmmaking was the collaboration.

We need to scream that from the rooftops. Getting that early conversation means that as filmmakers, it's just as important for us to come in and say, “No, that shouldn't be a visual effect shot.” And having that opportunity, rather than receiving something that’s already set… that early collaboration and trust, what about doing it this way? We all come with practical solutions because we've got a good understanding, like Gareth did, of how you can use visual effects really effectively, not gratuitously. And that's the bit that I would love to see.

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.