The Pixar senior scientist and lighting expert discusses the studio’s new physically-based lighting system, Global Illumination.
With the upcoming release of Pixar’s latest feature, Monsters University, much of the discussion on the studio’s recent technological innovations has focused on their new lighting system, Global Illumination. GI, as it’s known, represents a completely new approach to lighting for the studio. Instead of the time consuming and difficult application of potentially hundreds of pinpoint lights, artists can use a few “area” lights that provide a much more accurate physically-based and realistic effect. In addition to the visual benefits, GI provides filmmakers a fast sense of what the final lighting will look like much earlier in the production.
At FMX this past April, I had a chance to sit with Christophe Hery, a senior scientist and lighting expert, who came to Pixar three years ago from ILM to join the studio’s lighting and rendering technology development efforts. He shared his thoughts on the new GI tools as well as how the next generation of lighting advances needs to happen in collaboration with advances across the entire animation production process.
Watch Christophe’s interview exclusively on AWNtv. .
Dan Sarto: Tell us about some of the latest lighting advancements that you've helped engineer at Pixar?
Christophe Hery: I was hired in order to simplify the lighting setups. There was a DP on Monsters University who had decided to change the lighting approach. There were sometimes hundreds of lights in a shot, placed artistically. It was like painting, what the artists were doing. The setups were pretty heavy, pretty complex, hard to understand from one artist to another. So the idea was, “Can we use more modern technologies like Global Illumination and bounce the light around? Could we express lights that are more physical to the point where we don't need potentially hundreds of lights for a character? We may need only two or three. [Can we] let the computer bring some of the complex light transports into the image?” So that's basically what I brought.
I come from visual effects so I had been exposed to those techniques and I was able to bring them into the land of animation at Pixar. Once we had them in place on Monsters we used them on the short film The Blue Umbrella and pretty much every feature film afterwards. I can't completely predict the future but at this point we're using those lights now everywhere.
DS: Can you explain a bit more about Global Illumination and more physically-based lighting?
CH: Yeah, it's really fascinating. At Pixar, as I said, the lighting was very complex. For instance, working with early benchmarking shots on Toy Story 3, we were trying to understand what the lighters had done in their shots and how we could simplify that, what we should do nowadays with more modern techniques. In some of the scenes there were hundreds of lights. But sometimes the Sun was in the shot 10 times because the characters reacted differently to that light. So part of the idea of moving towards more physically-based techniques is also to unify what the material response is. You need materials that are energy conserving. You need shaders that are more physical, than can import and sample, that the notion of reflection versus specular is the same thing in nature even though in computer graphics historically we tune them separately. So, it was like trying to bring this back together in a more meaningful manner.
In the end, we had a new set of tools and so I had to re-train people. The language becomes more one of a live action DP in some ways. Like where you're talking in terms of exposures, you're talking in terms of really high dynamic ranges. Now, an intensity on the light is not one or two, it may be in the thousands, or 50,000. The sun would be at 64,000 or something like that in these new units. So I had to work really closely with the DPs and with the lighters to make the transition happen. In the end it's a new pallet in a way. They're moving a little bit from painting or a painterly approach to a more photographic approach. We're not removing tools. You can express the same thing in a way but then it's slightly displaced.
Let's take an example, if you don't mind. When you shoot indoors, let's say you shoot in a room where there's a window and the Sun comes in. There's a little bit of light coming from the outside into the room. In your camera, most of the frame is going to be dark. There's usually not enough light coming from outdoor into the room to illuminate, to properly expose the shot. A live action DP will usually enhance that light coming from the window by hiding a light on the ceiling out of the frustum of the camera, or behind the camera. So that's the type of cheat we needed to re-learn. We spent quite a bit of time understanding the setups from live action DPs, what they were doing, and re-learning that process, shooting the lighting in those ways. But what it brings at the end is a much richer image because the behavior of the light is more complex. It comes from different directions, its picking up the colors of the wall, picking up the colors of all the objects, bouncing around. So it's more complex in terms of richness of the image while at the same time the setups are easier. So you get probably 90% of the shot done very quickly. You don't have to struggle with secondary lights and things like that. And so you spend a lot of your time being artistic again.
Now, when I move a light, it’s because I really want this effect on the character, not because the material is not responding properly, not because what used to be a specular highlight that was working well in one environment doesn't work well in this new lighting environment. Now it's going to work in any environment. It's going to react naturally. There's this notion now of softness. Softness of the highlights, softness of the shadows. All this is controlled by the size of the light, by the emission of the light. You can put barn doors, you can put all those extra physical properties on the light to shape the behavior. So it's another richness and language and tool box for the users.
DS: Visual effects primarily involve photo-realistic visual productions, much different from feature film animation production. How different is working on animated features at Pixar from working on live action – visual effects driven features at ILM?
CH: Off the bat, I don't find it that different to be frank with you. I think even in live action and visual effects, the goal is to tell a story. It's not necessary to replicate reality, so the lights that you see on the set are not real lights either. They're just put there so that the characters, the actors will look good. So it's kind of the same to me. You need to focus the attention of the audience somewhere. An artistically driven approach to lighting is very strong at Pixar because we are a story driven company. Part of what can enhance a story in a shot in particular is the lighting, as well as the cinematography, the placement of the camera.
What I would say is a little bit different on the other hand is that in visual effects we're used to trying to capture data from what's there on the set. As a starting point, for instance, we would capture a 360 degree high dynamic range image, sort of a dome, to give us an idea of what the lights were that the DP had placed on the set to start with. We would cheat from there most of the time as well. But it gave us a really good starting point. We took that approach as well on Monsters, on The Blue Umbrella and on films that are that coming up next. But, we give them a special twist at Pixar where we can paint on this environment in real time and see the effect in the character. So even though we went more in a photographic way, we were still using a kind of painting approach in those tools, if that makes sense, so the artist could express really complex behavior by painting on this dome, this texture map and seeing the result on the character. That's one approach we took.
We could also start from acquired images. A lot of the turntables, a lot of the look development, we did them under IBL, high dynamic range images that I shot upstream so we could canonize the materials across the different assets. That's a very visual effects approach to a show. More and more you've seen visual effects studios doing animation lately, like Weta with Tintin and ILM with Rango. I'm not saying they're merging, but the approaches are becoming roughly the same. I mean granted Pixar has a long history of storytelling which obviously is very different than any other studio. But in terms of techniques, in terms of approaches to lighting, we are sort of moving towards the same direction right now. I hope that all these techniques are freeing the artists in the end because as I said they give them something really good, really fast. And so at that stage, they can move to being artistic, take liberties with those lights and express their shots, make them even better as opposed to spending time debugging something.
DS: What’s next for lighting technology development? What are the holy grails?
CH: Well, I think one of the big one that's coming up is trying to move towards real time. Not necessarily real time delivery of the final shots but interactivity for the tools so artists can really place light and see the effects of the lights directly. They can do animation with the light rigs without being slowed down by the tools. Luckily, we have lots of [powerful] hardware. There are GPUs and special CPUs that are really beefy now, so we can do a lot more than we used to be able to do 5 or 10 years ago. That’s one big thing. In terms of image richness and simulation of materials or lights, we can always do better. There are more light transports that we need to build into the system. But we need to model better, we need to texture map better as well. It's not just the light, it's not just the material. Everything has to work together. There is a design that needs to be there, there's an amount of detail that needs to be there. All that stuff has to sustain the story.
In the past, I worked a lot on Pirates of The Caribbean. For instance, at ILM, Davy Jones was a famous character that we did. The reason why Davy Jones worked in particular was that it was a whole design. There was this octopus thing. There was the voice of Bill Nighy and his performance behind it. There was all the detail we put in the model. Of course we had really complex light transport on the character as well subsurface scattering. We were rendering the eyes and everything, but the success of it was because we had all those elements together. I don't think I can tell you we need to do this new technique for lighting that's going to make the image so much better. We can't do that in isolation of not potentially re-addressing how we model or how we texture map, how we do all those other things at the same time. That's what fascinating about this field. It's a really a strong collaboration between all the different fields. Artists, engineers, modeling, animation, look development, storytelling and lighting have to work together. I think harnessing the power of the computer is one of the easiest things we can do. It's a hard problem but we can do it for sure. The rest is probably longer term.
--
Dan Sarto is editor-in-chief and publisher of Animation World Network.
Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.
