'Inspired 3D: Lighting and Compositing': An Interview with Dennis Muren

All images from Inspired 3D Lighting and Compositing by David Parrish, series edited by Kyle Clark and Michael Ford. Reprinted with permission.

This is the fourth in a number of adaptations from the new Inspired series published by Premier Press. Comprised of four titles and edited by Kyle Clark and Michael Ford, these books are designed to provide animators and curious moviegoers with tips and tricks from Hollywood veterans. The following is excerpted from Lighting and Compositing.

A Brief Introduction

If you know about computer graphics special effects, then you know the name Dennis Muren. During his years with Industrial Light + Magic, he has played an integral role in some of the most influential and ground-breaking special effects films ever created. Dennis has worked on special-effects films from the very dawn of the digital age. His efforts have not only been recognized with outstanding box office support, but also recognized by the Academy of Motion Pictures Arts and Sciences, with eight Oscars for Best Achievement in Visual Effects. The award-winning films ( Indiana Jones and the Temple of Doom, Inner Space, The Empire Strikes Back, The Abyss, E.T.: The Extra-Terrestrial, Terminator 2: Judgment Day, Return of the Jedi, and Jurassic Park) represent a wide range of achievements in effects work. Dennis was the 2nd cameraman on the original Star Wars movie, became a director of photography after that, and then found his niche as a visual effects supervisor in 1981. He's been providing us with amazing visuals ever since, and he was kind enough to share with me some of his secrets to success in the world of computer graphics.

Dennis Muren

In working with Dennis, I soon realized that he has a way of appealing to the child inside of us, who can look at the screen with the wonder and awe of seeing something for the very first time. Along with that ability, he is also able to appeal to an ever-increasing visual sophistication that the digital age has instilled in his audience. His creative energy and genuine excitement for the craft he has chosen continue to this day to be a driving force at Industrial Light + Magic.

David Parrish: One of the most valuable things that I know I learned from you was how to choose what to focus on when working on a shot. Could you tell me a little bit about how you look at a shot and determine whats important?

Dennis Muren: The first thing you need to do is to not think that the shot youre working on is going to be in isolation. Its one of 2,000 shots for a movie. A shot is there to tell a story, and the question is, whats the story that needs to be told for that two seconds or five seconds or ten seconds? Thats all there is to it, and its really that simple. There are a lot of elements in any one shot. There is a character, or maybe two or three, a background, some action, and some dialogue. But whats important about the shot? Its necessary to be able to distance yourself mentally from the shot and see it as though youve never seen it before. And I just do that. I can turn on this little switch, as though Im seeing something for the first time, but I know what the intent of the shot is. If I dont see it, then I have to figure out what the problem is. It may be that the camera is in the wrong position, it needs to focus on a different part of the scene, or the shot is just too complicated. The lighting plays a big role, since it can be lit in such a way that it can pull your eye to where you want to tell the story. Filmmakers do this all the time, but the audience doesnt realize it. People dont talk about it, but thats really what were doing when we set up a shot. When a DP is lighting a scene, hes lighting it so that the audience knows where to look. Also, something Ive always used, because our cuts here are so short, is a little formula. The audience should be able to see what the shot is within a half a second. Too many things on the screen can easily confuse the audience, so if we can follow that formula, then were way ahead. That comes with the composition, lens choice, where the characters facing, how hes posed and the lighting.

DP: What advice would you give a lighting artist on how to make a shot look right in these different scenarios: bright sunlight, underwater and a lightning storm?

DM: With bright sunlight it is best to first start by walking outside and looking at whats going on in the real world. Not just reading about it, but actually looking and figuring it out. Our light source, the sun, is very, very small and very, very far away. Theres not much light falloff going on since the sun is 93 million miles away. The other side has to be lit by some sort of ambient skylight or bounce light off of something nearby. This ambient light has either no shadows or very faint shadows on the fill side. Your key light, which is very strong, and maybe 5 stops underneath that (a stop is half the light) is how bright the fill side is. The color of that space also plays a role in determining what the ambient light sources contribute. In combining the CG with the background, your lighting must be from roughly the same point of view as in the background plate. If the shadow needs to follow the contour of something the characters walking by or the ship is flying over, that needs to be there, too. The shadow also needs to have all the same characteristics as the fill side of the character. It has color thats based on the environment, but without the sun. It is important to be attentive to those things. I spend a lot of time just looking outside at how the world acts and works. This is one of the important things that Ive done. There are books on the subject, but it sure doesnt hurt to go out and look at the world. Also look at paintings. Impressionistic paintings are really good because of the methods used for studying light. When looking at those paintings, you can see what you can get away with and still have shots look good or even better than if you did it technically correct. First youve got be able to get it to look real. If its appropriate, you can cheat it to look better than real and pull your attention to the character. In Jurassic, in that first shot of the Apatosaurus when it walks up to the tree and starts eating the leaves, we cheated the backlight on it (see Figure 1). The plate is shot very top lit, which means that the whole top of the dinosaur would have been lit up all the way down his shoulders and his underbelly would have been dark. When we did that, it just didnt look very good. I knew since most of the other stuff was so small in the frame, we could cheat the light back and give it more of an edge light. That edge light made it stand out and look much better. These sorts of cheats are okay as long as the audience doesnt know whats going on or see it as fake. Its good to try those things, but you have to know when you can get away with it.

[Figure 1] A dinosaur with light coming from behind to better identify its outline in the 1993 film Jurassic Park. All Jurassic Park images © 2002 by Universal Studios. Courtesy of Universal Studios Publishing Rights, a Divi

DM: For underwater lighting, a good example is the underwater sequence in Episode I. We didnt want the underwater scenes to look like they were in outer space. One of the ways to prevent this is to make sure things are kind of hazy. Its important to have sort of a fog effect so that things fade off into the murky distance. The trick is how much murk, because youve still got a story to tell. It is important to balance it so you can at least see the setting, but not have it look like outer space or crystal clear air. Another way to do it, as well as having the murk, so it doesnt look like its in fog, is to add little particulates floating around like little bits of debris in water currents. Its nice if that stuff actually has a current to it, with direction. Its not just floating, but its moving, so you get a concept of a large ocean current moving it. It makes everything look a lot more realistic. In addition, you can put little caustic type rippling effects if youre not too deep in the water. This helps remind you that youre in the water since you see the water ripples on everything. Another thing you can do is have your camera be like a hand-held type camera. Include a little panning and tilting, because most of the underwater footage we see, probably 99% of it, is shot handheld. This means youve got a little panning and tilting going on, but you also have camera roll happening all the time. By introducing those three things into it, you will help tremendously in giving the audience the feeling that theyre seeing something thats shot underwater. Something as simple as just adding a roll to the camera can make a big difference. If youre doing something thats at nighttime and dark, then you need to establish some sort of light to start out with. It may be the moonlight coming through clouds, because with a storm, its probably very cloudy. Its always nice in nighttime scenes to edge light something with a rim light, so you can identify the edges of the subject. The front of it can be pretty dark and pretty mysterious. Some of the shots with the T-Rex breaking through the gate and roaring in Jurassic are good examples. There are mainly two things going on in those shots. One thing is theres a little edge light on it, with the front side being really dark and mysterious. The second thing is there are a couple of eye lights in there eyes and not the rest of the model. When the dinosaur roared, we put a little light in its mouth (see Figure 2). It was a very weak light. The audience isnt even aware that they are seeing a light, but their eyes are drawn to the danger, which is the teeth and the jaws. Thats what you want to emphasize. Theres a weak little light source in there all the way through Lost World to make those nighttime scenes look really scary.

DM: The eye lights were used in both Jurassic Park and Lost World, but the mouth lights didnt come along until Lost World. The lighting scheme that Janusz [Kaminski, cinematographer on The Lost World] set up on Lost World was more radical, and it offered more opportunities. The eye and mouth lights are a really good trick. Again, you dont want the audience to think about it, and if you light it dimly enough, they will see it and not to be conscious of it. When youve got something thats kind of sketchy, you can add in the lightning flashes. They can be used sparingly but appropriately to highlight the drama of the scene. They will also fill in the side of the character that is normally dark in the shot. The part of the subject facing the camera could be dark with a little edge light, but then when the lightning hits you see all the muscles and detail. You really get a good chance to see the face, claws, and everything like that. And then its gone in a third of second, and you can use that as a great dramatic tool. What I like to do is keep shots sketchy so that by the end of it, you want to see more. Its what I call playing peek-a-boo with the cameras. As a shot goes on, you can actually direct the audiences eye from one point of the frame to another point, to another point, and so on. You can do that with lighting, performance, camera moves, or anything. With lightning, you can do it in an even more obvious way and it can be really, really effective. By picking the place that you put the light for the lightning flashes and deciding how many there are, you can get the details of the creatures flexing muscles to show up and look really frightening.

[Figure 2] Eye lights used in the 1993 film Jurassic Park.

DP: What kinds of details do you look for in the way light interacts with surfaces of CG creatures and objects that tells you if its working well or not?

DM: I try to have a reference. I always have a reference thats made, an actual object, because otherwise your mind will trick you. The more you work on anything, the more your mind will tell you its looking better. This can be a big mistake, because the audience only sees it once. You have to remember what the thing really looks like that youre trying to do. For The Abyss, we sculpted a pseudo pod character, cast it in clear resin, and had it there all the time. We knew what a clear object would look like and how it refracted light. We had a real object for studying the highlights along with every other detail. We did that for Jurassic, as well. We made the dinosaur models, and the TDs would have them on their desk. We also had film of them that we shot out on the set. Were always referencing back to what is real, and not what you think two weeks into a shot is real. Youll really be fooled if you do that, as you fall in love with your own work, even though its wrong. There are a million things to look at, but if you start with reality and can make a little set of it somehow, like with a maquette or reference photos, then your questions answer themselves. That way you know for sure if the glints are too bright or too wide, or if you can see too many shadows, or if the fill side is not lighting up correctly. One problem is if the skin of the animal makes it look like plastic or rubber. It may not look like CG, but it looks like a fake object instead of a creature. We had to figure out what real dinosaur skin looked like. We went to a local zoo and shot lots of footage of elephants and had all our guys go up and touch elephants so they could see just how much sheen was on it, how dusty they were, and how much detail was on it also (see Figure 3). This is what you need to do. Dont trust your memory. Its one of the most important things.

DP: So when youre getting elements, or when a TD is generating elements youve requested, do you rely heavily on 2D techniques to make those things better? Or do you prefer that the elements are lit and rendered as they will be in the final shot?

DM: It can kind of go either way. I think its sort of based on what the shot is. The comping tools are so great and the people are so good doing comping nowadays that you can almost shoot in any lighting and cheat it in the comp to get it right. I still try to shoot real elements as much as possible if we can get them to look good. Rain is a good example, since you can shoot that so easily. Weve got a good library of real rain, but other things, like complex contrails behind crashing airplanes, you can never really shoot correctly. Those need to be CG. Id say that if youre in strong sunlight or if theres a really dominant light source, then you need to customize it to the shot. If its a more general thing, though, you can be a lot sloppier with it to comp it in. Thats pretty much a judgment call as to what you can get away with. Itd be nice to do everything custom, but its probably going to be too expensive.

A lot of the things that were talking about are actually very insignificant to the shot. The important thing in the shot is that the planes crashing, not that its smoking out the back. You dont want to spend half your budget making the smoke. If theres some way that you can do that cheaper, even if its not quite correct but still looks real, then you sort of owe it to all of the other shots to not get bogged down on that one little detail in the shot.

[Figure 3] Dinosaurs benefiting from elephant skin reference, in the 1997 film Jurassic Park: The Lost World.

DP: So when you work on a film, how do you go about identifying your audience? Is it other people in the CG industry, you personally, your children? Who are you trying to please with these shots?

DM: Im primarily trying to please myself, but still maintaining an idea of who the audience is. It certainly isnt the CG world. Its the director and myself. I know a lot of audiences respond the same way I do to things, which is lucky for me. If they didnt, then Id be making art films somewhere and nobody would be seeing them. Theres nothing wrong with that, but that would be the reality of it. Because Ive got some common vision with the public, as do a lot of the directors I work with, Im able to share it with a greater number of people.

DP: When we worked together, you were always able to provide very clear verbal descriptions of what you were after visually. Do you think that ability to communicate comes from your traditional experience, or is it something youve developed more on the CG side, or is it natural for you?

DM: It certainly isnt natural. I never used to be able to do that. I directed a film when I was in college, and in talking with the actors I had to learn that. You make a choice at some point on whether you want to work for somebody else or you want to be the boss. If youre going to be a boss, you have to talk to people, and you have to learn how to do that well. Everyone offers guidance differently. I like to try to bring solutions to a dialogue, and I know some people dont. They will say heres what the problem is, you figure it out. Thats another way of working, and it may be easier actually on the TDs than the way I do it. I still wish I was directly involved in doing the shots. I feel like I can be making a contribution when I see some of things that sometimes the TDs cant see, and lead them in the right direction. Maybe the problem is that the key light is slightly off to one side, and the fill light needs to be a little more orange. Whatever it is, they just cant quite see it and I try to help them with that. Also, Im the one at that moment they need to please, so theyve got to work to make me happy, so that hopefully I can make the director happy. Youve got to be working for somebody, and its going to go faster if you can talk to them directly. The TDs and everybody else are part of a team. Within the team, the individuals are all trying really, really hard to make things look right. They all want to learn and do a good job, so people are open to suggestions. When things get hard is when they cant quite see it, or the directors changing his mind all the time, or Im changing my mind because somethings not working. Thats where it gets difficult. And then everyone has to have patience just to get through it. That goes all through filmmaking. That happens on live-action sets all the time. There are always situations where the directors uncertain and you have to go back and do something again, or the actor didnt like something and you have to do it again. Thats just part of working as a team, and that goes on with anything creative.

DP: After all youve done, all of the amazing work and Academy Awards, is it still fun for you?

DM: Yes! It is still fun. You know, I do think we need another big shot in the arm. Im not sure what thats going to be, but I like the idea of 3D IMAX films, if they can get those looking really great. Thats the thing Ive seen that has just really knocked me out. The moments when those 3D IMAX films have worked are like youre living the experience. Most of the time they dont work, but therell be moments in the film where its like youre there and youre doing it. Its just phenomenal. Id like to find something else like that, but I havent seen a big breakthrough coming up in the CG world. Weve got the same sort of tools and were just making them better and faster. Were going to be trying a digital human soon, but its all still kind of the same thing. Im still enjoying it, though, and there are still new challenges. I love spectacle. As long as there are spectacle movies where I can see things I cant see in the real world, Ill be doing them. Ill be working on them and enjoying them.

To learn more about lighting and compositing and other topics of interest to animators, check out Inspired 3D Lighting and Compositing by David Parrish; series edited by Kyle Clark and Michael Ford: Premier Press, 2002. 266 pages with illustrations. ISBN 1-931841-49-7. ($59.99) Read more about all four titles in the Inspired series and check back to VFXWorld frequently to read new excerpts.

David Parrish (left), Kyle Clark (center) and Mike Ford (right).

David Parrish went straight to work for Industrial Light + Magic after earning his master's degree from Texas A&M University. During the five years that followed, he worked on several major films including Dragonheart, Return of the Jedi: Special Edition, Jurassic Park: The Lost World, Star Wars: Episode I The Phantom Menace, Deep Blue Sea, Galaxy Quest and The Perfect Storm. After five years with ILM and a short stay with a startup company, he was hired by Sony Pictures Imageworks to work on Harry Potter and the Sorcerer's Stone.

Series editor Kyle Clark is a lead animator at Microsoft's Digital Anvil Studios and co-founder of Animation Foundation. He majored in Film, Video and Computer Animation at USC and has since worked on a number of feature, commercial and game projects. He has also taught at various schools including San Francisco Academy of Art College, San Francisco State University, UCLA School of Design and Texas A&M University.

Michael Ford, series editor, is a senior technical animator at Sony Pictures Imageworks and co-founder of Animation Foundation. A graduate of UCLAs School of Design, he has since worked on numerous feature and commercial projects at ILM, Centropolis FX and Digital Magic. He has lectured at the UCLA School of Design, USC, DeAnza College and San Francisco Academy of Art College.