Non-Photorealistic Computer Graphics (Modeling, Rendering and Animation)

This book by Thomas Strothotte and Stefan Sclechtweg comes across as quite a dilemma for me. On the one hand, its applied science obviously works (if not, we would not be enjoying the kind of software packages many of us are using day in, day out), and on the other hand, however, its "ontological moorings" are very weak indeed.

Overview

I will first briefly review the book's presentation in a historical context of "Non-Photorealistic Rendering" (NPR), as it offers much interesting information about the origins, present state and possible future (uses, areas of potential development, and so on) of this applied science.

The book provides much information of a very technical nature, including a plethora of algorithms, which will no doubt be of interest to those who are involved with designing software (something that seems to be on the increase with digital artists as well as the people who now try to escape the "Photoshop curse" by attempting to design their own tools based on priorities distinct from those behind the purposes of Photoshop). Here the declared target audience consists of computer science students, practitioners and researchers in the field.

Each chapter ends with a set of exercises, reminding us that both authors are teaching computer science at the University of Magdeburg in Germany, and that this book was first born as a series of courses taught both at Magdeburg, and at Simon Fraser University (Vancouver, Canada).

A serious shortcoming of not only this book, but of much of our applied sciences as a whole today is what Husserlian Phenomenology would label as "ontologizing" (Ontologisierung), a view that posits an "objective world" as the obligatory starting point of our investigations, rather than as "only one of" the many possible results of our deep seated intentionality. (More on this later.)

The book reviews both the "algorithmic" (computer science) and "user-centered" (humanities) facets of NPR. I submit that it is fairly lopsided on the side of computer science while weak on the user-centered one.

However, I still recommend reading/buying this book, if only because of its presentation of much of what is under the hood of what we use daily, something that could help us understand more of what supports the applications we use every day. The book is also recommended for the glimpses it can provide (by its absence) into all that is missing in modern applied science in terms of what makes us "human." Whoever says "user-centered" is implying a great deal about both epistemology and ontology, and this implied understanding of the underlying structure of experience is what I find particularly weak here.

The Book

Chapter 1 reviews the history of NPR, and offers an overview of its potential relevance today and in the future. It does show how many graphics applications were designed at first with the goal of photorealism in mind, and reveals that these goals receded during the 1990s to be progressively replaced by other less limited/limiting intentions. As the books states in the introduction, "One major direction in which attention has shifted is to view photorealism as just one of many rendition styles."

It defines some criteria by which to measure the worth of NPR, such as conveying meaning, clarifying the relationship between language and pictures and offering new products and services.

Much of this points to the fact that in the image-making "business," choice of what to show or not is of the essence, the work of a painter or illustrator is above all to "present" much more than it is to "represent." In this sense, I fully agree with the book when it posits that NPR is actually capable of reaching further than mere photorealism, even if (especially when) it is time to convey critical information, be it of a scientific or artistic nature.

Chapters 2 to 5 deal mostly with pixel manipulation and two-dimensional data structure, including natural media simulation. Those who, like me, spend much of their day in applications such as Studio Artist, Painter and Expression may be very surprised at the "under the hood" complexity of "simple" operations such as (for example) drawing a line with a Wacom tablet (I was).

This book made me much more appreciative of the work performed by the authors of my favorite digital tools, and that alone is worth the price of admission. It also showed me why there are some serious limitations to creative work inherent in the very ground on which these software packages are built, something I will try to address later in this review.

Chapter 6 talks about "2 1/2 D Data Structure," looking briefly at adding depth to objects portrayed in an image while remaining in two-dimensional data structures. This includes G-Buffers, Interactive Painting, 3D Parameters for 2D Dithering, and more.

Chapters 7 and 8 enter the exploration of three-dimensional information as encoded in geometric models for non-photorealistic computer graphics. This deals with the description of and operations for polygonal models, edge classification for NPR (here too a "point of view" is essential, and a classification of the relative importance of edges is paramount. The differentiation of this from that implies much subjective choice behind the programming of these operations).

Chapter 7 deals with computing intersections, determining global shape and much more. Lighting, lighting models and much more are at the core of chapter 8.

Chapter 9 enters the minefield of distorting images and models, one area that is loaded with meaning for animators and painters alike. It is fascinating reading, riddled with technical information, and shows how much all of this is based on the intentional organization of information, itself structured ("constituted") by our priorities.

Chapter 10 discusses a variety of applications of non-photorealistic computer graphics, such as Non-Photorealistic Animation, Architectural Illustrations, Rendering Plants, Illustrating Medical and Technical Texts and Tactile Rendering for Blind People. Much food for thought for animators in this one. "Non-Photorealistic Animation Based on Particle Systems" is one section (10.1.2) I am particularly interested in these days.

Chapter 11 offers the book's conclusion and presents "a conceptual framework for binding everything together." Topics are: Methodological Disclaimer, Mathematical Preliminaries: Equivalence Relations, Equivalence Classes, and Quotients, Physical Preliminaries: Communication via Light Rays, Neurobiological Context: Look-Ahead Sets and Look-Around Sets, A Model for Visual Communication, and finally, Summary and Practical Connection with NPR.

Ontological Underpinnings

It is in this chapter that the authors honestly state that "our only purpose is to inspire software engineers to come up with new ideas for visual communication and to facilitate arguing about such systems. Therefore, we think that our frivolous and speculative attitude towards mathematics, biology, perception theory, and cognitive psychology is, from the point of view of design methodology, at least partially acceptable."

Accepting the above statement, I will therefore offer my serious criticism of the book's ontological underpinning in a spirit of good-willed criticism contributing the little I can to this important debate and this essential research.

The book is riddled with statements that seem to be rooted in a "naive belief in an objective world" which, unavoidably, limits the scope of the software the book's science helps design.

I will take only a few of the many (many) examples present in the book to illustrate that: In "Before and After Photorealism" on page 3, the authors refer to an illustration of an ancient Egyptian mural in these terms: "Note how the artist has taken the liberty to draw the subjects in a way in which they cannot possibly have looked."

I share Giacometti's view that "to the Egyptians, their work had nothing to do with 'Style,' they painted what they saw as best as they possibly could." We have to be careful, especially when looking at other cultures' artifacts, to not fall into this "imperialist view of reality" that posits other cultures' worldview(s) as "primitive" or at least as "lesser" than our own. It is only if we posit our superstitious belief in an "objective world "as the norm for all that we can state that an ancient Egyptian mural depicts reality in ways that could not possibly correspond to the artist's (and the viewers') experience of "the real."

In a later example (a 1569 drawing of the city of Nîmes), the authors say: "These examples show how artists, either consciously or unconsciously, have taken advantage of being able to define a 'point of view.'" They go on to say, "Drawing by hand, it is possible to free oneself from physical constraints of reality and to convey an impression rather than just convey details of a scene's appearance."

I therefore have to ask: can one see without a point of view?

Before we consciously (or not, though this is a red herring) can "define" a point of view, we are already experiencing a point of view: nothing of what we see is free of a point of view, each and every one of us has or rather is as Merleau-Ponty rightly says, "a brand new point of view on the world."

Isn't every perception, by definition, a subjective experience?

Again, Merleau-Ponty made it quite clear (at least to me), "Perception is constitutive."

This points to the stance of the viewer as being a constitutive "part" of the experience, yet the book's science acts as if the object of experience (the visible world) could be accessible bereft of a viewer's built-in "subjectivity."

Much research has shown that a slight shift in priorities (a shift in "intentionality") causes the patterns of scanning to be altered drastically, and one's focus on a pre-determined aspect of a scene can and does make one totally oblivious to some aspects of the scene that are "right in front of one's eyes." (This is just a minor example that sacrifices to the "objective-world" golden calf.)

Also, and this is fundamental in this research, the body can help us "cater to the appearing as it appears" in ways that reach far beyond what reflection alone can reveal. Artists know that to work under the control of the usual "look-understand-do" is limiting one's work to the "already known," while shifting to "look-do-understand" opens up much of what is usually limited by language.

To say, "Drawing by hand, it is possible to free oneself from physical constraints of reality and to convey an impression rather than just convey details of a scene's appearance" is to imply once again that "impressions" are merely subjective, and that one's perception is not. To "just convey details of a scene's appearance" is far from being a pedestrian accomplishment, it belongs to the core of the best each and every one of us has to offer, our ("my") brand new point of view on the world.

What this implies is enormous and could contribute greatly if integrated in one's work at the level of one's Weltanschauung to the building of better models of perception (and therefore user-centered software packages).

The Non-Photorealistic Computer Graphics book has a lot to offer (again, I repeat that I recommend buying/reading it), but it also leaves a lot to be desired.

How can we reach beyond the limitations of the 19th century worldview if, while we aim at non-photorealistic computer graphics, we situate all the work within a Weltanschauung that naively believes (and posits) that the "objective world" is "real" and presents itself to us along the lines of photorealistic images?

"The eye does not work like a camera, and we do not all see the same thing." (This is a running theme in my "Notes from the Underground" series published here on AWN.)

A viewer is not a passive subject. Perception is in (a) constant flux, a constantly changing experience-ing in which we continuously swing from "Where is it?" to "A-ha, there it is!" What one sees is a direct product of what one is, while what one is is also changed by what one sees: "We derive meaning from the experience while simultaneously projecting meaning into it" (Merleau-Ponty).

As far as I am concerned, much of today's animation is obviously trapped in a box, a box made by our naive belief in a 3D world containing solid objects moving about in empty space. I hope that NPR can help us free ourselves of those limitations, but I see only little help coming from this book toward accomplishing that goal.

However, the authors promise a sequel to their book, and I sure hope they will try to review its underlying structure in the light of a better integration of "experience as flux in/of an infinite world" and therefore give us keys to designing software packages that will enter a closer correspondence with the world of "lived experience."

Bringing Into Practice

In light of this NPR book, here are a few quick examples of what three affordable current off-the-shelves applications can do, applications that can help us reach far beyond photorealism.

First are four strokes done in Corel's Painter 7, the software that is still king when it comes to natural media emulation. It was Painter (then at version 4) that made my transiting from natural media to digital (for health reasons) possible.

Here we have a paper texture showing through the oil pastel stroke, followed by the same image being diluted with simple water, then one stroke with an Impasto brush and another. As you can see this is a very rich programme. (www.corel.com)

I used the above four strokes as "source images" for four quick images in Synthetik's Studio Artist. I consider this application to be a paradigm shift in 2D work, including in animation as it has many functions that work on a timeline. It is so full of features, it would be the application I would pick if I had to own only one. (www.synthetik.com)

Studio Artist is being developed all the time, with many new features being added by its author, John Dalton. It offers amazing possibilities for intelligent rotoscoping as well. Unlike any other graphics apps, Studio Artist has an intelligent painting engine which is based on a deep knowledge of cognitive science. It is amazing to click on the "Action" button and see it go to work!

The metaphor behind Studio Artist is the music synthesizer, there's simply nothing like it out there! (But, at this point in time, it is for Mac only.)

The last tool I will mention is an old favorite of mine as well (since version 1); one that has recently received a great upgrade, Creature House's Expression 3.

Unlike the previous two applications, this one is vector based. It is loaded with features which enable the user to work with vectors while enjoying natural media like strokes and more, all in a very intuitive environment. Creature House is soon to ship another application based on the Expression's skeletal strokes but this time, aimed at the animation market (with a timeline and a very intriguing "in-betweening" engine, and more). LivingCels should be released soon. (www.creaturehouse.com)

The first of these images is the starting path, followed by three different strokes applied to it. Expression offers glorious transparent layers, text on a path, natural media looking strokes, and much much more.

Non-Photorealistic Computer Graphics (Modeling, Rendering, and Animation) by Thomas Strothotte and Stefan Schlechtweg. San Francisco, California: Morgan Kaufmann Publishers, an imprint of Elesevier Science (USA), 2002. 470 pages with illustrations. ISBN: 1-55860-787-0 (US$64.95)

Jean Detheux is an artist who, after several decades of dedicated work with natural media, had to switch to digital art due to sudden severe allergies to paint fumes. He is now working on ways to create digital 2D animations that are a continuation of his natural media work. He has been teaching art in Canada and the U.S., and has works in many public and private galleries.