In this month's edition of "The Digital Eye," Seth Rosenthal of Tweak Software discusses the significance of its first commercially available package, RV Playback.
Image courtesy of Deron Yamada. © 2004 DYA367.
The RV image and sequence viewer for vfx and animation production is Tweak Software's attempt to improve on one of the most basic tasks in computer graphics production -- looking at digital images. Tweak developed RV under the pressure of two strong influences: first deep experience with advanced CG pipelines and CG software development, and second, the intense challenge of building a small high-end vfx company from the ground up with extremely limited resources. The result is a very focused playback tool for digital artists that can be extended and customized for integration into proprietary pipelines.
High-end computer graphics production has always depended on extensive in-house software engineering. But contrary to the general perception, only a small part of the so-called R&D at leading facilities is focused on cutting-edge computer graphics research or creating breakthrough technology for the next big effect. The biggest return on in-house software development is the efficiency that results from optimizing the integration between commercial tools, in-house software, and computing resources. The important breakthroughs often come from figuring out how to make new techniques cost-effective on a large-scale within the constraints of a given pipeline. The most remarkable (yet un-remarked) accomplishment in CG production today is the astonishing ability of a few leading facilities to generate a massive volume of very difficult CG work. The key to this ability, though, is an elaborate division of labor among talented specialists combined with an ongoing process of pipeline refinement.
Overcoming Pipeline Entanglement
One of the quirky consequences of continual pipeline polishing is that the idiosyncratic refinements to homegrown tools take on very significant economic and practical value. For a widely used tool the integration value can easily outweigh the potential advantages of new, more sophisticated off-the-shelf products. This situation can be endlessly frustrating to vendors who struggle to understand why the most advanced facilities are sometimes indifferent to their latest products. Inevitably, facilities reach a point where they are held hostage by old tools that consume precious R&D resources in spite of offering no competitive advantage other than being deeply entangled in the pipeline.
The flipbook player is a good illustration of this phenomenon. There is great practical value of having a customized, tightly integrated playback tool because it touches virtually every corner of the pipeline and every CG discipline. As a result, homegrown playback tools tend to accumulate all kinds of facility specific features and inspire fierce artist loyalty. This makes them difficult to replace in spite of the relatively low cost of commercial tools.
Tweak developed RV so as to circumvent the problem of idiosyncratic entanglement: RV is an open, customizable playback platform that separates the user interface and control logic from the core multi-media technology. The source code for RV's UI is distributed with the program and the RV session file format is the open source GTO format. This makes it possible for facilities to tightly integrate RV into existing (or new) pipelines and retain the ability to make valuable pipeline -- or workflow-specific -- modifications, while relying on Tweak to maintain and extend the core technology and to keep up with changes in hardware and operating systems.
Seth Rosenthal.
Focus on the Basics
RV's design philosophy is a natural consequence of Tweak's approach to CG production and software engineering. When Christopher Horvath and Jim Hourihan founded Tweak Films after years immersed at ILM, they faced the prospect of taking on advanced, technically demanding vfx production work with no infrastructure and very limited resources.
Tweak approached this challenge by keeping a narrow, almost radical focus on the most basic requirements of the CG production pipeline. Rather than think about what tools to buy, they concentrated on fundamental issues like: how to store data, how to look at images, how to define shots, how to control procedural effects, how to build software that can be re-purposed and re-configured. The result was a collection of modular, scalable components that enabled a small crew of artists to handle high-throughput, advanced CG production tasks. Among other successes, this approach helped Tweak become the only small shop to ever receive the VES award for Best Single Visual Effect of the Year (shared with Digital Domain for work on the New York tidal wave sequence in ).
Many of the technologies that emerged from Tweak became part of the foundation for RV, these include:
The GTO file format for efficient storage and exchange of CG data, along with C++ and Python tools (now open-source and used by many CG facilities). GTO was designed as an optimal format for caching and exchanging geometric data. It served as the backbone of Tweak's pipeline, and has also become the format for RV's session files.
The Mu programming language for computer graphics. Mu is a long-term project of Jim's that was extended and widely used at Tweak. Mu is a high-level multi-purpose programming language for graphics.
A C++ application framework for computer graphics tools with built-in Mu scripting.
Extensive C++ libraries for image processing.
RVLUT, a kind of DI self-defense tool that solves for 2D or 3D LUTs from before and after versions of sequences.
Early versions of RV playback software and related tools for looking at images, converting image formats and publishing dailies with slates.
Early versions of RV caught the attention of friends and former colleagues in the industry and Tweak recognized that there was widespread desire for better playback tools among veteran CG artists. Tweak decided to re-engineer RV from the ground up and build a lasting, cross-platform playback system suitable for replacing the homegrown tools at the leading facilities and providing refined, customizable playback capability to smaller or newer facilities.
RV's core is a sophisticated, cross-platform, hardware accelerated compositing engine that makes aggressive use of graphics hardware. Courtesy of Seth Rosenthal.
RV 3.4
Tweak's focus on the basics and deep experience with pipeline development led to RV 3.4, which was released by Tweak Software on June 25. RV's core is a sophisticated, cross-platform, hardware accelerated compositing engine that makes aggressive use of graphics hardware. RV is heavily multi-threaded and it adapts to the hardware to maximize performance. RV's core can handle multiple color corrections (separate file, display and look LUTs), image transforms (gamma, exposure, hue, log-lin, etc.) and compositing operations (over, add, difference, tile, etc.) simultaneously in realtime. RV can work with film resolution imagery (including IMAX or larger), has extensive support for floating point and HDR imagery and also has native support for stereoscopic 3-D.
RV's user interface is developed entirely in the scripting language. This includes all of RV's menus, keyboard and mouse bindings and heads-up widgets as well as all of the logic that controls RV's behavior. Source code for the UI comes with RV and can be studied and modified by users. The scripting language controls RV through two basic mechanisms: first, by sending commands to the core library and second, by setting properties of the nodes in the RV Session DAG. It can also be used to read and write files, access the file system, get environment variables, make system calls and even draw directly on RV's display using OpenGL.
A powerful example of how RV can be customized is the source_setup module. Each time a sequence or image is loaded into RV, a source_setup function is called. This function can be customized to arbitrarily configure RV based on the image attributes, environment variables, or any other available information (perhaps a production database). This gives facilities an opportunity to exert very fine control over the playback and review of imagery. For example, LUTs or other color settings could be applied as a matter of policy based on show, sequence, shot or discipline. Compositors could have different default behavior than TDs. Dailies playback could be handled distinctly from desktop playback.
In a more extreme example of RV customization, one facility created tools using OpenGL to load 3D matchmove geometry and camera moves into RV and render them over background plates. More common modifications include creating custom menus and commands, generating RV session files from in-house browsers or editorial tools, and creating custom heads-up widgets to display particular image attributes or meta-data.
The Best of Both Worlds
We recognize from our background with both industrial-scale and boutique CG production that the pipeline is always more important than any single tool. It is a losing proposition for vendors to enforce hard-coded workflows in basic CG production tools. RV encapsulates this hard-won experience in the form of an artist-focused, open, customizable, playback tool that facilities can tailor to their needs. RV is just the beginning for Tweak Software. We are hard at work on the next version of RV and on new artist tools that embody Tweak’s philosophy of software design.
Seth Rosenthal is co-founder and president of Tweak Software. Prior to Tweak, Rosenthal spent seven years at Industrial Light & Magic where he supervised a team that made innovative contributions to visual effects filmmaking -- including projects that introduced new technology under the pressure of principal photography for directors such as Steven Spielberg and Ang Lee. He supervised projects including: motion capture (The Day After Tomorrow, Star Wars: Episodes I and II, The Mummy), Realtime virtual set integration with first unit photography ( Artificial Intelligence: AI), multi-camera photography for 3D reconstruction (Lemony Snicket, Minority Report, Pearl Harbor) and Photogrammetric set reconstruction and high dynamic range photography (Hulk).
