Raising the Bar: The Work Being Done to Improve Motion Picture Image Quality and Ensure Content is Viewed as the Filmmakers Intended
The increasingly rapid technological advancements in movie making are both exciting in their potential for new creative opportunities and challenging for those whose job it is to make it all work seamlessly. Delivering the images we create on the set at the highest possible quality levels, while ensuring the creative intent of the filmmakers, is ever more complex, especially with new delivery options like HDR and higher resolution displays, along with a plethora of new formats and shrinking production and post schedules. While everyone likes their secret sauce, industry standards and testing are vitally important to avoid total chaos. This essay is designed to explore some of the behind-the-scenes efforts within our industry to both create and help meet quality and interoperability benchmarks that will help to meet these challenges and protect creative intent.
Let’s start with the folks at Netflix. I was prompted to think about the work they’ve been doing in the area of technology standards when posting a recent news item about how a relatively inexpensive camera is now “Netflix approved.” Then when a longtime DCS member asked where he could find generally accepted distribution delivery specs, I again thought of Netflix and their efforts to not only create these benchmarks, but also to educate the community about how to meet them.
These efforts began in service of a better user experience for Netflix subscribers, and the company has since devoted considerable resources to creating specifications for such technologies as higher resolution capture and display, as well as HDR. In fact, their production and delivery requirements have become the de facto standard for high end digital production. In doing so, they helped to greatly accelerate the transition from HD to 4K, as no producer wanted to create a project that would not be capable of being carried someday by Netflix.
There was some push back along the way, however, mostly from Cinematographers who didn’t want to be told they couldn’t use their favorite digital cinema camera because it did not technically acquire in 4K, even though the camera had proven itself on hundreds of theatrical features. Yes, I’m thinking of the ARRI Alexa, but that controversy seems to be behind us now that ARRI has come out with several models capable of 4K resolution and techniques were devoloped to allow images from the legacy cameras to be used.
Netflix’s current programs include efforts like the Partner Help Center, which is a clearing house for such information, and where I referred my Producer friend looking for current delivery specs. They have also organized Netflix PTA, (Production Technology Alliance,) a program for manufacturers of products that generate or manage any kind of sound data, image data, or metadata from the entire production landscape, from pre-visualization to preservation. Any product that meets the established criterion is a candidate for the Netflix Production Technology Alliance logo. More recently, the company has joined an industrywide effort to establish a common vocabulary across professionals working in Virtual Production known as “The Virtual Production Glossary.” (All of the aforementioned information is freely available online, and links are available following this essay.)
However, Netflix is not alone in their mission to improve the creation and delivery of motion picture imagery. Many other entities are also working tirelessly to advance quality standards and make sure images are seen as the filmmakers intended. These include the ASC, the SMPTE, the Motion Picture Academy’s Science and Technology Council, the International Organization for Standardization (ISO), the UHD Alliance, and Dolby, among others.
HBO is another content provider that has done a lot of work in this area. They were testing cameras and workflows back when Netflix was still stuffing DVDs into the mail. Although their approach is different from Netflix, they also devote considerable effort to helping their content creator partners achieve the best possible quality and make sure their productions are seen as the filmmakers intended. Over the last couple of decades they have regularly produced The HBO Camera Assessment Series, in which they have painstakingly evaluated the best digital cinema cameras along with film capture so that they are ready to advise their partners on the best acquisition and post tools for each job.
With the knowledge gleaned from these camera assessments, Stephen Beres, SVP of Production Operations for HBO & HBO Max, along with his team, begin to advise their filmmaker partners as soon as an HBO project gets the green light. They do have some loose guidelines, such as no more than 20% of an HBO program should be Standard Def, but they are just not as concerned with tech specs as their colleagues at Netflix. Rather, they concentrate more on which format is best to tell the story.
A good example is the outstanding HBO series, “Winning Time: The Rise of the Lakers Dynasty,” where they used practically every format in the book to achieve their unique and impressive look. They shot mostly on film, 35mm and 16mm, but also mixed in Super8, as well as some standard def video tube cameras of the day like the trusty old Ikegami HL79 to authentically reproduce archival footage. If you haven’t seen the show, I highly recommend you check it out and you’ll see how they seamlessly integrated these diverse formats and managed to create a visual time machine in which to tell their story.
Of course, the Society of Motion Picture and Television Engineers (SMPTE), a global professional association of engineers, technologists, and executives working in the media and entertainment industry, have been creating and publishing technical standards for broadcast, filmmaking, digital cinema, and audio recording for over one hundred years now. They have published over eight hundred standards on everything from the size of the celluloid sprocket holes, to frame rates, time code, and the color bars we use to set up our monitors. As my friend Jim DeFilippis, a former Chair of SMPTE Hollywood Section, and owner of consulting firm, Technology Made Simple, likes to say, “Standards are very important…that’s why there are so many of them.” Jim also spent several years as the EVP of Digital Television Technologies and Standards for 20th Century Fox, so he is not one to make light of the work. Instead, he makes the point that technological progress would not be possible if such standards were not constantly evolving.
The American Society of Cinematographers is another organization that is over one hundred years old with a long history of efforts to shape motion picture technology. That work has grown exponentially since the transition from film to digital started over twenty years ago. Around that time, Cinematographer Curtis Clark, ASC took on the duties of Chair of the newly established ASC Technology Committee with the goal to give cinematographers a strong voice in the development of motion picture technology. This included collaborating with other organizations working to set standards, not just in production, but also postproduction, delivery, theatrical exhibition, and home display.
An early effort of the Technology Committee was working with DCI, (Digital Cinema Initiatives,) a consortium of the major studios tasked with setting standards for digital cinema projection. In 2005 the committee helped to create film and digital elements for DCI’s Standard Evaluation Material (StEM), a 12-minute production shot on 35mm and 65mm film for vendors and standards organizations to test, evaluate, and compare image compression and digital projection technologies. The idea was to come up with a standard for nascent digital projection technology that could try to match film print projection.
Another major contribution to this mission in 2009 had the ASC Technology Committee working alongside the PGA, (Producers Guild of America,) to produce their first Camera Assessment Series, putting a range of digital cameras to the test in order to compare against each other as well as 35mm negative. This was not so much a “shoot out” as a way for filmmakers to ascertain the strengths and weaknesses of each system so they could pick the right tool for each project. The Committee again partnered with the PGA on the Image Control Assessment Series in 2012. This project examined file-based workflows and color management. Another milestone was the development of the ASC CDL, (Color Decision List,) to enable consistent color correction data to be passed from the set to dailies and on to editorial post and final color.
Curtis Clark, ASC still leads what is now known as the ASC MITC, (Motion Imaging Technology Council,) where he continues to do the good work of trying to insure that the Cinematographer’s vision is carried from the set all the way through to the final delivery on the screen, whether that be in a theater or in the home. Recent efforts include the ASC MHL, (Media Hash List,) a system for verifying file integrity for data storage and transport, and the ASC FDL, (Framing Decision List,) a system to accurately preserve the filmmaker’s framing intent, a truly complex challenge due to all the various aspect ratios now extant.
Another Motion Imaging Technology Council project is StEM2, which was recently completed as a 17-minute short film called “The Mission” that is available for open industry access, (a link also follows this essay.) Whereas the original StEM was more of a film acquisition reference for theatrical exhibition, the latest edition is more specifically focused on digital imaging for both theatrical projection and emissive displays. StEM3 is now in the planning stages and will provide reference image material for Virtual Production and in-camera VFX calibration when working with LED volumes.
The ASC was also one of many groups assisting the Academy of Motion Picture Arts & Sciences on the development of ACES, (Academy Color Encoding System,) an industry-developed platform for color management and digital image interchange. The team at the Academy responsible for such efforts is known as the Science and Technology Council which is led by industry technology veteran Andy Maltz. Maltz has been working since 2003 to protect the interests of filmmakers. Besides ACES, he is credited as Co-author of The Digital Dilemma and The Digital Dilemma 2, the Academy’s examination of the challenges associated with the long-term preservation of digital motion pictures, and he again worked with the ASC on the SSI, (Academy Spectral Similarity Index,) to better understand and catalog the color rendition properties of various new light sources such as LED.
Andy Maltz also serves as Chairman of the International Organization for Standardization (ISO) Technology Committee 36 Cinematography. To be honest, the only time I had ever heard of “ISO” was as a rating for sensitivity to light of either film or a digital sensor. However, while this acronym is in fact a reference to the International Organization for Standardization, they do far more than define camera sensitivities; they also promote universal standards for measurements of all different types, on an international level. The Technology Committee 36 Cinematography helps set international standards for the motion picture industry codifying and giving international credibility to any standards that are adopted by the group.
I’m not sure it is proper to refer to new motion picture technology as a “can of worms,” but HDR, High Frame Rate, and Wide Color Gamuts, have certainly opened up a lot of new issues with regard to standards and the continuity of the viewing experience. For better or worse, more and more entertainment content is being consumed in the home rather than in a more controlled theatrical environment. There are a myriad of displays viewers might be watching the content on, anything from an iPhone to a 4K OLED with HDR. Many of the consumers’ displays do a lot of their own processing, sometimes boosting the peak luminance and raising black levels, or adjusting for a super bright part of the image, which can drive the rest of the picture down into complete blackness. Colors, highlights and shadows bleed together, so detail is even harder to see.
Part of the problem is that there is not a good standard for consumer display setup, it’s whatever looks good on the retail showroom floor, which is usually too bright with oversaturated colors. Unfortunately, that is what sells TVs and a large number of consumers leave their new sets in the factory default. When you add in the various motion interpolation schemes created to reduce motion blur for content such as sports, you’ve got a real mess. This has the very unfortunate side effect of making even the highest quality cinematic 24 fps narrative content look like a soap opera shot on a cheap video camera. With these many factors varying from screen to screen, how can filmmakers ever hope to deliver consistent quality? The good news is that there is work afoot to try to harness the power of metadata delivered along with the entertainment content to offer these abilities and display the creative intent of the filmmakers in the best way possible given the individual consumer display’s capabilities. Dolby, for one has developed a system to encode content so that it can be viewed correctly on displays, which they are campaigning to have manufacturers include on new TVs. Another such effort is being mounted by the UHD Alliance who have proposed an HDR10+ set of protocols.
They have also been promoting something called “Filmmaker Mode,” which will try to emulate as closely as possible what the filmmaker is seeing on set and in the DI suite. The display setup will be completely controlled by metadata without the ability to be overridden by the consumer. Of course, viewers can always turn this mode off if they don’t favor the creative intent of the filmmakers, but it may help avoid the whacked out settings that sometimes result from the less technically minded among us trying to noodle with the picture on their TV. Michael Fidler, the President of the UHD Alliance at the time, made a presentation on Filmmaker Mode at our last DCS Post Expo. This presentation includes interview clips from a host of notable filmmakers who are supporting the effort, including Martin Scorsese, Christopher Nolan, Ryan Coogler, Rian Johnson, Paul Thomas Anderson, Patty Jenkins, James Cameron, J.J. Abrams, Ava DuVernay, Judd Apatow, Ang Lee, Reed Morano, ASC, and the Duffer Brothers. You can view it here: https://vimeo.com/373226636
All the work I’ve described does not end when the content is released. With the rapid evolution of production and display technology it is important to make sure the filmmaker’s intent is included, archived, and translated for future consumption. The aforementioned entities continue the painstaking efforts to protect and preserve our images as we intend them to be seen now and in the future. Our industry owes them a large debt of gratitude and appreciation.
Previously referenced links:
Netflix Partner Help Center: https://partnerhelp.netflixstudios.com/hc/en-us
Virtual Production Glossary: https://www.vpglossary.com/
StEM2 (A short film, directed, produced and written by Jay Holben and Executive Produced by Curtis Clark, ASC for the American Society of Cinematographers Motion Imaging Technology Council. The StEM2 “The Mission” is intended for the evaluation and testing of theatrical projection systems (traditional lamp, laser and emissive screen), professional and consumer monitors and post image-manipulation software: https://www.youtube.com/watch?v=5bnZXQj1aeM
StEM2 Behind the scenes: https://theasc.com/asc/stem2