Screening the 2023 HBO Camera Assessment Series: A Decade of Testing

by James Mathers
Cinematographer and Founder of the Digital Cinema Society
(Excerpted from the August 2023 Digital Cinema Society eNewsletter)

The Digital Cinema Society was really gratified to have had the opportunity to share the HBO 2023 Camera Assessment Series with the L.A. area filmmaking community.  We organized three jam packed screenings sponsored by ZEISS Cinematography, one in the Paramount Theatre at Cine Gear Expo and two more in partnership with SMPTE Hollywood at the Academy’s Linwood Dunn theater.  The feature length production was never designed to be seen by the general public, or even the industry at large.  However, there are valuable conclusions that can be drawn by motion picture professionals when properly screened, and DCS jumped at the chance to help facilitate screenings when the HBO team informed us they wanted to share their results.

Projection has to be top notch in order to do the cameras justice for such critical comparisons, and proper evaluation also requires context.  Each screening needed to be followed by a Q&A which I was honored to moderate with the team from HBO who created the project. That included Director/DP Suny Behar, Executive Producer Stephen Beres, the SVP of Production Operations at HBO (WB-Discovery), and Erik Hansen, VP Media & Production Services at HBO (WB-Discovery).

DCS was chosen to help organize the screenings, with myself as moderator, due to our dedication to objectivity and trying to be what we refer to as “format agnostic.”  In that spirit, I didn’t want to previously share my thoughts publicly in order to let viewers draw their own conclusions.  However, many hundreds of image creators have now had the chance to view the tests and hear the filmmakers’ commentary and it looks like our participation has drawn to a close.  Also, given the limitations necessitated by the strict presentation standards and the fact that this material will never be available online, many more filmmakers who could benefit from the experience may never have the opportunity.  So, I think it is time for me to share some of my personal observations and conclusions after the many viewings I was lucky enough to take in. 

Basically, the HBO Camera Assessment Series, (CAS) is a project that seeks to document the performance of content creation tools that are in consideration on productions being produced in association with HBO, (now known as MAX, part of the Warner Bros./Discovery family of companies).  It is important to understand that it is not a “shoot out” to crown the overall winner, but instead a means to see which of these tools are best suited to various imaging scenarios so that filmmakers can choose the right tool for the job at hand.  

There are no charts, but instead intricately designed scenes with the same high production value that you might expect to see in HBO programming.  Each shot is painstakingly recreated for each of the imaging systems and edited to screen back-to-back, or occasionally with split screens, so they can be accurately compared. The color grade is basically a one-light based on ACES transforms and no special post tools such as noise reduction or power windows are employed.

Whereas Netflix tests various cameras in the market and then posts a spreadsheet online as to whether they are approved to acquire content produced for their network, the HBO approach is to be a little more accepting of various cameras and formats.  They of course have base technical requirements, but they put more emphasis on guiding their filmmakers to choose the ones that will best serve their creative and technical needs.

The first iteration of the CAS was created over a decade ago, with a new installment every year or two. CAS 2012 compared the ARRI ALEXA Studio, the Sony F-65, the RED Epic and Scarlet, the Sony PMW F-3, the Canon EOS C-300, as well as 35mm film, (which at that time was more of a mainstay for HBO productions). The test categories consisted of green screen and VFX, movement, available light, skin tone, dynamic range, and mixed lighting. In 2017, the cameras included the Panavision DXL, the Panasonic VariCam, the ARRI ALEXA 65, the Canon C700, along with the Blackmagic URSA Mini Pro, and the RED Weapon.  

The latest update, CAS 2023, Season 6, features five top of the line digital cameras, and again, 35mm film.  The digital cinema cameras include the ARRI ALEXA Mini, the Sony VENICE 2, RED V-Raptor, Blackmagic URSA 12K, and the new ARRI ALEXA 35.  While most of the cameras tested have been released within the last couple of years, Suny Behar explained that the ARRI ALEXA Mini was included again as a baseline.  Since the ALEXA Mini has been by far the most popular camera on HBO productions over the last decade, it makes a good comparative reference.   

This year’s tests included Skin Tone, Image Elasticity, Dynamic Range, a Color and Patterns test, and Available Light under nothing but moonlight.  Prior to, and interspersed with, the actual test footage are short interview clips with the HBO CAS Team explaining their process as well as a few Time Warner executives and several of their filmmaking collaborators including ASC DPs Rodrigo Prieto, Todd Banhazl, and David Klein among many others.

Film had been left out of several seasons but was added back this year due to renewed demand from filmmakers requesting to use it on HBO productions.  However, Stephen Beres raised the ire of some in the audience when he said in one of his on-camera interview clips that film was now a “specialty” format.  He was careful to explain in the Q&A following the screening that his determination was in no way meant to disparage film as a format and he doesn’t discourage its use.  Nevertheless, until recently with shows such as Euphoria and Winning Time, not too many HBO productions were considering shooting on film.  And although any cost differences are, in his words, “a rounding error” on major HBO productions, it can be more challenging to support a film project, if only because there is not as much film production happening, which has caused industry infrastructure to shrink.  He gave as an example that they had trouble rounding up raw stock while on location in Australia and needed to ship the footage back to one of the only film labs still operating in the world, FotoKem in Los Angeles.    

During the Q&A, I tried to bring out the great lengths the filmmakers went to in order to be fair to each camera system and format, especially film. The reason film capture was a sensitive subject for many in the audience was because it honestly did not perform too well whenever low light sensitivity was a factor.  In the available light tests, which were literally exposed with nothing but moonlight, there was almost no image at all, even though it was force processed to the equivalent of ISO 2000.  Film also did not fare too well in the Image Elasticity test where a cutaway of a two story house simultaneously showed a brightly lit kitchen, an averagely exposed living room, a dimly lit den, and an even darker attic.  Each camera system was then shown both over and under exposed by up to 4 stops. The bright kitchen looked fine, but darker rooms got pretty noisy on film compared to the other cameras when starved for light. This, in spite of the fact that ND was added to the digital cameras in order to bring them down to match base exposure and ISO rating of film.

Film, however, looked great in the skin tone test and also reacted pretty well to the mixed lighting test, where the camera weaves through an intricate indoor/outdoor, (and even underwater,) sequence lit by everything from a warm tungsten practical, uncorrected fluorescent, mercury vapor, neon, HMI, and blacklight.  And this gets me to my main takeaway from the 2023 CAS, that is to use the appropriate tool for each scenario.  All the cameras performed well and according to the HBO team, any could be used as an A-camera on one of their productions.  However, if what you have to capture does not allow for good control of the lighting environment and you need to shoot available light at night, you might want to think twice about using film. 

That is not to say that you can’t mix formats within a project; with the quality of modern digital cinema cameras and ever advancing post production abilities, there is no reason that a variety of cameras cannot be perfectly matched from scene to scene.  In fact, that is what the HBO team recommends to many of their collaborators. They have found that jumping to a camera like the RED Raptor when needed for its low light abilities, even though that is not the preferred camera on the rest of the show, has worked beautifully on Game of Thrones.  It has allowed them to cut their lighting budget and expand their shooting day when losing daylight.  

Another great example of creative mixing of formats is the HBO series, Winning Time: The Rise of the Lakers Dynasty, where they used practically every format in the book to achieve their unique and impressive look. They shot mostly on film, 35mm and 16mm, but also mixed in Super8, as well as some standard def video tube cameras of the day like the trusty old Ikegami HL79 to authentically reproduce archival footage.  

When a camera like the Blackmagic URSA 12K costs roughly one-tenth of what some of the others do, maybe you want to choose it when you have a tight budget, but need lots of cameras on hand, or want to use it when the camera is required to be more expendable.  Not to diss the Blackmagic, but it would make a more logical crash cam than the ALEXA 35 or the VENICE 2.  You might also want to choose the 12K URSA if you need the extra resolution for reframing in post, stabilization, or for virtual production background plates.  And from my perspective as an Owner/Operator, purchase price is a definite consideration.  While the ALEXA 35 and VENICE 2 were clearly in a higher class, the differences were pretty subtle, and there was no beating the RED Raptor when it came to low light performance.  

So as my late friend Geoff Boyle was fond of saying, “different horses for different courses.”  Which is to say, pick the right tool for every job.  I think we all owe the team from HBO including Stephen Beres, Suny Behar, and Erik Hansen, a huge debt of gratitude for sharing this valuable documentary with the community.  DCS would also like to thank ZEISS Cinematography, whose lenses were used exclusively on the project, for their financial support of the screenings. A special thanks is also due to Abigail Hall of Warner Media, who is Executive Assistant to Stephen Beres, and Bill Hogan of SMPTE Hollywood for going above and beyond to help coordinate our efforts, as well as Tim Sassoon and Enrique Del Rio for photos documenting the screenings.

The 2023 HBO Camera Assessment Series is an extremely valuable presentation that can help to inform what capture device and format are best for the various real world production scenarios we face.  I know it will inform my choices going forward, and to anyone who has not yet had the chance to see it, I highly recommend you do.  DCS will post details of any upcoming screenings we find out about.  Although we are not involved, we are currently aware of one in Potsdam, Germany and another in NYC as part of NAB Show New York:

CAS 2023 Screening in Potsdam, Germany September 15, 2023:

https://www.eventbrite.de/e/hbo-camera-assessment-series-2023-tickets-689272531697?aff=oddtdtcreator

CAS 2023 as part of NAB Show New York, October 25, 2023:

https://nabny23.mapyourshow.com/8_0/sessions/session-details.cfm?scheduleid=93