Simon Tindemans

The scene-referred workflow - a proposal

First published 25 January 2008; updated 25 May 2008; reformatted and shortened August 2012.

Many photographers claim that they want their photographs to look as realistic as possible. Taken literally, this implies a scene-referred image (explained in more detail below). Unfortunately, scene-referred images tend to look washed out and dark on displays and print, due to differences in contrast range and brightness. Hence, scene-referred images are commonly dismissed as no more than a technical curiosity.

In this essay I will argue that whilst scene-referred images are not very suitable as a photographic end product, they can fulfill a valuable role as an intermediate stage in the image processing workflow. Amongst other benefits, scene-referred intermediate images provide an effective way to 'insulate' the photographer and his/her style from the constant changes in digital camera equipment. Also, the use of scene-referred images suggests a natural workflow for HDR imaging and panorama stitching. Below, I describe the proposed workflow and a detailed list of potential advantages.

Scene-referred images

A typical outdoor scene encompasses a large range of brightness and color. The contrast range can easily reach 10,000:1 and some of the colors can be extremely saturated. That alone means that we cannot hope to reproduce this information on a typical print with a contrast range of approximately 200:1 and a much more limited color palette.

Traditionally, most photography has been destined for a specific output medium (print or projection). Because of this, photographers are used to target their work directly to this output medium and make the necessary adjustments at an early stage in their workflow. For example, increasing the development time of a negative film would boost the contrast of the prints. The digital equivalent of this process is the use of output-referred images: images that are processed from the beginning with a specific output in mind.

Contrast this with scene-referred images. A scene-referred image reflects the relative brightness differences and colors as accurately as possible, even if they cannot be reproduced on the targeted output medium. For most photographers, scene-referred images have been nothing more than a curiosity. The reason for this is that most scene-referred images simply cannot be represented accurately on a monitor. Because an 18% brightness level is considered medium grey, the monitor only has approximately 2.5 stops between grey and white. Many scene-referred images require more highlight headroom to be viewed properly. To illustrate this point, it may be worth pointing out that high dynamic range (HDR) images are also scene-referred, at least before they are tone-mapped to an output-referred image.

For more information on scene-referred images, see this white paper by the ICC. For those interested for a more thorough discussion, I highly recommend the following Adobe white paper by Karl Lang: Rendering the Print: the Art of Photography.

A workflow proposal

By now, it will have become clear that a scene-referred image is not desirable as a final product. However, it is nevertheless the most accurate representation of the colors and brightness distribution in the original scene. This property makes it attractive as an intermediate point in the workflow, for reasons I will describe below.

The scene-referred image is neither what comes out of the camera nor is it the final product. Instead, it can be used to naturally divide the image processing workflow into two distinct stages. The first stage is what I will call the scene reconstruction stage. This step is concerned with constructing a scene-referred image from the unprocessed data. Effectively, this removes the 'fingerprint' of the camera as much as possible. This is followed by a second stage: the creative processing. The following image illustrates the proposed workflow.

The scene-referred workflow and its two stages: scene reconstruction and creative processing. Also listed are the typical tasks one would perform in each stage.

Advantages

Splitting the workflow into a scene reconstruction and a creative processing step offers a number of advantages:

  • Scene reconstruction is performed only once The scene reconstruction step is independent of the output medium, so if an image is to be targeted to another output medium, this step does not need to be repeated.
  • The scene-referred image can satisfy objective quality standards There are (fairly) objective criteria for what constitutes an optimal scene-referred image. It can therefore be left to hardware and software vendors ('the engineers') to aim for the best possible image in this stage.
  • The scene-referred image is as realistic as it gets In news photography, trustworthiness of the image material is of the utmost importance. It is thinkable that a photojournalist would send along a scene-referred copy of every photo as a reference.
  • A common standard enhances interoperability By using a scene-referred image with its predictable properties as a central node in the workflow, it becomes easier to mix and match programs that 'understand' scene-referred images: HDR, panoramas and noise reduction are a few applications.
  • The creative processing stage decouples the photographic style from the camera Because the scene-referred image is essentially determined for a given input, all the things that define a photographers' processing style are condensed into the creative processing stage. The camera - in combination with the scene reconstruction step - thus becomes 'transparent' to the style. This has the additional advantage that photographers are no longer at the mercy of their camera manufacturer's product revisions for the look of their work.

Note that this proposal has analogies with the three-step sharpening process that is steadily gaining ground. However, the scene-referred workflow makes no distinction between the creative and output stages, because the properties of the output medium also define the boundaries for creative processing. In the specific case of sharpening the two are more or less independent, allowing one to separate the two steps.

Scene reconstruction

The scene reconstruction stage is the first step in the proposed scene-referred workflow. It's purpose is to remove the 'fingerprint' that the camera has left on the image. Obviously, there are limits to this process. For example, one cannot recreate detail that has not been resolved in the first place, or restore color nuances that have not been not registered. The output of this stage, the scene-referred image, should ideally contain all the information that is needed to recreate the same visual stimulus of the original scene - assuming a perfect output medium.

The process of undoing the distortions the camera has imposed on the captured scene naturally suggests a number of processing steps. Most of them are the usual controls that you will find in a RAW converter. Note, however, that tonal corrections are absent at this stage. Also, some operations that are usually done in third-party software, such as HDR merging and panorama stitching, are listed as part of this processing step, because they assist in the recreation of the original scene.

  • Bayer demosaicing
  • basic noise reduction
  • highlight recovery
  • lens correction
  • HDR merging
  • panorama stitching
  • white balance (to compensate for adaptation of the eye)
  • color correction (to a calibrated standard)

Note that these are all technical processes that benefit from a scientific/engineering approach. In addition, these steps can all be performed with relatively little manual intervention. The scene reconstruction step is therefore well suited to batch processing by specialized software.

Creative processing

The creative processing stage is concerned with transforming the accurate-but-bland scene-referred image into an appealing final image. Creativity is often bounded by the limits of the output medium, so the properties of the output medium (dimensions, contrast range, color gamut, viewing surround, etc.) generally have to be taken into account. For obvious reasons, there is no well-defined list for the things that should be done in the creative processing stage, but the following list of steps would be a good start.

  • cropping
  • tone mapping
  • compositing
  • selective sharpening
  • color manipulation (including B & W conversion)

The scene-referred workflow in the real world

The above may seem like an academic discussion on the benefits of an imagined process. To a certain extent that is true. I think that the two-stage process provides a good conceptual framework for developers and users that like to know the detailed properties of their tools. However, there are some real world applications where a scene-referred workflow makes sense right now. Here is a short listing of the cases in which I personally use this workflow:

  • Circumventing the saturation-affecting tone curves of common RAW converters Most tone mapping (curve) tools affect the saturation of an image in a complex way.  In order to satisfy my personal workflow needs and curiosity, I have created the Tonability plug-in for Photoshop that resolves this problem by replacing the standard tone mapping (curve) tools.
  • Combining multiple exposures into an HDR image If the individual files have already been tone-mapped, this complicates tonal relations in the merged HDR file. In addition, the saturation side-effects of the individual tone curves become very hard to correct for. It is easier and more consistent to combine different scene-referred images and perform the tone mapping only once, directly on the HDR data.
  • Panorama stitching When combining a number of RAW photos into a panorama this can lead to a quandary. You need to develop the RAW photos before you can hand them off to the stitching software, but you often don't know what settings to use, because you don't have the overview. This means you have to resort to a two-pass approach, which naturally fits the scene-referred workflow. I first create 16-bit scene-referred images with matching exposures, white balance and capture sharpening. Then, I stitch them together and re-import the result into Lightroom. At this point, I can use the exact same tone curve settings as for an ordinary RAW image.

This brings us to the problem of obtaining a scene-referred image in the first place. Most RAW processing software available today aims to be an all-in-one solution that takes you from the RAW file to the print. These programs mix and entangle elements from the scene reconstruction and creative processing steps described above. However, there are three RAW converters that I know of in which all of the creative processing steps can be bypassed in order to generate a scene-referred image. These three programs are: Adobe Camera Raw (used in Photoshop, Photoshop elements and Lightroom), LightZone, dcraw and UFRaw (a front end to dcraw). Below, I will show the necessary settings for Adobe Camera Raw, Lightroom and LightZone. I have no first hand experience with dcraw or UFRaw.

Conclusions

I have provided a number of arguments for the adoption of scene-referred images as an intermediate step in the digital imaging workflow. Such a workflow allows for a clean division of the image processing steps into a scene reconstruction stage and a creative processing stage. Besides its appeal as a conceptual tool, this approach has several real world advantages, for example in HDR creation and panorama stitching. Basic tools that make such a two-stage approach possible are currently available.

Due to limited software support, a scene-referred workflow is at the moment mostly relegated to low-volume specialist and evaluation uses. However, this should be sufficient to get a feeling for the possibilities and advantages. Native support for a scene-referred workflow in future software could make these advantages available to a broader audience.

 

About Zenfolio