recovering color from black and white sco590/ color from black and white ... information from black...

Download Recovering Color from Black and White sco590/ Color from Black and White ... information from black and white photographs in ... Both the computer graphics and photography communi

If you can't read please download the document

Post on 06-Feb-2018




2 download

Embed Size (px)


  • Recovering Color from Black and White Photographs

    Sven OlsenUniversity of Victoria

    Rachel GoldUniversity of Victoria

    Amy GoochUniversity of Victoria

    Bruce GoochUniversity of Victoria


    This paper presents a mathematical framework for recover-ing color information from multiple photographic sources.Such sources could include either black and white negativesor photographic plates. This papers main technical contri-bution is the use of Bayesian analysis to calculate the mostlikely color at any sample point, along with an expectederror value. We explore the limits of our approach usinghyperspectral datasets, and show that in some cases, it maybe possible to recover the bulk of the color information inan image from as few as two black and white sources.

    1. Introduction

    The modern tradition of photography started in the 1820s.Before Kodaks introduction of Kodachrome color reversalfilm in 1935, color image reproduction was a laboratorycuriosity, explored by a few pioneers such as Prokudin-Gorskii [1]. Tens of thousands of black and white plates,prints, and negatives exist from the hundred year time pe-riod between the birth of modern photography and thewidespread introduction of color film [15]. Currently, it isonly possible to create color images from early black andwhite data sources in those rare occasions when the originalphotographer contrived to create a color image by takingmatched exposures using red, green, and blue filters.

    In this paper, we explore how variations in spectral sensitiv-ity between different black and white sources may be usedto infer color information to within a known margin of error.We assemble a toolkit that scholars can use to recover colorinformation from black and white photographs in historicalarchives.

    The papers main technical contribution is the use ofBayesian analysis to calculate the most likely color at anysample point, along with an expected error value. This anal-ysis requires some knowledge of the types of light spectralikely to occur in the scene.

    We believe that, for many antique photographs, the intro-duction of accurate color information will greatly enhancetheir visual quality. Color recovery may prove invaluablein understanding archived materials of historical, cultural,artistic, or scientific significance.

    2. Related Work

    Both the computer graphics and photography communi-ties have long investigated techniques for colorizing blackand white photographs. Previous research in the computergraphics community proposed adding color to black andwhite images by either transferring color from a similarcolor image to a black and white image [13, 17] or usedcolor-by-example methods that required the user to assigncolor to regions [7, 14].

    Colorizing old black and white photographs can be as sim-ple as painting colors over the original grey values. Whiledifficult to do by hand, with the help of the right softwaretool, a convincingly colorized version of a black and whitephotograph can be created provided that a user selects rea-sonable colors for each object. Software capable of quicklycolorizing a greyscale image given minimal user interac-tion has been a topic of recent research interest, pioneeredby Levin et al. [7]. Nie et al. [9] presented a more com-putationally efficient formulation. Qu et al. [12] presenteda colorization system specialized for the case of coloriz-ing manga drawings, while Lischinski et al. [8] generalizedthe idea to a broader category of interactive image editingtools. Such colorization techniques naturally complementthe process of color recovery. Provided an appropriatelychosen sparse set of recovered colors, colorization can beused to create complete color images from any of the blackand white originals (see also Figure 2).

    In the field of remote sensing, it is sometimes necessary toconstruct color images from multispectral image data. Forexample, color images of the Martian landscape were gen-erated from the 6 channel multispectral image data returnedby the Viking lander [10, 11]. The basis projection algo-

  • 400 450 500 550 600 650 700 7500





    5TriX Pan and a Blue Filter: Effective Sensitivity Curves






    cm2 )

    Wavelength (nm)

    KODAK TriX Pan Film + Wratten 47

    KODAK TriX Pan Film

    (a) Effective sensitivity curves for two B&W sources (b) Initial Colors (c) Corrected Colors (d) Ground Truth

    Figure 1: Our mathematical framework allows the recovery of color with known error thresholds from as few as two images.This example uses a hyperspectral dataset to create two simulated photographs using KODAK Tri-X Pan film with and withouta blue filter (Wratten 47). The first image shows the effective sensitivity curves used for creating the two simulated imagesused as source data. The remaining three images show the default recovered colors, the recovered colors with statisticalerror correction, and ground truth color values for the dataset.

    rithm used to create those color images is nearly identicalto the projection technique that appears in our own methodssection (see Equation (4)). However, in our own framework,this initial result can be significantly improved using naturallight statistics.

    A technique developed by the photographic communitycombines several registered black and white images takenin rapid succession with red, green, and blue filters to cre-ate a color image. This method dates back to at least theearly 1900s when Sergey Prokudin-Gorskii set out to doc-ument the Russian Empire using a pioneering camera andred, blue, and green filters to capture three different imagesof the same scene. The slides were then projected throughred, green and blue filters of a device know as a magiclantern which superimposes the images onto a screen, pro-ducing a color projection [1]. While the projection equip-ment that Prokudin-Gorskii used to display his photographshas been lost, it is possible to extract color information fromthe surviving glass plates. Recovering color from Prokudin-Gorskiis photographs is relatively straightforward, becausewe can assume that the red, blue, and green channels inan output color image are just scaled versions of the red,blue, and green filtered photographs. In order to create colorimages, scholars working with the Library of Congress ad-justed the weightings of each color channel until the result-ing image looked sensible. In this work we provide a so-lution to the general problem of recovering color from anarbitrary collection of black and white photographs.

    3. Methods

    3.1. Terminology and Background

    3.1.1 Terminology

    In this paper, we use the convention of referencing per-photograph variables with a i subscript. To distinguish be-tween the different dimensions of a color space, we use a ksubscript.

    Given a spectral sensitivity curve, G(), we define itsideal response to incoming light having radiance distribu-tion V () to be,

    G()V ()d. (1)

    Under certain assumptions, equation (1) can be used tomodel photoreceptors with spectral sensitivity G(). In or-der for equation (1) to predict the response of a photore-ceptor, it must be the case that the incoming light doesnot change as a function of space or time. The activationlevel of a photoreceptor is influenced by the amount of timethat it is exposed to light, thus, for Equation (1) to predicta photoreceptors measurements, we must assume that thereceptor will always be exposed for a fixed length of time,and that the sensitivity of the photoreceptor does not changeover time. These assumptions are reasonable in the case of

  • Film/Filter Spectral Sensitivty Curves

    Input Images Partial Knowledge of the Scene

    Recovered Colors


    Figure 2: Application to non-aligned images. Our methodscan operate on data samples taken from non-aligned im-ages. Color recovery at point samples is possible, providedthat a user selects pixel sets that share a common materialand lighting properties, and the spectral sensitivity curvesof the film and transmission functions of any lens filters areknown. Such point color samples are a natural input forstandard colorization techniques.

    a CCD camera with fixed exposure time, but they are overlyrestrictive as a model for film photography.

    The CIE Colorimetric system defines the RGB and XYZcolorspaces in terms of ideal response equations [18].Specifically, if we assume that the light V () passingthrough a small patch of our image is constant over time,and Ck() is the sensitivity curve associated with the k-thdimension of the CIE RGB color space, then the k-th colorvalue of that patch should be,

    Ck()V ()d. (2)

    In a black and white photograph, an integral similar to equa-tion (1) plays a role in determining brightness at each point.The films spectral sensitivity and the transmittance func-tions of any applied filters will combine to form an effectivespectral sensitivity, which we denote as Fi. The relation ofthe ideal response of the film to the density of the developed

    negative is determined by the parameters of the developingprocess; see Section 3.3 for further discussion.

    3.1.2 The Linear Algebra of Response Functions

    Putting aside, for a moment, the question of whether or notit is possible to find our films ideal response values, wenow consider whether given such responses, it is possibleto approximate the color values at any point in the image.

    First, note that the set of all response functions forms aninner product space, with the inner product defined as,

    F G :=F ()G()d.

    Define R(V ) to be a linear transformation that maps radi-ance distributions V to the ideal responses of a set of filmsensitivity functions {Fi},

    R(V )i := Fi V.

    It follows that any color response function


View more >