Reflections of Reality in Jan van Eyck and Robert Campin

A. Criminisi†, M. Kemp‡ and S. B. Kang?†Microsoft Research, UK, ‡University of Oxford, UK, ?Microsoft Research, USA

antcrim@microsoft.com

Abstract

There has been considerable debate about the perspectival / optical bases of the naturalism pi-oneered by Robert Campin and Jan van Eyck. Their paintings feature brilliantly renderedconvexmirrors, which have been the subject of much comment, especially iconographical. David Hockneyhas recently argued that the Netherlandish painters exploited the image-forming capacities ofcon-cavemirrors. However, the secrets of the images within the painted mirrors have yet to be revealed.Using novel, rigorous techniques to analyse the geometric accuracy of the mirrors, unexpected find-ings emerge, which radically affect how we see the paintings as being generated. We focus on Jan vanEyck’s Arnolfini Portrait, and the Heinrich von Werl Triptych, here re-attributed to Robert Campin.The accuracy of the convex mirrors depicted in these paintings is assessed by applying mathemati-cal techniques drawn from computer vision. The proposed algorithms allow us also to “rectify” theimage in the mirror so that it becomes a normalised projection, thus providing us with a second viewfrom the back of the painted room. The plausibility of the painters’ renderings of space in the convexmirrors can be assessed. The rectified images can be used for purposes of three-dimensional re-construction as well as measuring accurate dimensions of objects and people. The surprising resultspresented in this paper cast a new light on the understanding of the artists’ techniques and their opti-cal imitation of seen things, and potentially require a re-thinking of the foundations of Netherlandishnaturalism. They also suggest that the von Werl panels should be re-instated as autograph works byR. Campin. Additionally, this research represents a further attempt to build a constructive dialoguebetween two very different disciplines: computer science and history of art. Despite their fundamen-tal differences, the procedures followed by science and art history can learn and be enriched by eachother.

1. IntroductionThe advent of the astonishing and largely unprecedented naturalism of paintings by the van Eyck broth-ers, Hubert and Jan, and of the artist called the Master of Flemalle (now generally identified with the doc-umented painter, Robert Campin), has always been recognised as one of the most remarkable episodesin the history of Western art. Various explanatory modes have been developed to explain the basis ofthe new naturalism and its roles in the religious and secular societies of the Netherlands in the early15th century. Amongst the new technical factors, the perfection and innovative use of the oil mediumare clearly seminal. In terms of meaning, the naturalistic integration of objects within coherent spacesallowed a profusion of symbolic references to be “concealed” within what looks the normal ensemble

1

a b c d

Figure 1: Convergence of orthogonals in paintings: (a)Masaccio,Trinita con la Vergine e San Giovanni,approx. 1426, Museo di Santa Maria Novella, Firenze, Italy.(b) The dominant orthogonals (superimposed inyellow) converge at a single vanishing point.(c) Jan van Eyck,Madonna in the Church, 1437-1439, StaatlicheMuseen zu Berlin, Gemaldegalerie, Berlin.(d) The orthogonals (marked) donot converge at a single vanishingpoint.

of a domestic or ecclesiastical interior. It has also been long recognised that the coherence of the spacesin Netherlandish painting did not rely upon any dogmatic rule, such as the convergence of orthogo-nals to the “centric” or vanishing point, which provided the basis for Italian perspective in the wake ofBrunelleschi (cf. fig. 1). The straight edges of forms perpendicular to the plane of the picture convergein a broadly systematic manner, but are not subject to precise optical geometry. The two paintings onwhich we shall concentrate, the so-calledArnolfini Weddingby Jan van Eyck (figs. 2a,b) and theSt. Johnthe Baptist with a Donorfrom the Heinrich van Werl triptych (figs. 2c,d), both demonstrate the way thatconvincing spaces could be created by imprecise means.

The effects and meaning of the new naturalism have been the subject of more attention than thequestions of the visual or optical resources used to compile the images and how such revolutionarynaturalism could be forged in the face of resistant conventions. Recently, David Hockney has arguedthat optical projections provide the key. Initially disposed to argue that the artist relied very directly onimages projected on to white surfaces by lenses or concave mirrors, Hockney now places more emphasisupon the projected image as the key breakthrough in revealing what a 3-D array looks like when flattenedby projection. In any event, he rapidly came to see that an image like Jan van Eyck’s would not havebeen projected as a whole for literal imitation, not least because the overall perspective of the paintings isconsistently inconsistent. Rather he advocated that the components in the image were studied separatelyin optical devices, to be collaged together in what he has called a “many windows” technique. Thearguments between Hockney and his detractors have become increasingly bogged down in personalisedpolemic which dogmatically uses varieties of technical evidence in the service of pre-determined stances,without much sense of improvisatory procedures which typify artists’ uses of the tools at the disposal.

2

a b c d

Figure 2: Original paintings analysed in this paper: (a)J. van Eyck,Arnolfini Wedding, panel, 1434, NationalGallery, London.(b) Enlarged view of the convex mirror in (c).(c) R. Campin,St. John with a Donor, panel,1438, Museo del Prado, Madrid.(d) Enlarged view of the convex mirror in (a).

Our intention is to enquire about Netherlandish naturalism along a different route. We will be un-dertaking a close analysis of the remarkable convex mirrors in the two paintings, using techniques ofcomputer vision to “rectify” the curved images in the mirrors. We will be asking what the rectifiedimages tell us about the spaces that the painter has portrayed - looking, as it were, into the interiorsfrom the other end - and what implications emerge for how we conceive the painters as composing theirimages. What emerges from our analysis seems to us to have unavoidable implications for how theystage-managed their ensembles of figures and objects. Along the way, we will suggest that the positionof the von Werl triptych in relation to Campin’s oeuvre and the works of his followers needs to be re-assessed. Our conclusions will have a bearing on the Hockney hypothesis, though in a permissive ratherthan conclusive manner.

Before we begin our analysis, it will be a good idea to introduce the two paintings which will beserving as our witnesses.

1.1. The Arnolfini WeddingThe painting by Jan van Eyck in the National Gallery, London, executed in oil on an oak panel, isgenerally though not universally identified as portraying Giovanni di Nicolao Arnolfini and GiovannaCenami. Arnolfini was a successful merchant from Lucca and representative of the Medici bank inBruges, who satisfied Philip the Good’s eager demands for large quantities of silk and velvet. Thedouble portrait has reasonably been identified as representing an event, namely their marriage or, moreprobably, their betrothal, as witnessed by the two men reflected in the mirror. Jan himself appears tohave been one of the witnesses, since a beautifully inscribed graffito on the end wall records the date,1434, and the fact that “Johannes de eyck fuit hic” (Jan van Eyck was here).

1.2. The St. John PanelThe other painting, theSt. John with a Donorin the Prado Museum in Madrid, uses the oil medium noless brilliantly than van Eyck. It is one of the two surviving wings of an altarpiece commissioned, as theinscription tells us, by Heinrich van Werl in 1438. Von Werl was a Franciscan theologian from Cologne

3

who became head of the Minorite Order (a conventual branch of the Franciscans). The right wing depictsSt. Barbara in an interior within which light effects are rendered with notable virtuosity. The main panel,probably containing the Virgin and Child with saints (one of whom would almost certainly have been St.Francis) is no longer traceable. It is likely that Heinrich had chosen St. John as his personal saint, to actthe intercessor who grants him the privilege of witnessing the Virgin through the door that opens to thesacred realm of the central panel. Two Franciscans, visible in the painted mirror, witness the scene. Theplay of domestic and sacred space would have been crucial in the reading of the triptych’s meaning, andin signalling the donor’s hoped-for translation to the heavenly dwelling of the Virgin and saints after hisdeath.

In the technical analyses that follow, we provide a geometric analysis of the two painted mirrors onthe basis of the optics of spherical and parabolic mirrors, and transform the images into normalisedrectilinear views of the painted interiors. The rectified images are subject to accuracy analysis, andthe artists’ possible manipulation of the effects is considered. Finally, we spell out the surprising andradical implications that emerge from the study of two such apparently subsidiary aspects of the compleximages.

2. Geometric analysis of painted convex mirrorsThis section presents some simple but rigorous techniques for the analysis of the geometric accuracyof convex mirrors in paintings. The algorithms employed here build upon the vast computer visionliterature, with specific reference to the field of compound image formation, termed catadioptric imag-ing [1, 10]. A catadioptric system uses a camera and a combination of lenses and mirrors.

The basic mirror-camera model. Given a painting of a convex mirror (e.g.figs. 2b,d), we model thecombination mirror-panel (or, similarly, mirror-canvas) as a catadioptric acquisition system composedby a mirror and an orthographic camera as illustrated in figs. 3a,b.

It is well known that curved mirrors produce distorted reflections (cf. figs. 3c,d) of the environmentthey are in. For instance, straight scene lines become curved (e.g.see the window frames in figs.2b,d).However, if the shape of the mirror is known, then the reflected image can be transformed (warped) insuch a way to produce a perspectively corrected image;i.e. an image as if it was acquired by a camerawith optical centre in the centre of the mirror1. But, while this is true for real mirrors, the case of paintedmirrors is more complicated. In fact, one should not forget that a painting is not a photograph, but theproduct of the hand of a skilled artist; and, even when it appears correct, its geometry may differ fromthat produced by a real camera.

In this paper we propose novel techniques for: i) Assessing the geometric accuracy of a painted mirrorand ii) Producing rectified, perspective images from the viewpoint of the mirror itself.

The basic algorithm may be described in general terms as follows:

1Straight scene edges are imaged as straight lines by a pinhole camera. In addition, strictly speaking, the correctivetransformation can be done with the knowledge of only the mirror shape if the imaging system has a single point of projection.Otherwise, additional knowledge such as scene depth has to be known as well.

4

Image Plane

Mirror

Optical axis

P

P

p d

m

a b c d

Figure 3: Catadioptric imaging model: (a) The mirror-panel pair is modelled as a catadioptric acquisitionsystem, composed by a curved mirror and an acquiring camera.(b) A ray emanating from a pointP in the sceneis reflected by the mirror onto the corresponding image (panel) pointpd. The ray is reflected off the mirror surfacein the pointPm. (c) A square grid reflected by a convex mirror appears barrel-distorted.(d) A goal of this paperis to “rectify” the image to produce a perspective image. In the case of fig. (c), we wish to recover an image of theregular grid, where all the black and white patches are perfect squares.

1. Hypothesizing the shape of the convex mirror fromdirect analysis of the distortionsin the original images.

2. Rectifying the input image (distorted) to produce the corresponding perspective one.

3. Measuring the discrepancy between the rectified image and the “perspectivally cor-rect” image to assess the accuracy of the painted mirror.

As it will be clearer in the remainder of the paper, these basic geometric tools will help us cast a newlight on the analysed paintings and their artists. To proceed, we need to make some explicit assumptionsabout the shape of the mirror. We explore three different rectification assumptions:

• Assumption A1: The mirror has a parabolic shape,

• Assumption A2: The mirror has a spherical shape,

• Assumption A3: The geometric distortion shown in the reflected image canbe modeled as a generic radial-distortion process.

For each of those three basic assumptions, from the direct analysis of the image data alone, our algorithmestimates the best geometric transformations which can “remove” the distortion in the reflected imageand rectify it into a perspective image.

It must be noticed, though, that from a historical point of view the parabolic can be discarded sincethe standard convex mirror of the time was cut from a blown glass sphere. However, the parabolic mirrorremains useful as a model in the analyses that follow.

5

2.1. Assumption A1: Parabolic mirrorThe parabolic shape assumption is convenient since it allows us to re-use considerable amount of tech-niques developed in the field of catadioptric imaging [1, 10]. In fact, it is possible to treat the paintedmirror as part of a parabolic mirror and orthographic camera catadioptric system (fig. 3a). Such a systembehaves as a single-optical-centre acquisition device2 and the corresponding rectification equations arestraightforward.

The diagram in fig. 4 describes a plan-view cross-section of the basic parabolic mirror-camera systemand simple algebra leads to the basic equations for undistorting the reflected image. Given the pixelcoordinateud of the distorted point in the original image we can compute the coordinateuc of thecorresponding corrected point by applying the formulae below:

uc =∆

wud with w =

h2 − r2

2h, (1)

where∆ is the distance of the camera/panel from the mirror focus,h is the parabolic parameter, andr isthe distance between the focus and the pointPm.

Notice that the above equations have been derived for the 2D mirror cross-sections only, but since themirrors are assumed to be perfectly symmetrical surfaces of revolution the derivation of the formulae forthe complete 3D case is straightforward.

2.2. Assumption A2: Spherical mirrorUnlike parabolic mirrors, spherical mirrors do not present a single point of projection but a whole locusof viewpoints which is called acaustic surface[14]. The rectification process becomes more compli-cated here, since it is now necessary to know the three-dimensional shape and depth of the surroundingenvironment.

However, the equations for rectifying the image reflected by a spherical mirror simplify if we assumethat the visualized points (e.g.the pointP in fig. 3b) lie at infinity; or more realistically, far away fromthe mirror itself. From a historical point of view this latter assumption is, obviously, more plausible andeffectively operates in our two chosen examples.

From an analysis of the diagrams in fig. 5 and some simple algebra we can derive the followingrectification equations:

uc = ud + (∆− γ)2udγ

r2 − 2u2d

with γ =√

r2 − u2d, (2)

where∆ is the distance (large) of camera/panel from the centre of the sphere,r the radius of the sphere,ud the coordinate of the distorted image point, anduc the coordinate of the corresponding correctedpoint.

2.3. Assumption A3: The convex mirror induces radial distortion on the imageplane

In this section, rather than reasoning about the 3D shape of the mirror, we try to model directly thedistorting effect that a convex mirror produces on the reflected image.

2The interested reader may refer to [9, 13] for other single-viewpoint catadioptric systems

6

Image plane (orth. camera)

P a r a b o l i c m i r r o r

O p t i c a l a x i s

Image plane

P a r a b o l i c m i r r o r Focus

O p t i c a l a x i s

Image plane

O

P

w h/2

u

u

r

d

c

P m

a b c

Figure 4: The geometry of parabolic mirrors: (a) Plan-view cross-section of a parabolic-mirror and the image(panel) plane.(b) All the rays coming from the orthographic camera are reflected by the mirror as rays whichintersect in a single point, namely thefocusof the mirror. The existence of a single focus makes parabolic mirrorsparticularly convenient, see text for details.(c) A point P in the three-dimensional scene is reflected by the mirrorinto its corresponding image point atud distance from the centre of the image. In other words,ud is the observed(distorted) coordinate. The corresponding, undistorted position is labelleduc. The centre of projection (focus) isdenotedO and the parabolic parameterh.

Image plane (orth. camera)

S p h e r i c a l

m i r r o r

O p t i c a l a x i s

Centre of sphere

Image plane

S p h e r i c a l

m i r r o r

Section of caustic surface

u u Image plane

O

r

d c

a b c

Figure 5: The geometry of spherical mirrors: (a) Plan-view cross-section of a spherical-mirror and the image(panel) plane.(b) All the rays coming from the orthographic camera are reflected by the mirror as rays which donot intersect in a single point, but in acaustic surface. This characteristic makes spherical mirrors harder to use,unless the 3D pointP is assumed to be at infinite distance (or very far) from the centre of the mirror.(c) ud anduc are the observed and corrected coordinates, respectively.

7

c

p r

p ’

c

p r

c

p

l

p ’ p

Image plane

d

c

e d e c

Figure 6: Mirror rectification via a generic radial model: A radial transformation maps a pointpd in the orig-inal image into the corresponding pointpc in the rectified (corrected) image by applying a radial transformationcentered in the image centrec, see text for details. The complete rectified image is obtained by applying thispoint-based tranformation to all the points in the original (distorted) image. Notice that the radial transforma-tion straightens curved edges,e.g.the edgeed in the original image is transformed in the corresponding edgeec

(straightened) in the rectified image. See also figs. 3c,d.

As shown in figs. 3c,d, in the image reflected by a convex mirror straight scene edges appear curved.Here we make the plausible assumption that the distorting transformation can be modelled as a standardradial distortion model. Radial distortion typically occurs in cameras with very wide-angle lenses [4].Cheap web-cameras are a good example.

A generic radial model can be described by the following equations (cf. fig. 6):

pc = c + f(l)(pd − c), (3)

wherepd is a 2D point on the plane of the original image andpc is its corresponding, corrected point.The quantityl = d(pd, c) is the distance in the image plane of the pointpd from the centre of the imagec and the functionf(l) is defined as

f(l) = 1 + k1l + k2l2 + k3l

3 + k4l4 + ...

Therefore, if theki parameters of the above model were known, this radial transformation could beapplied to the reflected image to “correct” the distortion and obtain the corresponding perspective image.It can be shown that a generic radial model subsumes both the parabolic and spherical models. This isdone by expanding the right hand sides of (1) and (2).

We can choose the complexity of the radial model by selecting how many and which of theki param-eters to use. In the results below, in order to keep the model simple, we have chosen to use only thek1

andk2 parameters.

2.4. Image rectificationEstimating the parameters. In all the three cases described above, in order to rectify the reflectedimage it is necessary first to estimate optimal values for the parameters of the described mathematicalmodels. The parameters that need to be estimated in the three cases are summarized in Table 1.

8

Assumption Parameters to be computed

A1. parabolic mirror h (i.e.parabolic parameter)A2. spherical mirror r (i.e. radius of sphere)A3. generic radial model k1, k2

Table 1: Mirror shapes and parameters to be computed.

As mentioned previously, we expect the rectification algorithm to straighten the curved images ofstraight scene edges such as window and door frames. Therefore, the best set of rectification parametersmay be computed as the set of values which best straightens some carefully selected edges in the original(distorted) images. Our semi-automatic rectification algorithm proceeds as follows:

1. A Canny Edge Detection [2] algorithm automatically detects sub-pixel accurate im-age edges;

2. A user manually selects some curved edges which correspond to straight 3D scenelines;

3. A numerical optimization routine such as Levenberg-Marquardt [11] automaticallyestimates the parameters values that best straighten the selected edges.

Rectification results. Results of the rectification of Campin and van Eyck’s mirrors are illustrated infig. 7 and fig. 8, respectively.

One first observation is that the three different mirror shape assumptions lead to very similar (but notidentical) rectifications. This can be explained by considering the fact that the tips of the parabolic andspherical mirrors are very similar in shape. In fact, in Optics, parabolic assumptions are made for lensesand mirrors when small portions of a spherical surface are being considered. If the visible parts of themirrors are just the tips, as in these two cases, it is not surprising that the rectification results are verysimilar. Furthermore, the fact that the spherical assumption and the parabolic one yield very similarrectification results validates the assumption of substantial distances between the 3D scene points andthe mirror, which needs to be made in the spherical mirror case. This is an indirect way of assessing thevalidity of the rectification approach we have undertaken.

Notice that horizontal flipping of the rectified images in figs. 7b,c,d and figs. 8b,c,d would producethe perspective images that an observer would see if standing in the place of the mirror in the depictedscene (e.g.the large window would be on the right-hand side).

The rectification process produces novel perspective images, from a different vantage point than thatof the artist; thus, in effect, giving rise to stereo views of the depicted scene. Given two views of thesame scene from two different viewing positions it is conceivable to apply standard three-dimensionalcomputer vision techniques [5, 6] to reconstruct complete virtual models of the depicted environment.Alternatively, single-view reconstruction [3] may be applied twice: to the image from the front (thepainting itself) and to the image from the back (the rectified mirror reflection) and the two resultingthree-dimensional reconstructions merged together to create a single and complete shoe-box-like virtualmodel of the scene.

9

a b c d

Figure 7: Rectification of the convex mirror in Campin’s St. John Panel: (a) Original (distorted) mirrorfrom fig 2b. (b,c,d) Rectified images.(b) Result of rectification using the parabolic assumption;(c) Result ofrectification using the spherical assumption;(d) Result of rectification using the radial model fitting; Notice thatin all the rectified images the edges of the window and door have been straightened.

a b c d

Figure 8: Rectification of the convex mirror in van Eyck’s Portrait: (a) Original (distorted) mirror from fig 2d.(b,c,d)Rectified images.(b) Result of rectification via parabolic assumption;(c) Result of rectification via spher-ical assumption;(d) Result of rectification via generic radial model fitting;

2.5. Comparing mirror protrusionsAs mentioned above, in the case of the spherical mirror assumption, in order to rectify the input, reflec-tion images it is necessary to compute the radiusr of the spherical mirror itself (Table 1).

Since we are working directly on the image plane withno additional assumptions on absolute scenemeasurements, it isnot possible to measure radii in absolute metric terms but only as relative measure-ments. For instance, we can measure the ratio between the radiusr and the radius of the disk in eachpainting (we calldiskthe visible part of a spherical mirror on the image plane,cf. fig. 9a).

Figure 9a shows a plan view of a spherical mirror sticking out a wall with the disk radius clearlymarked. Figure 9b shows a comparison between two spherical mirrors with the second mirror (namelymirror 2) bulgier than the first one. We now define a measure of theprotrusion(bulge) of a sphericalmirror as the ratioP between the disk radius and the radius of the sphere.

P =rdisk

r(4)

By inspection of fig. 9b, sincer2 < r1 and the disk radiusrdisk is the same, thenP2 > P1.From the protrusion measures reported in Table 2.5 for Campin’s and van Eyck’s painted mirrors we

can conclude that the two mirrors are very similar in terms of shapes (similar protrusionsP) but the

10

Image plane

O

r r disk

Wall

Visible section of spherical

mirror

r disk r disk r

Wall

M i r r o r 1

M i r r o r 2

2 r 1

a b

Figure 9: (a) We call the visible part of a spherical mirror adisk. The quantityrdisk is the disk radius.(b) A planview of two spherical mirrors sticking out a wall. Sincer1 > r2 the second mirror sticks out more and thus isbulgier than the first mirror.

Painting Protrusion factor,P = rdisk/r

Campin’sSt. John with a donor P = 0.83van Eyck’sArnolfini Wedding P = 0.78

Table 2: “Bulge” estimates for the two paintings.

former is slightly bulgier (greater value ofP) than the latter.This result should be taken with some care. In fact, in order to be completely certain of the degree

of protrusion of each mirror we need to be certain about the accuracy level of its rendered geometry.The following section will show that while we can be quite confident about the geometric correctness ofCampin’s mirror, the same cannot be said of van Eyck’s.

The protrusion measurements of Table 2.5 could also be used to compute the focal lengths of thetwo mirrors. In fact, elementary Optics teaches us that the radius of curvaturer of a mirror is twice itsfocal length. Therefore, from equation (4), if we knewrdisk we could computer and thereforef = r/2would be the focal length. Notice, though, that an absolute metric value forrdisk can be estimated onlyafter making further assumptions on shapes and sizes of objects that appear in the scene. This kind ofassumptions would make sense if, like in the case of a few paintings by Jan Vermeer, some of the depictedobjects still exist and can be measured [8, 12]. But in the case of the two paintings analysed in this paper,we feel that assumptions not supported by strong physical evidence may conduct to misleading resultson the value of the mirrors focal lengths.

2.6. Accuracy analysisThe previous sections have described how direct analysis of the image of a convex mirror can lead to theremoval of the inherent optical distortion in the original images and the generation of the correspond-ing, perspectivally-correct images. This section, instead, analyses the rectified images to assess theirgeometric consistency.

11

h 1 h 2

a b c

Figure 10: Accuracy of Campin’s mirror: (a) Rectified image of mirror from spherical assumption;(b) Straightlines have been superimposed to (a) to demonstrate the accuracy of the rectification process.(c) The accuracyof the rectified image is sufficient for extracting metric information out of the painting. By applying single viewmetrology techniques the ratio between the heights of the two monks is computed to beh1

h2= 1.38.

2.6.1 Accuracy of Campin’sSt. John Panel

Figure 10a (identical to fig. 7d) is the rectification of Campin’s mirror obtained from the assumption ofspherical mirror.

Straightening of curved edges. Figure 10b shows that images of straight scene lines now have becomeconsistently straight. Furthermore, the edge of the door and the three vertical edges of the window meet,quite accurately in a single vanishing point (far above the image). This demonstrates the extraordinaryaccuracy of the rendered geometry and the high level of skill exercised by Robert Campin. In fact, if itis difficult to paint in a perspectively correct way, painting a convex mirror with the degree of accuracydemonstrated in Campin’sSt. Johnrequires an extraordinary effort.

Measuring heights of people. The accuracy in Campin’s painting is sufficient for us to apply singleview metrology [3] techniques to measure the ratio between the heights of the two monks. The height-estimation algorithms in [3] applied to the image in fig.10a produce the ratioh1

h2= 1.38 between the

heights of the two monks; thus proving that the difference in the monks’ imaged height is not due justto perspective effects (reduction of farther objects) but to a genuine difference in their height. Thisdiscrepancy in height could be explained by the possibility that the farther monk is kneeling, but thedifferent scales of their heads suggests that Campin’s otherwise meticulous scaling of objects in themirror image has gone awry in this small detail

2.6.2 Accuracy of van Eyck’sArnolfini Portrait

Inconsistency of mirror geometry. As it can be observed in figs. 8b,c,d, whatever the parametersused, it does not seem possible to simultaneously straighten all the edges in Jan van Eyck’s mirror (seealso fig. 11). The most striking error can be observed in the leftmost edge of the large window in therectified images. In fact, while it is possible to straighten up the central and farthest vertical edges ofthe window frame, together with the edges of other objects in the scene, the left-most window edge

12

a b

Figure 11: Accuracy of van Eyck’s mirror: (a) Original image;(b) The best possible rectification with sphericalmirror assumption. Most edges have been correctly straightened (marked in yellow), but the left-most edge of thewindow (marked in red) has remained noticeably curved. This is indication of inconsistency in the geometry ofthe painted mirror.

remains curved. Figure 11 illustrates this concept more clearly. While all the edges marked in yellowin the rectified image in fig. 11b are straight, the edge marked in red is still clearly curved. This can beinterpreted as lack in geometric consistency;i.e. it is not possible for a physical parabolic or sphericalmirror to produce the kind of image observed in the original painting (fig 2).

Fig. 12 shows the effect of rectifying the reflected image by the spherical model with different valuesof the sphere radius (the mirror radiusr). As it can be seen, none of these values can straighten up all theedges at the same time. The same problem arises with the parabolic or generic radial model assumptions.

Incidentally, as predicted by the laws of Optics, changing the radius of the mirror has the effect ofchanging the focal length of the mirror-camera system (i.e. zooming effect, see fig. 12).

Correcting inaccuracies in van Eyck’sPortrait. Further inconsistencies characterise the geometry ofJan van Eyck’s mirror.

In fig. 11b, we observe that the bottom edge of the wooden bench and the bottom edge of the woman’sgown do not appear to be horizontal (as they are likely to be). Instead, these edges are horizontal inthe original painting (fig. 11a). However, this fact is physically impossible: these edges, due to theirproximity to the boundary of a convex mirror, should appear curved and oriented at an angle.

To explain this concept in easier visual terms, we have run the experiment illustrated in fig. 13. Thesteps taken were as follow:

1. Rectify the original mirror (fig. 13a) using the process described in the previous section and spher-ical assumption. The resulting rectified image (fig. 13b) shows the window, bench, and gownartifacts mentioned earlier.

2. Straighten these three edges manually via an off-the-shelf image-editing software. The resultingimage (in fig. 13c) is perspectivally and physically “correct” in that projected straight 3D edgesare now straight and the bench and gown edges (circled with a dotted line) are horizontal.

13

a b c

Figure 12: The effect of different radii: Rectifying van Eyck’s mirror by means of spherical assumption andvarying the radius of the sphere. Despite all the efforts there is no single value of the mirror radius that canstraighten up all the edges at the same time.

3. Warp the image in fig. 13c using the inverse of the spherical transformation employed in the firststep.

The resulting image in fig. 13d illustrates what the artist “should have painted”. When comparedwith the original painting, the image in fig. 13d is more consistent with the laws of optics applied to aspherical convex mirror and general assumptions about straight lines in an indoor environment.

In fig. 13d, the bottom edge of the bench more realistically follows a sloped curve and so does the bot-tom of the woman’s green gown. Furthermore, the curvature of the left-most edge of the large window ismuch less pronounced than in the original painting. Overall the image in fig. 13d looks more consistent.

The above analysis suggests the possibility that the artist has deliberately altered the geometry of someobjects in the painted mirror. The question is: why would Jan van Eyck do so?

Plausible answers may be: i) the artist accentuated the curvature of the left-most edge of the windowto convey a stronger sense of the bulge of the mirror (in fact, our analysis shows that Campin’s mirroris bulgier than that of van Eyck); ii) he may have painted the edges of the bench and gown as straight tomake them look less strange,i.e.accentuate reality.

In the interest of clarity, fig 14 shows enlarged versions of the images in fig. 13a and fig. 13d, respec-tively, with the edges of interest marked with different colours.

3. ConclusionWe have conducted a rigorous geometric analysis of the convex mirrors painted in theSt. Johnpanelfrom the Heinrich von Werl Triptych and Jan van Eyck’sArnolfini Wedding. It must be stressed thatthe results were obtained from direct analysis of the original paintings, without unnecessary (and oftenunconvincing) assumptions about the represented scene, such as the position and size of depicted objectsor heights of people. Finally, all the assumptions involved in this analysis have been made explicit,cross-validated and verified from a scientific as well as an historical point of view. The computer visionalgorithms we have employed have allowed us to analyse the shapes of the mirrors, compare them andtransform the reflected images into ones that correspond with orthodox perspective. We are thus provided

14

a b

c d

Figure 13: Correcting Jan van Eyck’s Portrait: (a) The original painted mirror;(b) The rectified mirror, seetext for details.(c) The left-most window edge is manually straightened. The bottom edges of the woden benchand that of the woman’s gown are manually made horizontal.(d) Image (c) is warped back by the inverse ofthe spherical model used to generate image (b). The resulting image (d) is the “corrected” version of van Eyck’smirror, one that more closely obeys the laws of Optics.

15

a b c

Figure 14: Correcting Jan van Eyck’s Portrait: (a) The original painted mirror;(b) Our corrected version, seetext for details. The left-most edge of the large window is less pronounced in the corrected version and the benchand gown bottom edges are correctly curved, consistently with the laws of Optics.(c) Enlarged version of fig. 13d,to show the “corrected” edges. To be compared to the original painting in fig. 13a.

with new views from the back of the depicted rooms. A rigorous comparison between the two analysedpaintings leads to the following conclusions:

• the geometry of Campin’s mirror is astonishingly good and is considerably more accurate thanthat of Jan van Eyck, which is nevertheless quite impressive;

• Campin’s mirror appears to be bulgier than van Eyck’s;

• it appears that Jan van Eyck’s has purposely modified the geometry of the image in certain partsof the painted mirror.

A series of notable art-historical consequences flow from these findings, most particularly with respectto theSt. Johnpanel.

3.1. The St. John PanelThe viewpoint used by Campin to paint the mirror is located on a vertical axis a small distance outsidethe right boundary of the mirror (see fig. 15a). The height and distance of the viewpoint are moreproblematic. The rectified view of the mirror produces a point of convergence for the windows at ahorizontal level equivalent to eye level of the standing St. John and nearest Franciscan3. However,the most prominent verticals in the rectified image (marked in yellow in fig. 15b) undergo upwardsconvergence towards a second, single vanishing point, which suggests that the painter’s eye level wasbelow the central axis of the mirror. Experimenting with an actual spherical mirror has confirmed that alower viewpoint, relatively close to the mirror, does indeed produce the effect of a convergence for theorthogonals on or close to the central horizontal axis of the mirror. This lower viewpoint is consistentwith that of the interior as a whole as represented on the panel, which is clearly seen from a position

3The horizontal edges of the window (marked in green in fig. 15b) converge roughly at a point on the right boundary ofthe mirror; approximately at the eye level of the reflections of St. John and the closest Franciscan in the mirror.

16

a b

Figure 15: Vanishing points in Campin’s St. John. (a) The edges of the left window suggest a vanishing pointa short distance to the right of the right edge of the painting.(b) In the image of the rectified mirror the horizontalwindow edges (in green) converge roughly to a point on the right boundary of the mirror, while the verticals (inyellow) converge upwards. The upwards convergence suggests a tilting of the “camera/panel”.

closer to the eye level of the kneeling donor and perpendicular to a point that lies decisively withinthe lost central panel (i.e. well outside the right boundary of the St. John panel). What we cannot telldecisively is how close the artist’s viewpoint was located to the mirror in the actual painting.

These observations suggest the following model for the artist’s procedure. To paint the staged scenein his convex mirror, Campin moved closer to the mirror than is inferred from the original set-up asportrayed as a whole in the panel. He observed it from a position just sufficiently outside the rightmargin of the mirror to avoid the problem of his head occluding significant portions of the image. Itmay be that he used the door as a “shield” from which he could observe the image with a minimumof intrusion. He also observed the view in the mirror from an angle below its horizontal mid-line,i.e.effectively tilting the picture plane or “camera” in such a way as to cause the upwards converge.

The fundamental veracity of the painted reflection extends to such details of the small portion of thedonor’s trailing drapery visible beyond the edge of the door, though the donor himself is hidden. Whileit may be possible to envisage in a general way what an imaginary view would look like in a sphericalmirror, it is inconceivable that such consistently accurate optical effects could have been achieved bysimply thinking about it. We are drawn to what seems to be the inescapable conclusion that the artist hasdirectly observed and recorded the effects visible when actual figures and objects are located in a specificinterior. This means that at some point, models must have been posed in exactly the positions occupiedin the painting and dressed in appropriate costumes - with the exception of the two Franciscan witnesses,who are in their normal habits. If this particular picture was achieved through such a stage-managed set-

17

up in real spaces, we may reasonably believe that other Netherlandish masterpieces of naturalism wereaccomplished in the same way.

The question of how the painter achieved such optical accuracy remains to be decided. To someextent, given the condensing of an image within the small compass of the mirror, and with the circularframe serving as a ready point of precise reference, the painter is faced with an easier task than when hesurveys the whole scene at its normal scale. On the other hand, if the optical image in the mirror coulditself be optically projected, this would facilitate the process. If the mirror were to be projected from thedistance implied in the painting, the definition of the image with contemporary equipment would havecome nowhere near delivering the required results. However, if, as suggested, he portrayed the mirror ina separate act, from a closer position, an optical projection remains a possibility.

We may note in passing, that such an astonishing achievement suggests a major mind and talent atwork. The recent tendency to see the von Werl wings as the works of a contemporary pasticheur ofCampin and van Eyck seems entirely unjustified.

3.2. The Arnolfini WeddingJudged on its own merits, the accuracy of the image in Jan’s convex mirror is striking. It is only bycomparison with Campin’s that it appears less than notable. Even with its faults, it is remarkable enoughto support (though less conclusively) the hypothesis that the image in the mirror was painted directlyfrom a stage-managed set-up in an actual space. The most notable of the artist’s departures from opticalaccuracy, the line of the nearest edge of the bench by the window, and the hem of the woman’s dress, canboth be explained as instinctive responses to the extreme effects visible in the mirror. In other words,the painter has very selectively played to what we expect the image to look like rather than preciselyfollowing its actual appearance. Such moves are common enough in Italian perspective pictures. Itmay also be that the border regions of the mirror used by Jan were less optically sound than the middlesections. It should also be noted, that the mirror reflection in his painting is portrayed from a viewpointperpendicular to or very near the centre of the mirror, which suggests that he did not adopt the expedientwe have suggested for Campin,i.e.moving closer to the mirror and slightly to the side in order to portraythe reflected scene more readily.

3.3. The Hockney Hypothesis and Wider ConsiderationsThe finding that Campin and van Eyck seem to have worked with a tableau vivant, using posed figures,actual objects and real interiors, does nothing to negate Hockney’s hypothesis that optical projectionfrom a concave mirror or less was used to make the paintings. But neither does it prove it. We are stillleft with the possibility that the painters “eyeballed” their scenes with miraculous accomplishment. Inany event, it remains to be demonstrated that the images in the painted convex mirrors could have beenprojected, using contemporary equipment, with a quality such as to provide such mini-masterpieceswithin each masterpiece. On the other hand, the fact that the Campin mirror may have been representedusing a different viewpoint from the interior in the panel as a whole lends some support to Hockney’s“many windows” theory. Our findings may be regarded as broadly permissive for Hockney’s theory, butstop short of providing incontrovertible support. The main implications of our work lies elsewhere.

The chief of the wider considerations increases the radical distance we can discern between the won-ders of illusion being accomplished in the Netherlands and Italy at precisely the same time. Italian

18

perspectival pictures, as pioneered by Masaccio, were achieved through the synthetic construction ofgeometrical spaces according to pre-conceived designs. Within the geometrical containers, the Italianpainters then inserted to scale separately studied figures or small groups (sometimes beginning with nudeor near-nude studies). By contrast, the leading Netherlandish reformers of representation may be seen asworking as literally as they could with what they could see. They are true champions of what Gombrichhas described as “making and matching”. Such literalism of matching, much to the taste of 19th-centurycritics like Ruskin, has not found favour with recent art historians, who prefer more complicated expla-nations. It is almost as if we are precociously entering the realm of Courbet, the 19th-century realist,who declared, “show me an angel and I will paint one”. Campin, five centuries earlier, seems to besaying, “to paint a St. John, I need to see one”.

References

[1] R. Benosman and S. B. Kang, editors.Panoramic Vision: Sensors, Theory, and Applications. Springer-Verlag, 2001.

[2] J. F. Canny. A computational approach to edge detection.Pattern Analysis and Machine Intelligence,8(6):679–698, 1986.

[3] A. Criminisi. Accurate Visual Metrology from Single and Multiple Uncalibrated Images. DistinguishedDissertation Series. Springer-Verlag London Ltd., September 2001. ISBN: 1852334681.

[4] F. Devernay and O. D. Faugeras. Automatic calibration and removal of distortion from scenes of structuredenvironments. InProc. of SPIE, volume 2567, San Diego, CA, July 1995.

[5] O. D. Faugeras.Three-Dimensional Computer Vision: a Geometric Viewpoint. MIT Press, 1993.

[6] R. I. Hartley and A. Zisserman.Multiple View Geometry in Computer Vision. Cambridge University Press,ISBN: 0521623049, 2000.

[7] D. Hockney. Secret Knowledge: Rediscovering the Lost Techniques of the Old Masters. Viking Press,October 2001.

[8] M. Kemp. The Science of Art. Yale University Press, New Haven and London, 1989. ISBN: 0-300-05241-3.

[9] S. S. Lin and R. Bajcsy. True single view point cone mirror omni-directional catadioptric system. InProc.Int’l Conf. on Computer Vision, volume II, pages 102–107, Vancouver, British Columbia, Canada, July 2001.

[10] S. Nayar. Catadioptric omnidirectional camera. InCVPR, pages 482–488, Puerto Rico, June 1997.

[11] W. Press, B. Flannery, S. Teukolsky, and W. Vetterling.Numerical Recipes in C. Cambridge UniversityPress, 1988.

[12] P. Steadman.Vermeer’s Camera: Uncovering the Truth Behind the Masterpieces. Oxford University Press,Oxford, 2001. ISBN: 0-19-215967-4.

[13] T. Svoboda, T. Pajdla, and V. Hlavac. Epipolar geometry for panoramic cameras. InECCV, pages 218–231,Freiburg, Germany, June 1998.

[14] R. Swaminathan, M. D. Grossberg, and S. K. Nayar. Non-single viewpoint catadioptric cameras: Geometryand analysis. Technical report, Columbia University, 2001.

19