three gradients and the perception of flat and curved...

Journal of Experimental Psychology: General Copyright 1984 by the1984, Vol. 113, No. 2, 198-216 American Psychological Association, Inc.

Three Gradients and the Perception of Flat and Curved Surfaces

James E. Cutting and Robert T. MillardCornell University

SUMMARY

Researchers of visual perception have long been interested in surfaces. Most psychologistshave been interested in the perceived slant of a surface and in the gradients that purportedlyspecify it. Slant is the angle between the line of sight and the tangent to the planar surfaceat any point, also called the surface normal. Gradients are the sources of informationthat grade, or change, with visual angle as one looks from one's feet upward to the horizon.The present article explores three gradients—perspective, compression, and density—and the phenomenal impression of flat and curved surfaces. The perspective gradient ismeasured at right angles to the axis of tilt at any point in the optic array; that is, whenlooking down a hallway at the tiles of a floor receding in the distance, perspective ismeasured by the x-axis width of each tile projected on the image plane orthogonal tothe line of sight. The compression gradient is the ratio of y/x axis measures on theprojected plane. The density gradient is measured by the number of tiles per unit solidvisual angle. For flat surfaces and many others, perspective and compression gradientsdecrease with distance, and the density gradient increases. We discuss the manner inwhich these gradients change for various types of surfaces. Each gradient is founded ona different assumption about textures on the surfaces around us.

In Experiment 1, viewers assessed the three-dimensional character of projections offlat and curved surfaces receding in the distance. They made pairwise judgments ofpreference and of dissimilarity among eight stimuli in each of four sets. The presence ofeach gradient was manipulated orthogonally such that each stimulus had zero, one, two,or three gradients appropriate for either a flat surface or a curved surface. Judgmentswere made for surfaces with both regularly shaped and irregularly shaped textures scatteredon them. All viewer assessments were then scaled in one dimension. Multiple correlationand regression on the scale values revealed that greater than 98% of the variance in scalevalues was accounted for by the gradients. For the flat surfaces a mean of 65% of thevariance was accounted for by the perspective gradient, 28% by the density gradient, and6% by the compression gradient. For curved surfaces, on the other hand, a mean of 96%of the variance was accounted for by the compression gradient, and less than 2% byeither the perspective gradient or the density gradient. There were no differences betweenresults for surfaces with regularly shaped and irregularly shaped textures, demonstratingremarkable tolerance of the visual system for statistical variation.

The differential results for the flat and curved surfaces suggest independent channelsof information that are available in the optic array to observers for their use at differenttimes and in different situations. We argue that perspective information seems to be mostimportant for flatness judgments because that information is a component of an invariantavailable to'viewers about flat surfaces. We also argue that compression is important forcurvature judgments because it reveals potential nonmonotonicities in change of slant,the angle between line of sight and a line orthogonal to the local surface plane. InExperiment 2 we show that when the height of a curved surface is diminished enoughto create a nearly monotonic compression function, viewers cannot distinguish such asurface from one that is flat.

Finally, we suggest two things with regard to the existing literature on surface perception.First, although psychologists have been very interested in the perception of slant for aflat surface, we argue that slant is a largely irrelevant variable for the perception of flatness.That is, slant information is functionally related to compression gradients, and compression

198

PERSPECTIVE AND FLATNESS, COMPRESSION AND CURVATURE 199

gradients appear not to be used by perceivers in their judgments of flatness in surfaces.Instead, relative slants are important in the perception of curved surfaces, where compres-sion accounts for almost all of the variance in viewers' judgments of curvature. Second,although researchers in the field of artificial intelligence have largely shunned the studyof gradients as irrelevant to the perception of surfaces, we believe that our results suggestthat the information in gradients is crucial for their perception.

How do we perceive the layout of surfaces?What information is available to us, and whatinformation do we use? Such questions haveintrigued psychologists for a third of a century(see Gibson, 1950) and they are increasinglycentral to artificial intelligence approaches tovision (see Bajcsy & Lieberman, 1976; Marr,1982; McArthur, 1982; Stevens, 198la; Wit-kin, 1981). Gibson and his followers took theposition that we perceive surfaces with regardto slant—the angle of incidence between theline of sight and a line orthogonal to any par-ticular point on the surface of regard, calledthe surface normal. It is slant, according toGibson (1950; but see 1979, p. 196), that givesrise to various gradients of texture and thatpromotes the perception of surfaces recedingin depth.1

Continuing interest in perception of slanthas generated a large literature (see, e.g., Att-neave, 1972; Beck & Gibson, 1955;Braunstein& Payne, 1969; Clark, Smith, & Rabe, 1955;Epstein, Bontrager, & Park, 1962; Hock,Graves, Tenney, & Stephenson, 1967; Free-man, 1966; Gillam, 1970; Olson, 1974; Per-rone, 1980; Phillips, 1970; and Rosinski &Levine, 1976). This literature generally showstwo things: (a) observers are not very good atjudging absolute slant, and (b) they are quitegood at judging differences in slant. The firstof these findings seems unsurprising and notparticularly relevant to the perception of sur-faces; the second is relevant, but has not, inour opinion, been applied to surface perceptionin an appropriate way.

With regard to the first point, for example,we believe that observers do not often needinformation about absolute slant. When look-

ing out onto a flat terrain, one finds that theslant of the surface of support is perpendicularto the line of sight at one's feet, and that itgrades uniformly to the horizon where it be-comes parallel to the line of sight. At all points,the surface is flat, but slant is changing. It isdifficult to understand how computation (orpick up) of absolute slant would be relevantto the perception of the shape of the surfacewithout other information, such as where par-ticular portions of surface are with respect tothe observer. Moreover, the observer needs toperceive only that relative slants and tilts atdifferent points in the visual field give rise tothe shape of that surface and to the relativedistances of various points on it.

With regard to the second point, the dif-ference between slants of two patches of surfaceis also insufficient evidence in itself to deter-mine whether these two parts of a surface arecoplanar or noncoplanar. Information aboutspatial arrangement must also be available. Inother words, the same change in slant, de-pending on overall spatial arrangement andon viewpoint, could indicate a flat surface, acurved surface, or even two unrelated surfaces.Slant, as it turns out, appears to be crucialonly to the perception of certain kinds of sur-faces, and we will return to its discussion later.In essence, we demonstrate that slant is largelyirrelevant to the perception of flatness in sur-faces.

A second and related literature has notmeasured perceptual sensitivity to slant. In-stead, it has dealt with the phenomenalimpression of three-dimensionality of recedingsurfaces (see, e.g., Attneave & Olson, 1966;Braunstein,. 1976; Marr, 1982; Stevens, 1981a;

Supported by National Institute of Mental Health GrantMH37467. We thank Julian Hochberg, Barbara Metiers,Kent Stevens, and Deborah Walters for their comments.

Requests for reprints should be sent to James E. Cutting,Department of Psychology, Uris Hall, Cornell University,Ithaca, New York 14853.

1 Researchers now realize that tilt—the angle betweenthe surface normal and true vertical as measured on aprojection plane orthogonal to the line of sight—is a sep-arate and equally important variable (Marr, 1982; Stevens,198la, 1983). Nevertheless, it is slant that has capturedthe attention of psychologists.

200 JAMES E. CUTTING AND ROBERT T. MILLARD

Vickers, 1971; Witkin, 1981). It is this liter-ature, we believe, that is a more appropriatestarting point in trying to answer the questionsposed at the beginning of this article—howwe perceive surfaces, what information isavailable, and what information we use. Typ-ically in these studies, researchers employ sys-tematic variation in surface textures to studythe effect on perception of surfaces recedingin depth. This variation grades with visual an-gle (or height in field where 90° is at the ho-rizon and 0° is at one's feet), and gives riseto the study of gradients.

Gibson (1950, p. 78) was the first to discussthe perception of surfaces in terms of "stimulusgradients of the density of texture and the sizeof objects." This long and cumbersome phrasewas then shortened to "density gradients" orsometimes "texture gradients." These gra-dients were taken, by Gibson, to be the primarysource of information for the perception ofsurfaces. But gradients have speciated sinceGibson first wrote of them. We now know, forexample, that the density gradient, per se, isnot very effective in revealing a surface re-ceding in depth (Braunstein, 1976; Marr, 1982;Stevens, 198la), but this jumps us too farahead in our discussion.

Assumptions and Gradients

First, we must consider the nature of thetextures that pepper the surfaces around us.Whether we are talking about walls, floors,greenswards, parking lots, tundras, deserts, orocean surfaces, three properties that generallyhold are the, following:

Assumption 1: Texture elements areroughly the same absolutesize.

Assumption 2: Texture elements lie flat ornearly flat on the plane.

Assumption 3: Texture elements haveroughly uniform, absolutespacing.

These are the properties that give rise tosystematic change in the optic array. Study offlat surfaces has revealed many sources of in-formation that grade with visual angle. Theseare all separate texture gradients. Six such gra-dients are most often discussed. Each is typ-ically measured vertically in the picture planeunder the assumptions that the surface has

neither tilt nor horizontal perturbations. Thefirst is the perspective gradient, and it followsfrom Assumption 1. Perspective, sometimescalled scaling, can be measured in several ways.The most secure is the horizontally measuredrelative optic angle for any given texture. Thisgradient changes as a direct function of threeparameters: d or the distance of a particulartexture from the observer's feet, e or the eyeheight of the observer, and t or the size (radius)of any given texture in the real world. Ofcourse, the additional assumption that texturesare roughly circular is also being made. Theoptic angle P for the perspective of any giventexture along the line of slant is expressed asthe following:

P = 2 • arctan[*/(rf2 + e2)1'2]. (1)

Convergence, or linear perspective, is a specialcase of this gradient, an idea that we will returnto in the discussion of perspective and flatness.The second is the compression gradient, whichis sometimes called foreshortening. It followsfrom. Assumption 2. Compression, the pro-jectively vertical optic angle C for any giventexture element along the line of slant, is afunction of these same parameters:

C = arctan[(d + t)/e]

- arctan[(rf - t)/e], (2)

It is customary to measure compressionagainst perspective, hence the compressiongradients used to generate stimuli in this studyare actually C/P at a given distance and eyeheight. But because Equation 2 in its presentform will be helpful throughout this presen-tation, we will use it to stand for the inde-pendent information in compression gradients.The third is the density gradient, which resultsfrom Assumption 3. Density is the number oftexture elements per unit solid visual angle.Ignoring the difference between spherical andplanar geometry and assuming that the tex-tures are equally spaced at an average of 2«units apart in all directions, measured fromtexture center to texture center, the relativedensity gradient D at any point along the lineof sight can be specified as the following:D = l/{arctan[(rf + n)/e]

— arctan[(c? — n)/e]}

X 2 arctan[«/(rf2 + e2)1'2]. (3)


Notice that if t and « are systematically related,D is a function of P and C. A fourth gradientis the size gradient but this is always a straight-forward multiplication of P and C, a fifth isthe motion gradient (see Braunstein, 1976;Gibson & Carel, 1952), and a sixth is the in-tensity gradient (see Horn, 1975; Marr, 1982;Todd & Mingolla, 1983). The latter two arebeyond the scope of this article. The firstthree—compression, density, and perspec-tive—can be specified independently for astatic surface, and it is these gradients thatconcern us.

It is generally known that the perspectivegradient is more potent than compression (e.g.,Rosinski & Levine, 1976; Vickers, 1971) forthe perception of a flat surface receding in thedistance, and that the density gradient is nei-ther necessary (Marr, 1982, p. 237) nor suf-ficient (Braunstein, 1976, p. 48). But we donot generally know the full ordinal rankingsof these three gradients in their use for per-ception. One of the goals of this study is todetermine the relative importance of each ofthese gradients in the judgment of flat sur-faces—with the expected outcome that theperspective gradient would be more importantthan the other two. Moreover, importance isassessed not only in ordinal terms, but in pro-portional terms as well through the use of var-ious scaling and regression techniques. A sec-ond goal of this study is to investigate the rel-ative importance of these three gradients forthe perception of curved surfaces. Little isknown about the perception of these gradientsin nonflat conditions (although Todd & Min-golla, 1983, have studied intensity gradientsfor curvature). One might expect thatcompression would be relatively more impor-tant for curved than for flat surfaces, but noother clues to possible outcomes seem avail-able. What follows is an empirical analysis ofthe relative import of three gradients in theperception of flat and curved surfaces.

Experiment 1: Perceptual Assay ofFlat and Curved Surfaces

MethodTen viewers participated in four tasks: the 2 authors,

and 8 members of the Cornell community who were naiveto the notion of gradients and who were paid for theirservices. Two tasks entailed judgments of the flatness oftextured surfaces receding in the distance, and two entailed

the judgments of the curvature of textured surfaces recedingin the distance. Crossed with this flat/curved distinctionwas another: In two tasks the viewers were presented withsurfaces peppered with regular octagons, and in the othertwo they saw surfaces with irregular octagons. The fourtasks were called flat/regular, flat/irregular, curved/regular,and curved/irregular. Tasks were self-paced and took about35 min each to complete. The order of participation wasbalanced in a partial Latin Square design (with 2 observersin each of two orders, and 3 in each of two others). Viewersparticipated individually for two sessions, each lasting 1hr 15 min. Each task was preceded by a four-item practicesequence to familiarize viewers with the stimulus rangeand with the responses they were to make.

Each task consisted of 56 trials: all possible pairwisecomparisons among eight different stimuli (28) taken inboth left-right (L-R) and right-left (R-L) configurations.Each viewer was presented with a different random orderof trials. On a given trial, the viewer made two responsesof stimulus comparison. First, he or she indicated whichmember of the pair (L or R) looked more like a flat (orcurved) surface receding in the distance. This judgmentwas called the preference judgment, and formed the basisof Case 5 Thurstonian scaling, discussed later. Second, heor she indicated the degree (1 to 9) to which the preferredstimulus revealed flatness (or curvature) over the nonpre-ferred stimulus. This judgment was called a dissimilarityjudgment, and formed the basis of metric multidimensionalscaling, discussed later. Viewers typed their responses ona console, and individual data files were arranged andstored by computer. These responses, which were clearlyinferential and not direct on the part of the observer, wereused to make interpretations about phenomenal impres-sions of the stimuli. Thus, direct perception (Gibson, 1979)was not being tested; only the information that could servedirect perception was being assessed.

Stimuli were generated on a Hewlett-Packard (HP)1000L computer, and displayed in an HP1350S vector-plotting display system. Each stimulus pair was generateduniquely for each trial, for each condition, and for eachsubject. They were shown side by side to the viewer, witha vertical line between them. Since the experimental roomwas moderately lit, the sides of the monitor could be seenas well, forming a frame around the screen image. Thus,the stimuli looked like representations of depth rather thanreal depth. Sample stimuli for each of the eight trial typesin each of the four conditions are shown in Figures 1 and2. The eight stimuli for the flat/regular and flat/irregular'tasks are shown in Figure 1, and those for the curved/regular and curved/irregular tasks are shown in Figure 2.In each case, Stimulus 1 indicated a stimulus with nogradients appropriate to the task judgment, Stimuli 2-4have one appropriate gradient apiece: Stimulus 2 showsonly a compression gradient, Stimulus 3 shows only adensity gradient, and Stimulus 4 shows only a perspectivegradient. Stimuli 5-7 have two appropriate gradients:Stimulus 5 has both compression and density, Stimulus 6has both compression and perspective, and Stimulus 7 hasboth density and perspective. Stimulus 8 has all three ap-propriate gradients. In Figure 1, those stimuli without aparticular gradient appropriate for flatness have a distri-bution of textures that is uniform from the bottom to thetop of the panel. Technically, this means that the gradientis appropriate for a surface orthogonal to the line of sight


«^s

no• appropriate

gradients

compressiononly

densityonly

4 5 6 7 8neraiprtivp compression compression density all

onlv and and and appropriate' density perspective perspective gradients

/. At the top are eight sample stimuli representing flat surfaces peppered with regular textures, andat the bottom are eight with irregular textures. (Stimuli 1 have no gradients appropriate for a receding flatsurface, instead they are those of a surface orthogonal to the line of sight; Stimuli 2-4 have one gradientapiece, with Stimuli 2 having a compression gradient, Stimuli 3 a density gradient, and Stimuli 4 a perspectivegradient; Stimuli 5-7 have two appropriate gradients, with Stimuli 5 having both compression and density,Stimuli 6 compression and perspective, and Stimuli 7 density and perspective; and Stimuli 8 have all threegradients. These representations, because of their reduction in size, are not nearly so impressive as surfacesreceding in depth as the original stimuli. Because of the use of polar projection and because of the reductionin size, the computed station point for each stimulus would be about 2 in. [5 cm] away from this page.)

O O

po

compressiononly

~

o oc> o

o oo oo

o

6 8density

onlyperspective

onlycompression compression density all

and and and appropriatedensity perspective perspective gradients

O

Z3.

O

Figure 2. The same arrangements as in Figure 1, but for curved surfaces. (Those at the top have regulartextures and those at the bottom irregular textures. Inappropriate gradients are those for a flat, recedingsurface. Thus, Stimuli 1 in this figure are the same as Stimuli 8 in Figure 1. Again, these sample stimuliare much reduced and hence less impressive than those seen by the participants in the experiment.)


(such as a slightly curved wall) and inappropriate for ahorizontal surface receding from view. Since the regularoctagons are slanted, the textures in Stimulus 1 may bethose on a Venetian blind. Again, one should note thatcompression values are considered as C/P, not simply Cin Equation 2. In Figure 2, those stimuli without a par-ticular gradient appropriate for curvature have a gradientthat is appropriate for the flat surface of Figure 1. Thus,in Figure 1 the conflict of information in Stimuli 2-7 isbetween gradients that specify horizontal flatness versusnear vertical flatness, whereas in Figure 2 the conflict isbetween gradients appropriate for curvature and those ap-propriate for horizontal flatness. In other words, Stimuli1 in Figure 2 are the same (in terms of gradients) asStimuli 8 in Figure 1. Because of their reduction in size,the sample stimuli of Figures 1 and 2 are not nearly asconvincing as those viewed by the participants.

The projective geometry of the viewing situation wasas follows: The viewer sat so that he or she saw the stimulifrom the projected station point. If one considers the unitof distance from the viewer's eye to the surface of theprojected plane as 1 eye height, then the front of thatplane was at a distance of 2.0 eye heights from the observer'sfeet and the back of that plane at 5.0 eye heights. Thedistance across the front of the surface seen through thevisible window was 0.16 eye heights, and the distance acrossthe back of the seen surface was 0.36 eye heights. For flatsurfaces all textures were viewed from a uniform eye height;for curved surfaces, the curvature was convex, followingan inverse cosinusoid (1 - cos /3) with an excursion depthof 0.14 eye heights. That is, the front edge and back edgesOf the viewed surface were at 1.0 eye heights, but in betweenthe surface sloped upward such that the middle was 0.86relative eye heights below the station point. For an observer1.83 m (6 ft) tall, with an eye height of about 1.7 m, theheight of the cosinusoidal hill was 0.238 m (or 9.37 in.).This hill was one dimensional, with variation only fromfront to back (or bottom to top in the picture plane). Therewas no tilt and no variation in tilt. The relations are shownin the top panel of Figure 3. Notice that there were noocclusions.

To avoid the powerful cues of linear perspective, textures(regular or irregular octagons) were placed on the surfacesat random locations within cells on a grid. Each cell onthe surface was a rectangle. These cells projected as trap-ezoids on the scope face for flat surfaces and curved surfacesalike. For flat surfaces, regular octagons (projected as nearlyellipsoidal) were 0.0IS eye heights in diameter; irregularoctagons were formed by generating a random numberbetween 0.01 and 0.02 eye heights and using that valueas a radius (r value) for each of eight angles (8 values)spaced 45° around a circle. Thus, the irregular shapes hadthe mean radial statistics of the regular shapes, but noinvariant properties per se. For the curved surfaces thetexture-size statistics were twice as great, and their numberabout half as many. The rationale for this change wascomputational: Because each stimulus was computeduniquely for each trial and because the computations fora curved surface are much more complex, the intertrialinterval would have been inordinately long—on the orderof a minute—had the textures been as plentiful in thecurved surface conditions.

If one considers the horizon to be at an optic angle of90° and a point directly beneath one's eyes to be at 0°,

then the window through which the textured surfaces wereseen was between 63° and 78°. Using textures at 45° (orthe horizontal distance outward of 1 eye height) as an

geometry of the setting

VIEWING WINDOW

distance in eye heights

flat surfaceperspective

curved surface

60 70 80

OPTIC ANGLE90

Figure 3. Geometry of the viewing situation and gradientsfor two surfaces. (The upper panel shows the viewed surfacebetween 2 and 5 eye heights away from the observer, sub-tending the optic array between 63° and 79°, if the horizonis considered to be at 90°. The hill for the curved surfaceis 0.14 eye heights in elevation. The middle panel showsthe relative values of each of the three gradients throughthe optic array from 45°, 1 eye height distant, to 90° forflat surfaces. In the text this is called optic angle a. Verticalmarkers indicate the region within the window of the displayscope. All gradients are arbitrarily normalized at 45°. Thebottom panel shows the relative values of the three gradientsfor the curved surfaces. What is labeled as compressionis the function generated by Equation 2. True compressionis actually C/P, or the outcome of Equation 2 divided bythat of Equation 1.)


arbitrary anchor for calculating gradient values, the changein the C, D, and /"gradients for a flat surface can be seenin the middle panel of Figure 3. These are generated fromEquations 2, 3, and 1, respectively, and assume no variationin size, flatness, or density of textures.2 Allowing for randomnormal variation in these measures, which is found fordensity in all four tasks and found for compression andperspective for the two tasks involving irregular textures,we might have introduced confidence bands around thesefunctions. Thus, one could consider these gradient func-tions as broad smears with only their central tendenciesshown in Figure 3. Notice that all three are different, butwithin the window used in these experiments the changein each of the gradients is roughly comparable. The gra-dients for the curved surface are shown in the bottompanel of Figure 3. Here one can see that the change inthe gradients is rather marked for compression and density,but less so for perspective. On the basis of these plots, onemight expect that all three gradients would be about equallyimportant for judging the flatness of a surface within therange of 63° and 78°, and that Cand D would be relativelymore important than P in judging the curvature of surfaces.The results of the following experiment show that neitherof these predictions is obtained.

Analysis

The data for each subject in each task wereassembled into two 8 X 8 matrices with majordiagonals missing: one matrix for the prefer-ence data and one for the dissimilarity data.Consider first the preference data. Phi coef-ficients were computed for the 56 judgmentsfor each of the 10 different subjects for eachtask. Mean phi coefficients were .593 for theflat/regular task, .468 for the flat/irregular task,.336 for the curved/regular task, and .319 forthe curved/irregular task. Since these meansare all reliable (p < .025 or better), one cansee that there is good correspondence amongthe viewers in their preference judgments onall tasks.

The 10 individual preference matrices werefolded and then averaged to create a singlehalf-matrix for each of the four tasks. Thosematrices were then subjected to Case 5 Thur-stonian scaling (see, e.g., Baird & Noma, 1978)in order to obtain one-dimensional preferencescale values for the eight different types ofstimuli in each task. The average discrepanciesfor each scaling solution were quite low, in-dicating (a) an excellent fit of the data by chi-square goodness of fit tests and (b) additivityof all judgments. The average discrepancy forthe flat/regular task data was .07, x2(21) =21.09, p > .35; that for the flat/irregular taskdata was .05, x2(21) = 6.59, p > .99; that for

the curved/regular data .07, x2(21) = 9.84,p > .98; and that for the curved/irregulardata was .07, X

2(21) = 8.79, p > .99. Thescale coordinates for the four tasks are givenin Table 1.

The dissimilarity data were treated differ-ently. The individual matrices were first foldedacross the diagonal, averaging those judgmentsof dissimilarity involving the same stimuliprovided they passed a consistency check. Thischeck can be described as follows: Consider atrial presenting Stimulus 2 on the left side andStimulus 3 on the right as a 2/3 trial, and thereverse configuration as a 3/2 trial. If the ob-server responded L4 on the 2/3 trial and R7on the 3/2 trial, then the observer consistentlychose Stimulus 2 over Stimulus 3, and thedissimilarities would have been averaged andwould have equaled 5.5. If, on the other hand,the observer responded L4 on the 2/3 trial andL7 on the 3/2 trial, then the observer was in-consistent and preferred Stimulus 2 once andStimulus 3 once. In such cases the dissimilarityjudgments would have been considered theabsolute value of the difference, divided by 2:Thus |4 - 7J/2 = |7 - 4|/2 = 1.5. The datain the resulting half-matrices were then nor-malized to reduce the effect of individual dif-ferences in the use of the l-tor9 rating scale.

To ensure that individual variation wasminimal, the half-matrices for all subjects wereintercorrelated for each task. Mean correlationfor the flat/regular task dissimilarities was .842;that for the flat/irregular data was .781; thatfor the curved/regular data, .729; and that forthe curved/irregular data, .692. Since thesemeans are all reliable (p < .001 or better), onecan see that the dissimilarity data also havegood intersubject correspondence.

The 10 individual half-matrices were thenaveraged to create a single half-matrix for eachof the four tasks. Those matrices were thensubjected to multidimensional scaling, with

2 Notice that what is indicated as compression in thelower two panels of the figure stems from Equation 2 forthe flat surface and a modification of it for the curvedsurface. Compression, as defined for the stimuli used inthese experiments, is divided by perspective. The twofunctions are separated here because, for reasons that arenot analytically obvious to us, C/P = kP, or C(of Equation2) = kP2 (of Equation 1). In other words, the perspective(P) and compression (C/P) functions would be identical.


Table 1Resulting Coordinates for Thurstonian Scaling of Preferences and Metric One-DimensionalScaling of Dissimilarities

Flat surface Curved surface

Regular Irregular Regular IrregularGradient

1.2.3.4.5.6.7.8.

C

01001101

D

001010

. 11

p

00010111

Prefer-ence

2.9022.3921.9411.4471.3940.8950.7540.000

Dissimi-larity

1.3611.1650.626

-0.1490.244

-0.451-1.076-1.719

Prefer- .ence

2.3792.1061.5921.2251.0220.9070.4480.000

Dissimi-larity

0.8240.5950.3270.0910.104

-0.034-0.674-1.026

Prefer-ence

1.3470.1960.9741.2250.000

-0.0211.064

-0.108

Dissimi-larity

1.268-0.883

0.9131.005

-0.767-1.002

0.742-1.276

Prefer-ence

1.3600.0871.1781.4060.2870.1891.0520.000

Dissimi-larity

0.992-0.810

1.1561.058

-1.202-0.918

0.751-1.028

Note. C stands for the compression gradient (actually C/P), D for the density gradient, and P for the perspective gradient. Presence ofan appropriate gradient is indicated by 1, inappropriateness by 0. See the text and captions for Figures 1 and 2 for definitions ofappropriateness. For both preference and dissimilarity values, the less the number, the more the stimulus revealed either a flat or curvedsurface.

the idea that solutions in one dimension wouldbe optimal. Nonmetric scaling was attemptedat first, but this led to several badly degeneratesolutions. Metric scaling was then tried so thatsmall dissimilarities would not be underem-phasized. Both Equation 1 and Equation 2stresses were employed. Stress Equation 1 ledto somewhat more satisfactory one-dimen-sional solutions, where stress for the flat/reg-ular task data was .20, that for the flat/irregulardata was .33, that for the curved/regular datawas .22, and that for the curved/irregular datawas .19. Although direct comparison with thefits of the Thurstonian scaling is not possible,it is fairly clear that these data do not fit aswell. With nonmetric scaling it is generallysuggested (Kruskal, 1968) that with StressEquation 1, one-dimensional stresses of .20are poor. Because metric scaling will almostalways lead to stress values that are quite abit higher than corresponding nonmetric val-ues, it is probably safe to say that these fits tothe data are fair.3 Equation 2 stresses for thefour data sets given previously were .41, .47,.35, and .31, respectively. These values areroughly in keeping with the idea that StressEquation 2 values are about twice those ofStress Equation 1. The scale coordinates forthe one-dimensional Stress Equation 1 solu-tions for each task are given in Table 1.

Next, the stimulus coordinate values in theThurstonian scaled solutions and in the one-dimensional metrically scaled solutions wereused as dependent measures in multiple cor-relation and regression. Dummy values (1 =presence; 0 = absence) of the compression,density, and perspective gradients were usedas independent variables, as shown in Table1. Here, the presence or absence of each ofthe three gradients was used as a predictor ofthe scaled preference data and of the scaleddissimilarity data in each of the four tasks.Thus, eight different multiple correlations werecomputed, four for the preference data andfour for the dissimilarity data. Each R wasoutstandingly high. The smallest of the eightR values was .992, meaning that more than98% of the variance in each of the scalingsolutions is accounted for by the three di-chotomous variables: compression, density,and perspective. Thus, virtually none of thevariance in the scale data is attributable tonoise, and we can partial out the variance ac-counted for to the three independent variables.

3 Shepard (1974, p. 385) suggested that too much em-phasis has been placed on these evaluative labels. Theutility of scaled solutions is not so much determined bythe "badness of fit," as by interpretability. As we dem-onstrate, the scale solutions here are highly interpretable.


Since the three gradients occur in the eightstimuli in completely orthogonal manners(each correlates with each other, r = .00), thesemipartial correlations are the same as thesimple correlations. These values squared arethe proportions of variance accounted for inthe scaled solutions. They are shown in Table2, and are the main results of Experiment 1.Because of the low discrepancies in the pref-erence scale solutions and the moderate stressin the metric scale solutions, we felt justifiedin assuming independence of the three gradientmeasures.

Results and Discussion

There is a striking comparability among thefour analyses for flat surfaces, and among thefour analyses for curved surfaces. Regardlessof whether one considers the preference dataor the dissimilarity data, or whether one con-siders the data for the surfaces with regularlyshaped textures or irregularly shaped textures,the results are virtually identical. For the judg-ments of flat surfaces, between half and threequarters of the variance is accounted for bythe perspective gradient, between one quarterand one third by the density gradient, and verylittle by the compression gradient. For thecurved surfaces, on the other hand, essentiallyall of the variance is accounted for by thecompression gradient, and virtually none byeither the density or perspective gradient.

The use of the metric one-dimensional scal-ing solutions, even though they had moderatestress, appears justified given the comparabilityof multiple correlation results with those forthe preference data. The correlation of thevariances accounted for in preference and dis-

similarity data is very high (r = .992). Giventhe relative independence of the two measures(the consistency check in the dissimilarity datakeeps them from being completely indepen-dent), and given the comparability of resultsacross regular and irregular textures (r = .991),the pattern of results becomes undeniable: Theperspective gradient is by far the most im-portant source of information for judging flatsurfaces, but there is some contribution fromthe density gradient; and the compression gra-dient is the sole source of information usedfor judging curved surfaces. The negative cor-relation for the variances accounted for be-tween flat and curved surfaces (r = —.776)suggests further that quite different informa-tion is being used in the two situations. Al-though it is difficult to separate weights fromscale values in this type of design, the ideathat the variances accounted for reflects per-ceptual impact is pursued.

There is some problem about the generalityof the curved surface results. After all, everyflat surface is equally flat, but curved surfacescan vary widely, and in two dimensions, notjust the one dimension (projectively vertical)found in these stimuli. However, we think ourresults generalize to other (and perhaps all)curved surfaces given the modest size andsmoothness of the slope used here, and giventhe power of the results. Once local orientationof the surface, or tilt, is determined (Marr,1982; Stevens, 198la, 1983), differences incompression along the axis of tilt will alwaysbe of use as the determinant of local curvature.Thus, compression should equally well be usedto determine curves in walls and curves of asurface of support in both one and two di-mensions (see, for example, Marr, 1982, p.

Table 2Percentage of the Variance Accounted for by the Three Gradients in the Scale Solutionsfor Flat and Curved Surfaces

Flat surface Curved surface

Regular Irregular Regular Irregular

Prefer- Dissimi- Prefer- Dissimi-Gradient ence larity ence larity M

Prefer- Dissimi- Prefer- Dissimi-ence larity ence larity M

CompressionDensityPerspective

11.425.962.3

3.623.172.8

7.435.156.6

3.028.766.5

6.428.264.6

95.43.10.6

96.40.91.7

95.11.40.4

97.80.70.1

96.11.50.7


217, Figure 3-56b). We investigate compres-sion and curvature in more detail in a laterdiscussion section.

Mismatch of gradient change and percep-tion. The pattern of results for the flat andcurved surfaces cannot be accounted for interms of how much each of the gradientschanges across the display, bottom to top.Consider first the flat surfaces. As shown inthe middle panel of Figure 3, all of the gra-dients change a substantial amount, as gen-erated by Equations 1-3. The ratio of initial-to-final values in the perspective gradients,going from the bottom to the top of the dis-plays, is 2.27. The final-to-initial values of thedensity gradient are a ratio of 4.20, and theinitial-to-final ratio for the compression gra-dient is also 2.27, when taken as C/P. Thus,whereas the perspective gradient changes nomore than compression and changes less thandensity, it accounts for most of the variancein viewers' judgments.

This mismatch between results and thechange in the gradients is also found for curvedsurfaces. Here, the comparison is between thevalue of each of the gradients for the curve atvarious optic angles compared with those val-ues at the same angles for the flat surface. Thisentails a comparison between the lower andmiddle panels of Figure 3. At the inflectionpoints, C is as much as 42% higher and 70%lower in value for the curved surfaces com-pared with the flat ones. But assuming texturesare sampled throughout the density function(an assumption we will return to), density isas much as 206% higher and 39% lower in thesame comparisons. Perspective is the gradientthat changes least, and only in one direction:It is as much as 16% higher in the upper middleof the display. Most interesting is the fact thatthe compression gradient changes somewhatless than density, yet it accounts for essentiallyall of the variance in the viewers' responses.

The questions arise, of course, as to whythe perspective gradient (/>) should be the pri-mary source of information for flatness andthe compression gradient (C/P), for curvature.These questions are dealt with later. But firstwe emphasize that independent sources of in-formation—the various gradients—are beingused for different purposes in different situ-ations. We believe mat the real world providesa plenum of information that can be used in

different ways and at different times. This studyis a testimony to the fact that mere specifi-cation of the information available to a per-ceiver is a foolhardy course of action if we areto account for why we perceive things as wedo. The study of invariants and of variants(like the gradients here) in perception is in-complete without the demonstration that thesesources of information are used by the per-ceiver. Their differential use, as shown in thisstudy, suggests that we must go beyond theinformation given and study how informationis used (Hochberg, 1982).

Against strong inference. Finally, this kindof experimental analysis shows that we shouldbe wary of certain demonstrational analyseslike those, for example, done by Stevens(1981a;Marr, 1982; see also Braunstein, 1976).Stevens demonstrated that density gradientsare neither necessary nor sufficient to the per-ception of a flat surface by showing that asurface with perspective and compression gra-dients but no density gradients reveals a flatreceding surface, but that a density gradientalone, without perspective and compression,does not (see Marr, 1982, pp. 235-237). Yetthe results of this study show that density andcompression gradients together reveal flatnessin a receding surface better than perspectivealone for two out of four measures (see Table1, Stimuli 4 and 5), even though perspectiveby itself is the single most important gradient.4

This kind of quantitative analysis demonstratesthe shortcomings of certain qualitative dem-onstrations. For example, necessity and suf-ficiency may not always be the firmest ofgrounds on which to build systematic theoriesof perception. The method of strong inference(Platt, 1964) can be too strong, losing impor-tant subtlety in phenomena and giving mis-

* It is the preference scaling values, but not those fordissimilarities, that show this effect. Notice that it goesagainst the notion that 65% of the variance is accountedfor by perspective and that 35% is accounted for bycompression and density combined. Our only response tothis conundrum is that these comparisons have compar-atively high local stress (discrepancy) around them, sug-gesting a small violation of additivity of gradient infor-mation. The raw data in both cases show that Stimulus4 (with perspective only) was preferred over Stimulus 5(with compression and density) in 65% and 70% of alldirect comparisons in the flat/regular and flat/irregulartasks, respectively.


impressions of the combinatorics of infor-mation for visual perception. Of course, giventhat we have used a single tilt, a single viewingwindow, and one-dimensional gradients, weare subject to the same criticism. The pointremains valid, however, and we will be onlytoo pleased when better and more accurateassessments than ours are made.

Let us now turn to why it is that perspectiveis the salient gradient for the perception of flatsurfaces, and compression that for curved sur-faces.

Perspective, Flatness, and an Invariant

There are probably many ways in which ahuman perceiver or a computer vision systemmight use gradient information to judge theflatness of a surface. Because our data showthat human perceivers appear to use the per-spective gradient as most reliable in this do-main and because the other important gra-dient, density, is under normal circumstancescomputationally related to perspective (com-pare Equation 1 with the last part of Equation3 under Assumption 3), our tack is to try todetermine how a perceiver may determineflatness from perspective.

We make the assumption that the visualsystem is extracting some information fromthe visual display, and that that informationis invariant. Although the notion of invariantsfor perception is not without problems (seeCutting, 1983), invariants can provide anchorsof information for perceptual systems that canguide perceptual process. We also explore theecological validity (Brunswik, 1956) of thesesources of information in hopes of providingfurther constraints.

Immediately, however, one should recognizethat gradients cannot, by themselves, be in-variants: Things that grade, or change, withvisual angle cannot also be constant, or in-variant. Instead, the particular variant in mind;the perspective gradient, is assumed to be as-sociated with some other specifiable infor-mation in the optic array that is invariant, orat least roughly so. We assume further thatthe invariant information in the optic array isgenerated by some uniform property of thetextured surface under scrutiny. The latter as-sumption is essentially the same as we madeearlier: Texture elements are roughly of the

same absolute size. This assumption then feedsinto the former: Since size—or in our case thehorizontally measured visual angle—will gradewith distance, we can use invariant relationsof size and distance in the analysis of flatness.

The invariant that we are looking for is aninformation source that specifies flatness. Oneway to tackle this problem is to consider eyeheight, or e in the equations found at the ,be-ginning of this article, as an invariant of thesituation where an observer'finds him- or her-self in relation to a plane. Eye height, or thedistance to the eye from the plane as measuredalong a line orthogonal to that plane, is con-stant in relation to all points on the surfaceof the plane. If we can specify an invariant ofeye height whose basis is in the variable per-spective gradient combined with some othervariable information, then we may have aplausible invariant that perceivers could use.

Let us begin with a modification of the per-spective gradient from Equation 1:

P = 2 • arctan[k/(d2 + e2)1'2], (4)

where we are no longer directly interested inthe radius of a given texture t, but insteadsubstitute a constant k for t. (The assumptionof roughly equal sized textures is still made.)Let us next define an entity mentioned pre-viously, optic a, which is the angle from anygiven texture on the plane to the eye and thento the point on the plane just beneath the eye.In this manner, the optic angle « for the ho-rizon is virtually 90°, and that for any textureat one's feet is near 0°. Thus, a is essentiallythe information sometimes known as heightin the picture plane. The value of a dependson two parameters already used before, dis-tance from one's feet (or the point directlybelow the station point), and eye height:

a = arcten(d/e). (5)

Rearranging Equation 5 to express these re-lations in terms of distance, we find that d =e - tan a. Substituting for d in Equation 4, andagain rearranging the terms, we get the fol-lowing:

tan(P/2) = k/(e2 - tan2 «.+ e2)1'2. (6)

Solving Equation 6 for eye height, we get thefollowing:

e = fc/[tan2(P/2)-tan2 a + tan2(P/2)]1/2. (7)


At this point, we need to simplify this cum-bersome expression. First, because the hori-zontal visual angle P of any given texture willnot be very large, rarely larger than a few de-grees of visual angle, tan2(P/2) will be verysmall. For example, tan 2° = .03, and tan2

2° = .001. Thus, the value of tan2(P/2) typi-cally will be insignificant, and could be omittedfrom the last part of Equation 7. With its re-moval, the squares and square root operationscancel. Second, because for small angles a tan-gent transformation is very nearly a lineartransformation, we can consider tan(P/2) tobe equivalent to P times a constant. Accu-mulating all the constants (k, the divisor of P,and the constant of the tangent transformationfor small angle) into a new constant c, we findthe following:

a). (8)

In this manner, eye height is very nearly astraightforward multiplicative function of theperspective gradient P for any texture at anypoint in the visual field and tan a, where a isthe optic angle to the texture associated withthe particular P value. The results of Exper-iment 1, we believe, demonstrate that the per-spective gradient and its P values are used byperceivers, and it is fairly obvious that viewersought to be aware of a, or where within thefield of view they happen to be looking withrespect to their feet or to the horizon. For anygiven plane and the relation of a station pointto it, P • tan a is very nearly constant and couldserve as (nearly) invariant information onwhich judgments about the flatness of a surfacecould be made. Now, this is not to say thatwe think eye height is (directly) perceived; ouronly point is that invariant eye height is acorrollary to the flatness of a plane, and thatan invariant e could be information that theperceptual system uses. What the individualsees, and what viewers report seeing in ourexperiment, is a flat surface.

The force of this exercise is to demonstratehow an invariant that generates the perspectivegradient may be useful in judging flatness. Ofcourse, one can proceed through the same ex-ercise for the other two gradients. In a sketchyway, let us consider density first. Because ofthe similarity between Equations 1 and 3, thesame general end will be reached as in Equa-tions 4-8, although not quite so simply. To

make this type of analysis work for densityone needs to exchange the assumption ofroughly uniform texture size for one of roughlyuniform spacing. In many situations, such aslooking at trees on a savannah or at rocks ona desert, the latter assumption may be lesswarranted than the former. This fact, and per-haps the slightly increased cumbersomenessof the end expression (analogous to Equation8), may contribute to the density gradientweighing less in our perceptual assay. Anotherpossibility is that the perspective gradient, asshown in the middle panel of Figure 3, is verynearly linear for optic angles greater than 45°,whereas density is nearly exponential. This factmay make perspective information much moretrustworthy than density as a perceptual an-chor.

Consider next the compression gradient.Because of the manner in which compressionis determined in Equation 2, distance d quicklyfalls out of the equation when rearrangementsare made, and there is then no way to substitutea, or the general height in the plane of a tex-ture, back into the equation. Thus, one cannoteasily fix eye height to compression valueswithin the visual field. It would seem thatwithout this anchor, one would be at a dis-advantage, if not a loss, to determine e, al-though perhaps some other specification offlatness could be made. Another problem withthe use of compression is that it assumes thattextures lie flat on the plane. Rocks, trees, andanimals are usually clear violations of this as-sumption, perhaps rendering compression in-formation somewhat less useful for this pur-pose. We return to the role of compression inperception of surfaces later.

This, of course, has been only a formalanalysis of why perspective may be so impor-tant in the perception of flat surfaces. We re-gard it as plausible, but not necessarily defin-itive. More empirical work is needed. The es-sence of our argument is that the perspectivegradient of textures on a flat surface is specifiedby uniform eye height, which in turn is co-specified by the flatness of the surface. Thismay seem circular in logical form, but it isnot. It boils down to the idea that the opticalhorizontal extent of textures or objects (ofroughly uniform size), in conjunction with in-formation about their height in the field, issufficient information to specify flatness. This


characterization is an unfamiliar form of afamiliar pictorial cue, linear perspective.

Notes on Linear Perspective

Linear perspective, particularly as revealedin the familiar example of railroad tracks dis-appearing into the distance, is measured ex-actly this way. That is, under conditions of notilt, the width between the tracks is directlyrelated to height in the picture plane (or a).Because the bed on which the tracks are laidis flat, e is constant, and the tracks themselvesproject linearly.

One other fact about linear perspectiveshould be considered. Linear perspective oc-curs under conditions where there is no vari-ation in absolute size of elements and wherethere is also no variation in absolute density.In the railroad track example, the rail texturesremain exactly the same absolute size as theyrecede from view, and the distance betweenthe middle of the rails (hence, density) is alsoexactly the same. In the present experimentswe have decoupled the two, generating novariation in size but statistical variation indensity for surfaces with regular texture ele-ments, and generating independent variationin size and density for surfaces with irregulartexture elements. In both situations perspectiveand density accounted for a combined 90% ofthe variance or better in scale values acrossall measures, suggesting further independenceof these gradients. That is, regularity in theperspective gradient accounted for no more ofthe variance than did perspective under sta-tistical variation (regular vs. irregular-shapedtextures). We anticipate that in viewing situ-ations with linear perspective, the coupledperspective and density gradients would alsoaccount for about 90% of the variance. Sucha result would be consistent with Attneave andOlson (1966), Braunstein (1976), and Gillam(1970).

Compression, Curvature, and a Variant

As with judgments of flatness, there areprobably many ways in which a human orcomputer vision system may make judgmentsof curvature. Because our data demand thatwe consider compression gradients alone, wewill do just that. But first consider some ar-

guments why perspective and density gradientsmay not suffice for curvature judgments.

The perspective gradient essentially registersdistance. In three-dimensional settings, verticaldeviations from flatness are relatively smallwhen compared with horizontal extent. Con-sider for a moment a plastic environment witha hill that can vary in its height: Potentialchanges in the overall distance from the eyeto the peak of this particular plastic hill arevery small, due to potential variations in theheight of the hill. These Pythagorean relationsare captured as [(e — h)2 + d2]''2, where thenew term h is the hill height. Since both e andh will usually be small compared with d, theoverall distance from eye to peak will be de-termined primarily by the horizontal distancefactor d. That this is true is shown by com-paring the perspective gradients in the middleand lower panels of Figure 3. The fact thatthe perspective gradient changes little when asurface becomes nonflat makes it an unreliablesource of information in registration aboutdeviations from flatness.

The density gradient also measures distance,in part, and it does this through spacing oftexture elements. Scrutiny of Equations 1 and3 shows that density and perspective are relatedif t and « are related, but comparison of Equa-tions 2 and 3 shows that the same is true fordensity and compression. It is this latter re-lation that contributes to the sharp inflectionshown in the bottom panel of Figure 3 for thedensity gradient. One may think that the peakin this gradient would be a salient local cuefor the registration of curvature. However, itmay be that this peak is mostly a function ofthe continuous representation of density asshown in the figure; in the actual stimuli aninterpolated density function may not benearly so articulated. That is, given that therewas only an average of 13 "rows" of textureswithin the 15° window that comprised theprojection of each stimulus, there is on averageless than 1 row per degree. Because the peak,as it would reveal a nonmonotonicity in thedensity function, is less than 2° wide, theremay be too few textures within this region toregister the much increased local density. Thus,statistical variations that are present in thesestimuli and that would be likely in the realworld may render density a less efficacioussource of information about curvature.


This leaves compression. Because there isno absolute shape that is constant for all hills,we will not look for invariant informa-tion about curvature. Instead, because thecompression gradient changes broadly andnonmonotonically for the curved-surfacestimuli, as shown for the C function in thebottom panel of Figure 3, aspects of this vari-able function will be entertained. The questionis: What is it about these changes that makescompression a salient source of informationabout curvature? The answer is that compres-sion is part of the crucial information thatsignals to the observer the relative orientationof the texture element with respect to the ob-server. Optical 6, or slant, is the angle betweenthe line of sight and the line passing throughthe point of regard that is orthogonal to thesurface, or the surface normal (see Rock, 1964;Stevens, 1981b; Witkin, 1981). It is dennedby the following relation:

$ = arccos(C/P), (9)where C and P are the Equation 2 and Equa-tion 1 gradients for any texture element at any

optic angle a. As mentioned earlier, C/P is thecompression gradient, properly measured.Optical 6 for both flat and curved surfaces,then, is a function of compression, and it isthe ratio of the functions shown in the middleand bottom panels of Figure 3.

Experiment 2: Assessment of Thresholds forCurvature Perception

The compression gradient is shown in Fig-ure 4 for several types of surfaces. Function1 represents the gradient for the curved sur-faces used in Experiment 1, and Function 4represents that for flat surfaces. Function 2 isthe gradient for a hill of the same shape asFunction 1, but only half the height. Function3 is for a hill only one quarter of the originalhill height. For hill heights only a little greaterthan that used in Experiment 1, the ratio dipsbelow zero, indicating that occlusions have oc-curred. Since P remains essentially linear, mostof the inflection in these functions is due tovariations in C.

85 70 75

OPTIC ANGLE

Figure 4. The compression gradients (C/P) for those surfaces shown in the top panel of Figure 3, and fortwo other surfaces used in Experiment 2. (Function 1 indicates the gradient for the curved surface used inthe experiment; Function 2 is for a surface of the same shape but half the height; Function 3 for one-quarterthe height of that for Function 1; and Function 4 for a flat surface. The compression gradient is theinformation available to perceivers to determine slant, the angle between the surface normal and the lineof sight. Observers can judge that hills corresponding to Functions 1 and 2 are curved, but they cannot forthat of Function 3.)


One can see that Function 1 is highly ar-ticulated. A large inflection occurs nearlyseven-eighths up the viewing window, and thelocation of this inflection can be discerned inFigure 2 for Stimuli 2, 5, 6, and 8. Indeed, allviewers in the experiment said that they wereparticularly aware of this region in their judg-ments of the curved surfaces. Going back toFigure 4, one notices that the smaller the hill,the less articulated the function.

We strongly suspected that the nonmono-tonicity of the compression function was thesource of information used by perceivers intheir judgments of curved surfaces. In orderto determine if there was merit in this idea,we ran four observers through a variant of thecurved-surface task employed in the recentlydescribed experiment, Using Stimulus Types7 and 8 from Figure 2 in both regular andirregular versions, we had observers makepairwise comparisons as to which stimuluslooked more like a curved surface. We startedthe observers with hills identical to those shownin the figure, with maximum elevations of 0.14eye heights. All performed perfectly at this hillaltitude, preferring Stimulus 8 over 7. We thenhalved the altitudp (0.07 eye heights), whichgenerated the compression function shown asFunction 2 of Figure 4. Overall performancefell to 73%, but was still reliably above chance.Halving the hill height again (0.035 eyeheights), generating Function 3, reduced per-formance to chance. Again, there were nostrong differences between regular and irreg-ular textures on the surfaces; in fact, irregularlytextured surfaces with hill height elevations of0.07 eye heights were slightly more discerniblethan were their regularly textured counter-parts. We find it remarkable that the visualsystem is so little perturbed by statistical vari-ations found in the irregularly shaped textures.

We take these data, in connection with thefunctions shown in Figure 4, as indicating thatobservers determine the presence of curvaturesin surfaces by discerning whether there arediscernible nonmonotonicities in the com-pression gradient. Moreover, discernible non-monotonicities are present when observingsurfaces that have smooth curvatures whosemaximum coplanarities are only 0.07 timesthe eye height, within a window that is between2 and 5 times the eye height. Thus, in onestatic view a viewer about 1.67 m (5 ft 5 in.)

in height could discern a smooth curvature ina tiled floor of about 0.1 m (4 in.) in height,when that bump occurred throughout a dis-tance of 3 to 7.7 m (10 to 25 ft). With lesssmooth bumps of more local extent but thesame height, more striking changes incompression would occur, perhaps even gen-erating occlusions. It seems likely that thesewould be even more discernible. At greaterdistances, because of the geometry of occlu-sions, such hill sizes may be even more de-tectable than they are between 2 and 5 eyeheights. Add motion gradients and the abilityto detect noncoplanarities would surely getmuch more acute.

In summary, then, compression (measuredagainst perspective) determines the orientationof a particular patch of texture in relation tothe line of sight of the viewer. This angle, calledoptical 6 (Flock, 1964), is the much researchedentity called slant. It appears that changes inslant, per se, are not so effective in surfacejudgments as are changes in the changes ofslant. Slant for a flat surface cannot be relevantbecause we know that compression accountsfor so little variance in these judgments. Thus,we believe that most of the literature on slantjudgments has misapplied the phenomenallyimportant aspect of slant: Slant does not con-cern the phenomenal impression of flat sur-faces, it concerns the impression of curvedsurfaces.

Conjectures About ProcessThere are few inferences that we can make

about perceptual process, but some logicalconsiderations about process are in order. Per-spective and compression, as we have con-strued them, are orthogonally measured gra-dients. Perspective is measured at right anglesto the axis of tilt, and compression is the ratioof information along that axis as a functionof perspective. Thus, the registration of tilt(Stevens, 1983) is logically prior to registrationof gradient information shown useful in Ihesestudies.

Given roughly circular textures that projectas ellipses on the image plane, the line alongthe minor axis of any texture element willsuggest a local axis of tilt (Stevens, 1981b,1983; Witkin, 1981). Marr (1982, p. 234) sug-gests that this information is: extracted ex-plicitly, and we agree. This axis, if general to

PERSPECTIVE AND FLATNESS/COMPRESSION AND CURVATURE 213

the optic array, sets up the coordinate systemfor measuring the three gradients. So long asall textures agree in this measure (and all didin the studies reported here) the entire planewill have the same tilt. Small perturbations intilt by one texture in a field of others withidentical tilts are probably quite salient, sinceviewers are quite good at determining the tiltof individual patches of texture (Stevens, 1983;see also Witkin, 1981). These perturbationsindicate violations of global coordinates forperspective and compression measures, andamong other things would immediately suggestthe plane is not flat.

Following Marr (1982) and his students, wesuggest that tilt is extracted locally from eachtexture and from all textures in parallelthroughout the array. Global comparisonswould then be made across the array. We sug-gest further that within any region of the opticarray where a surface has generally uniformtilt (and such regions are not at all rare in thenatural environment) comparisons amongtextures will be made for perspective andcompression. Comparisons along the perspec-tive gradient are relevant for determinationsof flatness; comparisons along compression,for curvature. We make no claim that all theinformation in any gradient is used in makingjudgments about flatness or curvature, onlythat some of it must be used because it is thesegradient relations that carry all the informationabout textures in our displays. We do suggest,however, that all the gradient, information islikely to be registered in relatively early stagesof visual processing. In other words, we suggestthat gradients are there (in the stimulus andvisual system) to be used.

It is certainly possible that other kinds ofprocesses are used by the human visual systemin surface perception. Marr (1982, pp. 234-236), for example, doubts the efficacy of gra-dients in this domain, suggesting that they arederived measures that may have little to dowith perceptual process. We will return to thisidea several times while addressing three otherissues.

Three Other Considerations

We wish to address three other points: thatof local versus global analysis of textures, thatof mode of projection (orthographic vs. per-

spective), and that of the separability of gra-dients in the natural environment.

First, consider the issue of local versus globalanalyses. The issue here is often couched interms of priority—which comes first? Whereasthis may be an important issue in some sit-uations (Navon, 1977; Pomerantz, 1983), wedo not regard it as important here. What isimportant, we believe, is the level of analysisthat is most pertinent to perceiver and percept.For machine vision, Marr (1982) and others(Stevens, 198 la; Witkin, 1981) seem tostrongly prefer local approaches to global ones,and emphasize the utility of local analyses.That is, analysis begins on each local texture.The orientation of the texture is then discernedand the relations to other textures, computed.The relations to other textures are guided, inpart, by the assumption that neighboring ele-ments have smoothly changing differences be-tween them (Marr, 1982, pp. 44, 49). Thissmoothness assumption, we believe, almostguarantees measurable (and even differentia-ble) gradients. Gradients, in our view and assuggested above, are the global informationmeasures of surfaces. Since one needs texturesbefore one can have gradients of textures, wehave no qualm with the necessity of local op-erations before global ones. Our point is thatcertain relations among the local elementsemerge after this analysis, that these relationsare the gradients, and that the gradients arethe phenomenally relevant level of analysis.Claims against the effectiveness of gradients(e.g., Marr, 1982, pp. 233-239) would appearto be based in assertions about priority in thelocal/global issue, in incomplete understandingof the richness in different kinds of gradients,and in the failure to realize that the smoothnessassumption is precisely what can be tested byresearch on gradients.

Second, we believe that perspective (or polar)projection is absolutely essential to researchon the visual perception of surfaces. We> findit odd that most machine-vision approacheshaye used orthographic (or parallel) projection(Marr, 1982; Stevens, 198la; Ullman, 1979;Witkin, 1981; see also Sedgwick, 1980, andTodd, 1982, for analyses of projective meth-ods). Our objections to orthographic projec-tion in general are twofold: (a) Most objectsunder our scrutiny in the real world are rel-atively near to us. Perspective projection cap-


tures variation in nearness; orthographic pro-jection simulates infinite viewing distance, asituation approximated only when lookingthrough telescopes at small, distant objects.Thus, orthographic projection is "unnatural."(b) Orthographic projection nullifies all threegradients. That is, it is unnatural in a particularway. Under orthographic projection, and underthe assumption that textures are roughly thesame absolute size, flatness, and spatial sep-aration, the gradients cannot possibly be usefulto a perceiver because they will not be presentin the stimulus. Yet we believe that our resultsshow that gradient information is crucial injudgments of flat surfaces receding in the dis-tance.

To be sure, Ullman (1979, chap. 4) proposeda hybrid "polar-parallel" scheme, where localanalyses are carried out in an orthographicsystem of coordinates and then combined laterby comparing different axes of projection intoa polar system. This scheme, according to Ull-man (1979), is both computationally man-ageable and perceptually plausible. We haveno qualm with either of these points, nor withthe general scheme, but it does seem inelegant.Why not start with a polar scheme rather thanadding the extra step of converting to it? Ull-man (1979, p. 155) has three replies: He claimsthat a polar perspective system at initial stagesof image analysis (a) is inherently susceptibleto errors because of the relative smallness ofperspective eifects, (b) is usually computa-tionally more complex than a parallel system,and (c) leads to performance superior to thatusually attained by the human visual system.In response, we would say that (a) because theresults of our experiments suggest that smallprojective effects perfusing large regions of theoptic array can lead to large perceptual effects,redundancy can efficiently overcome mea-surement error in the image, (b) given ourbasic ignorance about the algorithms of thevisual system and our increasing knowledgeabout their complexity, there is no real reasonto feel constrained in our choice of compu-tational algorithms in biological systems bythe ease of implementation of them on digitalcomputers (see Marr, 1982, chap. 1), and (c)assessments of what the visual system can doare always filtered through responses, inter-vening processes, and motivational concernsthat can, and almost certainly do, degrade our

estimates of its ability. In other words, althoughwe feel that Ullman's polar-parallel schemeis workable, a purely polar scheme may beequally workable for a complex biological ma-chine, and the latter scheme is our estheticpreference. Esthetic judgments aside, theremay be little empirical justification to choosebetween the two.

Third, some researchers may object to ourorthogonal manipulation of gradients. Theworld, they may argue, does not present itselfto viewers with noncorrelated sources of in-formation as we have over the course of Ex-periment 1. We have two responses to thispoint, (a) Our data, in the sense that the scalesolutions show additivity of gradient infor-mation, do not show any systematic interac-tions or synergies of information. Thus, view-ers appear to treat the information separably.Of course, a counter to this response is thatthe demand character of the experimental set-ting and of the stimulus sets encouraged in-dependence. Our parry, then, is simply to statethat this is exactly what we hoped to show:Viewers can treat the various, independentlyspecified gradients as independent, (b) Moredeeply, however, we note that the three gra-dients investigated here are based on inde-pendent assumptions about the real world.Some of these assumptions hold in some en-vironments, but not in others, and the matchacross environments and assumptions seemsfar from perfect. Only in highly artificial en-vironments—-for example, when looking attiled floors inside buildings—are these threegradients indivisibly yoked.

In natural environments the three gradientsare, at best, moderately correlated. For ex-ample, consider the case when looking out asecond-story window at a piazza crowded withsightseers, where each person is considered atexture element. The assumption of equal ab-solute size of people-textures seems likely tobe more valid than the assumption of equaldistances among people-textures, which inturn is more valid than the assumption thatthe people lie flat on the plane (which is pa-tently false although they do stand on the flatplane). In contrast, consider the case whenlooking at the litter on the floor of an arenaafter a sporting event or a rock concert, wherethe pieces of litter are the texture elements.The assumption that the litter lies flat on the


plane seems likely to be more valid than eitherthe assumption about equal spacing or thatabout equal size. It seems likely, however, thatin the first case a flat surface is implicit, wherethe planar elements consist of people's headsabove the piazza; and in the second case theflat surface is equally implicit, where the planarelements consist of the shreds of paper andplastic. Thus, we believe that the world is fur-nished in such a way that gradients are sep-arable. Given the less-than-complete couplingamong the three assumptions, and their cor-responding gradients, separable sources of in-formation may be necessary for systematicperception in the less-than-systematic envi-ronments that we find ourselves in every day.

Conclusions

Throughout this article we have concernedourselves with three assumptions. The first as-sumption is that texture elements on a surfaceare of roughly the same size. By this we meanthat although there may be textures differingin size as boulders and blades of grass, mostall boulders are of the same size and most allblades of grass are of the same size. We donot see that statistical variation is a problem,and to a remarkable degree the human visualsystem doesn't either: The results with ran-domly shaped (and thus differently sized) tex-tures were no different than those with reg-ularly shaped textures on either flat or curvedsurfaces. This size assumption is the basis forthe regularity of the perspective gradient. Forflat surfaces, two regularities fall out of theserelations. First, the optically measured hori-zontal extent of any texture element is a func-tion of distance; and second, distance is afunction of optical angle a, or height in theplane. These relations, of course, are not per-fectly invariant, but they are generally so andcan serve as the information source that per-ceivers use about the flatness of a surface.

The second assumption is that texture ele-ments lie flat or nearly flat on the plane. Thisassumption is probably justifiable in fewer sit-uations than is the first assumption; as men-tioned earlier, trees, animals, and many otherobjects that we find on surfaces are not func-tionally flat. Even grass yields a compressiongradient inverse to that of flat textures: Look-ing down on the blades foreshortens them

more than looking at them a few yards awayfrom one's feet. But absolute flatness againstthe surface need not always be attained, be-cause the randomly shaped textures can beinterpreted as objects, like rocks, which eruptout of the surface to somewhat differentheights. Also, compressionlike gradients canbe found elsewhere than on the surface of sup-port. A flock of sheep seen from a distancereveals a compressionlike gradient as measuredacross the flanks and backs of each animal.The flatness assumption, of course, is the basiswe use for compression gradients. As discussedearlier, it is also the basis of measures of slant.We believe that the data of the present studydemonstrate that compression is not an im-portant gradient for the perception of a flatsurface, but that it is crucial for the perceptionof curvature. What observers appear to discernare changes in the changes of compression, orthe second derivative. More particularly, non-monotonicities in the compression gradientappear to be salient local sources of infor-mation about curvature. Unlike the use ofperspective information, especially as it revealsthe invariant of eye height and hence the flat-ness of a plane, the use of compression in-formation appears to entail no invariant.Variants, particularly local inflections in thegradient, appear to be most important.

The third assumption is that texture ele-ments have roughly uniform spacing. Becausewe were interested in perspective gradients,and not linear perspective, we chose to staggerthe texture elements in all of our displays,making the statistical separation between ele-ments uniform or nonuniform depending onour purposes. It seems to us that the assump-tion of even roughly equal distribution maybe the most problematic of the three assump*tions for most situations. The problem maynot be so much that objects in the world arenot roughly uniformly spaced; they may in-deed be in more situations than not. But theuniformity of spacing may be out of scale tothe purposes of the animal. Acacia trees maybe uniformly spaced on a savannah, but thisprobably does not mean that their spacingwould be a good source of information forflatness or curvature judgments. There are fartoo few of them. Perspective would be betterfor the former judgment, and treetop heightfor the latter. Relative densities may be either


too low as in this case, or too high in othersas in the case of blades of grass (Flock, 1964),for density values to be perceptually relevant,or even perceptually discernible. The data ofExperiment 1 appear to indicate that densitygradients are of secondary importance, andprobably useful only insofar as they reinforceperspective gradients, as in the case of linearperspective.

References

Attneave, F. (1972). Representation of physical space. InA. W. Melton & E. Martin (Eds.), Coding processes inhuman memory (pp. 283-308). New York: Halsted Press.

Attneave, E, & Olson, R. K. (1966). Inferences aboutvisual mechanisms from monocular depth effects. Psy-chonomic Science, 4, 133-134:

Baird, J. C., & Noma, E. (1978). Fundamentals of scalingand psychophysics. New Yjrk: Wiley.

Bajcsy, R., & Lieberman, L. (1976). Texture gradients asa depth cue. Computer Graphics and Image Processing,5, 52-67.

Beck, J., & Gibson, J. J. (1955). the relation of apparentshape to apparent slant in the perception of objects.Journal of Experimental Psychology, 50, 125-133.

Braunstein, M. L. (1976). Depth perception through motion.New York: Academic Press.

Braunstein, M. L., & Payne, J. W. (1969). Perspective andform ratio as determinants of relative slant judgments.Journal of Experimental Psychology, 81, 584-590.

Brunswik, E. (1956). Perception and the representative de-sign of psychological experiments, Berkeley: Universityof California Press.

Clark, W. C., Smith, A. H., &Rabe, A. (1955). Interactionof surface texture, outline gradient, and ground in theperception of slant. Canadian Journal of Psychology, 9,247-253.

Cutting, J. E. (1983). Four assumptions about invariants.Journal of Experimental Psychology: Human Perceptionand Performance, 9, 310-317.

Epstein, W., Bontrager, H., & Park, J. (1962). The inductionof nonveridical slant and the perception of shape. Journalof Experimental Psychology, 63, 472-479.

Flock, H. R. (1964). A possible basis for monocular slantperception. Psychological Review, 71, 380-381.

Flock, H. R., Graves, D., Tenney, J., & Stephenson, B.(1967). Slant judgments of single rectangles at a slant,Psychonomic Science, 7(2), 57-58.

Freeman, R. B. (1966).. Absolute threshold for visual slant:The effect of stimulus size and retinal perspective. Jour-nal of Experimental Psychology, 71, 170-176.

Gibson, J. J. (1950). The perception of the visual world.Boston: Houghton Mifflin.

Gibson, J. J. (1979). The ecological approach to visualperception. Boston: Houghton Mifflin.

Gibson, J. J., & Carel, W. (1952). Does motion perspectiveindependently produce the impression of a recedingsurface? Journal of Experimental Psychology, 44,16-18.

Gillam, B. (1970). Judgments of slant on the basis of fore-shortening. Scandinavian Journal of Psychology, 11, 31-34.

Hochberg, J. (1982). How big is a stimulus? In J. Beck

(Ed.), Organization and representation in perception (pp.191-218). Hillsdale, NJ: Erlbaum.

Horn, B. K. P. (1975). Obtaining shape from shading in-formation. In P. H. Winston (Ed.), The psychology ofcomputer vision (pp. 115-166). New %rk: McGraw-Hill.

Kruskal, J. B. (1968). How to useM-D-SCAL, a programto do multidimensional scaling and multidimensionalunfolding. Unpublished manuscript. Bell TelephoneLaboratories, Murray Hill, NJ.

Marr, D. (1982). Vision. San Francisco: Freeman.McArthur, D. J. (1982). Computer vision and perceptual

psychology. Psychological Bulletin, 92, 283-309.Navon, D. (1977). Forest before trees: The precedence of

global features in visual perception. Cognitive Psy-chology, 9, 353-383. '

Olson, R. K. (1974). Slant judgments from static and ro-tating trapezoids correspond to rules of projective ge-ometry. Perception & Psychophysics, 15, 509-516.

Perrone, J. A. (1980). Slant underestimation: A modelbased on the size of the viewing aperture. Perception,9, 285-302.

Phillips, R. J. (1970). Stationary visual texture and theestimation of slant angle. Quarterly Journal of Exper-imental Psychology, 22, 389-397.

Platt, J. R. (1964). Strpng inference. Science, 146, 347-353.

Pomerantz, J. R. (1983). Global and local precedence:Selective attention in form and motion perception.Journal of Experimental Psychology: General, 112, 516-540.

Rosinski, R. R., & Levine, N. P. (1976). Texture gradienteffectiveness in the perception of surface slant. Journalof Experimental Child Psychology, 22, 261-271.

Sedgwick, H, A. (1980). The geometry of spatial layoutin pictorial representation. In M. Hagen (Ed.), The per-ception of pictures, (Vol. 1, pp. 33-90). New York: Ac-ademic Press. '

Shepard, R. N. (1974). Representation of structure in sim-ilarity data: Problems and prospects. Psychometrika,39,373-421. ;

Stevens, K. A. (1981a). The information; content of texturegradients. Biological Cybernetics, 42, 95-105.

Stevens, K..A. (1981b). The visual interpretation of surfacecontours. Artificial Intelligence, 17, 47-73.

Stevens, K. A. (1983). Surface tilt (the direction of slant):A neglected psychophysical variable. Perception & Psy-chophysics, 33, 241-250.

Todd, J. T. (1982). Visual information about rigid andnonrigid motion. Journal of Experimental Psychology:Human Perception and Performance, 8, 238-252.

Todd, J. T, & Mingolla, E. (1983). The perception ofsurface curvature and direction of illumination frompatterns of shading. Journal of Experimental Psychology:Human Perception and Performance, 9, 583-595.

Ullman, S. (1979). The interpretation of visual motion.Cambridge, MA: MIT Press.

Vickers, D. (1971). Perceptual economy and the impressionof visual depth. Perception & Psychophysics, 10,23-27.

Witkin, A. P. (1981). Recovering surface shape and ori-entation from texture. Artificial Intelligence, 17,17-45.

Received'May 9, 1983Revision received August 3, 1983 •

three gradients and the perception of flat and curved...

Documents