Dr Evelyn Goldsmith is aResearch Fellow inthe Department of VisualDommunication, BrightonPolytechnic

Author's address:Department of VisualCommunicationBrighton PolytechnicGrand ParadeBrighton BN2 2JY

Evelyn Goldsmith 1980

Information designJournalVol 1, 204-213

204

Comprehensibility of illustration- an analytical model

Evelyn GoldsmithDepartment of Visual Communication, BrightonPolytechnic

I have read the paper by Clive Ashwin (1979) inthe first issue of this journal with a great deal ofinterest, not least because of its basic premise thatillustrations can be analysed and that, on both apractical and theoretical level, it may often behelpful to do so. For some years I have myselfbeen developing a model for the analysis ofillustration, and more than once have encounteredthe suggestion that such a proposition 'is neitherpossible nor desirable. My own feeling isthat ananalytical framework should be seen as a valuablethinking tool for illustrators, students, publishers,teachers and anyone else who produces or usesillustration: the very opposite of the formulas and

.superficial guidelines which critics of researchseem to assume are the inevitable outcome.

There are, of course, different ways of, anddifferent reasons for, taking illustration apart. Mystudy grew from an interest in illustration'ssupportive educational role, particularly where it isintended to help adult beginning readers identifywords; and although considerations of preferencesfor various styles are certainly not irrelevant in thiscontext, I decided to try and construct a modelthat would most usefully lend itself to theevaluation of the comprehensibility of illustration.The result used alongside, or perhaps integratedwith, Ashwiri's proposals could provide the basis

It is suggested that"themeans to analyse iliustrationwould be of great benefit to students, illustrators,publishers and others, and a model· is presentedidentifying /2 elements which the author believescontribute to the comprehensibility of pictures,particularly those intended to be in support oftext.

for a comprehensive analytical structureencompassing considerations of the cognitive andaffective aspects of illustration.

Here briefly are the main points of the modelpresented in full in my thesis (m!l4$ij1ii!ii:;1~78);:

Analytical proposals 1: Semiotic levels( suggest that comprehension of a picture involvesthree different levels of response rL'A response tographic signals as a discr-iminable image or set ofimages. 2-)A response to images in terms of themeanings the artist intended to set down. J}Aresponse to the artist's meanings in terms of all theelcvant previous experience and present judgment

of the viewer.The similarity to a passage in the Bullock

Report (1975) describing the skills involved in thecomplex process of reading is deliberate. Theadaptation is to underline my argument thatcertain parallels can be drawn between theunderstanding of text and of pictures.

My first proposal therefore is that the study ofsupportive illustration should be undertaken atthree levels; and since Morris (1938) has provideda suitable basis for such a distinction in his theoryof signs, I have adop-ted his terminology. The firstlevel, known as syntactic, has already been welldescribed by Ash win. It could be summarised as

the level which does not presuppose any recognitionor identification of image. The second and thirdlevels, both referred to by Ashwin as semantic, arcdesignated by Mortis as semantic and pragmatic.These appear to correspond respectively withAshwirr's 'semantic properties which are reasonablyobjective, verifiable and stable', and 'a moresophisticated semantic level, that of interpretationof the image in terms of psychological, emotional,aesthetic or moral significance' (p.53). This higherlevel implies interpretation by a viewer; ind at thiss"tagecomprehension of a pictu~~n:!.~y depend ondevelopmental, culturalii1d-srmilar factors. Thereis a sad little cartoon by Charles Addams whichexemplifies the three levels clearly. It shows twounicorns stranded on a small mound surroundedby water, with an ark sailing away in the distance.The syntactic level requires an ability to distinguishthe bounds of the separate images and to perceivepictorial depth -,The semantic level demandsrecognition of the images as unicorns, water, andan ark. But the point of the drawing is lost at thepragmatic level if the viewer is' unaware of thestory of the Flood and the current scarcity ofunicorns. It must be understood, of course, that inpractice the levels are often interdependent, andthat in any ease consideration of the higher levelsautomatically presupposes the existence of thosebelow.

Analytical proposals 2: Visual factorsUnity:Attempts have often been made to establish a'grammar' of Visual forms, most of which (egl)on;dis.,'l97J') take docs, lines and so on as theirbase. However, I agree with M'oriis(1946) whoargues that the component lines of a painting areno more signs than are the individual phonemes ofa spoken language. I have called this first factorunity: a term I am not completely satisfied with,since it could cause confusion. However, I havebeen unable to think of any other suitable wordwhich means 'oneness', and I hope that thecontext in which it is used will resolve potentialambiguity. Unity, then, will refer to any area in a

picture which might be recognised as having a .separate identity, even if the identity is notknown. The degree of separateness is obviouslygoing to vary with the level of discussionappropriate to the intention of the picture or theinterest of the viewer: for example, in a landscapethe single image might be 'cow', while in a portrait

: it might be 'eye' or even 'pupil'.

LocationAs soon as there is more than one image in anarray, a second factor has to-be considered: thespatial relationship between the images. Itparticularly encompasses the various devicesavailable for depicting pictorial depth, such asoverlap, relative distance up the picture plane, andthe different forms of gradient identified byGipson (1950'): gradients of tone, size, clarity,texture and so on.

EmphasisThis factor also refers to the relationship between'images, but is hierarchical rather than spatial.Many of the experiments relating to the learningeffects of pictures set out to test simplicity versuscomplexity of visual material (egHolmes, 1963,;Dwyer, 1972:). But as Fuglesang (1973)' points out,it is often not the simplicity or complexity in itselfwhich is important, but the amount of relevantdetail. Just as a verbal exposition sometimes needsto consider a number of issues in order to presentan argument in its true perspective, so a drawingcan lose much of its communicative value if in anattempt at simplicity it is denied an appropriatecontext. But in neither situation need the increasein complexity imply a resultant move towardsconfusion. The successful visual communicatorwill make clear the successive levels of importancein even the most complex pictorial representation;and such control is surely preferable to whatAmheim. (1954) calls poverty of abstinence.

Text parallelsThe three factors so far outlined all refer to thepictorial aspect of the communication. But earlier

205

Figure 1:Analytical model showinginteraction between factorsand levels

I mentioned that my concern was with supportiveillustration; and the analysis would be incompletewithout consideration of the relationship betweenthe-picture-and its text. If. as in the case of booksfor beginning readers, the picture is thought of asa translation of the text, or vice versa, andresponses to the picture do not reflect reasonablyaccurately the meaning expressed verbally, thenfor some reason the translation is inadequate. Textparallels are proposed as the factor which promotesverbalisation of the translation process. It is thestage at which the assessor of the picture/textrelationship looks at the text and asks himselfwhich of the textual elements can be portrayeddirectly, which can be suggested indirectly, andwhich cannot be portrayed at all.

Levels Syntactic Semantic Pragmatic

FactorsUnity 1 2 3Location 4 5 6Emphasis 7 8 9Text parallels 10 11 12 Analytical model

A model can now be constructed, based on theinteraction between the four factors identified andthe three semiotic levels at which each factor canbe considered. This gives a total of 12 elementswhich I propose contribute to the compre-hensibility of supportive illustration (Figure 1).

The numbers shown at each of the co-ordinatepositions on the model refer to the sections belowwhich give a very brief outline of the compass ofeach element. Where appropriate I have includeda pair of examples in which the particular elementhas been isolated as a variable, so that the first ofthe pair represents its less comprehensible aspect.

Figure 2A: Syntactic unity:line print

206

Figure 28: Syntactic unitY:halftone print

Syntactic unityAlthough some psychologists claim that there is noperception without recognition, I would suggestthat in some circumstances a mark on a surfacecould be generally accepted by all obververs as oneimage rather than a group of images: it would havewhat RuhinU91S) calls 'thing-character'. Inpractice, of course, images arc rarely simple.Printed photographs, for example. are reproducedby a halftone process, and in drawings andpainting contours are often left open and boundsto images implied rather than expressed.

Some of these conventions can rely on theobserver's expectations, and in this case it could beargued that without recognition the image wouldassume a different shape in the mind of theobserver, more closely related to its retinalprojection: that is, it would probably be interpretedliterally as a wiggly line or a flat patch.

Figure 3A: Semantic unity : few distinguishing featuresFigure 38: Semantic unitv : strong distinguishing features

.... ;...-;

Nevertheless, some perceptual tendencies havebeen shown to exist which are claimed by thegestalt theorists to be a result of organisationalfactors present in the visual cortex. The theoriesare well-known, and it may be sufficient here torecall that two of the major phenomena identifiedare the tendency to group marks, by proximity,similarity and so on; and to complete figures. Fordetails, reference may be made to almost any writeron perceptual psychology Cqt Vernon,.19.;S2;Zusne, i970}.

The implications of this for incomplete orunderstated images must be apparent. In Figure2A the tolerance of perceptual processes has beenstretched quite considerably; yet of 38 people towhom I have shown this line print.ronly four couldnot idcntify it correctly.

Semantic unityIn syntactic unity it is only necessary to be able todistinguish one from among a number of images,or to discern that what ~ay appear to be discrete.marks are in fact intended to represent a singlefigure. Semantic unity, however, requiresrecognition of the image, and representationadequate for this purpose relies not on slavishimitation of an object, but on the clarity ofdistinguishing features which give relevantinformation.

The number or strength of distinguishingfeatures that any object possesses controls theextent to which its image can be distorted. In thecase of many round objects such as certain fruits,for example, a single feature such as colour orscale may be crucial; whereas a picrure of a humanbeing can undergo extensive modification (eg as ina cartoon by Gerald Scarfe) without becomingunrecognisable.

In Figures 3A and 3B the fruits werephotographed to the same scale and in exactly thesame conditions, and the prints were made toduplicate the degree of focal distortion. Thequality of focus can be seen to be irrelevant in thecase of the banana, but important for the positiverecognition of the apple. Wrong identificationshave included orange,-peach, and even pea.

207

Figure 4A: Pragmaticunity: impoverishedinformation

Pragmatic unityRelying on probabilities in our environmentenables us to make rapid decisions. If everypercept had to be checked we could scarcelyfunction at all. But this implies familiarity with asituation, reliant on such factors as age, culture,training, interests and so on. F-~iliarity carloTcourse work for or against the viewer. Most of usat some time have been misled by faulty perceptionwhich has been a result of expectations that tumout to be unwarranted.

On the whole, however, familiarity providessupport in the form of understanding of contexts.An example is given in Figures 4A and 4B. In 4Bimpoverished information is supplemented by acontext which leaves no room for doubt as toidentity 1 provided that the viewer is aware of thestory of the Garden of Eden.

6Figure 48: Pragmaticunity: the benefit ofcontext

.:••..

208

Syntactic locationThere are many cues [Q depth in a picture, some ofwhich were mentioned earlier, and all of whichhave been known to painters for centuries. Towarrant inclusion at the syntactic level the devicefor depiction must be independent of the viewer'srecognition of the image; and one of the mostcompelling is the gradient of texture usc.~_~~evariable in Figures SA and 5B.

I have used baubles in this example to avoid theinevitable assumption that the objects are at thesame horizontal level ; thus removing relative .upward location in the picture plane as a potentialcue to depth in SA.

Figure 5A: Syntactic location: gradient cue missingFigure 58: Syntactic location: gradient cue present

Figure 6A: Semanticlocation: scale cue missing

Figure 68: Semanticlocation: scale cue present

Semantic locationThis does not refer to the way in which locationaffects the semantic level of reading a picture, butthe way in which the semantic level affects theassessment of location. In a line drawing with fewindications of depth, a useful cue can be variationsin size of a known object. However, this can be a-wcak cue, and Hoch f;> erg (1.9'64) has shown thatalthough in a laboratory situation differently-sizedplaying cards will suggest that some are moredistant than others, a contradictory frame ofreference locating them on one plane will easilynegate the previous assumption.

In Figures 6A and 68 all cues to distance are'removed with the exception of known size, and itis expected that in 6B the extra information willfacilitate judgements of relati~e distance.

Pragmatic locationIn some cases, the process of inferencecharacteristic of the pragmatic level can becounted on to correct the sort of perceptualanomalies that are most prevalent in the work ofphotographic novices, but not unknown in moreprofessional circumstances. In viewing the realworld, in order to see a reflection in a shop

window or an object in the background, we haveto change the focus of our eyes quite substantially;and unless we make a conscious effort to do thissuch events will go unnoticed. However, presentedin two-dimensional form, no change of focus isneeded and the disturbing element becomesapparent.

.The extcn t to which the un wanted juxtapositioninterferes with the information value of thepicture depends on two things: the experience ofthe viewer in reading pictures, and the acceptabilityof the portrayed situation. If a literal interpretationof a scene would be absurd, it will usually bediscounted, but dislocations unnoticed by theartist or photographer can lead to misinterpretation.

In an experiment by Smith and Watkins (1972)to test ways of drawing children's attention todangerous objects in the home, a complex drawingof a kitchen shows a young child reaching uptoward the handle of a steaming saucepan on thestove. However, one subject, noticing the child'shand level with the gas taps, assumed it wasturning them on and listed this as the hazard.

In Figures 7A and 7B the paintbrushes are inboth cases in a jar on a shelf about 15 inches awayfrom the man. In 7A, however, the juxtapositionleads to misreading, while in 7B it does not.

209

F igu re 7A: Pragmaticlocation: juxtapositionwhich could be misread

Figure 78: Pragmaticlocation: unambiguousjuxtaposition

210

Syntact ic emphasisI am using the term emphasis to describe thcaspect of pictorial organisation that helps theviewer extract relevant information. It is largelyconcerned with attracting and directing attention;and one way of classifying attention identified byJames (1890) is by making a division betweenobjects of sense, which attract sensorial attention,and ideal or representational objects demandingintellectual attention.

These two categories correspond usefully withthe syntactic and semantic/pragmatic levelsadopted here, and it becomes a simple matter toidentify the types of visual stimulus appropriate tothis particular element. Attention can be attractedby contrast of all kinds: tone, colour, size,direction, etc; position within the picture frame,isolation and so on i and directed by linear devicessuch as edges of roads, swords, arms, or echoes ofcolours or shapes that tend to group in the mannerdescribed by the gestaltists.

In Figure 8B attention is attracted by contrastof organisation. The slightly displaced spot has nosemantic value, and in 8A receives no attention atall. People asked to mark a spot which attractsthem in Figure 8A concentrate on the centre four,and in particular the spot top left of centre: aphenomenon confirmed byJ3;Jtindt(l~'4:S:) usingeye movement cameras.

Figure 8A: Syntacticemphasis: regular visualarray

Figure 8B: Syntacticemphasis: irregularitycausing attraction

eeee_eeeeeeeeee_

eeeeeeeeeeeeeeee

Semantic emphasisEye movements have been studied extensively bya number of researchers, notably Buswell (1935)who· confirms that of all representational imagesthat having most universal attraction is the humanface: in particular, the eyes. The direction ofattention at the semantic level is a simple matter inany picture containing human beings: onceattracted to the eyes of the main figure the naturaltendency is to follow the direction of gaze. Moresubtly, direction may be achieved at the syntacticlevel: an example of the two levels workingtogether in the organisation of the picture,

Pragmatic emphasisSince the pragmatic level depends upon theexperience and interests of the individual viewer:...•itmight be thought difficult to evaluate this clemdntof an illustration. However, certain considerationsarc common to large groups of people, and it isworth remembering that some perceptualphenomena mentioned in connection with thesyntactic level may be influenced by developmentalor cultural factors.

The tendency of viewers to look top left ofcentre in Figure 8A is likely to be.partly due toreading habits. Also under this heading can comeemphasis which is arranged arbitrarily: colour-coding, for example, as in the London Undergroundsign system.

Syntactic text parallelsThe use of the word syntactic in connection withtext parallels should not suggest an analysis of thetext itself. The syntactic level here refers to therelation between the pictorial and the verbal signs.

As with syntactic location, the concern is withspatial relationships: the position on the page, orseries of pages, -of the picture relative to its text.Smith and Watkins (1972) carried out a well-planned series of experiments designed to test thiselement. Some interesting points emerged, but thefindings are complicated to summarise and onewould do well to read the original report,

211

Brandt (1945) also did some investigations withan eye movement camera which demonstrated thatif a picture and text are placed horizontallyadjacent the text receives more attention if thepicture is on the left. Brandt proposes that this isdue to the picture's having greater attraction valuethan'the text. This means that attention is initiallydrawn to the left, and reading habits then make itmore natural to move to the right.

Semantic text parallelsIn semantic unity consideration is given to thequestion of distinguishing features of an object,and problems of identification. However, teachersusing flash cards containing a single image andword are familiar with the danger of not wrongbut alternative identification.

In an experiment to exemplify this element, Idrew two pages of objects, one containing itemsthat could be expected to produce a reliable singleresponse, and the other containing objects havingseveral alternative names. A picture of an apple,for example, was consistently named correctly,while a jumper was labelled jumper, jersey,pullover, woolly and sweater.

Pragmatic text parallelsIn a book in which the intention is that the text"should be closely supported by the illustration, itcould be said that the text and illustration areintended to be symbolic of the same reference.The fact that the mode of sy~bolisat;on isdifferent and the reference often not concretemakes for difficulties, however.

I have made an analysis of the first group ofSchoneil's(932) 'essential spelling' list comprisingabout 400 words which it is proposed form thebasic vocabulary of. children. Even in this mostconcrete of levels I could find only abou t 120words that could be directly illustrated (of whichonly 80 could be called reliable). Of the remaining280, almost half would defeat the most ingeniousartist: for example 'hope', 'only', 'yet', 'soon';while the rest present problems that arc notinsuperable, given reasonable context.

212

It is the function of text parallels to considerthis context; and at the pragmatic level thecommon experience of a known readership can beassumed. It is not possible to draw 'a cold' forexample, bu t we can observe not only its symptomsbut the habits of people suffering from such anailment.

This element implies in the artist theobservational powers to recognise the significanceof quite small informational cues; and in theviewer the experience to notice and take advantageof such cues when they are offered. Much of thedepiction may rely not on denotation, the literalcomponents of the drawing, but on connotation:the invisible elemen t brough t to the drawing bythe viewer. A pin man with a hat and stick, forexample, will often be described by youngerviewers as 'an old man'.

It is an awareness of such connotations thatmakes reasonable correlation between text andpicture possible at all; and if a picture fails it maybe because pragmatic text parallels have not beensufficiently clearly thought out. Once a decision ismade how to suggest a certain state - 'old', 'tired','a cold' and so on - the responsiblity lies with thepictorial factors. If it is agreed that a pin man willlook old given a hat and a Stick, and nobodyrecognises the hat as a hat, then it is the fault ofone of the levels of unity; but if everyone gives acorrect literal description of the drawing withoutincluding the word 'old' then the fault is with textparallels. Either it must be agreed that 'old' cannotbe suggested, or another way must be found to doit.

ConclusionI have suggested that it is important to be able toanalyse illustration, and have proposed a modelthat would facilitate this process. There is onlyroom here for brief examples of each clement, andit is in any case doubtful whether any list couldever be exhaustive.

It is likely that most established illustratorsalready have the experience to know whether ornot their drawings will work in the context in

which they are set; but I feel that a method ofanalysis could be an invaluable aid to students whomay not be aware that emphasis, for example, .functions at different levels, and who may beunwittingly dissipating attention by allowingsyntactic and semantic forms to cancel each otherout.

Analytical studies of renowned artists couldyield understanding of principles that could beadapted to other styles and applications, andperhaps make their own work less reliant on directderivation. Control of complexity, and informed

. Ashwin CliveThe ingredients of style incontemporary illustration: acase studyInformation Design Journal1979, vol 1,51-67

Brandt HermanThe Psychology of SeeingNY: Philosophical Library1945

Bullock ReponSee: Education and Science,Department of

Buswell GTHow People Look at PicturesUniversity of Chicago Press,1935

Dondis Donis AA Primer of Visual LiteracyCambridge Mass: MIT Press1973

Dwyer Francis M JrA Guide for ImprovingVisual ised InstructionLearning Services. StateCollege, Pennsylvania 1972

Education and Science,Depanment of

A Language for Life: Reportof the Committee of Inquiryappointed by the Secretary ofState tor Education andScience (Bullock Report)London: HMSO 1975

experimentation, could result in more excitingartwork in the field of information. Publishers,teachers, resource? centres and so on could findanalysis and the terminology pertaining to ithelpful in evaluating and discussing pictures thatthey arc proposing to use.

The model outlined here was designed toquestion the comprehensibility of illustration, butit could easily be used for other purposes, andpossibly integrated with Clive Ashwin's analysis ofstyle to provide a basis for a more comprehensiveapproach .

Fuglesang AApplied Communication inDeveloping Countries. Ideasand ObservationsUppsala Sweden: The DagHammarskjold Foundation1973

Gibson JJThe Perception of the VisualWorldBoston: Houghton 1950

Goldsmith EwlynAn Analysis of the ElementsAffecting Comprehensibilityof Illustrations Intended asSupportive to TextUnpublished CNAA PhDthesis. Brighton Polytechnic1978

Hochberg Julian FPerceptionNJ: Prent ice-Hall 1964

Holmes Alan CA Study of Understanding ofVisual Symbols in KenyaOVAC London 1963

James WilliamThe Principles of PsychologyNY: Dover 1950 (First edition11390)

Morris CharlesFoundations of the Theory ofSigns vol 1 no 2Foundations of the Unity ofScience InternationalEncyclopaedia of UnifiedScienceUniv. of Chicago Press 1938

Morris CharlesSigns, Language and BehaviorNY: Braziller 1946

Rubin ESvnoplevede FigurerCopenhagen 1915. Translatedin Readings in Perception edBeardslee DC and WertheimerMPrinceton 1958

Schonell Fred JThe Essential Spelling ListLondon: Macmillan Education1932

Smith J and Watkins HAn Investigation into SomeAspects of the Illustration ofPrimary School BooksTypography Unit, Univ ofReading 1972

Vernon MDA Further Study of VisualPerceptionCUP 1952

Zusne LeonardVisual Perception of FormNY: Academic Press 1970

213