overview of imaging science - pnasproc. natl. acad. sci. usa vol. 90, pp. 9746-9750, november1993...

5
Proc. Natl. Acad. Sci. USA Vol. 90, pp. 9746-9750, November 1993 Colloquium Paper This paper was presented at a coUoquium entitled "Images of Science: Science of Images," organized by Albert V. Crewe, held January 13 and 14, 1992, at the National Academy of Sciences, Washington, DC. Overview of imaging science ROBERT N. BECK Center for Imaging Science, The University of Chicago, Chicago, IL 60637 ABSTRACT The traditional disciplines of science are grounded in the observation and measurement of object prop- erties. Recent advances in digital computer technology have spawned numerous computer-based imaging systems that ex- tend the range of observation and measurement into realms that would otherwise be inaccessible. More importantly, the same set of principles, concepts, strategies, and methods mar be used to address the generic issues involved in the production and use of all digitized images. Recognition of this fact is giving rise to the new discipline of imaging science, with its own inteUectual agenda. In recent years, the production and use of images have become increasingly essential to advances in science and technology, to biomedical research and clinical practice, to commerce and industry, to entertainment and advertising, and to communication and education in virtually every field. In short, images are now commonplace-an important and ubiquitous part of everyday life and culture-and undoubt- edly will remain so. Despite this fact, imaging science does not yet exist as a well-defined discipline with its own intellectual agenda; rather, it is currently at about the stage in its evolution that computer science was forty years ago. At that time, compu- tation was equally commonplace, but few, if any, Depart- ments of Computer Science existed within our universities, and virtually no one was referred to as a "computer scien- tist." However, once the principles, concepts, and methods common to all digital computations were recognized, and were empowered by advances in electronic circuit compo- nents that could be switched rapidly between binary states, attention was focused on such general issues as alternative computer architectures and the solvability of various classes of problems, as well as on the more specific issues, such as optimization of computer design to perform computations more rapidly, accurately, and reproducibly. In parallel with advances in computer hardware, workers in many fields took up the task of developing software to meet their own interests and needs, many of which required improved means for producing digitized images and extracting information from them. As a result, workers scattered throughout the tradi- tional disciplines have contributed to the explosion of ad- vances in computer-based imaging hardware and software that we see today. In turn, these advances have resulted in a growing recognition of the similarity of issues that must be addressed, whether one deals with images produced by a computer-based microscope, telescope, or medical imaging system. It is primarily this recognition, coupled with a general desire to reduce redundancy of effort by building upon advances made by others, that is rapidly giving rise to The publication costs of this article were defrayed in part by page charge payment. This article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. §1734 solely to indicate this fact. the exciting new discipline of imaging science, which is destined to have an ever-increasing impact on all our lives. My purpose in this overview is to provide one perspective on how imaging science might be defined, to identify its scope and its underlying principles, concepts, strategies, and meth- ods, to define its unique intellectual agenda (which are still being carried out in an ad hoc fashion by workers in numer- ous departments of our universities, national laboratories, and industries, while functioning in relative isolation), and finally, to suggest what might be done to accelerate further advances and, ultimately, to establish imaging science as an important new discipline. Conceptual Basis of Imaging Science We begin with the generally accepted premise that all of the traditional disciplines of science are grounded in the obser- vation and measurement of object properties, both static and dynamic, and note that the earlier development of imaging tools such as the optical microscope, telescope, and photo- graphic film extended the range of natural vision and enabled us to observe, to record, and to measure certain otherwise inaccessible object properties by means of their images. More recently, dramatic advances in computer science and tech- nology, and in radiation detectors, have facilitated the de- velopment of computer-based imaging systems, which have further increased enormously the number of object properties that can be observed, as well as the accuracy and reproduc- ibility with which they can be measured. And although imaging science is currently not well defined in terms that are generally agreed upon, from our perspective it encompasses the detection, spatiotemporal localization, recording, dis- play, visual observation, and measurement of an ever- increasing number of important object properties from im- ages that are obtained from an ever-increasing variety of imaging modalities. As a result, there is a growing need for a set of unifying principles, concepts, and methods for imaging science. Before I proceed, it is also worth noting that, as an example, just as the objects of interest to a biological scientist are living systems, the "objects" of interest to an imaging scientist are imaging systems. Both the biologist and the imaging scientist are concerned with developing a better understanding of the structure and functions of a broad class of complex material systems, and to that end, each attempts to develop principles, concepts, and methods that are broadly applicable and easily communicated. As it happens, the "objects" of interest to imaging scientists are also among the tools used for the study of objects of interest within all of the traditional disciplines, including biology. To avoid confusion, I will refer to the "objects" of interest to imaging scientists as imaging systems, whereas all other objects will be referred to simply as objects, be they living or nonliving. On this basis, objects, imaging systems, images, and imaging science may be defined, or characterized conceptually, in the following terms: 9746 Downloaded by guest on October 8, 2020

Upload: others

Post on 30-Jul-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

  • Proc. Natl. Acad. Sci. USAVol. 90, pp. 9746-9750, November 1993Colloquium Paper

    This paper was presented at a coUoquium entitled "Images of Science: Science ofImages," organized by Albert V.Crewe, held January 13 and 14, 1992, at the National Academy of Sciences, Washington, DC.

    Overview of imaging scienceROBERT N. BECKCenter for Imaging Science, The University of Chicago, Chicago, IL 60637

    ABSTRACT The traditional disciplines of science aregrounded in the observation and measurement of object prop-erties. Recent advances in digital computer technology havespawned numerous computer-based imaging systems that ex-tend the range of observation and measurement into realmsthat would otherwise be inaccessible. More importantly, thesame set of principles, concepts, strategies, and methods marbe used to address the generic issues involved in the productionand use of all digitized images. Recognition of this fact is givingrise to the new discipline of imaging science, with its owninteUectual agenda.

    In recent years, the production and use of images havebecome increasingly essential to advances in science andtechnology, to biomedical research and clinical practice, tocommerce and industry, to entertainment and advertising,and to communication and education in virtually every field.In short, images are now commonplace-an important andubiquitous part of everyday life and culture-and undoubt-edly will remain so.

    Despite this fact, imaging science does not yet exist as awell-defined discipline with its own intellectual agenda;rather, it is currently at about the stage in its evolution thatcomputer science was forty years ago. At that time, compu-tation was equally commonplace, but few, if any, Depart-ments of Computer Science existed within our universities,and virtually no one was referred to as a "computer scien-tist." However, once the principles, concepts, and methodscommon to all digital computations were recognized, andwere empowered by advances in electronic circuit compo-nents that could be switched rapidly between binary states,attention was focused on such general issues as alternativecomputer architectures and the solvability of various classesof problems, as well as on the more specific issues, such asoptimization of computer design to perform computationsmore rapidly, accurately, and reproducibly. In parallel withadvances in computer hardware, workers in many fields tookup the task ofdeveloping software to meet their own interestsand needs, many of which required improved means forproducing digitized images and extracting information fromthem. As a result, workers scattered throughout the tradi-tional disciplines have contributed to the explosion of ad-vances in computer-based imaging hardware and softwarethat we see today. In turn, these advances have resulted in agrowing recognition of the similarity of issues that must beaddressed, whether one deals with images produced by acomputer-based microscope, telescope, or medical imagingsystem. It is primarily this recognition, coupled with ageneral desire to reduce redundancy of effort by buildingupon advances made by others, that is rapidly giving rise to

    The publication costs of this article were defrayed in part by page chargepayment. This article must therefore be hereby marked "advertisement"in accordance with 18 U.S.C. §1734 solely to indicate this fact.

    the exciting new discipline of imaging science, which isdestined to have an ever-increasing impact on all our lives.My purpose in this overview is to provide one perspective

    on how imaging science might be defined, to identify its scopeand its underlying principles, concepts, strategies, and meth-ods, to define its unique intellectual agenda (which are stillbeing carried out in an ad hoc fashion by workers in numer-ous departments of our universities, national laboratories,and industries, while functioning in relative isolation), andfinally, to suggest what might be done to accelerate furtheradvances and, ultimately, to establish imaging science as animportant new discipline.

    Conceptual Basis of Imaging Science

    We begin with the generally accepted premise that all of thetraditional disciplines of science are grounded in the obser-vation and measurement of object properties, both static anddynamic, and note that the earlier development of imagingtools such as the optical microscope, telescope, and photo-graphic film extended the range of natural vision and enabledus to observe, to record, and to measure certain otherwiseinaccessible object properties by means oftheir images. Morerecently, dramatic advances in computer science and tech-nology, and in radiation detectors, have facilitated the de-velopment of computer-based imaging systems, which havefurther increased enormously the number ofobject propertiesthat can be observed, as well as the accuracy and reproduc-ibility with which they can be measured. And althoughimaging science is currently not well defined in terms that aregenerally agreed upon, from our perspective it encompassesthe detection, spatiotemporal localization, recording, dis-play, visual observation, and measurement of an ever-increasing number of important object properties from im-ages that are obtained from an ever-increasing variety ofimaging modalities. As a result, there is a growing need for aset of unifying principles, concepts, and methods for imagingscience.Before I proceed, it is also worth noting that, as an

    example, just as the objects ofinterest to a biological scientistare living systems, the "objects" of interest to an imagingscientist are imaging systems. Both the biologist and theimaging scientist are concerned with developing a betterunderstanding of the structure and functions of a broad classof complex material systems, and to that end, each attemptsto develop principles, concepts, and methods that are broadlyapplicable and easily communicated. As it happens, the"objects" of interest to imaging scientists are also among thetools used for the study of objects of interest within all of thetraditional disciplines, including biology. To avoid confusion,I will refer to the "objects" of interest to imaging scientistsas imaging systems, whereas all other objects will be referredto simply as objects, be they living or nonliving. On this basis,objects, imaging systems, images, and imaging science maybe defined, or characterized conceptually, in the followingterms:

    9746

    Dow

    nloa

    ded

    by g

    uest

    on

    Oct

    ober

    8, 2

    020

  • Proc. Natl. Acad. Sci. USA 90 (1993) 9747

    Objects. Objects are generally three-dimensional (spatial-ly), but may be regarded as having four (or more) dimensionswhen temporal (or multiple additional) attributes are consid-ered. At any point in time, an object may be characterized byits physical/chemical/isotopic composition at each point inthe space it occupies. Alternatively, an object may be char-acterized and classified by properties related to its compo-sition, such as self-radiance, reflectance, fluorescence, re-laxation time, opacity/transmittance, conductance, oracoustic impedance at each point-i.e., by the numericalvalues associated with properties that can be detected andlocalized in space and time. [Additional object propertiesmay be derived or inferred from the reconstructed images. Asexamples drawn from the field of medical imaging: x-raycomputed tomography (CT) permits the quantitative deter-mination of local values of the concentration of electronsfrom the measured local attenuation coefficient of tissue;magnetic resonance imaging (MRI) permits the local deter-mination not only of the concentration but also of the mobilityand the chemical bonding of elements such as hydrogenwithin tissues; positron emission tomography (PET) permitsthe local determination of glucose metabolism, blood vol-ume, blood flow, oxygen utilization, receptor binding, andother parameters of tissue function. Similar statements canbe made regarding measurements made with various micro-scopes used for the study of materials and biological speci-mens, satellite- and space-probe-based telescopes and radarsystems for studies of the earth and planets, and varioustelescopes for astronomical observation and measurement.Clearly, objects ofinterest range from elementary particles toclusters of galaxies. The solar system, and especially theearth and all its living forms, are objects of immediateinterest. The human body and particularly the human brain-objects of immense complexity-are of very special interest.Properties of all such objects are being studied currentlywithin the traditional disciplines of science by means ofcomputer-based imaging systems that provide informationneeded for an understanding of their structure and functions.As a result, much of what we know about ourselves and theobjects in the world around us has been derived from images.]Imaging Systems. Imaging systems perform the function of

    mapping certain object properties into image space. Thesemappings, or images, are always incomplete in the sense thatonly detectable properties are mapped, and these are fre-quently mapped as projections into a plane, rather than intothree-dimensional image space. In addition, these mappingsare always inaccurate in the sense that imaging systemsinvariably introduce some degree of blurring, distortion ordeformation, interference, and artifact production. More-over, these mappings are always imprecise, or irreproduc-ible, in the sense that random statistical fluctuation, or noise,is always present to some degree.

    Images. Images (including the imperfections introduced bythe imaging system) may be characterized by the spatial (andpossibly the temporal) distribution of properties such as lightreflectance, color, brightness, or opacity/transmittance ateach point-i.e., by the numerical values associated withsuch properties. Some images are entirely synthetic, beingproduced by computer simulation, based on mathematicalmodels of objects and, in some cases, hypothetical imagingsystems. (Original works of visual art, including photo-graphic images of objects, may be regarded as objects, andare judged by aesthetic criteria. On the other hand, repro-ductions may be regarded as images, and are evaluated interms of their correspondence to the original.)Imaging Science. Imaging science is concerned with the

    principles, concepts, strategies, and methods of image for-mation and the measurement of object properties from theirimages. On a very concrete level of abstraction, the goal ofimaging science is to make images that are more accurate

    representations of objects, by reducing blurring, distortions,artifacts, and interference; more reproducible, by reducingrandom fluctuations or noise without increasing observationtime; more complete, by producing three-dimensional imagesofthree-dimensional objects, and by increasing the number ofobject properties that can be imaged; and more understand-able, by rendering and displaying images in a manner thatmakes the extraction, interpretation, and assimilation ofinformation more accurate and reproducible and more effi-cient for the observer.

    Generic Issues of Imaging Science

    Much of the intellectual content of imaging science is asso-ciated with the generic issues that must be addressed, or stepsthat must be taken, in the optimal production and use ofimages in specific applications. Topics related to the genericissues are generally associated with the traditional disciplinesof mathematics, statistics, computer science, physics, engi-neering, biophysics, and biopsychology, although importantcontributions to imaging science continue to be made byworkers in such fields as astronomy, geophysical science,materials science, the biomedical and radiological sciences,and the graphic arts. Below is a brief sketch of the genericissues and some of the topics that are relevant to a deeperunderstanding of these issues.As has been suggested already, image formation, in the

    most general sense of the term as it is used here, involves theconcept of the image as a mapping of some detectable objectproperty into image space-i.e., with the classification ofobjects in terms of their detectable physical, chemical,and/or isotopic properties, alternative strategies for perform-ing the mapping, the mathematical concepts and assumptionsused for modeling ofthese strategies, and the classification ofimperfections inherent in all such mappings. [Relevant topicsinclude the following. Mapping in real space by geometry (raytracing), diffraction imaging, the solutions of forward andinverse problems, point measurements (scanning), and pro-jections. Linear and nonlinear systems, shift-invariance.Images of point and line elements, sensitivity, spread func-tions, conVolution integrals and the convolution theorem.Mapping in frequency space by continuous and discrete Abel,Fourier (FFT), and Hankel transforms; spatial-frequencyresponse; cascaded stages; conventional and generalizedtransfer functions; wavelets. Image imperfections that limitthe accuracy and reproducibility of measurements of objectproperties: blurring, distortion, interference, artifacts, andnoise.]Development of the comprehensive understanding of im-

    age formation that is needed to optimize mappings requiresdetailed attention to all of the steps involved. Generally, thesteps that are taken in digital imaging procedures include thefollowing.

    Image-Data Acquisition. Acquisition requires consider-ation of the physical principles that govern the detection andlocalization of a particular object property, as well as theparticular strategy to be used in image formation. Theseconsiderations govern the design of the front-end hardware ofthe imaging device (e.g., the optical components of a micro-scope) which, in turn, determines the sensitivity, resolution,and other quality characteristics of the imaging system.[Relevant topics include the following. Radiation sources andtheir properties (particles and waves): thermal (black-body),laser, nuclear, synchrotron, acoustic, seismic, etc. Waveequations, propagation of radiation, and the effects of tur-bulent media: absorption, scattering, refraction, reflection,diffraction, coherence, and interference. Radiometry andphotometry. Collimators, coded apertures, lenses, opticalelements, and adaptive optics. Radiation detection: film,semiconductors, charge-coupled device (CCD) arrays, scin-

    Colloquium Paper: Beck

    Dow

    nloa

    ded

    by g

    uest

    on

    Oct

    ober

    8, 2

    020

  • Proc. Natl. Acad. Sci. USA 90 (1993)

    tillation detectors, transducers, antennas, and phased arrays.Efficiency, linearity, and dynamic range of radiation detec-tors. Sampling theory: Whittaker-Shannon theorem andNyquist limit, ideal vs. integral sampling, convex projection,and aliasing. Poisson processes and shot noise, spectraldensity, correlation function, Wiener-Khintchine theorem,and signal-to-noise ratio. Spatial, temporal, and energy res-olution. Trade-offs between sensitivity and resolution, noiseand contrast. Windowing and multichannel spectroscopicapproaches. Use of mathematical models and computersimulation of stochastic and nonstochastic processes foroptimization of parameters of the image-data acquisition(detector) system for a particular class of objects.]Image Recovery. Recovery-reconstruction and process-

    ing-involves the mathematical concepts and algorithms forproducing an image from the acquired data (e.g., imagereconstruction from projections), and for processing theimage to improve its quality or usefulness (e.g., to sharpenedges and to reduce distortion, interference, artifacts, andimage noise). [Relevant topics include the following. Recon-struction by analytic, statistical, and probabilistic algorithms:inverse problems; Radon transforms; limited-angle recon-struction/ill-conditioned problems; singular value decompo-sition; wavelets; regularization; filtered back-projection;Fourier methods; expectation maximization/maximum like-lihood (EM/ML) algorithms; Bayesian approaches; use of apriori knowledge and heuristic assumptions; Gibbs priors;maximum entropy; and closure phase. Noise smoothing:running mean and median filters and matched filters. Edgesharpening: Wiener and Metz filters, unsharp masking, con-trast enhancement, and motion deblurring. Multimodalityimage registration and fusion: weighted aggregation of im-ages. Nonlinear operations: background subtraction and his-togram equalization. Computer simulation of algorithms.]Image Recording and Distribution. Recording and distribu-

    tion involve the mathematical algorithms for image compres-sion (i.e., reduction of redundant information in images thatare to be stored, networked, and/or transmitted), imageclassification schemes and search strategies for rapid re-trieval, and network architectures and strategies for control-ling access to and the sharing of stored images, software, andfacilities. [Relevant topics include the following. Data com-pression: orthogonal transforms of Walsh, Hadamard, andKarhunen-Loeve; Huffman and Q-coding; wavelet analysis;and fractals. Storage media: film, magnetic tape/disks, op-tical disks, and CD/ROM. Image formats and standards anddocumentation. Entropy and information theory, error cor-rection, and packet switching. Electrical, optical, and micro-wave links and interfaces. Networks and protocols. Picturearchiving and communications systems (PACS). Neural net-works and associative memories.]Image Display/Visualization. Display and visualization in-

    volve strategies for three-dimensional display of images ofreal objects and dynamic processes, and/or of hypotheticalobjects and processes obtained from computer simulationsbased on mathematical models, as well as the graphic displayof multiparameter data sets representing complex phenom-ena-e.g., demographic and economic data. [Relevant topicsinclude the following. Physical modeling; surface and volumerendering; depth coding; shape, texture, shading, and mo-tion/animation; perspective; and parallax. Three-dimen-sional display by stereoscopy and holography (static anddynamic). Virtual reality. Dynamic range of display: black/white, pseudocolor, and color. Conventional cathode-raytube, high-definition television, touch-screen, and liquidcrystal displays. Hard copy: impact, ink jet, and laser print-ers; lithography; xerography; and facsimile. Graphics andimaging workstations. Interactive display of multimodalityimages, registered and fused.]

    Image Observation and Observer Performance. This stepinvolves the response characteristics ofthe eye-brain systemand measures of the ability of the observer to perform visualtasks, as well as strategies for the development ofinteractive,analytic, diagnostic, and adaptive software to facilitate ob-server performance, based on knowledge of human vision.[Relevant topics include the following. Spatiotemporal re-sponse of the human visual system, photopic and scotopicvision, and spectral sensitivity. Visual-attentional and per-ceptual biases and contrast, color, and depth perception.Visual cognition and memory, fading of stabilized retinalimages, visual masking, and visual illusions. Image interpre-tation and understanding. Modeling of the visual system.Observer performance by receiver operating characteristic(ROC) analysis, which may also be used as a criterion forimage evaluation.]Image Analysis. Analysis involves strategies for the extrac-

    tion of qualitative and/or quantitative information about anobject from its image(s) and the automation ofthese strategiesto assist or to replace the observer. [Relevant topics includethe following. Signal-detection theory and statistical-decisiontheory. The ideal observer. Extraction of information: edgedetection; perimeter, distance, area, and volume measures;segmentation; region growing; shape from shading, texture,and motion; morphologic analysis; pattern recognition; cor-relation; feature/factor extraction; multivariate analysis anddiscriminant functions; moments; derived parametric/functional images; object recognition; image understanding;expert systems and artificial intelligence schemes; neuralnets; computer vision; automated image analysis; and robot-ics.]Image Evaluation. Evaluation involves measures of image

    quality that can be used as criteria for the (goal-dependent)optimization ofparameters associated with each ofthe abovesteps. [Relevant topics include the following. Image qualitymeasures based on signal-to-noise ratio, correspondencebetween object and image (e.g., correlation, least squares),likelihood ratio; Hotelling trace, information content, com-plexity, accuracy and reproducibility of measures of objectproperties, and risk-to-benefit and cost-to-benefit ratios.Optimization of parameters: method of steepest decent andsimulated annealing. Observer performance/ROC analysis(see Image Observation, above).]

    It is useful to think of these steps as raising generic issuesthat cut across all imaging modalities, some of which areidentified in Fig. 1. [In some modalities (e.g., those employ-ing film as a detector/recorder to produce a planar projectionfrom a single view), some of the steps (e.g., image recon-struction) may not be taken explicitly and separately. Itshould be noted also that the human visual system is a veryspecial imaging modality in its own right, although it does not"display" images in the conventional sense. However, whenit views images obtained from another modality, it is mostappropriately regarded as a cascaded stage that operates inseries with the other modality; i.e., images from othermodalities are objects for viewing and imaging by the humanvisual system.]

    Intellectual Agenda of Imaging Science

    An abstract general theory of image formation (i.e., fromacquisition to display) can be formulated, based on thewell-known principles of linear systems and the concept ofmapping, that is applicable (as a first-order approximation) toall imaging modalities. In an earlier discussion paper*, wehave sketched the theory, identified its underlying assump-

    *Beck, R. N. & Crewe, A. V. (1988) The Emerging Field ofImagingScience. Unpublished discussion paper, available on request.

    9748 Colloquium Paper: Beck

    Dow

    nloa

    ded

    by g

    uest

    on

    Oct

    ober

    8, 2

    020

  • Colloquium Paper: Beck Proc. NatL Acad. Sci USA 90 (1993) 9749

    Z0

    > ELU E

    E (U!j.E

    E

    'ZOZ14 V.2,U a

    C '6 ESa I 3 I >,- .1. I>,:2Lv m0 E &.9 t-E Eim z 6 E

    R 35 AaV.2 C C- xCo VI 153%.0 'a A Is1r; IW E'E.5

    ra L? 24z 0 I

    C.C40M72 0279 -ti E

    M 0 0 VS -CP ro-< o 2 C. .5la laco Z% CC

    zz to C) >. o E-Co co C .8Ld E 8.9= 8 go)0 9 vLU M tAOzm r-3:

    .,s 3t V%8 AS

    0E Vcco.ra C Ez

    .r- to I -5 1 2!.-Ebr 1-4.8 -a - 'O I In M toE crEui r- CL(3

    4c 79 .8 -6 CZ1:2 C. co r-it E-03-2 'M -'s ci92 9 0H-

    co CZCr-M - . 1 1.0 *.AC ..-E 'o -io S10. %_ 11= § .1,S) 02 bo-6 -RE 0 %_ ZEco ZV. 6 C E:5z -0 E .S E0 1 . o R Lra r.- C.=,a - 8- S .2 ; -.. 9 > r-M 16 0. H.92olo - co0 -6 co - ME M -6 0189= 71 21% - 16. 0 0-_ -U.r2- .79 > E 0(35 g g 6 9 0 'cm >.-- Eti :F. .2 0 a.W.F.IE t - 16 S? = -6& F c.. . E IV '%R. '2 uJ 0 E' 2 Sel E r- mo0

    C :91 L? COC;W Ul 4-4

    LU 00 E 27 7'.

    0 A

    uo

    13I .;. 1. co

    0 V VC §

    E O

    0ui bo u0 1 W A !G0 z

    oz Cs z 0 CZ4 E'S o O.- 4) z Em U--J >-M E -9 a: IC E 9C U0, CL: (0 9 9 2CL z = 8 g a :.: ui -9 .en, 2- = Uzi > E So '>, t.H. > U)8 z 0M"ou ,U_ > r- r Z ro- ._ a: 9 LU.6M oc o k C.)r- > LLJ -J m 4a 0)'6, cqsCD 0 2L r

    Am r- (L E. 0 0

    V -~~~~~~~~1I - ~ -Wl .06-' I '

  • Proc. Natl. Acad. Sci. USA 90 (1993)

    tions, and shown how it may be applied to several classes ofobjects. A more detailed theory, in which the displayed imageis expressed as a function of parameters associated with thesteps leading to it, is required for optimization of theseparameters. In short, the general theory may be viewedsimply as a conceptual frame within which a detailed analysisof any imaging modality can be mounted.On a more detailed level of analysis, a major component of

    the intellectual agenda for imaging science includes not onlythe development of a quantitative understanding of all of theseparate steps but also quantitative knowledge of the com-plex ways in which they are interdependent. For example, itis clear that an understanding of the spatiotemporal and colorresponse of the human visual system is needed for optimizing(in the mathematical sense) the parameters associated withthe steps of image-data acquisition, processing, and display,in order to achieve some desired, application-specific goal-e.g., to maximize the likelihood that radiologists will seesubtle lung lesions on chest films. It is also clear that the largenumber ofparameters involved obviates a purely experimen-tal approach to their optimization; thus, computer simulationof all of the steps in the imaging process appears to be themost effective approach. For each modality, this will requirethe formulation ofan equation describing the desired measureof image quality as a function of these parameters, for aparticular class of objects to be imaged, and for a particularstrategy for accomplishing each step. Whereas, in principle,the parameter values associated with each step could then beoptimized, in practice, this has not been achieved for anyimaging modality, primarily due to a lack of quantitativeknowledge ofthe interdependencies among the various steps.

    In brief, ..a comprehensive understanding must includequantitative knowledge of the following: the class of objectsand the particular property to be imaged; the effects ofattenuation and scattering of radiation within the mediumseparating the object and the imaging device (which affect theinput radiation spectrum); the relationships among the vari-ous parameters (including those that affect spectral sensitiv-ity, spatial, temporal, and energy resolution, depth of field,linearity, and stationarity) for the various detector materialsand configurations that might be employed in image-dataacquisition; the effects of the various strategies for imagereconstruction, processing, compression, display, and anal-ysis; as well as the response characteristics and limitations ofthe human visual system and the variability of these amongobservers. Currently, much of the research being done inimaging science is confined to the optimization of parametersassociated with specific steps, under quite restricted condi-tions. While this research is yielding a better quantitativeunderstanding of specific issues, very little has been done todate to investigate the interdependence of the steps. Ulti-mately, this formidable challenge needs to be addressed.

    Finally, particularly exciting aspects of the intellectualagenda for imaging science are the discovery of additionalobject properties that can be imaged, the invention of newstrategies for imaging, and the development of new systems.These are active areas of research which, in recent decades,have resulted in a remarkable array of new microscopes,telescopes, and medical imaging systems that have acceler-ated the production of new knowledge in many fields.

    Education in Imaging Science

    Currently, there is little organization or funding of educationin imaging science per se, and, particularly in this sense,

    imaging science has not yet emerged as a recognized disci-pline. Nevertheless, a number of outstanding programs fo-cused in specific areas, such as optics, have existed for manyyears, particularly in schools of engineering and technology.In view of the fact that imaging science draws upon, andserves, so many traditional disciplines, it is perhaps notsurprising that even the University of Chicago (which has noengineering schools) offers more than 75 courses in topicsdirectly relevant to some aspect of imaging science. Thesecourses are offered by 14 departments of the Physical,Biological, and Social Sciences Divisions and the Humani-ties. Historically, there has been little interaction acrossdepartments and less across divisions. It is probably accurateto say that a similar situation exists in most major universitiesand to conclude that there is little or no shortage of relevantcourses with which to make a beginning; what is lacking is anorganizational structure that will foster the emergence ofimaging science. We are attempting to remedy this throughthe Center for Imaging Science, which was established forthis purpose, and which has begun to explore the formationofa multidisciplinary Committee on Imaging Science that willdevelop graduate and postgraduate programs, based substan-tially on existing courses and organized conceptually aroundthe generic issues of imaging science.

    In contemplating the nature of an educational programfocused on imaging science, we see strong similarities be-tween certain aspects of imaging science and those of com-puter science. Specifically, computation is grounded in math-ematics and is essential to all of the traditional disciplines (aswell as to imaging science). Modern computation is based onhigh technology, largely in the form of digital computers.However, our Department of Computer Science is not con-cerned solely, or even primarily, with the technologicalaspects of computation, but with deeper underlying issues.Examples are alternative and optimal computer architecturesand algorithms; computational complexity and the solvabilityof various classes of problems; the potential of machinelearning and artificial intelligence schemes; and, ultimately,such questions as What is knowable through computation?

    Similarly, imaging is grounded in physics and mathematics,and images are essential to all of the traditional disciplines ofscience, as well as to medicine, the graphic arts, and educa-tion. Most modern imaging methods are based on severalhigh technologies, including digital computers. However, themultidisciplinary Committee on Imaging Science would notbe concerned solely, or even primarily, with technologicalaspects of imaging, but with deeper underlying issues. Ex-amples are alternative and optimal ways of acquiring anddisplaying images of various classes of objects and processes;the potential for imaging additional object properties that arecurrently inaccessible; the potential for extraction of moreinformation from images by human vision and by automatedcomputer-vision methods of image analysis; and, ultimately,such questions as What is knowable about objects from theirimages? How can images and words best be used synergis-tically to convey knowledge through educational programs inother fields? and What will be the impact on our culture ofbringing together, in a more intensive convergence, thesehistorically divergent ways of knowing and of communicat-ing? Some of these questions will be addressed in the finalpresentation of this colloquium: "Issues of imaging sciencefor future consideration."

    9750 Colloquium Paper: Beck

    Dow

    nloa

    ded

    by g

    uest

    on

    Oct

    ober

    8, 2

    020