a guide to picture
TRANSCRIPT
-
8/2/2019 A Guide to Picture
1/20Copyright 1997, Tektronix, Inc. All rights reserved.
A Guide to PictureQuality Measurementsfor Modern
Television Systems
-
8/2/2019 A Guide to Picture
2/20
-
8/2/2019 A Guide to Picture
3/201
Introduction
Convergence is a popular termthese days. It has many defini-tions and many factors dependingon ones perspective and tech-nology base. From a generalistspoint of view, convergence may
be defined as the comingtogether of communication,computer and television tech-
nologies to provide informationof any kind to any location.One of the major focal points ofconvergence is the need for acomplete new technology forthe evaluation of modern televi-sion systems. In this guide, theaspects of video testing are pre-sented based on an understandingof the complete television systemincluding production, compres-sion, decompression and thedisplay or reuse of the originalprogram. The need for continuingapplication of traditional videotesting methods is explainedalong with their limitations forthe identifying the artifactsintroduced by video compression.With the variety of video com-pression methods in use and
being developed, there is arequirement for picture qualityassessment methods which areindependent of the compressionalgorithm and its related artifacts.An overview of subjective testing,which uses a panel of observers,is presented as it has been themainstay for video compressionsystem development. Due to the
complexity and variability ofsubjective testing there is a strongrequirement for an objectivemeasurement instrument muchas we use today for traditionaltelevision systems. Advantagesand limitations of proposedobjective testing algorithms arepresented leading to the conclu-sion that a method based on arepresentation of the humanvisual system is required for
best results. To complete theguide, implementation of an
objective picture quality assess-ment algorithm in a practicalmeasurement instrument isshown to require a combinationof traditional video technologyand modern computer techniques.
-
8/2/2019 A Guide to Picture
4/202
Compressed Television Systems
Compression is Nothing New. Thereare two reasons to compresstelevision video signals, practicallimitations of processing speed(bandwidth) and cost of trans-mission or storage resultingfrom the required bandwidth.Today, the availability of high-
speed semiconductors andintegrated circuits make thelatter reason most important innearly all applications. Virtuallyall video compression methodsutilize the limitations of thehuman visual system toremove the less visible pictureinformation that might otherwise
be present.
As broadcast television wasbeing developed, display ratesof 50 or 60 pictures per secondwere considered necessary. To
provide sufficient visual infor-mation each picture was judgedto need about 500 displayelements (now called pixels) ineach direction.1 To generate andtransmit such a sequence ofpictures in analog form wouldrequire a processing speed andtransmission bandwidth ofabout 10 MHz which was diffi-cult for the available technology
and excessive for the availableradio frequency spectrum.
The first practical televisionbroadcast systems used a formof two-to-one bandwith reduction,or compression, called interlace.Instead of sending 50 or 60frames per second, each frameis divided in to two fields contai-ning half the total number oflines. The lines in the first fieldare every other line from theframe, say lines 1, 3, 5... and thelines in the second field fill inthe missing lines during thesecond field as shown in Figure 1.Picture degradation due tointerlace is in the form of anartifact known as inter-linetwitter, however the quality isquite satisfactory for entertain-ment video viewed severalpicture heights away from thedisplay device.
In the 1950s, color televisionwas developed. A single colorpicture requires three images,specifically red, green, and blue(RGB) for light emitting devicessuch as cathode ray tubes (CRT).Starting from the full progres-sive scan picture, this wouldrequire a 30 MHz bandwidth toprovide the desired picture rate.
Again, interlace is used to reducethe bandwidth to 15 MHz for ananalog RGB system. Within astudio the signals are carriedon three separate cables at5 MHz or more bandwidth each,as shown in Figure 2. A funda-mental compression schemeused in color television is totranslate the three color signals
into the color-differencedomain where the picture isrepresented by a luminance(equivalent to the earlier mono-chrome) picture and two colordifference pictures, R-Y andB-Y. Another name for thissystem is YUV, Y for luminanceand U, V for the two colordifference signals. Again usingthe limitations of the humanvisual system, in this case lesscolor than luminance visualacuity, the bandwidth of the
color difference signals isreduced by 50% for a total YUV
bandwidth requirement of 10MHz. Today, YUV signals areused in both analog and digitalforms and have very little visibledegradation compared to inter-laced RGB video. Both formsare known as component videowith YUV being used formost applications.
Camera LinearMatrix
Red
Blue
Green
Y
R-Y
B-Y
RGB - Component Video - YUV
Quadrature
Modulationat SubcarrierFrequency
Chroma2-5 MHz
Y 0-5 MHz Composite Video
0-5 MHz
R-Y 0-1.5 MHz
B-Y 0-1.5 MHz
Field 1,Odd
NumberedLines
Field 2,Even
NumberedLines
Camera VideoSignal
Figure 1. Interlaced scanning. Figure 2. Analog television compression.
1The Americas, Japan and Korea use the 525-line, 60 field/s system while most of the rest of the world uses the 625-line 50 field/s system.
-
8/2/2019 A Guide to Picture
5/203
Color television in the 1950s,and until recently, required fur-ther compression both to fit inthe allocated 6 MHz bandwidthof transmission channels and to
be compatible with the installedbase of monochrome televisionsets. To accomplish this task thetwo color difference signals arefurther reduced in bandwidth to
about 1.5 MHz each and quadra-ture modulated on a subcarrierwhich is subsequently added tothe luminance signal producingcomposite video. CompositeNTSC and PAL2 produce verygood entertainment qualitycolor video in a 5 MHz band-width, a compression ratio ofsix to one from the ideal pro-gressive scan RGB video. Thefinal two to one compressionfrom component to compositedoes bring with it noticeable
picture degradation includingchroma information seen as in-correct luminance and visa versa.3
Today, using modern digitalcompression methods, four ormore excellent quality digitalcomponent television signalscan be delivered to the homewithin the same 6 MHz trans-mission channel. If derived froma high quality digital component
measurement instrument to pro-vide a result that is independentof the compression method used.
The compression method that isbecoming dominant today iscalled MPEG-2, defined by theMotion Picture Experts Groupand standardized by both theInternational StandardsOrganization (ISO) and the
International ElectotechnicalCommission (IEC). MPEG-2 is
based on the Discrete CosineTransform (DCT) method incombination with powerfultemporal compression tech-niques.5 Although some applica-tions may be best served byother compression methods,MPEG-2 is expected to be themost widely used method in theforeseeable future. This is
because it is an agreed standardthat is either optimum or goodenough for a wide variety ofapplications, a large amount ofeffort is going into the develop-ment of chip sets for lower costencoders and decoders, and theforthcoming large installed basewill be attractive for manyequipment manufacturers andapplication developers.
source, the resulting multipletelevision signals have a notice-able quality improvement overthe single 6 MHz bandwidthcomposite video signal.
Digital Compression Methods.Digital video became a reality in1973 with the invention of thecomposite-based digital time
base corrector for video taperecorders. In the early 1980s, aworldwide digital componentvideo standard was developedrequiring 216 Mb/s or 270 Mb/sdepending on the use of 8-bitor 10-bit sample values. Thisstandard is commonly known asRec. 6014. It is the dominantsampling structure for digitaltelevision and its use is showingrapid growth for all types ofapplications. Since the approvalof the Rec. 601 standard, much
research and development hasbeen directed towards digitalvideo data rate reduction resultingin a variety of video compressionmethods. Each of these com-pression methods has its ownadvantages, disadvantages andpicture degradation characteris-tics. It will be important for anygeneral purpose picture quality
2NTSC (National Television System Committee) is used in most 525-line countries and PAL (Phase Alternating Line) is used in many, but not all, 625-line countries.3For a more complete description of basic digital t elevision see "A Guide to Digital Television Systems and Measurements", Tektronix literature number 25W-7203.4Rec. ITU-R BT.601, Encoding Parameters of Digital Television for Studios. Originally it was CCIR Recommendation 601 but has been changed to Recommendation ITU-R BT.601. Rec 601 is used throughout
this guide (and is much easier to say).5See "MPEG-2 Fundamentals for Broadcast and Post-production Engineers" for a brief description of the MPEG compression method. Tektronix literature number 2AW-1061.
-
8/2/2019 A Guide to Picture
6/20
The Modern Television System.
A simplified block diagram of amodern compressed televisionprocessing and transmissionsystem is shown in Figure 3.Television nominally consists ofaudio and video; however, thesystem may include data andcontrol signals (not shown inthe figure) hence may be
thought of as a multimediasystem. One-direction transmis-sion is shown. A multiplicity ofmethods are depicted, particu-larly in transmission, makingthis diagram an overview ofmany types of applications. Nospecific compression method isshown, however the MPEG-2transport stream is shown in thetransmission area since it can beconsidered a general purposemultiplexing scheme capable ofcarrying any type of compressed
video and audio.Analog RGB video is producedin the camera and processedinto one or more of severalpossible formats; analog com-posite, digital composite, analogcomponent or digital compo-nent. Full-bandwidth digital
video is an extremely importantpart of the television systemtoday. Program productionprocesses must be full-bandwidthdigital (or analog) in order tomanipulate the picture toproduce desired artistic results.
Following program production,the television signal may becompressed for storage, efficienttransmission or intra-facilityinterconnection in digital form.Typically this will be MPEG-2compression resulting in anMPEG transport stream (MTS)that may be multiplexed withother MPEG transport streams fortransmission or interconnection.
New systems for RF transmis-sion of television signals usedigital modulation schemeswhich are generally more robustfor the same transmitted power
and provides the digital channelfor compressed television signals.It is important to note, that evencompressed digital video broad-casting to the home often startswith full bandwidth digitalvideo to drive the bit-rateefficient, statistically-multiplexedcompression system.
The broadband telecommunica-tion system provides a varietyof transmission methods.Traditionally these have beenvoice channel oriented withspecial data mapping for digitaltelevision signals. Althoughdirect mapping of the MTS intothe digital telecommunicationshierarchy is in the process of
being standardized, it is expect-ed that asynchronous transfermode (ATM) will become thepreferred method of inter-facili-ty video transfer.
Video testing in this moderntelevision system is not just amatter of developing new tech-niques to evaluate the effects ofcompression. The significantportion of the system utilizinganalog and full-bandwidth digitalsignals requires application oftraditional analog and recentlydeveloped digital test methods.To determine picture qualityimpairments caused by com-pression, a video measurementsystem must take into accountthe various signal formatchanges affecting the videothroughout the system.
4
Camera CCU CompressTransport
Multiplexer
MTS
V-RGB
V
AAudio
V
A
MTS
OtherPrograms
MTS (Point-to-Point or Network)
RFDemod
RFModulator
ATMGateway
ATMGateway
Satellite, Cable,Terrestrial Transmission
BroadbandNetwork
QPSK
QAM
TDM
QPSK
QAM
TDM
TransportDemux
DecompressMTS V
A
Distribution QualityTVSet
PRODUCTIONRecording
EditingSpecial Effects
PRODUCTIONRecording
Editing
Special Effects
Contribution Quality
V
A
MTS
OtherPrograms
Figure 3. Modern television system.
-
8/2/2019 A Guide to Picture
7/20
Figure 4. Analog video system.
Video Testing Concepts
Growth of Functional Layers. Overthe half-century of widespreadtelevision use there has been arelatively simple model foranalyzing analog video systems.Figure 4 shows a basic blockdiagram of the analog videosystem, its functional layers and
test methods. Testing is per-formed at one interconnectiongenerally carrying a compositePAL or NTSC signal. A singlemeasurement instrument cananalyze both the operationalaspects, such as signal level orcolor balance, and the data for-matting which is the synchro-nizing signal part of the samevideo waveform. This analysisof the signal quality through thetransmission path using a suiteof test signals does an adequate
job of characterizing resultingpicture quality. The idea of asuite of test signals is important.No one test signal will charac-terize the system and someexpert interpretation as well asvisual inspection of the resultingpictures is required. For intra-facility transmission of signalson coaxial cable, a separatepiece of test equipment, thetime domain reflectometer, isused to ensure the continuity ofthe physical layer. Long range
transmission is by amplitude orfrequency modulation on acarrier, however the resultingchannel characteristics for thevideo are still determined byanalog measurements such asthose specified in ANSI T1.502.6
5
6ANSI Standard T1.502 "System M-NTSC Television Signals Network Interface Specifications and Performance Parameters."
CameraComposite
Encoder
RGBPAL/NTSC Transmission
orOperation
Recordor
Display
PAL/NTSC
VIDEOOperational Monitoring
Technical Measurements
SYNCHRONIZING WAVEFORMSTechnical Measurements
PHYSICAL LAYER (Coax)Technical Measurements
Functional Layers
Waveform Monitorsand
Measurement Sets
TDRs
RGB
CameraPAL/NTSC Transmission
orOperation
Recordor
Display
PAL/NTSC
VIDEO
SIGNAL CODING (Rec 601)
DIGITAL FORMATTING (Rec 656)
Functional Layers
Waveform Monitorsand
Measurement Setswith
Analog or DigitalCapabilities
TDRs, OTDRs
StudioInterconnector Operation
EncodeD/A
Rec 601/656 Rec 601/656
DIGITAL WAVEFORM (Rec 656)
PHYSICAL LAYER (Coax/Fiber)
Operational MonitoringTechnical Measurements
Technical Measurements
CompositeEncode orRec 60 1/656
Format
DecodeA/D
Figure 5. Hybrid digital/analog video system.
-
8/2/2019 A Guide to Picture
8/20
With the advent of digital televi-sion over the past 15 years, amore complex system blockdiagram and set of functionallayers has been required asshown in Figure 5. The analogsignal is converted to digital inaccordance with a samplingstandard such as Rec. 601.Formatting and studio intercon-
nection of the digitized signalfollow a related standard, Rec.656,7 leading to a extension inthe functional layers and thevariety of tests to be performed.For operational purposes, themonitoring of analog videosignal properties is still key;however, this signal must beprocessed from the digital data.
Where testing of the analogsignal required only that variousparameters be measured on asingle waveform, digital testingrequires analysis of the digitalwaveform, digital data formattingand digital signal coding inaddition to the resulting analogsignal. Again a suite of testsignals is required, expanding
the suite needed for analog-onlytesting. Although all those mea-surements can be performedwith a signal instrument, suchas the Tektronix WFM601M,there is significant processing
between each pair of layers withdifferent analysis methods foreach layer as well. Prior to theadvent of digital compression
techniques, transmission of thishigher quality signal was handled
by compression back to thecomposite analog domain. Theanalog-to-digital and digital-to-analog conversion does introducesome signal quality degradation
beyond that of the basic NTSCor PAL analog signal.
With the convergence of televi-sion and telecommunication,not only are there many morefunctional layers for the testengineers to consider but thereare various possible paths withdifferent layers. Figure 6 showsa few of the possible functionalpaths and layers.
Serial Digital Interconnect (SDI)is the Rec. 656 worldwidestandard used for serial digitalvideo. The SMPTE WorkingGroup on Packetized Television
Interconnections is developing amethod of carrying packetizeddata over the same cabling andswitching hardware calledSDDI (for Serial Digital DataInterconnect). A networkingtype interconnect for the televi-sion facility, being considered
by ANSI and SMPTE, is FibreChannel which provides highspeed, large packet sizes andreasonably priced hardware. TheSynchronous Digital Hierarchy(SDH) telecom methods are well
established worldwide and candirectly carry the MPEG-2 trans-port stream with simple dataformatting, although there ispresently no standard. Lookingtoward the future, ATM is theexpected method for transmis-sion of packetized data, certainlyfor long distances and perhapswithin a studio.
6
Data,etc.
MPEGAudio
Encoding
MPEGVideo
Encoding
TELEVISION SIGNALSVideo, Audio and Related Test Signals
Analog to Digital Conversion
MPEG System (Transport Stream)
SignalDataFormating
OtherAudio
Encoding
OtherVideo
Encoding
DVB, GA,Other
Call
Setup,SwitchingControl
Transmitter and Receiver Electro/Optic Circuits
Copper or Fiber and Connectors
SwitchingHardware
Fibre ChannelSDI
SDDI
SDH/Sonet
ATM
AAL(1or 5)
PhysicalLayer
TypicallyHardware
TransmissionData
Formatting
Figure 6. Modern television functional layers.
7Rec. ITU-R BT.656, "Interfaces for Digital Component Video Signals in 525-line and 625-line Television Systems Operating at the 4:2:2 level of Recommendation 601."
-
8/2/2019 A Guide to Picture
9/20
Three key testing layers can bedefined for the modern televi-sion system as shown in Figure 7.Each has its own subset of moredetailed testing layers. Videoquality for compressed televi-sion systems is a much morecomplex matter than just usingthe indirect measurementmethods for uncompressed
video. This will be covered indetail in subsequent sections ofthis guide. Once the picture has
been compressed, the resultingdata is formatted for intra-facilityconnections. Examples for theuse of such connections are:program interchange betweenvideo disk servers or severalvideo/audio encoders sendingsingle program transportstreams to a multiplexer to pro-duce a multi-program transportstream for satellite broadcasting.
This is an appropriate layer forprotocol testing because the dataformatting can be quite complexand is relatively independent ofthe nature of the uncompressedsignals or the eventual conversionto inter-facility transmissionformats. For a majority of thetelevision transmission systemsthe MPEG-2 transport stream isthe common denominator at thecompressed data level. Thesyntax and semantics for boththe compressed data and the
transport stream are welldefined. Typical protocol testingequipment, such as theTektronix MTS 100, will be botha source of known valid, orspecifically invalid, signals andan analyzer which locates errorswith respect to a defined standardand determines the value of var-ious operational parameters forthe stream of data. There are anumber of possible inter-facilitytransmission methods as previ-ously described. Many are well
established, such as SDH/Sonetand cable television, with avariety of effective test equip-ment available. ATM is anemerging technology with newtest equipment on the market
and under development.Adaptation of traditional com-munication test equipment toanalyze or interconnect withMPEG-2 transport streams is onthe horizon.
Video Quality. There are severaldimensions of video qualitymeasurement methods that needdefinition. These are summa-rized in the table below.
Subjective measurements arethe result of human observersproviding their opinion of thevideo quality. Objective mea-surements are performed withthe aid of instrumentation,manually with humans readinga calibrated scale or automaticallyusing a mathematical algorithm.
Direct measurements areperformed on the material ofinterest, in this case, picturesand are also called picturequality measurements.
Indirect measurements aremade processing speciallydesigned test signals in thesame manner as the picturesand are also called signalquality measurements.
Subjective measurements areonly done in a direct mannersince the human opinion of testsignal picture quality is not par-ticularly meaningful. (Of course,expert viewing of full-field testsignal pictures is useful as a wayto determine signal distortionsnot for their aesthetic value.)
In-service measurements aremade while the program is
being displayed, directly by
evaluating the program materialor indirectly by including testsignals with the program material.Out-of-service, appropriate testscenes are used for directmeasurements and full-fieldtest signals are used forindirect measurements.
7
SIGNAL TEST FUNCTION
NTSC/PALComponent(Rec. 601)
MPEG-2Transport Stream
Telecommunicationor RF Channel
Video Quality
Protocol Analysis
(Origination Side Shown,Destination Side Similar)
TransmissionSystem
Video Input
MPEG Compressionand
Transport StreamFormatting
ConnectionsINTRA-Facility
INTER-FacilityConnections
ATM Output RF Output
ChannelFormattinge.g. OC-3
SwitchFormattinge.g. ATM
Cable orSatelliteChannelCoding
andModulator
Figure 7. Functional testing layers.
Subjective Direct
(Picture Quality)
Objective Direct(Picture Qualilty)
Objective Indirect(Signal Quality)
In-Service Out-of-Service
Program
Material
ProgramMaterial
Vertical IntervalTest Signals
Test
Scenes
TestScenes
Full FieldTest Signals
-
8/2/2019 A Guide to Picture
10/20
Although there is a modestamount of compression appliedto the NTSC and PAL compositesystems, they are considereduncompressed in today's termi-nology. Signal quality (objectiveindirect) measurements are areasonably good way to deter-mine the picture quality forsuch uncompressed systems.
That is, there is a strong mathe-matical correlation betweensubjective measurements madeon pictures from the system andobjective measurements madeon a suite of test signals usingthe same system. The correlationis not perfect for all tests. Thereare distortions in compositesystems, such as false colorsignals caused by high frequencyluminance, which are not easilymeasured by objective means.Also, there are objective mea-
surements which are so sensitivethey dont directly relate tosubjective results. However,such objective results are oftenvery useful because their effectwill be seen by a human observerif the pictures are processed inthe same way a number oftimes. An example would bemultiple generations using ananalog video tape recorder.
The reason signal quality mea-surements work with analog and
full-bandwidth digital systemsis uncompressed systems arelinear.8 That is, the system
behavior is time invariant,signal independent and super-position applies. Signal qualitymeasurements are made with asuite of test signals whoseresulting distortions willdetermine transmission channelor video processing characteris-tics. These test signals can bevery short, as an example, oneline in the vertical interval. Signal
quality of the uncompressed
video remains critical in systemsthat use compression forseveral reasons:
The input to a video compres-sion codec must be accurate,in compliance with approp-riate standards, and of as higha quality as possible toprovide for efficient encoding.
Video processing such asadding titles and specialeffects can not be accomplishedin the compressed domain.
Production facilities will notbe fully compressed due tothe cost and quality ofcompression codecs.
The only way for differentcompressed formats to beinterchanged is at the full
bit-rate level.
This leads to a strong require-
ment for testing of the analogand full bandwidth digitalportions as well as the sophisti-cated compression andtransmission systems.
With the advent of compresseddigital video systems the situa-tion has become more complex.Signal quality testing will notwork for the compressionencoder/decoder part of the sys-tem. Traditional test signals arerelatively simple compared to anatural scene and are easilycompressed with little distor-tion or loss. Due to the ease ofcompression, these signals donot evaluate the encoder/decoderprocess. As an example, signal-to-noise ratio is not a reliablemeasure of picture quality, it isnot a constant for a given systemand it can give completelymisleading results. Thereforepicture quality measurementsrequire a direct method, usingnatural scenes, or an equivalentthereof, which are much more
complex than traditional testsignals. These complex scenesstress the capabilities of theencoder resulting in non-lineardistortions that are a function ofthe picture content.
Use of digital compression hasexpanded the types of distor-tions that can occur in themodern television system.
Quantization noise which isalso present in full-bandwidthdigital systems is oftenincreased by the compressionsystem bit rate reductionprocess. Blockiness is a checker-
board pattern that may occur inDCT-type compression systems.Loss of resolution is common
because the compressionsystems use the human visualsystem limits of acuity as aguide for removing informationfrom the picture. Therefore,greater compression generallymeans less resolution. Althoughhuman acuity is less for chroma,the uncompressed picture hasalready used some of that lati-tude and compression systemsoften squeeze the chroma evenmore than the luminance. Edge
busyness is another effect ofquantization since more infor-mation is removed from thehigh-resolution parts of the pic-ture producing noise on edges.When that noise is displaced bythe compression processing intonearby flat areas it is sometimescalled mosquito noise. Motionrelated artifacts, such as jerki-ness or misplaced blocks ofpixels, are present in systemswhich use temporal compres-sion either based on sophisticat-ed motion compensation orsimply dropping frames becausethere are not enough bits avail-able in low bandwidth systems.9
8
8Analog systems are not perfectly linear, however they are quite good and sensitive objective testing can be used to determine the small amounts of non-linearity.9A list of impairment terms and other definitions may be found in ANSI T1.801.02 Digital Transport of Video Teleconferencing, Video Telephony Signals - Performance Terms, Definitions and
-
8/2/2019 A Guide to Picture
11/20
With the broader range of distor-tions to measure and the desireto optimize program distribu-tion both technically and eco-nomically, the field of subjectivemeasurement has expanded.Some of the subjective measure-ments even include an elementof program quality as well aspicture quality as will be
discussed in detail later in thisguide. Since signal qualitymeasurements will not do thejob, objective picture qualitymeasurements are needed.Expanded types of signal qualitymeasurements are not appropri-ate to cover the new subjectivemethods. In fact, with theincreased ideas for subjectiveevaluation it may be true thatthe traditional signal qualitymeasurements no longer have asstrong a correlation with subjec-
tive requirements. There doesnot appear to be any plan toexpand or re-test the signalquality measurement methodssince there is so much work todo in developing objective pic-ture quality methods. Such pic-ture quality measurement meth-ods must, also, have strong cor-relation with subjective mea-surements and cover a reason-ably broad range of subjectiveconsiderations. It is expectedthat picture quality distortions
too small for the human to seewill be measured and providean indication of the performanceof concatenated systems.
Picture Quality Testing
Subjective Testing. Televisionprograms are produced for theenjoyment or education ofhuman viewers so it is theiropinion of the video qualitywhich is important. Informaland formal subjective measure-
ments have always been, andwill continue be, used to evaluate
system performance from thedesign lab to the operationalenvironment. Even with all theexcellent objective testing meth-ods available today for analogand full-bandwidth digitalvideo, it is important to havehuman observation of thepictures. There are impairmentswhich are not easily measured
yet are obvious to a humanobserver. This situation certainlyhas not changed with the addi-tion of modern digital compres-sion. Therefore, casual or infor-mal subjective testing by areasonably expert viewerremains an important part ofsystem evaluation or monitoring.
Formal subjective testing hasbeen used for many years with arelatively stable set of standardmethods until the advent ofdigital compression subjectivetesting described in Rec. 500.10
In brief, a number of non-expertobservers are selected, tested fortheir visual capabilities, showna series of test scenes for about10 to 30 minutes in a controlledenvironment and asked to scorethe quality of the scenes in oneof a variety of manners.Subjective testing is used for
both quality assessment, systemperformance under optimumconditions, and impairmentassessment under non-optimumperformance due to transmis-sion limitations. In a moderntelevision system that incorpo-rates compression, the picturequality is not a constant overtime. Picture quality is a func-tion of the complexity of theprogram material and, in thecase of statistical multiplexing,the moment to moment operationof the transmission system.Considering this time varyingproperty and the number of new
impairments, the defined andproposed measurement methods
have grown in recent years. Inaddition to selection of themeasurement method there are anumber of other proceduralelements for which alternateapproaches are available. Theseare such things as: viewingconditions, choice of observers,scaling method to score theopinions, reference conditions,
signal sources for the testscenes, timing of the presenta-tion of the various test scenes,selection of a range of testscenes, and analysis of theresulting scores. Selection of theparameters for each of theseelements is related to theintended application of thetelevision system and leads to acomplex maze of possibilities.A description of the varioussubjective measurement methodsprovides some insight.
Double Stimulus ImpairmentScale (DSIS) Observers areshown multiple reference-scene, degraded scene pairs.The reference scene is alwaysfirst. Scoring is on an overallimpression scale of impair-ment: imperceptible, percepti-
ble but not annoying, slightlyannoying, annoying, and veryannoying. This scale is com-monly known as the 5-pointscale with 5 being imperceptibleand 1 being very annoying.
Double Stimulus ContinuousQuality Scale (DSCQS) Observers are shown multiplescene pairs with the referenceand degraded scenes randomlyfirst. Scoring is on a continuousquality scale from excellent to
bad where each scene of thepair is separately rated but inreference to the other scene inthe pair. Analysis is based onthe difference in rating foreach pair rather than the
absolute values.
9
10The standard for subjective measurements is ITU-R BT.500 "Methodology for the Subjective Assessment of the Quality of Television Picture". First issued in 1974 and formally known as CCIR Rec. 500, version 7
of this document covers all of the past and proposed methods for subjective testing.
-
8/2/2019 A Guide to Picture
12/20
Single Stimulus Methods Multiple separate scenes areshown. There are twoapproaches: SS with no repe-tition of test scenes and SSMRwhere the test scenes arerepeated multiple times.Three different scoring methodsare used:
Adjectival the 5-gradeimpairment scale, howeverhalf-grades may be allowed.
Numerical an 11-gradenumerical scale, useful ifa reference is not available.
Non-categorical acontinuous scale with nonumbers or a large range,e.g. 0 - 100.
Stimulus Comparison Method Usually accomplished withtwo well matched monitors
but may be done with one. Thedifferences between scene pairsare scored in one of two ways:
Adjectival a 7-grade,+3 to -3 scale labeled: much
better, better, slightly better,the same, slightly worse,worse, and much worse.
Non-categorical acontinuous scale with nonumbers or a relation-number either in absoluteterms or related to astandard pair.
Single Stimulus ContinuousQuality Evaluation (SSCQE) A program, as opposed toseparate test scenes, is contin-uously evaluated over a longperiod, 10 to 20 minutes. Datais taken from a continuousscale every few seconds.Scoring is a distribution of theamount of time a particularscore is given. This methodrelates well to the time variantqualities of todays compressed
systems, however it tends tohave a significant content ofprogram quality in addition tothe picture quality.
In one evaluation, Rec. 601video, which has been consid-ered to be essentially perfect forthe past fifteen years, was givena quality rating above 90% foronly 14 minutes out of a 20minute program.
In addition to these definedmethods, there are two newapproaches that start to bridge
the gap between subjective andobjective picture quality mea-surements. They are picture-content failure characteristicsand composite failure charac-teristics of program and trans-mission conditions. These will
be discussed in the section onobjective measurements.
Advantages of subjective testingare;
valid results are produced forboth conventional and com-
pressed television systems,
a scalar mean opinion score(MOS) is obtained, and itworks well over a wide rangeof still and motion pictureapplications.
Weaknesses of subjective testingare;
a wide variety of possiblemethods and test elementparameters must be considered,
meticulous setup and control
are required, many observers must be
selected and screened, and the complexity makes it
very time consuming.
The net result is subjective testsare only applicable for develop-ment purposes. They do notlend themselves to operationalmonitoring, production linetesting or trouble shooting.
Objective Testing. The need for anobjective testing method of pic-ture quality is clear; subjectivetesting is too complex andprovides too much variability inresults. However, since it is theobservers opinion of picture
quality that counts, any objec-tive measurement system musthave good correlation withsubjective results for the samevideo system and test scenes. Aswith subjective testing, nearlyall objective testing methods donot claim to measure picturequality directly but provide anindication of how a degraded
picture or scene compares witha reference picture or scene.Such comparisons tend toeliminate the aspect of programquality from the measurements.
Over the past few years a widevariety of methods have beeninvestigated for objective testingof picture quality in compressedvideo systems. The methodsproposed may be roughlydivided into two categories,feature extraction and picturedifferencing, each of which may
be implemented in a varietyof ways.
Feature extraction uses amathematical computation toderive characteristics of a singlepicture (spatial features) or asequence of pictures (temporalfeatures). This usually resultsin an amount of data per picture(say, a few hundred bytes) thatis considerably less than usedto transmit the compressedpicture. The calculated char-
acteristics of the reference anddegraded pictures are thencompared to determine anobjective quality score.
Picture differencing uses amatrix-based mathematicalcomputation to process eachpicture or sequence of pictures.The resulting data representsa filtered version of the picturescontaining an amount of datasimilar to the original pictures.Usually, the pixel-by-pixeldifference between filtered
versions of the reference anddegraded pictures is used todetermine an objective qualityscore. In some cases, it may bethe difference between thereference and degraded picturesthat is filtered.
10
-
8/2/2019 A Guide to Picture
13/20
Figure 8 shows how the twobasic methods might be used inan objective measurement sys-tem. The advantage of the fea-ture extraction method (8a) isthe calculated characteristics ofthe reference (input) picturemay be sent through the trans-mission channel along with thecompressed picture for objective
scoring at a remote location.Because of this advantage, thefeature extraction method has
been vigorously pursued, some-times in combination with thepicture differencing method.However, research at Tektronixand other laboratories has shownthat certain picture differencingmethods (8b) provide objectivescores that correlate best withsubjective results.
It is important to note, neitherof these methods can be guaran-teed to always give the correctpolarity of the change in picturesalthough virtually all systemsproduce picture degradation.There are examples where apicture with noise or otherartifacts is improved by filtersat the input to a compressionsystem resulting in a net pictureimprovement through the com-pression/decompression process.
Some of the concepts of thefeature extraction method are
codified, for luminance only, ina recently approved AmericanNational Standards Institute(ANSI) standard.11 The standardmay be considered a tool box ofobjective measurement methodsproviding a set of performanceparameters where each parameteror combination of parameters issensitive to some uniquedimension of video quality orimpairment type. The scopeof the standard statesDiscrimination between two or
more similar systems is beyondthe accuracy of the objective
measurements defined in thisstandard at this time. Furtherwork by the members of theANSI committee has beenreported12 indicating that a com-
bination of feature extractionand picture differencingmethods give the best results.Even with these extensions, themethods to be used should be
chosen depending on the appli-cation to provide the bestcorrelation between subjectiveand objective scores.
Another significant approach tofeature extraction has beendeveloped and reported in thelatest proposed revision to theinternational subjective testingstandard, Rec. 500, as appen-dices Picture-content failurecharacteristics and Compositefailure characteristics of programand transmission conditions.They introduce the concept of
criticality which is a measureof the complexity of the picturesto be compressed. The idea isthat pictures with more criticality(complexity) will be more diffi-cult to compress and will resultin lower picture quality.
This approach is compressionmethod dependent as well asapplication dependent. Differentcompression methods willproduce a different picturequality for the same input criti-cality. Even the same method,for instance MPEG-2, willproduce different results ifparameters are changed such asgroup of picture length or rela-tive number of bits allowed forluminance verses chrominance.The method of calculatingcriticality will be dependent onapplication much as the featureextraction methods are whenapplying the ANSI techniques.
11
11ANSI Standard T1.801.03-1996 "Digital Transport of One-way Video Signals, Parameters for Objective Performance Assessment".12ANSI T1A1.5/96-121 "Objective and Subjective Measures of MPEG Video Quality".
ReferencePicture
Compression Codec DegradedPicture
FeatureExtraction
FeatureExtraction
FeatureComparison
Objective
Score
Low Bandwith Data
(a Few 100 Bytes)
Figure 8a. Feature Extraction
ReferencePicture
Compression Codec DegradedPicture
ImageProcessing
ImageProcessing
PictureData
DifferenceObjective
Score
Picture Data
(Many kBytes)
Picture Data
(Many kBytes)
Figure 8b. Picture Differencing
-
8/2/2019 A Guide to Picture
14/20
is worse with a value of 27.10.Therefore, MSE is not an appro-priate picture differencinmethod for objective picturequality measurements.
Although picture differencingmethods based on featureextraction parameter calculationshas been shown to improve onthe basic ANSI approach, theresult is not application or tech-nology independent. Numerousresearchers have indicated thatthe way to achieve technologyindependence and provide goodcorrelation between subjectiveand objective measurements isto have the test instrumentperceive and measure videoimpairments in the same manneras a human observer. In otherwords, filtering for the picturedifferencing method should usea model of the human visualsystem (HVS). Application ofsuch a model will provide animage quality metric that isindependent of video material,types of impairments and thecompression system used.
As previously stated, certainpicture differencing methodsprovide better objective picturequality measurement correlationwith subjective results. Themost obvious picture differencingmethod is to simply subtract thetwo pictures without any filteringor processing. If the differenceis zero, the pictures are identical.
When the pictures are differenta mean square error (MSE) can
be calculated on a pixel by pixelbasis, a larger MSE indicates agreater difference betweenreference and degraded pictures.Another way to express thisdirect picture difference isPSNR which computes the logof the ratio of the square of thepeak signal (255hex in an 8-bitsystem) to the MSE much as isdone for signal to noise ratio(SNR) in an analog system. This
method has some practical usesand some significant failings.For a very constrained system,say bit rate change only, MSEwill increase with decreasingpicture quality. Also, designers
may find it useful to view thepixel value differences in pictureform when looking for designproblems. However, it is wellknown that MSE can give acompletely false indication. Asan example, consider the com-parison of two types of degrada-tion; one is the addition of asmall amount of random noise,
say five quantizing levels, andthe second is the addition ofsomewhat less blockiness, saytwo quantizing levels. The latterimpairment will have a smallerMSE value, however observerswill consider the noisy pictureto have little degradation wherethe blockiness will be quiteapparent as a significant degra-dation. An example of this mea-surement is shown in Figure 9.Codec A provided an outputimage with a MSE value of
21.26 but a significant amountof blockiness whereas codec Bprovides a much better lookingpicture with a small amount ofadded noise, however the MSE
12
Input Image Codec A: MSE = 21.26 Codec B: MSE = 27.10
Figure 9. MSE measurement examples.
-
8/2/2019 A Guide to Picture
15/20
Application of the Human VisualSystem. Researchers at the DavidSarnoff Research Center (SarnoffLabs) have devoted significantresources, over a number ofyears, to studying the humanvisual system and applying theknowledge gained to televisiondisplay and picture quality eval-uation. Based on this work, the
Just Noticeable Difference (JND)image quality metric has beendeveloped for automatically andaccurately assessing the percep-tual magnitude of differences
between a test and referencesequence.13 Figure 10 shows anoverview of the JND modelarchitecture. The inputs are twosequences of arbitrary lengthwhich are separately processed(filtered) to the DifferenceMetric box near the bottom ofthe diagram where the differ-
ences between the processedsequences are used to developthe JND maps and JND numericvalues. An example is shown inFigure 11. Image A is the refer-ence and image B is the degrad-ed picture, image C is the JNDmap. Note the distortion of thenumbers on the trolley car andthe corresponding bright area inthe JND map. Also note thesolid line on the ground to theleft of the trolley car which has
become a dotted line in the
degraded picture. In the JNDmap, a series of dots shows thenoticeable difference betweenthe two pictures.
For the JND image quality metriccalculation, each field of thesequence is represented as a trioof RGB images. In the first stage,labeled Front End Processing,
the voltage units are transformedto light output units to obtainluminance (Y), and then to thepsychophysically defined quan-tities of the CIE L*u*v* uniformcolor space to obtain the twochannels (u*, v*) of the modelschrominance pathway. In the nextstage of the model, labeled PyramidDecomposition, each sequence
is filtered and down-sampledusing a Gaussian pyramid
operation to efficiently generatea range of spatial resolutions forsubsequent filtering operations.Next, the Normalization stagesets the overall gain with a time-dependent average luminance, tomodel the visual systems rela-tive insensitivity to overall lightlevel and to represent sucheffects as the loss of visual sen-
sitivity after a transition from abright to a dark scene.
13Material for this section of the guide is excerpted from the paper Vision Model-based Assessment of Distortion Magnitudes in Digital Video by J. Lubin, M. Brill and R. Crane, presented at the Made to Measure
96 symposium, Montreux, Switzerland, November 1996.
R' G' B'
Front End Processing
Pyramid Decomposition
Y u* v*
Normalization
Oriented Contrast Flicker Contrast Chromatic Contrast
Contrast Energy Masking
JNDValues
DataPooling
ChromaJND Map
LumaJND Map
DifferenceMetric
Identical Process
R' G' B'
Figure 10. JND image quality metric architecture.
A B C
Figure 11. JND map example.
13
-
8/2/2019 A Guide to Picture
16/20
After normalization, threeseparate contrast measures arecalculated; oriented, flicker andchromatic. In each case, thecontrast is a local difference ofpixel values divided by a localsum, approximately scaled as afunction of pyramid level so theresult is 1 when the image con-trast is at the human threshold.
This establishes the definitionof 1 JND, which is passed tosubsequent stages of the model.(The JND unit of measure isfunctionally defined such that 1
JND corresponds to a 75% prob-ability than an observer viewingtwo images multiple times would
be able to see the difference.)
In the Contrast Energy Maskingstage, each contrast image issubjected to a point non-linearity,the gain of which is controlled
by the response across otherresolutions and channels. Thisgain-setting is included tomodel visual masking effectssuch as the decrease in sensitivityto distortions in busy image
regions. The parameters of thepoint non-linearity at this stageare fit according to contrastdiscrimination data in whichthe contrast increment neededto detect the change in contrastis measured as a function of thecontrast from which the changeis made.
At the Difference Metric stage,outputs from the test and refer-ence sequences are combinedvia a simple difference operatorand then summed across pyramidlevels and channels to returnthe number of JNDs in bothluma and chroma. Separate JNDmaps for luma and chroma can
be pooled into one map andsummary statistics can beobtained. Such statistics would
be mean JND, max JND andQ-norm, which allows a general-ized approach to mean andmax calculations.
The JND image quality metricprovides all the facilitiesrequired for a robust objective
picture quality measurementmethod. It includes the threenecessary dimensions for evalu-ation of dynamic and complexmotion sequences; spatial analy-sis, temporal analysis and fullcolor analysis. By using a modelof the human visual system in apicture differencing process,results will be independent of
the compression process andresulting artifacts. This is partic-ularly important in concatenat-ed television systems which areexpected to involve several dif-ferent compression methods.14
Objective measurement methodsthat rely on a model of the com-pression codec or evaluate spe-cific types of artifacts will havevery limited application in suchsystems. In addition to beingappropriate for overall systemmeasurement, it is expected that
combining the results of theJND image quality metric forseparate parts of a concatenatedsystem will provide a usefulindication of overall performance.
14
14An example of up to ten different compression methods in a complete television system is described in the paper "Why is Objective Evaluation Needed for Compressed Digital Video", by C. Dalton, presented at the
Made to Measure '96 Symposium, Montreux, Switzerland, November, 1996.
-
8/2/2019 A Guide to Picture
17/20
System Approach to ObjectiveTesting. Objective testingrequires a valid algorithm, suchas the JND image quality metric,as its foundation. However,implementation of a real-worldmeasurement system mustinclude a number of otheraspects such as: reference scenemotion sequences, a physical
source for the reference scenes,format conversions, scenechanges due to processing in thenon-compressed parts of thesystem, and accurate alignmentof pictures as an input to themeasurement algorithm. Anoverall block diagram of themeasurement system is shownin Figure 12 for application ofthe JND image quality metric. Areference sequence is suppliedto the system under test from asource such as a video recorder
or other picture generatingequipment (providing a definedvideo quality). Objective mea-surements of picture quality
including temporal aspects ofthe human visual system should
be possible with about twoseconds of video sequence.However, subjective assessment
by an expert viewer may also bedesired so the test sequencesource should provide five ormore seconds of continuousvideo which may be repeated or
palindromed for longer viewing.At the system output, thedegraded image is captured inthe picture quality measurementinstrument which also has acopy of the reference sequence.Reference and degraded picturefiltering, differencing and datapooling is accomplished withextensive compute power andthe results made available by anappropriate human interface orcomputer data connection.
Input to the system under test isa number of short referencesequences used in a directmeasurement technique, that is,
actual pictures are used ratherthan test signals. Multiple teststimulus is also the approachfor analog or full bandwidthdigital systems which use anumber of test signals in eitherdirect or indirect measurements.For picture quality measure-ments, the different referencesequences will represent various
applications for the system andtypes of program material. Someexamples are: sports followingthe action with backgroundmoving, sports stationarycamera with the action moving,scenes with high detail, panningand zooming on high detailscenes, rotary motion with colorsnot easily handled by somecompression systems, subtleskin tones and lighting, andscenes with variable amounts ofnoise content. One requirement
is the test material be such thatthe system being measured isworking at or near the limits ofits capabilities. This has always
15
5+ SecondsVisual
Sequence
Copy of Reference
Sequence
2 Seconds ofReferenceSequence Test
Results
ComputeEngine
Degraded Test Frames
Transport DecoderEncoder
TestSequence
Source
Figure 12. Objective measurement system.
-
8/2/2019 A Guide to Picture
18/20
been done with traditionalanalog measurements (an exam-ple would be use of the 2-Tpulse) and is even more impor-tant to stress the non-linearcharacteristics of video com-pression systems. Althoughscenes that break either thecompression system or measure-ment method will be of some
interest to find the outer limitsof the system, they are notappropriate for repeatable andconsistent measurements.
Studies which compare subjec-tive and objective picture qualitymeasurements generally concludethere is a moderately wide vari-ation in subjective results. Thisconclusion is often emphasized
by one or more scenes whosesubjective quality does notprovide good correlation withobjective measurements.Certainly it would be desirableto develop an objective methodwith no algorithm-breakingscenes, however standardizationof well behaved and truly repre-sentative scenes should providevery useful results. Consideringthat some program material doesnot correlate with signal qualitytest results in todays analogsystems (striped shirts near thesubcarrier frequency) and thatobjective tests for compressedvideo systems are predicted to
be only 90% to 95% accurate, itwould seem appropriate for theindustry to agree on a variety of
standardized motion sequencesfor objective measurement ofpicture quality. This will allowdevelopment of very useful, ifnot perfect, picture quality mea-surement equipment. An ANSIstandard defines a number ofscenes for testing of video con-ferencing systems15 and thereare a number of standards orga-
nizations working on definitionof a wider variety of test scenes.It will be very important to havea set of standardized test scenesso measurement data willcorrelate between differentmanufacturer's test equipmentand all systems designed forsimilar applications.
In order to make objective picturequality measurements, it isnecessary to insure the two videosequences are presented to theimage quality metric calculationin much the same manner asrequired for subjective tests.That is, gain and dc level of
both the luminance and chromi-nance must be closely matched.In addition, there must be tem-poral alignment and very accu-rate spatial alignment. Theselatter two requirements are dueto the need to do a type of a dif-ferencing process between videoframes as done with PSNR andthe JND image quality metric
model. Spatial alignment to anaccuracy of one-twentieth of apixel is required. Format con-
version may be required as partof the matching process. Manycompression systems have ana-log composite NTSC or PALinputs and/or outputs. Sincecomposite encoding and decodingproduces artifacts in the picturewhich are independent of thecompression system (althoughthey may well affect operation
of the compression coder) thereare two further requirements forthe picture quality measurementinstrument: an excellent qualitycomposite decoder and a refer-ence sequence that includes thecomposite artifacts. Experimentsconducted at Tektronix haveindicated that picture qualitytesting where composite encodeand decode processes areincluded will tend to mask mea-surements of compressionsystems with small amounts of
degradation, such as, MPEG-2main profile @ main level with
bit rates in the 12 Mb/s to 15Mb/s range. This appears to be areasonable result since those bitrates represent the highest qualityof entertainment video, eitherperfect NTSC/PAL or very goodcomponent video. Systems thatdont incorporate compositesignals and provide a Rec. 601input/output can be evaluatedfor very small picture degrada-tions (suitable for studio program
production contribution quality)
16
15ANSI T1A1.801.01-1995, "Digital Transport of Video, Teleconferencing/Video Telephony Signals, Video Test Scenes for Subjective and Objective Performance Assessment".
-
8/2/2019 A Guide to Picture
19/20
based on the JND imagequality metric.
Use of specific reference scenesmeans that testing will be out-of-service. This paradigm forvideo testing will not be popularwith those who have, for manyyears, used vertical interval testsignals (VITS). Although in-service testing with the actualprogram material would belogistically possible in someapplications (monitoring adirect broadcast satellite systemat the up-link location) it mightnot provide meaningful resultsfor a majority of the programmaterial which does not stressthe system. Beyond that is anoperational parameter that maynot be satisfactory with generalprogram material. Time to makethe measurement is an important
feature in test equipment. If thepicture matching; gain, spatialalignment, etc., is to be doneon program material, a largeamount of compute time will berequired to make correlationcalculations. This is in additionto the time required to just makethe measurement after the twovideo scenes are correctly
matched. Therefore, it is proposedthat some known alignmentsignals, or calibration stripes, beadded to the video sequencesfor rapid picture matching asshown in Figure 13. It is expectedthat future advancements incompute power and measure-ment algorithm optimizationwill allow in-service testing forapplications where the reference(input) and degraded (output)video is available at the mea-surement instrument. This is
important for statistically multi-plexed encoding systems where
bit rates are shared betweenprograms with the potential thatany part of program could bestressful to the encoding processdue to the bit rate allowed.
Picture Quality MeasurementInstruments
The need for objective measure-ment of picture quality (degra-
dation with respect to a reference)is well established and immediate.Formal and informal subjectivepicture quality assessment has
been used to develop, test,install and operate todays com-pressed television systems. Inthis guide, we have emphasizedthe continuing need for tradi-tional test methods and
described the new methodsbeing proposed for objectivemeasurement of picture quality.Tektronix and Sarnoff Labs arecooperating on the developmentof a picture quality measure-ment product based on the JNDimage quality metric and thesignal processing required foroperation within the completemodern television system.
This is an exciting new mea-surement paradigm for the tele-
vision and telecommunicationsindustries. Please contactTektronix to express your interestin this technology so you can beinformed as more technical andproduct information becomesavailable. Further theoreticalinformation and experimentdata will be disseminated byrevisions of this guide, paperspresented at conferences,informational seminars andpublication of articles in journalsand magazines.
17
Figure 13. Calibration stripes (top of picture).
-
8/2/2019 A Guide to Picture
20/20
Copyright 1997, Tektronix, Inc. All rights reserved. Tektronix products are covered by U.S. and foreign patents, issued and pending.Information in this publication supersedes that in all previously published material. Specification and price change privileges reserved.
For further information, contact Tektronix:
World Wide Web: http://www.tek.com;ASEAN Countries (65) 356-3900; Australia & New Zealand 61 (2) 888-7066; Austria, Eastern Europe, & Middle East 43 (1) 7 0177-261; Belgium 32 (2) 725-96-10;Brazil and South America 55 (11) 3741 8360; Canada 1 (800) 661-5625; Denmark 445 (44) 850700; Finland 358 (9) 4783 400; France & North Africa 33 (1) 69 86 81 08; Germany 49 (221) 94 77-400;Hong Kong (852) 2585-6688; India 91 (80) 2275577; Italy 39 (2) 250861; Japan (Sony/Tektronix Corporation) 81 (3) 3448-4611; Mexico, Central America, & Caribbean 52 (5) 666-6333;The Netherlands 31 2356 95555; Norway 47 (22) 070700; Peoples Republic of China (86) 10-62351230; Republic of Korea 82 (2) 528-5299; Spain & Portugal 34 (1) 372 6000; Sweden 46 (8) 629 6500;Switzerland 41 (41) 7119192; Taiwan 886 (2) 765-6362; United Kingdom & Eire 44 (1628) 403300; USA 1 (800) 426-2200
From other areas, contact: Tektronix, Inc. Export Sales, P.O. Box 500, M/S 50-255, Beaverton, Oregon 97077-0001, USA (503) 627-1916