Communication Acoustics: Paper ICA2016-391

Views on Sound Quality

Alexander Raake(a)

(a)Audiovisual Technology Group, TU Ilmenau, Germany , alexander.raake@tu-ilmenau.de

Abstract

The talk presents an overview of research on Sound Quality evaluation. In related research,sound quality refers to the judgment of auditory events with regard to internal references. Soundquality is closely related with Quality of Experience (QoE), that is, quality judgments address-ing auditory experiences. The talk reviews and discusses different research streams regarding"sound quality" and QoE, including work on product sounds, audio and speech technology. Theanalysis addresses assessment using human listeners as measurement entity as well as ap-proaches involving quality models, that is, algorithms that target predictions of human judgments.Perspectives for future research are provided, as well as considerations on limitations of practicalassessment strategies.

Keywords: sound quality, Quality of Experience, evaluation, tests, models

[Views on Sound Quality]

1 IntroductionSound quality evaluation has been a topic since the early days of sound generation, includingthe evaluation of musical instruments, technical systems such as the telephone, the gramo-phone or more recent audio coding and transmission. For example, the Bell receiver from1876 was used in the first telephone system, followed by the carbon microphone invented byEdison in 1877 – reportedly much better sounding than its predecessor (see [32]).

When addressing sound quality , a human recipient of sounds is considered, who uses the re-ceived acoustic signals to extract features and assign meaning, to interact with his environment,to communicate. As such, sound quality obviously is an important component of Communica-tion Acoustics. It should be noted that all sounds evaluated in terms of sound quality can beconsidered as product sounds, or at least, produced sounds.

A straight-forward example for a sound quality issue is a too low signal-to-noise ratio, whichmay affect the audibility or understanding of the wanted sound, and may be associated withannoyance due to the unwanted sound. Hence, communicability associated with its productionplays a crucial role [27]. Interestingly, humans dispose of different means to compensate for theeffect of degraded reception when they produce the wanted sound themselves during speaking.For example, in noisy environments, interlocutors speak in Lombard speech, that is, raise theirvoice and stress syllables differently from normal speech (e.g.[20, 16]). The speaker often isnot aware of actively applying means of situational sound quality improvements. It is importantto note that the respective recipient of such context-adapted speech will typically find it to benormal and appropriate, hence of high sound quality

These aspects are mentioned here to provide examples of how basic control mechanisms arenaturally used by humans to improve the resulting sound quality . Of course there are manymore such examples related with the creation of sound or audio, such as the musician adjustingher playing based on what she perceives, or the sound engineer controlling the results of herrecording or mixing process.

In technical engineering contexts, physical measurements were often used to evaluate and pos-sibly control certain processing steps or technology settings, such as sound pressure levels,frequency responses, decay times, or signal delays. With the creation of more complex techni-cal systems, perception and sound quality as integral components of Communication Acousticsare more and more used for system evaluation – less so however for optimization or control.In complex audio technology contexts such as the design of spatial audio systems or audioconferencing systems, automatic sound quality evaluation and respective control mechanismsare challenging, and hence here listening tests may be the only alternative.

In this paper, different aspects of sound quality are revisited. In particular, the role of thecreation and mixing process as a part of the end-to-end chain (recording, coding, storage /transmission, enhancement, reproduction, ...) is considered. The concept of quality has beencomplemented by that of Quality of Experience (QoE) in the past decade. The two concepts

2

will be revisited and related with research reported in the literature in the past years. Thepaper presents conceptual considerations, and is accompanied by another paper on the as-sessment of sound quality for spatial audio technology such as Wave-Field Synthesis (WFS)or stereophonic systems [30].

2 QualityIn the recent past, efforts were made to clarify the definitions related with quality and themore recent Quality of Experience. The term quality has different connotations, dependingon the context it is used in (see work by Parasuraman et al., Reeves & Bednar, Blauert &Jekosch, Jekosch, Martens & Marten [26, 31, 3, 13, 21]). In the following, a summary of theconsiderations the author provided in [29] is given.

A noteworthy review of quality definitions was provided by Reeves and Bednar in 1994 [31].The most relevant definition according to their view is “the extent to which a product or servicemeets and/or exceeds a customer’s expectations” (see also [21]), which they refer to as adefinition from the service marketing domain. Along those lines, they state that services weremost difficult to include in previous definitions of quality up to their publication in 1994.

Some years earlier, it was stated that “only customers judge quality” and “all other judgmentsare essentially irrelevant” (cited by [31] from [40]). A related wording was introduced into stan-dardized definitions by ISO in 2000 [6], stating that quality “is the ability of a set of inherentcharacteristics of a product, system or process to fulfill requirements of customers and otherinterested parties”. Interestingly, the current definition of Quality of Service (QoS) by the ITU-Treflects the same view as the ISO-definition of quality, now from a manufacturer’s or serviceoperator’s perspective. Quality of Service is defined as the “totality of characteristics of atelecommunications service that bear on its ability to satisfy stated and implied needs of theuser of the service”.

The views on quality from the nineties are reflected in Jekosch’s definition of quality for her workon voice and speech quality. Here, quality is defined as [13]: “The result of the judgment ofthe perceived composition of an entity with respect to its desired composition.” As stated alsoin [29], it is assumed here that this definition exclusively addresses perception that involvessensory processing of external stimuli.

3 Quality of Experience (QoE)Based on different considerations around the need for a perception- and user-related coun-terpart of QoS (as defined by ITU-T, see above), the term Quality of Experience (QoE) wasintroduced to the ICT / multimedia field in the early 2000s, and later standardized by ITU-T inRec. P.10. Since the respective definition was considered inadequate by many experts, an im-proved definition was developed within the European COST Action Qualinet, as a collaborationof approximately 20-30 scientists, captured by the author of this paper in[25]. In a later update,QoE has been defined as [25, 29]): “Quality of Experience is the degree of delight or annoy-

3

ance of a person whose experiencing involves an application, service, or system. It resultsfrom the person’s evaluation of the fulfillment of his or her expectations and needs with respectto the utility and/or enjoyment in the light of the person’s context, personality and current state.”

According to this definition, QoE applies to a judgment of experiencing in terms of “... theindividual stream of perceptions (of feelings, sensory percepts and concepts) that occurs ina particular situation of reference.” [29]. This definition reflects that experiencing can havehedonic (≡ feelings) and pragmatic (≡ concepts) aspects.

As pointed out in [29], the definition is closely related with the concepts of quality of experi-ence and experienced utility by Kahnemann [15]. According to Kahnemann, experienced utilityrefers to judgment in terms of good/bad of a given experience, related with individually per-ceived “pleasure and pain”, “point[ing] out what we ought to do, as well as determine what weshall do” (Kahneman [14], making reference to Bentham, [1]). Experience(ing) here may relateto painful medical investigations such as colonoscopy [15], or pleasant phases of experiencing,for example when attending a concert, or to the quality of life at large [15]. Here, it is interest-ing to note the German translation of “quality of experience” as used by Kahneman, Qualitätdes Erlebens, which explicitly reflects that experiencing is a process. “Quality of experience”assessment can be made “moment-based”, or “memory-based” (“remembered utility”) [14].

4 Complementing conceptsTo better distinguish quality and QoE, the authors in [29] have amended the term quality ac-cording to Jekosch’s definition to quality (based on experiencing). For sound events, quality(based on experiencing) becomes sound quality . Then, sound quality corresponds to basicaudio quality as used in standards such as MUSHRA [7] or PEAQ [9].

Sound quality according to the definitions in this paper reflects the case where the evaluatingperson is aware of the technical system or at least the form / carrier [12] of the sound, andassesses it directly, for example when she tries out different audio systems for a purchase ina store, or when she takes part in an audio quality listening test. Real Quality of Experienceaccording to the above implies that the listener is not necessarily aware of how the listeningexperience is influenced by the technology used during different steps such from recording toreproduction. Due to the general difficulty of direct Quality of Experience assessment, most ofthe literature from the audio technology domain is on sound quality .

When a person listens to sounds involving some kind of technical system, she can typicallytake two perspectives, focusing (a) on the medium that is employed, in terms of an artifact,for example paying attention to the sound features related with a certain audio system whenreproducing a musical piece, or (b) on the auditory scene or musical piece presented to her.Mausfeld [23] has described this as the “dual nature” of perception.

Sound quality involves sensory processing of sound. In [29], the complementary term assumedquality is defined as “the quality and quality features that users, developers, manufacturers orservice providers assume regarding a system, service or product that they intend to be using,or will be producing, without however grounding these assumptions on an explicit assessment

4

of quality based on experiencing.” This term was introduced since often times the evaluation oreven choice of a given multimedia technology is made with regard to specifications or physicalassessment criteria instead of own perception or experiencing of resulting stimuli.

5 Quality formation, internal references and expectationsIn all of the above concepts, both related with sound quality and QoE, the evaluating personcompares percepts with internal references. In the following, a brief review of concepts relatedwith internal references is provided, building on the considerations in [29].

5.1 References and layer model

The work on speech quality by Jekosch has been accompanied by parallel work on soundquality at a larger scale by Jekosch and Blauert [3]. Additional concepts discussed in [3] relatewith the character of references, and how these may differ depending on the respective soundquality assessment context.

A classification scheme of aural quality references according to four different layers of abstrac-tion was proposed in [2]: (1) Aspects of psychoacoustics, (2) aspects of perceptual psychology,(3) aspects of physical acoustics, (4) aspects of communication sciences.

Here, psychoacoustics references relate with fundamental concepts such as loudness, spectralbalance, spaciousness (locatedness, source width, etc.), absence of artifacts. These featuresdo not form aural objects as such, but are only associated with them, or underly their formationprocess. According to [2], the layer “perceptual psychology” refers to the object-formation andscene-analysis step. Instead of analytic listening as for the psychoacoustic properties, listenersfocus on object properties and aspects such as their constancy and plausibility (for example interms of identity). Blauert argues that the work of Tonmeisters and sound-engineers is mainlyhappening at this level.

At the third layer of abstraction lies the analysis in terms of physical acoustics. This classifi-cation may appear counter-intuitive at first, since this physical level is typically assumed to liebelow any other level, based on how persons process acoustically (physically) presented infor-mation. However, Blauert here addresses the description-component of this view, and for this,mathematical abstraction is required. The general motivation of including this level here is dueto the fact that physical descriptors appear to be good correlates of certain perceived featuresof aural quality.

According to Blauert, the highest level of abstraction relates with the meaning associated with ascene or (aurally) perceived message. Here, intra- and inter-personal as well as larger culturaland social aspects come into play, and the received signs as of the semiotic view according toJekosch [13, 12] are interpreted as a whole. It is at this level that the experiencing in terms ofthe above perspective is fully involved.

In essence, sound quality as defined by the author of this paper encompasses the layers 1 and2, in terms of quality (based on experiencing)[29], in line with the definition by Jekosch [13], and

5

according to the considerations in Sec. 2, in line with the term Basic Audio Quality [8, 35, 24].In turn, layer 3 is related with the aspect of assumed quality as defined above [29]. Along thoselines, layer 4 is related with the considerations around the definition of Quality of Experience.

5.2 Multidimensional view of sound quality

As outlined for example in [17], two sounds can be distinguished based on their loudness,pitch, timbre, spaciousness and duration. The totality of these features describes the perceivedcomposition, perceived nature or character of a sound [13, 11, 17, respectively]. In essence,this reflects a multidimensional view of auditory events (compare layers 1 and 2 above).

Accordingly, the effects that certain audio technology such as spatial audio systems or codingand transmission may have on sound quality is multi-dimensional, too. Specific terminology hasbeen suggested by Jekosch in this regard, distinguishing quality features from quality elements[13]. Quality elements are, so to speak, the knobs and screws that a designer of the technology,service or system has at hand that may ultimately influence sound quality or QoE – see theQoS definition by ITU-T above. Quality features are the perceptual features relevant and usedby a person for judging sound quality (or QoE).

The quality-feature-based, analytical view is related with the large body of sound quality re-search inspired by sensory evaluation from fields such as the food and beverage industries.Examples include work on speech quality [22, 36], concert hall acoustics [19], spatial audioquality [37, 18, 38, 39], and audiovisual quality evaluation [34]. In these works, multidimensionalanalysis using techniques such as attribute scaling (with and without prior attribute elicitation),multidimensional scaling or mixed-methods approaches are used to determine the perceptualfeature space associated with sound quality . In a subsequent step referred to as (external)preference mapping (e.g. [4]), the resulting dimensions may be mapped to uni-scale soundquality ratings obtained using methods such as MUSHRA [8], Absolute Category Rating [10]or so far non-standardized quality-rating methods (see e.g. [39]). Often times a so-called ideal-point of the multidimensional feature space can be found that represents the best-possiblesound quality (e.g. [22]). In principle, finding this point can be considered as an implicit way ofdetermining the dimensions associated with the perceptual reference.

A complementary review of the literature on expectations and references from the field of mar-keting is provided in [29]. There, the person’s role as customer is addressed.

6 Closing remarksThe terms sound quality and QoE were reviewed from the authors perspective. Most of the pa-per was directly or indirectly focused on assessing quality involving human listeners or users. Itis noteworthy to mention that there are a number of approaches to automatically predict soundquality based on technical system parameters (e.g. [27] for speech), or real or test signals, forexample using some human-inspired, more or less complex bottom-up feature extraction (e.g.[35, 33, 30]). For more details, the reader is referred to the original works, and the author’sconsiderations on modelling in [30, 28].

6

person

technology media

creation

presentation

creator

context

perceptual event

signal event

quality event

desired features/references

mental state

behavior

behavior

per-cep-tion

evalu-ation

person

technology media

presentation

context

perceptual event

signal event

judgment (quality)

description (features)

quality event

desired features/references

mental state

behavior

per-cep-tion

evalu-ation

en-coding

en-coding

Figure 1: (Top) Quality perception in the context of creation/production. The person and creatormay be identical. Quality of experience or quality based on experiencing will be used by thepersons involved in the creation as target for optimization. (Bottom) Quality during listening.In case of random perception, no “encoding” into sound quality judgments or a description willbe performed, and both sound quality and QoE may “happen” in the mind of the person, de-pending on the attention to the sound or the overall experience. In case of controlled listeningas in a sound quality test, the impression of sound quality will be encoded into a description(sensory evaluation, see above) or a quality judgment, for example on a rating scale.

7

There are numerous challenges regarding sound quality and QoE evaluation. These are re-flected to some extent already by the outlined concepts and definitions provided above. Oneparticular challenge needs to be pointed out: In the past, sound quality evaluation has mostlybeen done using a “fidelity-paradigm” (see also [30]). However, with new technologies such asbinaural re-synthesis (“binaural audio”) or loudspeaker-based sound field synthesis, there is nosuch explicit reference. Due to today’s listening habits, the internal reference is likely basedon prior experience with technology such as stereophonic audio. Also, no established au-dio recording, mixing and processing process exists for new audio technology paradigms (see[30]). This results in dedicated requirements on the end-to-end processing chain for audio forrespective tests and modelling campaigns [5], and assessment techniques that do not involveexplicit reference stimuli, as for example the MUSHRA-inspired but reference-free approach in[39], or full paired-comparison preference tests as described in [30].

The complexity of sound quality and QoE in a person’s mind and its (external) assessmentis illustrated in Fig. 1. A conceptual model of a person is shown, for example influencing thesound herself, rating sound quality in a quality test, or naturally listening to some music in aconcert or at home.

There obviously is the need for more research on sound quality and QoE, with regard to testmethods as well as modelling of the resulting data towards automatic prediction.

AcknowledgementsThis research has been supported by EU FET grant TWO!EARS, ICT-618075 (see www.twoears.eu). The authors wish to thank Sascha Spors, University of Rostock, and Jens Blauert, Ruhr-University Bochum, for fruitful discussions.

References

[1] J. Bentham. An Introduction to the principle of morals and legislations. Blackwell, 1948,Reprinted Oxford, UK, 1789.

[2] J. Blauert. Conceptual aspects regarding the qualification of spaces for aural perfor-mances. Acta Acustica united with Acustica, 99:1–13, 2013.

[3] J. Blauert and U. Jekosch. Concepts behind sound quality: Some basic considerations.In: Proc. Internoise 2003, KR–Jeju, 1:72–79, 2003.

[4] J. D. Carroll. Individual preferences and multidimensional scaling. volume I, pages105–55, 1972.

[5] C. Hold, H. Wierstorf, and A. Raake. The difference between stereophony and wavefield synthesis in the context of popular music. In Audio Engineering Society Convention140. Audio Engineering Society, 2016.

[6] International Organization for Standardization. ISO 9000:2000, Quality managementsystems: Fundamentals and vocabulary., 2000.

8

[7] International Telecommunications Union. ITU-R Recommendation BS.1534: Method forthe subjective assessment of intermediate quality levels of coding systems.

[8] ITU–R BS.1534-1. Method for the Subjective Assessment of Intermediate Quality levelof coding systems. International Telecommunication Union, CH–Geneva, 2003.

[9] ITU–R Rec. BS.1387. Perceptual evaluation of audio quality (PEAQ). InternationalTelecommunication Union, CH–Geneva, 2001.

[10] ITU–T Rec. P.800. Methods for Subjective Determination of Transmission Quality. Inter-national Telecommunication Union, CH–Geneva, June 1996.

[11] U. Jekosch. Basic concepts and terms of “quality", reconsidered in the context ofproduct sound quality. Acta Acustica united with Acustica, 90(6):999–1006, 2004.

[12] U. Jekosch. Assigning meaning to sounds: Semiotics in the context of product-sounddesign. In: Communication Acoustics (J. Blauert, ed.), pages Springer, D–Heidelberg,forthcoming, 2005.

[13] U. Jekosch. Voice and Speech Quality Perception — Assessment and Evaluation.Springer, D–Berlin, 2005.

[14] D. Kahneman. Experienced utility and objective happiness: A moment-based ap-proach. In I. Brocas and J. D. Carrillo, editors, The psychology of economic decisions,pages 187–208. Oxford University Press, 2003.

[15] D. Kahneman. Objective happiness. In D. Kahneman, E. Diener, and N. Schwarz, ed-itors, Well-Being: The Foundations of Hedonic Psychology, pages 3–25. Russell SageFoundation, 2003.

[16] H. L. Lane, B. Tranel, and C. Sisson. Regulation of voice communication by sensorydynamics. J. Acoust. Soc. Am., 47(2):618–624, 1970.

[17] T. Letowski. Sound quality assessment: Concepts and criteria. Preprint 87th Audio En-gineering Society (AES) Convention, (Paper D-8, Preprint 2825):USA–New York, Octo-ber, 1989.

[18] A. Lindau, V. Erbes, S. Lepa, H.-J. Maempel, F. Brinkman, and S. Weinzierl. A spa-tial audio quality inventory (saqi). Acta Acustica united with Acustica, 100(5):984–994,2014.

[19] T. Lokki, J. Pätynen, A. Kuusinen, H. Vertanen, and S. Tervo. Concert hall acousticsassessment with individually elicited attributes. The Journal of the Acoustical Society ofAmerica, 130(2):835–849, 2011.

[20] E. Lombard. Le signe de lelevation de la voix. Ann. Maladies Oreille, Larynx, Nez,Pharynx, 37(101-119):25, 1911.

[21] H. Martens and M. Martens. Multivariate analysis of quality. Wiley, Chichester, 2001.[22] V.-V. Mattila. Ideal point modelling of the quality of noisy speech in mobile communi-

cations based on multidimensional scaling. In: Proc. 114th Audio Engineering Society(AES) Convention, March 2225, NL–Amsterdam, 2003.

[23] R. Mausfeld. Conjoint representations and the mental capacity for multiple simultane-ous perspectives. In H. Hecht, R. Schwartz, and M. Atherton, editors, Looking into pic-tures: An interdisciplinary approach to pictorial space, pages 17–60. MIT Press, 2003.

9

[24] J. H. Michael Schoeffler. About the impact of audio quality on overall listening expe-rience. In Proceedings of the Sound and Music Computing Conference (SMC), pages58–53, 2013.

[25] Q. Q. W. Paper. Qualinet White Paper on Definitions of Quality of Experience. COSTAction IC 1003, S. Möller, P. Le Callet and A. Perkis (eds.), Lausanne, CH-Switzerland,1.1 edition, 2012.

[26] A. Parasuraman, V. Zeithaml, and L. Berry. A conceptual model of service quality andits implications for future research. Journal of Marketing, 49(Fall 1985):41–50, 1985.

[27] A. Raake. Speech Quality of VoIP – Assessment and Prediction. John Wiley & SonsLtd, Chichester, West Sussex, UK, 2006.

[28] A. Raake and J. Blauert. Comprehensive modeling of the formation process of sound-quality. In Proc. IEEE QoMEX, Klagenfurt, Austria, 2013.

[29] A. Raake and S. Egger. Quality and quality of experience. In S. Möller and A. Raake,editors, Quality of Experience. Advanced Concepts, Applications and Methods, chap-ter 2. Springer, Berlin–Heidelberg–New York NY, 2014.

[30] A. Raake and H. Wierstorf. Assessment of audio quality and experience using binaural-hearing models. In Proceedings 22nd International Congress on Acoustics (ICA), 2016.

[31] C. A. Reeves and D. A. Bednar. Defining quality: Alternatives and implications.Academy of Management Review, 19(3):419–445, 1994.

[32] D. L. Richards. Telecommunication by Speech. Butterworths, UK–London, 1973.[33] F. Rumsey, S. Zielinski, P. Jackson, M. Dewhirst, R. Conetta, S. George, S. Bech,

and D. Meares. QESTRAL (part 1): Quality evaluation of spatial transmission andreproduction using an artificial listener. In 125th Conv. Audio Eng. Soc., 2008.

[34] D. Strohmeier, S. Jumisko-Pyykkö, and K. Kunze. Open profiling of quality: A mixedmethod approach to understanding multimodal quality perception. Advances in Multi-media, 2010(Article ID 658980):28 pages, doi:10.1155/2010/658980, 2010.

[35] T. Thiede, W. Treurniet, R. Bitto, C. Schmidmer, T. Sporer, J. Beerends, C. Colomes,M. Keyhl, G. Stoll, K. Brandenburg, and B. Feiten. PEAQ - the ITU standard for objec-tive measurement of perceived audio quality. J. Audio Eng. Soc., 48:3–29, 2000.

[36] M. Wältermann, A. Raake, and S. Möller. Quality dimensions of narrowband and wide-band speech transmission. Acta Acustica united with Acustica, 96(6):1090–1103, 2010.

[37] H. Wierstorf, A. Raake, M. Geier, and S. Spors. Perception of focused sources in WaveField Synthesis. Journal of the Audio Engineering Society, 61(1-2):5–16, 2013.

[38] N. Zacharov, T. Pederson, and C. Pike. A common lexicon for spatial sound qualityassessment latest developments. In Proceedings IEEE 8th International ConferenceQuality of Multimedia Experience (QoMEX), pages 1–6, 2016.

[39] C. ZacharoZacharov, Nick, F. Melchior, and T. Worch. Next generation audio systemassessement using the multiple stimulus ideal profile method. In Proceedings IEEE 8thInternational Conference Quality of Multimedia Experience (QoMEX), pages 1–6, 2016.

[40] V. A. Zeithaml, A. Parasuraman, and L. Berry. Delivering quality service. The FreePress, New York, USA, 1990.

10