multivariate analysis of sensory dataetd.dtu.dk/thesis/222770/ep08_84.pdf · as evaluated by ten...

Multivariate Analysis of Sensory Data

Phucan Le

Kongens Lyngby, 2008

| iii

Technical University of Denmark Informatics and Mathematical Modelling Building 321, DK-2800 Kongens Lyngby, Denmark Phone +45 45253351, Fax +45 45882673 [email protected] www.imm.dtu.dk

iv |

Summary

In this thesis a sensory profiling data set describing the quality of fish on 17 different sensory attributes

as evaluated by ten assessors is analyzed with main focus on the differences between products (fish

given different feed and stored on ice a different amount of days. Especially the difference resulting

from different feed is of interest. The analysis will be done by a number of multivariate methods.

Comparison of the performance of the methods on the given data set is also an objective of the thesis,

but only secondary.

The data analytical strategy involves a descriptive statistical analysis to obtain an overview of the

distribution and standard deviations of the scores for each sensory attribute along with the correlations

between pairs of attributes. Especially the presence of multicollinearity is of interest as the performance

of the multivariate methods depends on this.

A Principal Component Analysis (PCA) is employed in order to visualize the main tendencies of variation.

3-way and 4-way univariate mixed models are analysed in order to model multivariate test statistics. 3-

way and 4-way mixed model MANOVA are done together with Canonical Variate Analysis (CVA) in order

to visualize the main tendencies in the same way as done with the PCA, but taking the error structure

into account and with p-values for difference in products.

A 50-50 MANOVA is done using the principles of both PCA and MANOVA, hereby obtaining test statistics

on dimension reduced data.

The statistical reliability and predictive validity of the product differences are obtained by (M)ANOVA

and cross validation.

Similar data structures are observed in the various multivariate (and univariate) methods with slight

differences. Odor, flavor and texture attributes differentiated the fish samples and the different type of

feed had an effect on the sensory evaluation of the fish.

vi |

Resume

Et sensorisk profilerings data sæt der beskriver kvaliteten af fisk, givet forskelligt foder og islagret

forskellige antal af dage, evalueret af 10 dommere på 17 forskellige sensoriske attributter analyseres

med henblik på at undersøge for forskelle mellem produkter. Specielt forskelle forsaget af de forskellige

typer af foder er af interesse. Analysen udføres ved hjælp af forskellige multivariate metoder.

Sammenligninger af metoder ud fra deres brugbarhed til analyse af dette data sæt vil også blive lagt

vægt på dog kun sekundært.

Data explorative analyse udføres for at skabe overblik over distributioner og standardafvigelser af de

sensoriske attributer. Korrelationer mellem attributterne og specielt tilstedeværelsen af

multicollinearitet vil blive undersøgt da dette har stor betydning for brugbarheden af de forskellige

multivariate metoder.

En Principal Component Analyse (PCA) udføres for at visualisere hovedtendenser af variation i data.

3-vejs og 4-vejs univariate mixede modeler undersøges med henblik på senere modellering af test

statistikker for mixede multivariate modeller.

3-vejs og 4-vejs mixed model MANOVAer udføres på dette grundlag sammen med Canonisk Variate

Analyse (CVA) for at visualisere tendenser I data på same made som ved PCA, bare med fejlstrukturen

taget højde for og med p-værdier for test af produkt forskel.

50-50 MANOVA udføres som en kombination af metoderne PCA og MANOVA, hvorved p-værdier for

test på dimensionsnedsat data opnås.

Den statistiske validitet af modellerne opnås (M)ANOVA og kryds validering.

Lignende resultater opnås ved brug af de forskellige multivariate metoder med små afvigelser. Det kan

konkluderes at de anvendte lugt, smag og tekstur attributter beskriver forskellene mellem produkter og

at typen af foder påvirker den sensoriske kvalitet af fisk.

viii |

Preface

This thesis was prepared at Informatics Mathematical Modelling, the Technical University of Denmark in

fulfillment of the requirements for acquiring the Master of Science degree in engineering.

This thesis deals with the analysis of a sensory profiling data set describing the quality of fish on a

number of sensory attributes as evaluated by a number of assessors. Analysis is done by various

multivariate methods.

The thesis is divided in two parts. In Part 1 a brief introduction of the models used in the analysis is

given. In Part 2 the analysis by the methods described in Part 1 is done.

Phucan Le

x |

Acknowledgements

First and foremost I would like to thank my mum and dad. I would never have done this without you.

I would also like to thank my brothers and sisters for doing cool stuff like complaining over not being

mentioned in my thesis.

I would also like to thank my advisor Per Bruun Brockhoff for excellent guidance.

Finally I would also like to thank Grethe Hyldig for the data.

xii |

Contents

Summary ................................................................................................................................................ iv

Resume .................................................................................................................................................. vi

Preface ................................................................................................................................................. viii

Acknowledgements ................................................................................................................................. x

Contents................................................................................................................................................ xii

Part 1 Theory ........................................................................................................................................... 1

Chapter 1 Sensory Profiling Data ............................................................................................................. 3

Chapter 2 Principal Component Analysis (PCA) ........................................................................................ 7

2.1 Data representation in PCA ................................................................................................................ 8

2.2 The principle of PCA: Geometric approach ......................................................................................... 8

2.2 Properties of PCA ............................................................................................................................. 10

2.4 Residual analysis .............................................................................................................................. 11

2.5 Validation by cross validation .......................................................................................................... 11

2.6 The principle of PCA: The algebraic approach .................................................................................. 12

2.7 calculating the principal components ............................................................................................... 13

Chapter 3 Multivariate Analysis of Variance (MANOVA)......................................................................... 15

3.1 One-way Models.............................................................................................................................. 16

3.1.1 Univariate one-way ANOVA ...................................................................................................... 16

Tests of significance ....................................................................................................................... 17

3.1.2 Multigroup one-way MANOVA .................................................................................................. 18

Tests of significance ....................................................................................................................... 18

Wilks’ Test Statistics....................................................................................................................... 19

| xiii

Roy’s test ....................................................................................................................................... 20

Pillai’s test ..................................................................................................................................... 21

Lawley-Hotellings tests .................................................................................................................. 21

3.2 Unbalanced one-way MANOVA ....................................................................................................... 21

Summary of the former tests ............................................................................................................. 22

Chapter 4 Mixed Model ANOVA ............................................................................................................ 25

Estimating G and R in mixed model .................................................................................................... 26

Estimating β and γ in the Mixed Model .......................................................................................... 26

Inferential tests ................................................................................................................................. 26

Chapter 5 Canonical Variate Analysis (CVA) ........................................................................................... 28

5.1 Principle of CVA ............................................................................................................................... 29

5.2 Calculationg the canonical variates .................................................................................................. 29

5.3 CVA and PCA .................................................................................................................................... 29

Chapter 6 50-50 MANOVA ..................................................................................................................... 31

Part 2 Analysis of Fish Data .................................................................................................................... 35

Chapter 7 Fish Data ............................................................................................................................... 37

Design of data ................................................................................................................................... 39

Missing values Imputation ................................................................................................................. 39

Chapter 8 Initial Explorative Analysis ..................................................................................................... 41

Panel assessment .............................................................................................................................. 43

Chapter 9 Multivariate Analysis by PCA ................................................................................................. 51

Outliers ............................................................................................................................................. 51

Scaling or no scaling .......................................................................................................................... 52

the final model .................................................................................................................................. 53

results ............................................................................................................................................... 54

Other PCA models on subsets of data ................................................................................................ 56

PCA on subset of data not containing Time=0 and Time=12 ............................................................... 56

Chapter 10 Univariate Analysis by Mixed Model ANOVA and ANCOVA .................................................. 59

10.1 3-way univariate model ................................................................................................................. 60

Test of fixed effects ........................................................................................................................... 61

Test of random effects parameters .................................................................................................... 61

xiv |

Post hoc analysis ............................................................................................................................... 61

Validation of model ........................................................................................................................... 62

Residuals ....................................................................................................................................... 62

Variance homogeneity ................................................................................................................... 62

Outliers .......................................................................................................................................... 64

Normality of random effects .......................................................................................................... 64

10.2 4-way univariate mixed model ANOVA .......................................................................................... 66

10.2.1 4-way ANOVA ............................................................................................................................. 68


10.2.2 3-way mixed model ANCOVA ...................................................................................................... 70

10.2.3 Validation of 4-way ANOVA and 3-way ANCOVA ......................................................................... 71

10.3 Discussion of results ...................................................................................................................... 72

Chapter 11 Multivariate Analysis bu Mixed Model MANOVA with CVA .................................................. 73

11.1 3-way mixed model MANOVA ........................................................................................................ 74

Tests of fixed effects .......................................................................................................................... 74


Validation of model ........................................................................................................................... 79

11.2 4-way mixed model MANOVA ........................................................................................................ 79

Test of fixed effects ........................................................................................................................... 79


11.3 Comparison with PCA .................................................................................................................... 85

Chapter 12 Multivariate Analysis by 50-50 MANOVA ............................................................................. 87

Chapter 13 Conclusion and Discussion ................................................................................................... 89

Appendix A ............................................................................................................................................ 91

Appendix B ............................................................................................................................................ 92

Appendix C ............................................................................................................................................ 94

Appendix D ............................................................................................................................................ 97

Least square mean estimates ................................................................................................................ 97

Pair wise comparisons using tukey adjustment ...................................................................................... 98

Appendix E .......................................................................................................................................... 113

Mixed model ANOVA and ANCOVA from Chapter 10 ........................................................................... 114

3-way mixed model ANOVA ............................................................................................................. 114

| xv

4-way mixed model ANOVA ............................................................................................................. 114

3-way mixed model ANCOVA ........................................................................................................... 114

3-way mixed model MANOVA with CVA .......................................................................................... 115

4-way mixed model MANOVA with CVA .......................................................................................... 116

Part 1

Part 1 Theory

| 3

Chapter 1

Sensory Profiling Data

In a sensory profiling dataset different products are assessed on a number (often large) of attributes by

a trained panel consisting of a number of assessors. If more than one assessment is done these make up

the replicates. In conventional sense the data is described by three design variables: Product, Assessor

and Replicate, and response variables given by the attributes.

In a complete design the data structure for the independent variables is as illustrated in Figure 1

Figure 1 Design variables: Product assessor replicate

In the balanced case the total number of samples is found by multiplying the number of products,

assessors and replicates.

Disregarding the replicates for a moment, the data including the attributes (response variables) can be

expressed as in Figure 2

4 |

Figure 2 the data represented as tables by 1 assessor (left) and by the entire panel (right)

For a single assessor the data can be summarized as in Figure 2 left. Here the products are given scores on

the attributes by a single assessor. The replicates could be envisioned by imagining as many tables of

this kind as there are replicates or simply as a product added to the list.

The data for the entire panel can be summarized as in Figure 2 right. Here there is a data table similar to

the one described in Figure 2 left for a single assessor for each assessor.

The main objective the sensory profiling dataset is to answer the question of difference in the products.

The methods used and the analysis should be made having this in mind.

The setup of the data leaves different reasonable ways of using the data. As discriminating amongst the

products is the overall objective, information regarding products should not be compromised.

Reasonable alternations should hence only concern averaging unfolding attributes and assessors

Two cases for attributes are relevant:

Univariate analysis by viewing only one attribute at a time

Multivariate Analysis in the case that all attributes are considered as a whole

Three ways of viewing the assessors are relevant:

| 5

One assessor at the time

The panel mean

The entire panel, all assessors

According to which case is being considered, different methods are applicable. These will be described

in the following section. In Table 1 they are listed according how they will be used in the analysis of the

data in this report:

1 assessor Panel mean Panel

Univariate

ANOVA Mixed model ANOVA Mixed model ANCOVA

Multivariate

PCA

Mixed model MANOVA Canonical Variate Analysis 50-50 MANOVA

Table 1 Statistical methods

A special feature in data of this kind is the presence of a panel. Even if the overall aim of the data is to

investigate for product differences or similarities means should be taken to ensure that the panel

delivers data that live up to certain criteria. No clear consensus exists on this issue but a number of

suggestions that are intuitively clear for an acceptable panel performance are:

Repeatability

Same products get same scores

Compare Replicates

Validity

For a single assessor to be in agreement with the panel (on a mean)

Ability to score products the same on average with other panel members

For the panel not to include too many assessors who disagree

Panel homogeneity (but assessors ensure consensus rather than

Discrimination

Ability to give different products different scores

| 7

Chapter 2

Principal Component Analysis (PCA)

PCA is a multivariate technique applied to a single set of variables. The method seeks to maximize the

variance of a linear combination of the variables. It provides the most compact representation of all the

variation in a data table. This is done by summarizing the original variables into much fever and

informative variables called scores. These new variables are linearly weighted combinations of the

original X-variables. The loadings contain the weights used for each X-variable and thus reveal the

influence of individual X-variables.

It can be used as a dimension reducing method, representing multivariate data as a low dimensional

plane. It was first described by Pearson as finding “lines and planes of closest fit to systems in space”

[Jackson, 2003]. Statistically PCA finds lines, planes and hyper planes in K-dimensional space that

approximate the data as well as possible in a least squares sense (this is equivalent to maximizing the

variances).

In section 2.1 it will be presented from a geometric point of view as a rotation of coordinates. In sections

2.6 and 2.7 an algebraic approach will be given. Understanding the link between these two approaches

is important for full understanding of the link with Canonical Variate Analysis in Chapter 5.

The concept of cross validation is described in section 2.5.

8 |

2.1 Data representation in PCA With n observations and p random variables,

pxxx ...,,, 21 the data is represented by the np data

matrix X

pnp

n

xx

xx

1

111

X

Without loss of generality it will be assumed throughout this chapter that X is centered by variable. The

variables can then be viewed as having zero mean.

The sample covariance matrix, S, of X is given as

ppp

p

ss

ss

1

111

S

Where the components of S, jks ,

n

i

kikjijnjk xxxxs1

11

represents covariances between the variable components jx and kx . The components iis are the

variance of component ix . If two components jx and kx of the data are uncorrelated, their covariance

is zero ( jks = kjs = 0). S is by definition always symmetric.

It may be noted that PCA can be done on the correlation matrix R as well.

2.2 The principle of PCA: Geometric approach In the following the method of PCA is described as a coordinate rotation. It might be useful reviewing

Appendix A on characteristic roots and vectors before proceeding.

The data table, the X matrix, with p rows and n columns can be represented as a swarm of points (n

points) in a p-dimensional space. The centering of the variables assumed in the former section,

resembles transforming the origin of the original axes to the point x .

In the following the indexing of the columns of X and the elements of X is given only by the descriptive

index. That is the n observation vectors 1x , 2x ,…, nx and the p coordinates 1x , 2x ,…, px for each of these

is given so:

| 9

nxxX 1

and

px

x

1

1x.

The p axes can be rotated by multiplying each ix by an orthogonal pp matrix A:

i

T

i xAz

Finding the orthogonal matrix A that rotates the axes, so that the new variables (the principal

conmponents) pzz ,...,1 in XAZT are uncorrelated, can be done by letting the covariance matrix of Z,

zS , be diagonal. That is

2

2

2

0

0

00

2

1

pz

z

z

z

s

s

s

S

The sample covariance matrix of xAzT is given by

SAAST

z (2.1)

By the property that a symmetric matrix S is diagonalized by an orthogonal matrix of its eigenvectors

and the resulting diagonal matrix contains eigenvalues of S (see appendix A)

p

T

0

0

00

2

1

DSCC , (2.2)

Where pii ,...1, are the eigenvalues of S, C is the orthogonal matrix whose columns are normalized

eigenvectors, ia , of S

naaC 1

From (2.1) and (2.2) it is seen that the matrix that diagonalizes yS is the transpose of the matrix C:

10 |

T

p

T

T

a

a

CA 1

The principal components are the transformed variables xaxaT

pp

T zz ...,,11 in XAZT . The

eigenvalues p ,...,1 of S are the sample variances of the principal components xaT

iiz . That is

1

2 izs

xaTz 11 has the largest sample variance and T

ppz a the smallest.

2.2 Properties of PCA Because of the way the solution is constructed the method of PCA has a number of useful properties.

1. ii

T

i ,1aa

2. Any two principal components xaT

iiz and xa

T

jjz are orthogonal for ji .

The first two properties follow from the property that A is orthonormal.

3. The principal components are uncorrelated in sample, that is the covariance of iz and jz is zero:

jis j

T

izz ,021

Saa

4. The principal components are not scale invariant. If variables are standardized before calculating

eigenvalues and eigenroots (finding principal from the correlation matrix R) then these principal

components are scale invariant.

5. Because i ’s are variances of the PC’s it makes sense to talk of “the proportion of variance

explained by the first k components:

p

j

jj

p

p

k

s

riancevaofproportion

1

1

1

1...

...

...

since )(1

Strp

j

i

.

6. Inversion of the PC model

The equation XAZT may be inverted so that the original variables may be stated as a

function of the principal components because A is orthonormal and hence A-1=A. That is

AZX

| 11

2.4 Residual analysis The last two properties of section 2.3 lead to a principal result in PCA. In the last property x will be

determined exactly only if all the principal components are used. It is possible to get an estimate of x if

only k<p principal components are used, explaining the proportion of variance corresponding to the first

k λi’s given in property 5.

The model for the first k principal components is given by:

EAZX (2.3)

Where E is a nk matrix of residuals forming

the part of X not explained by the model forms the nk matrix E of residuals. Geometrically the

residuals is given by the distance between each point in k space and its point in the plane. As before X is

a nk matrix of variables, A is the kk matrix of the first k eigenvalues and Z is the nk matrix of

transformed variables the principal components.

2.5 Validation by cross validation The residuals in the residual matrix E in (2.3) are a measure of how much of the variance the principal

component model describes with a given number of principal components. It is not however an

indication of neither how well the model will perform on a new set of data, nor of the stability of the

model. In order to assess these questions validation of the model is required. The most correct way to

do this is to test the model on a new data set. That is a calibration set, Xcal to make the model and a

validation set, Xval to test it on. In this case the calibration and validation residuals are given by

n

XXe

calcal

cal

2)ˆ(

n

XXe

valval

val

2)ˆ( (2.4)

In most cases a validation set is not available. In data with few samples a sound validation option is full

cross validation also known as Leave One Out (LOO) validation with Jack-Knifing. This will be described in

this section.

In LOO validation as many sub models as there are samples are tested, each time leaving only one

sample out and using this as the validation set. With a data set with n samples, n sub models each

12 |

containing n-1 samples are tested. The squared difference between the predicted and the X value for

each omitted sample is summed and averaged in the same sense as in (2.4)

This validation residual or equivalently the validation explained variance by the model is a good measure

of how good the model will perform on other sets of data.

The concept of studying the variation the full model and the different sub models computed with Aopt

principal components during cross validation is a modification of the established Jack-knife technique

[Martens, 2001]. The deviations between the full model and the individual local submodels are called

partial perturbations. These are a good indication of the stability of the model and can be used in order

to find possible outliers.

As was mentioned above the primary goal of validation is

Estimating the predictive ability of the model

Assessing parameter stability

Furthermore validation can be used to

Optimize the model by determining the number of optimal principal components to use

Defining limits for outlier warnings

For details on other validation methods e.g. Test set validation suited for larger data sets, segmented

cross validation for intermediate sized data sets and leverage corrected validation see [Esbensen, 2006].

2.6 The principle of PCA: The algebraic approach In this section PCA will be (re)presented as an iterative method, where eigenvalues and vectors are

obtained sequentially starting with the largest eigenvalue and its associated eigenvector, then the

second largest eigenvalue and its associated vector and so on. This is done so subsequent vectors are

uncorrelated with existing ones.

This alternative representation may seem redundant but is done in keeping with understanding later

chapter on Canonical Variate Analysis.

Given the linear combination

PCA seeks a vector a1 that maximizes the variance of z. Since the variance of z1 is has no

maximum if a1 is unrestricted the function to maximize is given by

Subject to the constraint

xazT

11

11 SaaT

11

111

aa

SaaT

T

| 13

Subsequent vectors are found

Subject to the constraint

And the additional constraints

2.7 calculating the principal components The eigenvalues and eigenvectors can be calculated by means of eigenvalue decomposition. The

maximum value of the eigenvalue is given by the characteristic equation

(2.5)

The eigenvectors are then found by the expression

(2.6)

It is worth noticing that there is no inverse of S involved before obtaining eigenvectors for the principal

components. Therefore S can be singular, in which some of the eigenvalues are zero and can be ignored.

A singular S would arise for example when n<p. The tolerance for a singular S is a very important aspect

for the use of PCA.

111 aaT

i

T

i

i

T

ii

aa

Saa

1i

T

i aa

jij

T

i ,0Saa

0 IS

0aIS

| 15

Chapter 3

Multivariate Analysis of Variance (MANOVA)

In this section univariate anova is extended to MANOVA in which more than one variable is measured on

each experimental unit. It is not the purpose of this section to present the model of the MANOVA in its

full detail, but rather to outline the basic principles regarding hypothesis testing which may serve as a

framework for later sections.

16 |

3.1 One-way Models In this section univariate ANOVA is reviewed before covering the multivariate MANOVA.

3.1.1 Univariate one-way ANOVA In the balanced one-way ANOVA we have a random sample often referred to as group of n observations,

each of g normal populations with equal variances 2 .

g groups with n observations

Sample 1 from

),( 2

1 N

Sample g from

),( 2gN

1ky 1gy

kny gny

Total 1y gy

Mean 1y gy

Variance 2

1s 2

gs

For each group total, iy , and mean , iy , are calculated

n

j

iji yy1

n

j

ij

in

yy

1

With the overall total, y , and mean , y , calculated as

gn

ij

ijyy,

1,1

gn

ij

ij

gn

yy

,

1,1

The k samples are assumed independent. This along with the assumption of common variance is

necessary to obtain an F-test.

The model for each observation is

ijiijy

| 17

iji , gi ,...,1 and nj ,...,1

Where ii is the mean of the ith population.

Tests of significance

The statistical significance test of interest is the hypothesis of no group difference (in group means):

gH 10 : against the alternative jia jiH :,:

If the hypothesis is true all ijy are from the same population with the distribution ),( 2N and two

estimates of 2 can be obtained.

One based on sample variances si , gi ,...,1 pooled within-sample estimate of 2

11 ,

22

1

22

ng

nyys

gs

ji i iijg

i

iwithin (3.1)

and the other based on sample means, gyy ,,1

1

2

22

g

yynSs i i

ybetween

1

22

g

gnynyi i

(3.2)

When sampling from a normal distribution2

withins , a pooled estimate based on the g values of si, is

independent of 2

betweens which is based on the iy ’s.

Since 2

withins and 2

betweens are independent and both 2 , their ratio form an F-statistic

)1(

122

22

2

2

ngnyy

ggnyny

s

sF

ij i iij

i i

within

between

within

between

within

between

MS

MS

ngSS

gSS

)1(

1 (3.3)

Where i ibetween gnynySS 22 is the between sample sum of squares,

ij i iijwithin nyySS 22 the within sample sum of squares,

18 |

1 gSSMS betweenbetween and )1( ngSSMS withinwithin the corresponding sample mean

squares.

The F-statistic is distributed )1(,1 nggF when H0 is true. H0 is rejected if FF , where is the

significance level.

3.1.2 Multigroup one-way MANOVA Assume g independent random samples of size n, obtained from p-variate normal populations with

equal covariance matrices

Sample 1 from

),(2

1pN

Sample g from

),(2

g

N

11y

1gy

n1y gny

Total 1y gy

Mean 1y gy

The model for each observation vector is expressed

ijiij y

iji , ijevar

, gi ,...,1 and nj ,...,1

Tests of significance

Comparison of the mean vectors of the g samples for significant differences, hypothesis of no group

difference at all

gH

10 : against the alternative jia jiH :,:

As in the univariate case the between and within sample sum of squares withinSS and betweenSS are

given by (3.1), (3.2) and (3.3).

In the multivariate case the between sample and within sample sum of squares matrices are betweenSS

and withinSS defined as

| 19

i

T

iibetween yyn yySS

i j

T

iijiijwithin yyyySS

Assuming there are no linear dependencies in the p variables, the rank of the pp matrix betweenSS is

the smaller of p and degrees of freedom, dfbetween=g-1. That is, betweenSS can be singular.

The rank of the pp matrix withinSS is p unless dfwithin =g(n-1) is less than p.

The within group error covariance matrix is estimated by

withingng

SS

1ˆ

Wilks’ Test Statistics

The likelihood ratio test of g

H 10 : is given by

total

within

SS

SS

Wilks’ , can be expressed in terms of eigenvalues of betweenwithinSSSS1

as follows

s

i i1 1

1

,

Where betweendfps ,min and p is the number of variables (dimension).

An approximate F-test is given by

1

21

1

1

df

dfF

t

t

Where

betweendfpdf 1 ,

22

12 betweendfptdf ,

12

1 betweenbetweenwithin dfpdfdf and

20 |

5

422

22

between

between

dfp

dfpt

An approximate test is given by

ln1/

2

12

betweenwithin dfpdf

which has a 2 distribution with dfbetween degrees of freedom. 0H is rejected if 22

.

Roy’s test

In the union intersection we seek the linear combination ij

T

ijz ya that maximizes the spread of the

transformed means i

T

iz ya relative to the within sample spread of points. As SaaT

zs 2 , a is found

as the vector that maximizes

ggn

gF

within

T

between

T

aSSa

aSSa 1

This is maximized by 1a , the eigenvector corresponding to 1 , the largest eigenvalue of betweenwithinSSSS1

.

This gives

1

1

1max

g

ngF

a

Fa

max has no F distribution as it is maximized over all possible linear functions.

To test g

H 10 : based on i , Roy’s union intersection test also called Roy’s largest Root test

is used

1

1

1

(3.4)

The eigenvector 1a corresponding to 1 is used in the discriminated function yaTz 1 since this best

separates the transformed means

i

T

i yz 1a , gi ,,1

The coefficients of 1a can be examined for an indication of which variables contribute most to separating

the means

| 21

Pillai’s test

The Pillai’s statistic is given by

s

i i

ibetweenbetweenwithin

s trV1

1

1

SSSSSS

Pillai’s test statistic is an extension of Roy’s statistic given by (3.4). If the mean vectors do not lie in one

dimension, the information in the additional terms, ii 1 , si ,...,3,2 may be helpful in rejecting

0H .

An approximative F-statistic is given by

)(

)(

12

12s

s

Vssm

VsNF

Approximately distributed )12(),12( sNsmsF .

Lawley-Hotellings tests

The Lawley-Hotellings statistics is defined as

s

i

ibetweenwitin

s trU1

1)( SSSS ,

where pdfs between,min .

An approximate F-statistic is given by

12

122

sms

UsNF

s

Approximately distributed )1(2),12( sNsmsF .

3.2 Unbalanced one-way MANOVA

The balanced extended to the unbalanced case, in which there are in observation vectors in the ith

group.

The model becomes

ijiij y

iji , gi ,...,1 and nj ,...,1

22 |

The mean vectors

n

j i

ij

in1

yy

g

i

n

j

ij

N1

yy

, where

g

i

inN1

The betweenSS and the withinSS matrices are calculated as

i

T

iiibetween n yyyySS

(3.5)

i j

T

iijiijwithin yyyySS (3.6)

Wilk’s and other tests have same form as in specified above for the balanced one-way MANOVA using

betweenSS and the withinSS from (3.5) and (3.6).

In each test

1 kdfbetween and

k

i

iwithin kndf1

Summary of the former tests The measure of group differences with respect to df within group variability can all be expressed by the

eigenvalues s ...1 of the matrix

betweenwithinSSSS1

where withinSS is not singular.

The four test statistics can be summarized as follows

Pillai :

s

i i

isV1 1

Lawley-Hotelling :

s

i

i

sU1

)(

Wilk’s lambda :

s

i i1 1

1

| 23

Roy’s largest root : 1

1

1

Where 1,min,min gpdfps between .

Note that for all four tests pdfwithin assumed that the number of eigenvalues is equal to the minimum

of the dimensions of the variables space and the number of groups-1.

| 25

Chapter 4

Mixed Model ANOVA

The MANOVA models used will be based on univariate analogous. To describe the data mixed model

analysis of variance is deployed.

The general linear model ANOVA assumes independent and identically distributed errors with 0 mean

and 2 variance. The mixed model ANOVA extends the general linear model by allowing a more flexible

specification of the covariance matrix of . The mixed model ANOVA allows for both correlation and

heterogeneous variances, while still assuming normality.

Let the n x 1 vector y describe the n observations. Then the mixed model for can be written as

ZXy

The matrix notation for a mixed model uses two design matrices; One design matrix X to describe the

fixed effects in the model and one design matrix Z to describe the random effects in the model. If n

denotes the number of observations in the data, p the number of fixed effect parameters in the model,

and q is the number of random effects coefficients, then X has dimension n x p and Z has dimension n x

q. The p x 1 vector, , and q x 1 vector, , are the coefficients for the fixed and random design

matrices, respectively.

key assumption in the mixed model ANOVA is that and are normally distributed with

0

0

E

R0

0G

V

The variance of y is, therefore, V = ZGZ' + R.

26 |

Estimating G and R in mixed model Estimation in the mixed model cannot be done by least squares as in the generalized linear model as

there are three additional unknown parameters besides , namely , G and R.

In many situations, the best approach is to use likelihood based methods exploiting the assumption that

and are normally distributed [Hartley and Rao, 1967]. In the following the restricted (also known as

the residual) maximum likelihood method will be considered, because it accommodates data that are

missing at random [Little, 1995]. In the balanced cases, the random effect parameters are estimated

without bias, and for this reason the REML estimator is used in mixed models [Brockhoff, 2007].

The REML log likelihood function is:

2log2

'2

1'log

2

1log

2

1, 11 pn

REML

RVRXVXVRG (4.1)

where r = y - X(X'V-1X) - X'V-1y and p is the rank of X.

The estimates of G and R are denoted and , respectively in the following.

Estimating β and γ in the Mixed Model

To obtain estimates of and , the standard method is to solve the mixed model equations

[Henderson, 1984]:

yRZ

yRX

GZRXXRZ

ZRXXRX1

1

111

11

ˆ'

ˆ'

ˆ

ˆ

ˆˆ'ˆ'

ˆ'ˆ'

The solutions can also be written as

yVXXVX11 ˆ'ˆ'ˆ

ˆˆ'ˆˆ 1XyVZG

If G and R are known then ̂ is the best linear unbiased estimator (BLUE) of and ̂ is the best linear

unbiased predictor (BLUP) of gamma – here “best” means minimum mean squared error *Brockhoff,

2007].

Inferential tests The hypothesis for test of fixed effects are often expressed as the linear combination of the model

parameters

cH ':0 L against the alternative cH ':1 L (4.2)

| 27

Where L is a matrix, or a column vector with the same number of rows as there are elements in and c

is a constant.

If (4.2) is true and inserting the BLUE then

LXVXLL11'',0ˆ' Nc (4.3)

With this distribution the Wald test can be constructed as

)'ˆ'('')'ˆ'(111 ccW LLXVXLL (4.4)

Where W is approximately 2 distributed with 1df degrees of freedom. 1df equals the number of

parameters eliminated by 0H (rank of L). The F-test becomes

1df

WF (4.5)

Combining (4.5) with Satterthwaite’s approximation of the denominator degrees of freedom, 2df , the p-

value for 0H is given by [Brockhoff, 2007]

FFPP dfdfH 210 , .

The test for random effects can similarly be based on a Wald Z statistic, which is valid for large samples.

Another alternative is the likelihood ratio 2 . This statistic compares two covariance models, one a sub-

case of the other. With A as the full model, and B as the sub-model the test statistic is given by

A

REML

B

REMLBAG 22

Where A

REML and B

REML are the two negative restricted/ residual log likelihood values from equation

(4.1). Asymptotically BAG follows a )1(2 distribution, when B differs from A with one variance

parameter.

28 |

Chapter 5

Canonical Variate Analysis (CVA)

CVA is a widely used method for analyzing group structure in multivariate data. Krzanowski (2001)

summarizes that the objective of CVA is to, “provide a low-dimensional representation of the data that

highlights as accurately as possible the true differences existing between the m subsets of points in the

full configuration”. CVA finds a weighted sum of the variables whose between-groups variation is

maximized with respect to its within-groups variation.

| 29

5.1 Principle of CVA

With g groups CVA finds the 1p vector 1a maximizing the ratio

11

11

1

maxaSSa

aSSa

bwithin

T

between

T

subject to

1iwithin

T

i aSSa

Where the between-groups covariance matrix SSbetween and the within-groups covariance matrix SSwithin are given by

T

i

g

i

iibetween n ))((1

xxxxSS

,

g

i

n

j

T

iijiijwithin

i

1 1

))(( xxxxSS

where ix is a vector of sample means for the ith group, and x is the overall mean. The notation SSbetween

and SSwithin is chosen for compatibility with Chapter 3.

Successively find ia ,...,2i ),min( pgk , maximizing the ratio

iwithin

T

i

ibetween

T

i

i aSSa

aSSa

amax

Subject to

IASSA between

T

Where A is a kp matrix with columns ai.

5.2 Calculationg the canonical variates The matrix of canonical variates is obtained by finding the eigenvectors of

betweenwithinSSSS1

as it is done in for PCA in section 2.7. Note that the inverse of SSwithin is used in finding the eigenvalues.

Hence SSwithin cannot be singular.

5.3 CVA and PCA Another method for looking at the variance of the variables of a single data set is the PCA reviewed in

Chapter 2. It is intuitively perceivable that CVA resembles a PCA expressed on group level. Campbell ant

30 |

Atchley (1981) argue that one can view a CVA as a PCA performed on group means in the space

obtained by transforming the variables by the Mahalanobis transformation, that is

s, where Sxx=SSbetween+SSwithin. In this space Euclidian distance equals Mahalanobis distance, where

Mahalanobis distance between two group means i

and j

is defined as ji

T

ji , where

is a pp within-groups covariance matrix. Further in this space IS **xx. The principal components

of the group means in this transformed space correspond to the canonical variates.

| 31

Chapter 6

50-50 MANOVA

When nq number of responses exceed the number of observations the tests of (classical) MANOVA

collapse. When several of the responses are highly correlated the tests perform poorly.

50-50 MANOVA method, suggested by Langsrud (2001), is designed to handle these cases.

The concept of the 50-50 MANOVA is to handle the problem of collinearity by PCA of the response data.

The original response variables are then replaced by a few principal components on which an ordinary

MANOVA can be performed (can be understood as a reverse PCR on multivariate response).

Existing MANOVA tests are altered so the MANOVA on principal components have statistically correct

tests.

This is done in the Appendix B.

The singular value decomposition of Y is

s

i

T

iii

1

vuY

Where ),min( qns is the rank of Y. When the tests in (B2) is valid q<n and therefore s=q.

The p-value in (B2) for the test (B1) is invariant under the transformation

s

,,1Y

That is, the original responses Y nay be replaced by the principal components scores s

,,1 . When

q>n-m-p (n, m and p given as in Appendix B) the test (B1) cannot be performed, but a valid test can be

performed using the right number of principal components.

By letting

32 |

ppmn

pp

0

IX~

Tnm 1,...,~

zzY

s

i

T

iii

~

1

~vu

Where ),min(~ qmns is the rank of Y~

. The above property of principal components give

s

pvpv 1

,~~

,~

XYX

When there are too many responses for (B1), (B2) an alternative test may be based on the first k

principal components and the p/value is computed as

s

pvpv ~~,~~

,~

1XYX (6.1)

For simplicity assume that mnq and hence mns ~ . Let M denote the orthogonal matrix of the

n-m principal component scores

Tmn

~~1M

And the rows of XM~

denoted by mn ,...,1 . Then

Tmn XM~

and

kkmn

kk

k 0

IMYM 1

1

~

This transformation will not change the p-value given in (6.1). The p-value is hence

kkmn

kkT

mnpv0

I, (6.2)

Which is the reverse of that given in Appendix B (B2) regarding response and predictor variables. As the

function pv is symmetric regarding X and Y arguments. The two expressions are (reverse but) similar and

(6.2) can analogously to Appendix B (B2) be viewed as a test comparing the variability of k ,...,1 with

| 33

the variability of mnk ,...,1 . Similarly k ,...,1 are called the hypothesis observations and the

mnk ,...,1 the error observations.

The choice of k is handled by introducing a group of d buffer observations that will not be involved in the

expression SSerror.

The p-value will then be calculated as

kdkmn

kkT

mndkkpv0

I,1

Compared to (5.28) the number of responses has been reduced from q to k. The number of rows of the

matrices is reduced by d, which can be viewed as a reduction in the error degrees of freedom from n-m-

p to n-m-p-d. For the hypothesis the degrees of freedom, p, remain unchanged.

The rule for choosing k is given by

1. Choose k=1 if 90.0~

~

1

2

2

mn

i

i

i

otherwise choose the smallest k>1 so that 50.0

~

~

1

2

1

2

mn

i

i

k

i

i

2. Choose 23 kpmnd (truncated)

This is a simplified form of the 50-50 MANOVA. It can be modified to cases q<n-m where the rules

should be modified accordingly.

| 35

Part 2

Part 2 Analysis of Fish Data

| 37

Chapter 7

Fish Data

The sensory profiling data set to be analyzed represents sensory assessment of fish quality. The samples

consist of 38 different products evaluated by 10 different assessors twice (some products in four

replicates). There are 17 sensory variables whereof five are of the type odor, seven of flavor and five of

texture. The attributes are scored on a 15 centimeter ordinal scale. The scores are then measured and

reported with two decimals.

The replicates are not blocked in different sessions, however if replicate 1 is missing, replicate 2 is also

missing for all attributes by the same assessor and product. In total there are 128 samples that have not

been evaluated. This gives a total of 632 (38 x 10 x 2 – 128) samples.

Table 2(a) shows the structure of the design variables, product, assessor and replicate, set up as in Figure 1

There is one of these for each Product. That is 38 in all.

The response variables are described by Table 2(b). For each of the 17 attributes the number, type and

name are given.

38 |

Product Replicate Assessor

1 1

1 2

1 3

1 4

1 5

1 6

1 7

1 8

1 9

1 10

2 1

2 2

2 3

2 4

2 5

2 6

2 7

2 8

2 9

2 10

Table 2 (a) Design variable structure and (b) response variables

The variable product can furthermore be partitioned into two factors: feed and time. Here feed signifies

what type of fish feed the fish were given and time signifies how many days the fish were stored on ice

prior to consummation. There are of 7 different fish feeds and 5 different storage times. The make up of

every product according to feed and time is given in Table 3.

Number Name Feed Time Number Name Feed Time

1 Blå_0 Blå 0 20 gul_12b Gul 12

2 Blå_12a Blå 12 21 gul_3a Gul 3

3 Blå_12b Blå 12 22 gul_3b Gul 3

4 blå_3a Blå 3 23 gul_5 Gul 5

5 blå_3b Blå 3 24 gul_7 Gul 7

6 blå_5 Blå 5 25 hvid_0a Hvid 0

7 blå_7 Blå 7 26 hvid_0b Hvid 0

8 grøn_0 Grøn 0 27 hvid_12 Hvid 12



11 grøn_5 Grøn 5 30 jis_3 Jis 3

12 grøn_7 Grøn 7 31 rød_0 Rød 0

13 grå_0a Grå 0 32 rød_12 Rød 12

14 grå_0b Grå 0 33 rød_3 Rød 3

15 grå_12 Grå 12 34 rød_5 Rød 5

16 grå_5 Grå 5 35 rød_7 Rød 7

17 grå_7 Grå 7 36 grå_3 Grå 3

18 gul_0 gul 0 37 hvid_3 Hvid 3

19 gul_12a Gul 12 38 Sort Sort

Table 3 Products listed by number, feed and time

Number Type Name

1 Odor Earthy

2 Cooked potato

3 Sourish

4 Sour

5 Muddy

6 Flavor Earthy

7 Mushroom

8 Cooked potato

9 Sourish

10 Sweet

11 Green

12 Muddy

13 Texture Flaky

14 Firm

15 Juicy

16 Fibrousness

17 Oiliness

| 39

Design of data From inspection of Table 3 it is seen that for the feeds Blå, Grøn,Ggrå, Gul, Hvid and Rød there is a

product for each of the five different storage times. For the combinations Blå 12 and 3, Grå 0, Gul 12 and

3 and hvid 12 there are two products, the difference indicated by index a or b. For the Feeds Jis and Sort

only one product is available. Jis with time 3 and for Sort no storage time is indicated.

In Table 4 it can be seen how the 38 products are distributed in a more schematic form. Each product is

indicated by a X. There are only 37 Xs as Sort has no storage time specified.

Storage time 0 3 5 7 12

Feed

Blå X X X X X X X

Grøn X X X X X

Grå X X X X X X

Gul X X X X X X X

Hvid X X X X X X

Rød X X X X X

Jis X

Sort Table 4 The products indicated by their feed and time

For six feeds it is possible to view data in a full factorial design.

Only Jis and Sort do not have the property of being tested for all times. These are not intended to be

analyzed on the same terms as the other feeds and are furthermore expected to be taken out of the

analysis if proven to be outliers.

The goal of the analysis is to decide whether there is difference between the different types of feed. In

the case of design variables being Product, Assessor and Replicate this translates to difference in

Product.

Missing values Imputation As mentioned before there is a total of 128x17 missing values. Where possible analysis is carried out on

the raw data, but in cases where a full balanced data set is required this will be done on an imputed data

set.

The imputations are done by replacing the missing value with the product mean across all samples for

the attribute in question. This does not change the mean of the group, but it does reduce the within

group heterogeneity on the sample. In analysis of variances this way of imputation increases the

likelihood of Type 1 error [Tinsley, 2000].

Because of the nature of the missing values is that all 17 attribute values for a sample either present or

missing, the question of missing values in this data set simply translates to missing samples. By taking

the 128 samples in question out, we are left with a full data set, unbalanced but without missing values.

| 41

Chapter 8

Initial Explorative Analysis

Before the methods described in Part 1 is done initial analysis of the data is carried out.

The normality of the variables can be assessed by viewing histograms of the scores given. In Figure 3 the

histogram for the attribute earthy(o) is given along with the normal probability plot.

Figure 3 Histogram and normal probability plot for attribute earthy(o)

As is seen no unacceptable indication of non-normality is visible. The remaining 16 variables are

inspected by similar plots and all proved acceptable.

The attributes means, standard deviations, minimum values and maximum values are given in the

following table

Variable Mean Standard deviation Minimum value Maximum value

Earthy(o) 3.9059335 1.2750875 1.3500000 9.7500000

42 |

Cooked potato(o) 3.9059335 0.9737972 1.3500000 6.9000000

Sourish(o) 2.7469937 0.8037691 0.6000000 7.2000000

Sour(o) 1.6084652 0.8349841 0 5.2500000

Muddy(o) 1.5868671 0.9206049 0 6.0000000

Earthy(f) 5.1517405 1.3708611 1.0500000 8.7000000

Mushroom(f) 4.2498418 1.3042597 0.7500000 7.9500000

Cooked potato(f) 4.3383703 1.0549837 1.8000000 7.6500000

Sourish(f) 3.2862342 0.9657570 0 7.8000000

Sweet(f) 3.5043513 0.8869100 0 7.8000000

Green(f) 2.5424051 0.9203790 0 7.0500000

Muddy(f) 2.2179589 1.1079197 0 6.7500000

Flaky(t) 7.4420886 2.7592146 0 12.7500000

Firm(t) 6.2271361 2.4412599 0 12.3000000

Juicy(t) 7.1691456 2.4068933 0 12.9000000

Fibrousness(t) 3.9168513 1.8429409 0 10.3500000

Oiliness(t) 4.3115506 1.4276590 0 9.3000000 Table 5 Mean, standard deviation, minimum value and maximum value for the 17 response variables

The data in Table 5 can be summarized in mean and standard deviation plot for each attribute

Figure 4 Plot of mean and standard deviation for the 17 attributes

The minimum and maximum value along with the 25% percentile, median and 75% percentile is

described by the box plot in Figure 5

Figure 5 Box-plot for the 17 attributes across all samples

| 43

In both of the plots above the difference between the three types of attributes is visible. The five first

(counting from left) attributes are odor attributes, the following seven are flavor and the last five are

texture attributes. The odor attributes and the flavor attributes seem to be comparable both in level and

variation. The texture attributes however differ from the other two by larger means especially for

attributes Flaky, Firm and Juicy. Furthermore the variance for all texture attributes is seen to be

considerably larger than for the other attributes.

From the box plot it is seen that the texture attributes are scored on a large percent of the scale (from 0-

13 out of 15), whereas the range covered by the other two attribute types is limited within 2

centimeters.

The correlations between the dependent variables are given in Fejl! Henvisningskilde ikke fundet..

Three pairs of variables have a correlation above 0.6. These are Cooked Potato(o)-earthy(o), earthy(f)-

earthy(o) and muddy(f)- mushroom(f). Three pairs with a correlation above 0.5. Four pairs with

correlation above 0.4. The rest all have a value less than 0.4. very few variables in the data set are highly

correlated. Contrary to what could be expected the data does not show any signs of multicollinearity

amongst the 17 response variables.

[Everitt, 2002] suggests the variance inflation factors

21

1

i

iR

VIF

as a measure of multicollinearity. Here Ri is the multiple correlation coefficient from regression of the ith

response variable on the rest of the response variables. A VIF value above 10 is suggested as an

indication of multicollinearity. These are calculated for every attribute and no multicollinearities are

found.

Panel assessment As part of the initial explorative analysis the panel is assessed. By viewing the one-way ANOVAs on the

factor Product for each assessor, the criteria described in Chapter 1 can be assessed by the following:

1. repeatability: MSE

Low MSE: good ability to deliver same scores on replicates

3. discrimination: F=MSProduct/MSE

High F-value: good ability to discriminate between samples

Furthermore the validity can be assessed by viewing the correlation of the assessor and the entire panel.

That is

2. validity: correlation with panel mean

44 |

A slight relaxation is considered to be acceptable. Instead of doing the 10x17 ANOVAs and reporting the

results, PanelCheck1 is used on the imputed data set. As mentioned earlier for analysis of variances this

way of imputation increases the likelihood of Type 1 error. But with the knowledge of the nature of the

missing data for this particular data set the results are expected to be a

Optimistic measure of repeatability

As two missing replicates will have the same value in the imputed data set

Optimistic measure of validity

Missing values will all be replaced by the product mean that is likely to correlate with the panel

mean

Pessimistic measure of discrimination

Missing values will all be replaced by the SAME value: the product mean

The correlation is likewise done in PanelCheck and on the imputed data.

In the following plot the 10x17 MSE values are plotted grouped by assessor to compare assessors in

their ability to reproduce results across replications

Figure 6 MSE values from one-way ANOVAs, grouped by assessor

Assessors 3, 4, 5 and 10 all have MSE-values less than 2 on all attributes. Assessors 1, 2, 7 and 9 all have

MSE values less than 5 on all attributes. Finally assessors 6 and 8 have one or more attributes with MSE

value larger than 10. On a whole the panel shows acceptable ability to repeat replicates, with the

possible exception of assessors 2 and 6 and 8 in particular. These show poor ability to repeat scores on

1 software package designed to analyze panel performance by Matforsk

| 45

what seems to be mostly texture attributes. This seems to be the common trend across all assessors.

The ability to reproduce scores is relatively better for flavor attributes and odor attributes than for

texture attributes.

This is illustrated better in following plot that plots the same MSE values but grouped by attribute.

Figure 7 MSE values from one-way ANOVAs, grouped by attribute

As expected from Figure 6 it is the texture attributes that the panel have most difficulties in reproducing.

Especially the attributes Flaky(t), Firm(t) and Juicy(t) differ widely from replicate to replicate. A natural

explanation of this may lie in preparation of the replicates. A well cooked fish may very well be more

firm and less juicy than a rawer one.

From viewing the MSE values it is concluded that assessors 2, 6 and 8 show poor abilities in reproducing

results on the same products. The texture attributes are the ones that the panel has the most difficulty

in “recognizing” the products on.

In order to assess the panels ability in discriminating amongst products the F-values from the one-way

ANOVAs are summarized in a plot. As before the F-values are grouped by assessor (Figure 8) and by

attribute (Figure 9).

46 |

Figure 8 F-values from one-way ANOVAs grouped by assessor

In Figure 8 the F values from the one-way ANOVAs for assessors are plotted and grouped by assessor to

compare assessors in their ability to discriminate samples.

Assessor 4, 5 and 10 has the largest F values. Furthermore the F values for most attributes are over a

significance level of 0.05. These assessors have the best ability to discriminate between products in the

panel. Assessors 1, 2, 3, 7 and 9 have acceptable F values with most attribute F values over a significance

level of 0.05. Assessor 6 and 8 have a small number of attributes with F values over the 0.05 significance

level and are not very able at discriminating between products.

| 47

Figure 9 The F-values from one-way ANOVAs grouped by attribute

From Figure 9 it is seen that the assessors are more successful at discriminating products based on

texture attributes and then on flavor attributes than on the odor attributes. This should be seen in the

light of the results from Figure 7. Good results in repeatability and poor results in discrimination seem to

be positively correlated. The opposite similarly seems to be the case.

From the F-values it is concluded that assessor 6 and assessor 8 do not only have poor ability to replicate

their scores on the same products, they also have poor ability in discriminating between different

products. The rest of the panel does not seem to have the same difficulties with assessors 4, 5 and 10

being the best performers on both criteria.

In order to assess the panel homogeneity correlation plots showing the assessors scores along with the

panel mean are viewed.

48 |

Figure 10 Correlation plot for assessor 1 on the attribute earthy(o)

In the correlation plot above the correlation of the scores from assessor 1 and the rest of the panel is

assessed on the attribute earthy(o). The line corresponds to the panel mean. The white dots the other

assessors’ scores, the red dots the assessor in questions scores for given attribute. A good assessor is in

agreement with the panel if the red dots fall on the panel mean line. Left of the line signifies that the

assessor over scores, right of the line indicates that the assessor underscores.

From Figure 10 it is seen that assessor 1 scores in accordance with the panel on the attribute earthy(o).

For one assessor there are as many of these plots as there are attributes. This gives a total of 10x17

plots. They will not be shown here but the results from viewing them will simply be stated.

Assessor 1: In agreement with panel on most attributes with a slight tendency to over score on T

attributes

Assessor 2: In agreement with panel on most attributes with large spread on T (flaky, firm, juicy) but so

are rest of panel

Assessor 3 and 4: In agreement with panel

Assessor 5: In agreement with panel on most attributes but with a big tendency to underscore on T

along with big spread

Asessor 6: Mostly in agreement, large spread on T (first three)

Assessor 7: in agreement with panel, larger spread on T (first four) over score on oiliness (t)

Assessor 8: In agreement with panel, large spread on T

Assessor 9: Has tendency to either under score or over score

| 49

Assessor 10: In agreement with panel, good spread a slight tendency to over score O and F intensity

From this it can be concluded that the panel’s homogeneity is acceptable. The attributes that cause the

most disagreement are the texture attributes which are also the attributes the assessors are best at

discriminating samples on.

| 51

Chapter 9

Multivariate Analysis by PCA

To compensate for differences in interpretation of scale between individual assessors data is averaged

over assessors and replicates before PCA is employed. The score-plot and the loading-plot are given

below.

Figure 11 Score-plot of initial dataset averaged by Product (left), correlation loading plot (right)

Outliers The final model will be validated using Martens full cross validation with jack knifing. This is closely

related to choosing the number of principal components to use in the model. The optimal number of

principal components in this model is 15.

Another use of the uncertainty test is to find outliers. The score plot with the Hotelling T2 95%

confidence ellipse is shown in Figure 11 (left) and in Figure 12 the influence plot for the 15th principal

component is given.

The influence plot shows the residual variance and the leverage of the 38 samples for the model with

the optimal number of principal components (15). The leverage is defined by the distance between a

projected point (or projected variable) and the model centre.

52 |

Figure 12 Influence plot for PCA with Sort

As can be seen the sample Sort lies outside the the Hotelling T2 ellipse. Furthermore Sort is seen to have

leverage close to 1 in the influence plot. On the basis of this Sort is taken out of the model. No other

samples deviate extremely from the others.

Scaling or no scaling The loading plot in Figure 11 (right) shows, that the most influential variables are all texture variables.

From the mean and standard deviation plot given in Figure 4 it was seen that the variables (attributes)

differ widely in both mean and particularly in variance. A possible division of the data set is to partition it

into three sets: one for all of the odor attributes, one for all of the flavor attributes and one for of all the

texture attributes. Another possibility is to scale the complete data set. If the data is standardized by the

inverse of the standard deviation

ijscaled

ij

yy

the different variances are leveled out. The PCA is then done on the correlation matrix R instead of the

correlation matrix S

A scaling of the variables may prove advantageous as it will level out the effect of the different variables.

On the other hand the texture variables may be more pronounced in differentiating between samples

and this information may be lost in an analysis of a scaled dataset.

A scaled and an un-scaled analysis are done. The explained variance as function of the number of

principal components in the model for the two models is given below.

| 53

Figure 13 Explained variance for un-scaled model (left) and scaled model (right)

The blue bars are the calibrated variances, the red bars show the validated variances when Martens full

cross validation with jack knifing is applied. Viewing the validation variances it can be seen that the un-

scaled model performs better than the scaled one. Furthermore the suggested number of principal

components is 14 and 15 respectively, for the un-scaled and the scaled model.

Figure 14 correlation loading plots for the un-scaled model (left) and scaled model (right)

Viewing the loading plots it is furthermore seen that the texture variable dominance is not as large as

the one witnessed in Figure 11 above. This suggests that the sample Sort had a misleading effect and was

correctly taken out of the analysis.

On the basis of the comparisons above the un-scaled model is chosen. No further outliers are detected,

This is seen by viewing the Hotelling T2 95% confidence ellipse in the score plot (not given here) and

furthermore underlined by the influence plot for the 14th principal component along with the stability

plots for the scores and the loadings respectively.

the final model

Figure 15 Influence plot for the model without Sort at optimal number of principal components

From the influence plot in Figure 15 no samples are seen to deviate too far from the others. The residual

are at the most found to be 0.004 and the highest leverage is less than 0.7.

The stability plots for the scores and the loadings will give a possibility to assess the uncertainty of the

model.

54 |

Figure 16 stability plot for samples (left) and variables (right)

The two plots are apparently not so telling. But in fact they are. This is a good thing as the partial

perturbations are all satisfactory.

results

Figure 17 score plots for un-scaled model indicated by feed (left) and storage time (right)

58% of the total variation in the data is described by the two first principal components. There seem to

be three clusters. One to the left along the first principal component. One on the top right and one in

the bottom right. No distinct patterns in feed or storage time are found dividing the three groups.

From the score plot in Figure 17(right) where the samples are indicated by different storage times,

time=12 in brown, time=7 in light blue, time=5 in green, time=3 in red and time=0 in blue, a clear trend

is seen along the first principal component. Five of the samples with time 12 are grouped together to

the left, the intermediate time samples are in the middle and the samples with no storage time are

grouped to the right. When viewing the second principal component another time trend is seen. With

samples with time 3 at the bottom and moving upwards till the last three samples with time 12 in the

top. No samples with time 0 seem to be strongly described by the second principal component.

The division of the intermediate samples is not clear cut. Another time trend seems apparent along the

second principal component ranging from samples with time 3 at the bottom and samples with time 12

(other than the ones along the first principal component) at the top.

By viewing the same score-plot in Figure 17 (left) with the samples indicated by type of feed a clear cut

pattern is not observed. A little misleading the color indication is as follows: Blå in Blue, Grå in green,

Grøn in red, Rød in grey, Jis in brown, Gul in light blue and Hvid in pink.

| 55

The samples with feed Gul seem to group in the negative region of both the first and the second

principal components. The most part of the other samples are in first and fourth quadrant (positive first

principal component).

The corresponding loadings are seen in the loading plot.

The correlation loading plot takes the amount of variance explained into account. The outer ellipse is

the unit-circle and indicates 100% explained variance. The inner ellipse indicates 50% of explained

variance. This plot explains the structure of the data in terms of variables.

First of all it is seen that the two attributes Fibrousness and Firm (both texture attributes) are positively

correlated. They lie between the two ellipses and explain most of the variation along the second

principal component. The two attributes Sour(o) and Muddy(f) are the only ones to the left of the plot

along the first principal component. They are negatively correlated in particular the attribute Earthy(f).

This attribute is somewhat positively correlated to all other attributes not mentioned with the exception

of Muddy(o).

There is no sign of strict correlation of variables given by their type (odor, flavor and texture) or types

that are clearly more dominant than the others. This is seen as an indication of no need to split op the

data as suggested before.

Relating this to the score plot it is noticed that the two groups of samples with time=12 are explained by

large values for Sour(o) and Muddy(f) or large values for Fibrousness(t) and Firm(t) respectively. These

attributes seem to be dominant for fish that has been stored for more days. Fresher fish is described

mainly by the other qualities.

Samples with time 0 is described by large values of Earthy(f) and most samples are described by the

attributes in quadrant 1 or 4 respectively.

Inspection of the other principal components doesn’t give any results on difference in samples by feed

either.

The analysis of the un-scaled full data set (without Sort) explained the samples with regard to the time

effect more than anything. The feed effect was either found to be non existing or “drowned” by the time

effect. The three clusters form a circle almost not just a horse shoe as could be expected for two factor

data as this. This leads to the conclusion that the data set is not optimal in describing the Feed effect.

The analysis is not completely useless though. Valuable information can be read from the plots.

samples with feed Gul and Jis seem to be described by large values of the attribute Sour(o).

samples with large storage times are described by large values in attributes of Sour(o) and

Muddy(f) or in Fibrousness and Firm textures.

56 |

Samples with small storage times are described by the rest of the attributes except for

Muddy(o)

Time is a large effect in the analysis

Other PCA models on subsets of data The PCA model mainly explained the difference in variables according to the storage time. In order to

exploit the difference in samples due to the different types of feed other subsets of the data are

exploitet. The subsets are chosen to minimize the effect coming from the storage time and hence to find

“weaker” patterns within the data set.

Only those samples with Time=12 to see what variables explain these, then all samples without

the ones with Time=12 with and without the variables found in to be highly influential for the

samples with Time=12.

Only the ones with same time

Only samples with same Feed

Only on variables that was not highly influential in first run as they seem to explain the Time

effect more than the Feed effect

An assessor at a time

All of the possibilities mentioned above have the disadvantage of not exploiting the data to its fullest

extent. If the data contains misleading data that only contributes to the noise the right possibility will

however be the right choice.

Neither of the suggestions above gave positive results in regard to explaining the effect of different

Feed.

PCA on subset of data not containing Time=0 and Time=12 From the score plot by time another PCA is found interesting. The one exploring the second principal

component without the effect from the samples with time 0 and 12.

The score- and correlation loading plot for this PCA is given below. The explained variance by validating

is found to be lot smaller than for the model on the full dataset. This is probably caused by the fact that

the current model has fewer samples.

| 57

Figure 18 plots for the model without samples with time 0 and 12. Correlation loading plot (top left), explained variance (top right), Score plots indicated by feed (bottom left) and time (bottom right)

In these score plots the effect from feed is clear.

The time effect is still evident dividing the samples in those with time 7 and those with time less than 7.

The samples with time 7 are explained by large values of the attributes Firm and Fibrousness texture. Jis

is described by Sour(o) and Green explained by Muddy(o).

Viewing the second and the third principal components a clearer pattern emerges. In these plots

samples with feed Blå and Grøn are similar and explained by Juicy(t), Sourish(f), Cooked potato(f),

Sweet(f) and Oiliness(t). Hvid and Grå are also similar and described by Muddy(f), Sourish(o), Muddy(o)

and Cooked potato(o).

Figure 19 score plot (left) and loading plot (right) for second and third principal component

As the validation of this model is not acceptable the conclusions made in this model should not be

expected to hold for a similar data. In order to investigate this it is recommended that a similar sensory

profiling is done with storage time in the range of a couple of days and not nearly two weeks. Same

attributes and samples but more replicates or for example four consecutive days. From a data set in this

form valid conclusions on difference in feed can be expected to be found however.

Reinspecting the original score plot in Figure 17 this pattern might have been spotted. But it seems to be

across both the first and second principal component and very sought at the least.

Unless it is the time effect (or the feed time interaction) that is of interest it is not recommended that

the samples are made on largely different storage times, as the time effect is quite dominant and seem

to erase all other variance there may be.

| 59

Chapter 10

Univariate Analysis by Mixed Model ANOVA and ANCOVA

In this chapter the univariate mixed models that describe the data are presented and analyzed. In

section 10.1 the data is modeled by the three factors Product, Assessor and Replicate in a 3-way ANOVA.

In section 10.2 the property of the factor Product to be described by the factors Feed and Time is

exploited resulting in 4-way mixed models. In section 10.3.2 however the factor Time is modeled as a

covariate.

60 |

10.1 3-way univariate model In this section the products are considered ignoring the information on fish feed and storage time. This

leaves three factors

Product: 38 levels

Assessor: 10 levels

Replicate: 2 levels

As replicate is taken at random of each product, it is modeled as nested within product.

The prime goal of the analysis is to test for product differences, so the main factor product is considered

fixed. As the individual levels of assessors are not of interest, but only the variance, the main effect

assessors is considered random. As mentioned before replicates are random and hence all interactions

likewise assumed random.

Let m

ijkY denote the score on the k’th replicate of product given by assessor. The data previously

described for a single attribute, m, can then be described by a univariate mixed model ANOVA given by

` ijkikijjiijk PRPAApY )( (10.1)

where is the overall mean for the attribute in question. The product main effects, ip , i=1,…,38

represent the differences in level between the average score for the different products. The assessor

main effects,jA , j=1,…,10 represent the assessor variation. The product-assessor interaction

ijPA

expresses the variance of assessors in measuring differences in products, and ikPR )( , k=1,2 expresses

the replicate variation. The error term ijk represents the residual variation.

The variation of the random effects can be written

2,0 Aj NA

2,0 PAij NPA

2,0 RPik NPR

2,0 Ak N

The model can be described in a factor structure diagram given below:

[A×p] [A]

[I] [O]

[R] p

Figure 20 Factor structure diagram for the 3-way mixed model ANOVA

| 61

In Figure 20 the random factors are indicated in brackets. This diagram is helpful in determining the tests

of the fixed effects.

Test of fixed effects Carrying out the mixed model analysis corresponding to the model given by (10.1) gives the following ANOVA table for the fixed effect for the attribute earthy(O):

Source Numerator df Denominator df F value P-value Product 37 271 2.06 0.0006

Table 6 ANOVA table for fixed effects for Earthy(O), 3-way mixed model

From the ANOVA Table 6 ANOVA table for fixed effects for Earthy(O), 3-way mixed model it is seen that the product effect is statistically significant.

Test of random effects parameters Covariance parameter Estimate Lower 95% CI Upper 95% CI

Assessor 0.3058 0.001535 0.6102

Product*Assessor 0.2684 0.1155 0.4213

Replicate(Product) -0.0235 -0.06853 0.02154

Residual 0.9933 0.8477 1.1803

-2 Restricted/Residual log Likelihood 1923.4 Table 7 Covariance parameter estimates for Earthy(O), 3-way mixed model

From Table 7 Covariance parameter estimates for Earthy(O), 3-way mixed model it is seen that the estimate of the replicate effect it is very close to 0. The covariance structure is therefore reduced. In Figure 20 this can be envisioned directly by removing [R] from the model. The results of the reduced model gives the following

Covariance parameter Estimate Lower 95% CI Upper 95% CI

Assessor 0.3060 0.001551 0.6104

Product*Assessor 0.2798 0.1302 0.4295

Residual 0.9702 0.8350 1.1412

-2 Restricted/Residual log Likelihood 1924.2 Table 8 Covariance parameter estimates for Earthy(O), reduced model

A restricted likelihood ratio test is used to compare the two variance structures (Chapter 4), with p=0.3711. This indicates that the replicate effect is non-significant and can be left out of the model.

Post hoc analysis The least square means estimates for product are in Fejl! Henvisningskilde ikke fundet. together with

the Tukey-Kramer adjusted pair wise comparisons. No statistically significant differences between two

different products were found for Earthy (O). This is probably due to the small significance level for the

individual tests in order to protect the family wise type I error, as a consequence of the large number of

pair wise tests performed.

62 |

Validation of model The validation of the model follow the outline in section 10.3.

Residuals

Normality of the standardized residuals is checked by viewing histograms with the fitted normal curve

along with normal probability plots (QQ-plots) of the residuals. This is done in Figure 21

Figure 21 Histogram (left) and QQ-plot (right) for standardized residuals for reduced model

In Figure 21 it is seen that the residuals seem to be symmetrical distributed. The normal distribution

seems to fit quite well, while the tests for normality (Table 9) rejects the hypothesis for normality. These

tests however, are sensitive for small deviations in larger datasets [Brockhoff 2002 ], so the significance

is not necessarily relevant. Furthermore, the straight line in QQ-plot indicates a good fit to the normal

distribution with the exception of eight extreme points.

Test Statistic P-value

Shapiro-Wilk 0.989098 0.0001

Kolmogorow-Smirnov 0.047819 < 0.0100

Cramer-von Mises 0.291452 < 0.0050

Anderson-Darling 1.56188 < 0.0050 Table 9 Tests for normality for reduced model

Variance homogeneity

The variance homogeneity can be investigated by plotting the residuals vs. the predicted values and vs.

the levels of quantitative effects.

| 63

Figure 22 Predicted values vs. standardized residuals for 3-way mixed model

In Figure 22 of the residuals vs the predicted values it is checked for whether the variance depends on the

means. A satisfactory plot should show

1. No patterns

2. Evenly distributed around 0

3. 95% of the standardized residuals between ±2 and 99.9% inside ±3

4. Variance not changing

The standardized residuals as a function of the predicted values is trumpet shaped for predicted values

smaller than 3.5. This is probably due to the large amount of zero scores in the data set and should not

be taken as indication of unsatisfactory residuals. Beyond this point it levels out nicely. The residuals are

distributed evenly and symmetrically around zero. Most of the residuals are between ±2, and 6 residuals

are outside ±3 (>0.1%). This indicates that the assumption of variance homogeneity is not fulfilled.

Figure 23 Standardized residuals (3-way mixed model) vs product (left) and assessor (right).

Figure 23 of the residuals vs. factor levels is checked for group dependant variance homogeneity. Across

products there does not seem to be clear differences in variability. For the assessors, the variation of

assessor 6 and 8 seems to be larger than the rest of the panel. In order examine the influence of the

residuals outside ±3 the model is re-run without these outliers. An alternative approach to determine

the possible outliers is to apply the Mahalanobis distance [Tinsley, 2000].

64 |

Outliers

Outliers are dealt with in the following way:

1. Detect

2. Check external reason

3. Redo analysis without extreme observations and compare

Figure 24 Histogram and QQ-plot for the standardized residuals for 3-way mixed model without outliers.

The histogram and the QQ-plot are similar to the once in Figure 24, except the tails in the QQ-plot are

closer to the line. Furthermore the tests for normality are no longer rejected. For the residual plot

against the predicted values the trumpet shape is less obvious and the standardized residuals are all

within the limits ±2, which indicates that the assumptions for the model are fulfilled.

The outliers that were excluded from the analysis were all from assessor 6 and 8, which corresponds to

conclusion in Chapter 8 panel assessment. When comparing the estimated product levels the

differences are very small, except for the products grøn_7, grå_0b and grå_7 which were 9, 4 and 3%

smaller respectively without the outliers.

Normality of random effects

The normality of the random effects is checked indirectly by evaluating the QQ-plots of averages of the

assessor and the product by assessor interaction. Both effects roughly follows a straight line in the

probability plot which indicates that the assumption is fulfilled.

| 65

Figure 25 QQ-plots for random effects assessor (left) and product by assessor interaction (right).

The analysis, post hoc and validation done for the attribute Earthy(o) is repeated on the 16 remaining

attributes and the results are summarized in the following table.

Factor

Mixed model ANOVA for

Earthy

Ckd

po

tato

Sou

rish

Sou

r

Mu

dd

y

Earthy

Mu

shro

om

Ckd

po

tato

Sou

rish

Sweet

Gree

n

Mu

dd

y

Flaky

Firm

Juicy

Fibro

usn

ess

Oilin

ess

Product + + + + + + + + + + + + + + +

Assessor + + + + + + + + + + + + + + + + +

Product*Ass + + + + + + + + + + + + + + + + +

Replicate(product) + + + + Table 10 Results from the 17 mixed model ANOVAs

In Table 10 a plus sign + indicates that the given factor is statistically significant for the mixed model

ANOVA modeled for the given attribute. The models can be read by the columns for the different

attributes. It is seen that the factor Assessor and the Product x Assessor interaction are statistically

significant for all models. The factor Product is seen to be statistically significant for all attributes except

two: Muddy(o) and Sweet(f). The factor Replicate is tested non significant for all attributes except four:

Muddy(o), Cooked Potato(f), Sweet(f) and Muddy(f).

These results will be used in the analogous multivariate analysis (see section 11.1) but first the 4-way

univariate models will be discussed.

66 |

10.2 4-way univariate mixed model ANOVA In this section products are considered as represented as a combination of feed and storage time. It

should be mentioned that the product Sort does not have a storage time specified and will not be part

of the analysis in this modeling. In the raw data there are a total of 14 measurements on the product

sort. The degrees of freedom will therefore be reduced by 14 in comparison with the 3-way models.

All other products are explicitly explained by both a feed and a storage time.

The factors considered are

Feed: 7 levels

Time: 5 levels

Assessor: 10 levels

Replicate: 2 levels

As before a replicate is considered random and is modeled nested within feed and time. Both feed and

time are considered fixed along with their interaction effect. All other effects are considered random.

Figure 26 Scatter plot of Earthy (O) as a function of Time with regression line.

Figure 26 shows the scatterplot for the attribut earthy (O) against the Time. It is seen that the variation for

times 0, 3, 5 and 7 looks similar in size, and that the variation at Time=12 is smaller. The regression line

is slighly decreasing.

| 67

Figure 27 Earthy (O) vs. Time. Individual patterns (left) and regression lines (right) for the different feeds.

The individual patterns (Figure 27, left) show a tendency of a curvature. Hvid and Grå has an upwards

curvature and Blå, Rød, Grøn and Gul shows a downwards curvature, if any. There seems to be a

grouping of products by feed: Hvid and Grå, Blå and Rød, Grøn and Gul.

From the regression lines by Feed (Figure 27, right) the grouping mentioned above is evident.

Furthermore there seems to be a decrease with Time. The slopes of the different lines look similar

except Grøn and Jis. It should be noted that Jis is only evaluated at a single time point, Time = 3. This

holds throughout the analysis.

Despite the fact that the three graphs all indicate a decrease with Time, the curvatures and the variation

in the scores it is questionable whether there is a linear relationship or not. Therefore the analysis

carried out both as an ANOVA and as an ANCOVA.

68 |

10.2.1 4-way ANOVA When using the same notation as before the model for the 4-way mixed model ANOVA becomes

ijkliklijljlijjilliijkl FTRAFTATFAAfttfY )( (10.2)

Where if is the fixed Feed effect, i= Hvid, Grå, Blå, Rød, Grøn, Jis and Gul and lt is the fixed Time effect

in days, l=0, 3, 5, 7 and 12. The interaction of the two effects ilft is also fixed. All other main effects and

interaction terms are random.

The factor structure diagram for this model is given by

[A×t] [A]

[I] [A×f×t] [A×f] t [O]

f×t f

[R]

Figure 28 Factor structure diagram for the 4-way mixed model ANOVA

The analysis is done following the same principles as section 10.1 the results are presented in the

following on the analysis of the results for the attributes. As the results from this analysis is to be used

to define the model for the MANOVA, the parameter estimates for the individual attributes are not

shown, but only which effects are significant.

Factor

4-way Mixed model ANOVA for

Earthy

Ckd

po

tato

Sou

rish

Sou

r

Mu

dd

y

Earthy

Mu

shro

om

Ckd

po

tato

Sou

rish

Sweet

Gree

n

Mu

dd

y

Flaky

Firm

Juicy

Fibro

usn

ess

Oilin

ess

Feed + + + + + + +

Time + + + + + + + + + + + +

feed*time +

Assessor + + + + + + + + + + + + + + + +

Feed*Assessor + + +

Time*Assessor + + + + +

Feed*Time*Assessor + + + + +

Replicate Table 11 4-way mixed model ANOVAS

In Table 11 the fixed factors, feed, time and feed*time, are listed first and random factors after. As in Table

10 a plus sign + indicates that the factor is statistically significant.

| 69

From Table 11 it is seen that the factor Feed is only statistically significant for 7 attributes and non

significant for 10 attributes. The Time effect is considerably larger and only non significant for 5

attributes.

The factor Assessor is statistically significant for all attributes except for attribute Sour(o). The

interactions are mostly non significant. Replicate is non significant for all attributes.

Post hoc analysis The Tukey-Kramer adjusted pair wise comparisons in relevant cases indicate that samples fed with Blå

are the same as factors fed with Grøn and Rød, that samples fed with Hvid equals samples fed with Grå.

Jis and Gul does nor resemble any of the other feeds.

70 |

10.2.2 3-way mixed model ANCOVA That all feeds (except jis) are evaluated for all five storage times 0, 3, 5, 7 and 12 the factor storage time

can be considered as supplementary measurement on each experimental unit. From the plots in Figure 27

Earthy (O) vs. Time. Individual patterns (left) and regression lines (right) for the different feeds. it is seen that a linear

relationship between the storage time and the scores for attribute Earthy(o) may exist. In this section

the 4-way model is modeled taking this possible relationship into account. The storage time it is

modeled as a covariate.

The model for the 3way ANCOVA is written using the same notation as before, except the Time effect is

included as a covariate.

ijklikijjijklilliijkl FRFAAxfttfY )( (10.3)

It should be noted that the only fixed interaction terms with the covariate effects are included in the

model. The analysis is done following the same principles as section 10.1. The test of the covariate effect

ilft tests the hypothesis of equal slopes and the test of the covariate lt tests the hypothesis of

existence of a linear relationship.

The results of the 3-way ANCOVA are shown below for the attributes.

Source

3-way mixed model ANCOVA for

Earthy

Ckd

po

tato

Sou

rish

Sou

r

Mu

dd

y

Earthy

Mu

shro

om

Ckd

po

tato

Sou

rish

Sweet

Gree

n

Mu

dd

y

Flaky

Firm

Juicy

Fibro

usn

ess

Oilin

ess

Feed + + + + +

Time + + + + + + + + + + + + +

feed*time + +

Assessor + + + + + + + + + + + + + + + + +

Feed*Assessor + + + + + + + +

Replicate + + Table 12 Mixed model ANCOVAs

For the 3-way mixed model ANCOVA it is seen that the factor Feed is only statistically significant for 5 of

the attributes. This is two less compared to the results from the 4-way mixed model ANOVAs. The rest of

the factors are compared to the 4-way mixed model ANOVAs quantitatively the same with a slight

favour to the ANCOVAs.

The estimated slopes are listed in the table below for the relevant models

| 71

Variable Slope Earthy -0.04600 Cooked potato -0.03354 Sourish -0.02240 Sour -0.02801 Earthy -0.06399 Mushroom -0.09129 Sourish -0.05684 Sweet -0.03032 Green -0.05317 Muddy 0.07041 Flaky -0.04113 Juicy -0.08609 Table 13 Estimated slopes in the 3-way mixed model ANCOVAs

From the slopes listed in Table 13 it is seen that the slopes for the different models vary from -0.09129 to

0.07041. Most models have negative slope the only exceptions Fibrousness(t) and Muddy(f).

10.2.3 Validation of 4-way ANOVA and 3-way ANCOVA The validation of the 4-way ANOVA and 3-way ANCOVA is done similarly as described in section 10.1.

The QQ-plots for the standardized residuals (Figure 29) for both models show that the models fit

adequately.

Figure 29 QQ-plot of 4-way ANOVA (left) and 3-way ANCOVA (right).

Comparison of residuals show that the two models fit are almost identical, which could be due to the

covariate Time only has five different levels in this data set, and therefore is explained equally well by a

fixed effect.

72 |

10.3 Discussion of results In this section the results from the 3-way ANOVA, 4-way ANOVA and 3-way ANCOVA mixed models are

summarized and discussed. The results concerning the fixed effects are of interest in terms of

interpretation of which attributes account for product differences.

The random effects are of interest for later modeling of multivariate tests.

First the results from the 3-way mixed model ANOVA is summarized. Only attributes Muddy(o) and

Sweet(f) are not found to discriminate amongst products. Compared to the other two models this can

be translated into the factors Feed and Time combined . Assessor is found to be significant for all

attributes, replicate is found non significant for most attributes and the Product x Assessor interaction is

likewise significant for all attributes. This interaction translates into the interactions Feed x Assessor and

Time x Assessor and Feed x Time x Assessor in the other models.

Generally the results of the two 4 factor models are very similar. Neither the Feed x Time effect or the

replicate effect is significant in most of the models. The Time x Feed interaction (different slopes in the

ANCOVA model) is only significant in a very few cases. Unexpectedly, the Feed effect is only significant in

7 and 5 of the 17 attributes in the ANOVA and ANCOVA model, respectively. Time is generally significant

in all models. The variance term Assessor x Feed interaction is significant in 3 and 8 of the 17 attributes

in the ANOVA and ANCOVA model, respectively. Finally, Assessor is significant in all models.

Feed effect is found in attributes Earthy(o), Earthy(f), Flaky(t) and Oiliness(t) in both models.

Furthermore the attributes Cooked Potato(o), Sourish(o), Sour(o) and Sourish(f) are found to

differentiate between products for the ANOVA and the ANCOVA respectively.

Comparison with the 3-way mixed model ANOVA results are as expected. Once the proper “translations”

are made the results are very similar.

| 73

Chapter 11

Multivariate Analysis by Mixed Model MANOVA with CVA

in this chapter the univariate mixed model ANOVAs are extended to the multivariate analogues. In

section 11.1 a 3-way mixed model MANOVA is analysed with a CVA in section 11.1.1. In section 11.2 a 4-

way mixed model MANOVA is analysed together with CVAs in section 11.2.1.

74 |

11.1 3-way mixed model MANOVA The model for the 3-way mixed model MANOVA is formulated based on the 3-way univariate mixed

model ANOVAs. From section 10.1 results

ijkikijjiijk )(PRPAApY (11.1)

Where is the mean vector for the attributes. The product main effects, ip , i+1,…,38 represent the

differences in level between the average score for the different products. The assessor main effects, jA

, j=1,…,10 represent the assessor variation. The product-assessor interaction ijPA expresses the

variance of assessors in measuring differences in products, and ik)(PR , k=1,2 expresses the replicate

variation. The error term ijk represents the residual variation.

Tests of fixed effects The test for effect of product is tested up against the error term product by assessor interaction. This is

the analogue test performed in the univariate case where the model has been reduced of the factor

replicate (see section 10.1).

The test statistics for this test is given in Table 14

Statistic Value F Value Num DF Den DF P-value

Wilks' Lambda 0.02134986 1.80 629 4010.6 <0.0001

Pillai's Trace 2.94306619 1.53 629 4590 <0.0001

Hotelling-Lawley Trace 5.61148236 2.25 629 2758.2 <0.0001

Roy's Greatest Root 2.51250109 18.33 37 270 <0.0001 Table 14 Test statistics for 3-way mixed model MANOVA

It is seen that the Product effect is highly significant with p < 0.0001 for all four multivariate test

statistics. Compared to the results in section 10.1 the Product was not significant for the attributes

Sweet (f) and Muddy (o). This underlines the strength of using multivariate analysis of variance as

opposed to univariate.

Post hoc analysis Often univariate ANOVAs are performed after showing product differences. This have already been

done in the modeling of the multivariate models. Most attributes were found to differentiate between

products, but no products were found to be different.

The post hoc analysis for the MANOVA can be done similarly as for the univariate case i.e. performing

pair wise comparisons between products. In this report however, it is chosen only to analyze the data

further with a canonical variates analysis, CVA described in Chapter 5, that does not rely on pre specified

combinations of the variables.

The score plot for the canonical variables indicated by feed is shown below.

| 75

Figure 30 Score plot for canonical variables 1 and 2 from CVA of 3-way MANOVA indicated for different feed.

It is seen that product Sort is a possible outlier. As mentioned in Chapter 7, this could be expected.

Furthermore Sort was found to be an outlier in the PCA analysis. The CVA is redone without Sort.

The score plot for the CVA without the sample Sort is shown below.

76 |

he

Figure 31 Score plot for canonical variables 1 and 2 indicated for different time when excluding Sort.

In Figure 31 the products are color coded by their Time. Samples with Time=12 is shown in brown, Time=7

shown in red, Time=5 shown in orange, Time=3 shown in yellow and Time=0 shown in green (Note: this

is not the same color codes used in the PCA section). It is seen that there is a clear trend along the

canonical variable 1. The larger times to the left and descending towards right.

| 77

Figure 32 Score plot for canonical variables 1 and 2 indicated for different feed.

Figure 32 corresponds to Figure 31 except the 37 products are coded by the Feed. Blå is shown with the

color blue, Grå is shown in grey, Grøn is shown in green, Gul is shown in yellow, Hvid is shown in pink, Jis

is shown in orange and Rød is shown in red. A trend, however less obvious, is seen along canonical

variable 2. Hvid and Grå in the top, followed by Rød and Blå , and finally Gul, Grøn and Jis.

78 |

Figure 33 Structural loading plot for CVA without Sort

As the main interest is in what variables are similar and which are different, only the structural loading

plot is given above. The attributes are coded by their number. The attribute name can be seen from Table

2 (b).

Viewing the structural loading plot along with the score plot in Figure 31 the attributes Muddy(f), Firm(t)

and Sour(o) describe the samples with large Time. On the other end of the scale attributes Mushroom(f),

Sourish(f), Green(f) and Sweet(f) are positively correlated and describe samples with small Time

(Time=0). These attributes are hence negatively correlated with the attributes describing the samples

with large Time: Muddy(f), Firm(t) and Sour(o).

The attributes Flaky(t) and Juicy(t) are likewise positively correlated. They have large values for samples

with small Time. Comparing with the score plot in Figure 32 these attributes are seen to explain the

samples with Feed Hvid and Grå, while the attributes in the bottom are likely to explain the samples Gul,

Jis and Grøn.

Comparing this with the results from the PCA will be done in section 11.3.

| 79

Validation of model The validation of the multivariate setting covers

1. Multivariate normality (Sampling distributions of the dependent variables and all linear

combinations of them are normal).

2. Homogeneity of covariance matrices

3. Linearity (linear relationships between all pairs of dependent variables exist)

4. No multicollinearity (correlation r < 0.8 and vif < 10)

5. No singularity (a dependent variable is a linear combination of other dependent variables)

The MANOVA test is fairly robust to departures from multivariate normality, why this section will focus

on validation of equality of variance matrices and multi collinearity. The latter is the most important

validation step of the MANOVA [Rencher, 2001].

In Chapter 8 it was shown that based on the VIF criteria there are no multicollinearity between the

attributes. The correlations were likewise all found to be under 0.8.

11.2 4-way mixed model MANOVA From the results from section 10.2 it was seen that the 4-way ANOVA and the 3-way ANCOVA models

differed very little. As the slopes in the ANCOVA model were found to be very close to zero and

furthermore to be ascending or descending for different attributes only the ANOVA will be modeled in a

multivariate analogue.

When using the same notation as before the model for the 4-way mixed model MANOVA becomes

ijkliklijljlijjilliijkl )(FTRAFTATFAAfttfY (11.3)

Where if is the fixed Feed effect, i= Hvid, Grå, Blå, Rød, Grøn, Jis and Gul and

lt is the fixed Time effect in

days, l=0, 3, 5, 7 and 12. The interaction of the two effectsilft is also fixed. All other main effects and

interaction terms are random.

Test of fixed effects The effect of interest is primarily the Feed. Also the Time and the Feed x Time interaction will be tested.

The test statistics used in the univariate case is modified to accommodate the MANOVA for the data set.

As were seen from section 10.2.1 and 10.2.2 the Replicate effect were shown to be non-significant for all

attributes. The Feed x Assessor interaction is significant for more than half the attributes. The Time x

Assessor interaction is likewise tested out for almost half the attributes.

In the following the Replicate will be left out of the test statistic calculations. The two tests for Feed and

Time Will be done by Feed x Assessor, Time x Assessor respectively. The interaction Feed x Time x

80 |

Assessor was non-significant for all attributes. The multivariate test for the effect Feed x Time will be

carried out in the multivariate case non the less.

The test of the Feed effect


Wilks' Lambda 0.05327347 1.36 102 206.54 0.0322

Pillai's Trace 2.08203250 1.25 102 240 0.0844

Hotelling-Lawley Trace 4.60435857 1.51 102 126.34 0.0135

Roy's Greatest Root 2.46206468 5.79 17 40 <.0001 Table 15 Test statistics for test of Feed effect

As can be seen from the table above the Feed effect is significant on a 0.05 significance level in all

multivariate test statistics apart from Pilai’s Trace. Compared to the univariate tests already carried out

in section 10.2.2 where Feed was only significant for 7 out of 17 attributes, the two results differ.

The test for no overall Time effect, gives the result shown in Table 16


Wilks' Lambda 0.02786711 1.68 68 76.88 0.0135

Pillai's Trace 2.05896145 1.37 68 88 0.0809


Roy's Greatest Root 6.37157069 8.25 17 22 <.0001 Table 16 Test statistics for test of Time effect

The Time effect is significant on a 0.05 significance level in all test statistics except from Pilai’s Trace.

This result matches the results from the univariate tests carried out in section 10.2.1 where Time was

found to be significant factor for 12 out of 17 attributes.

For completion the test for no overall Feed x Time effect is listed below.


Wilks' Lambda 0.08962842 1.04 340 1742 0.2942

Pillai's Trace 2.12410575 1.04 340 2482 0.2979


Roy's Greatest Root 0.51675620 3.77 20 146 <.0001 Table 17 Test statistics for Feed x Time effect

The Feed x Time effect is non-significant in all tests except from Roy’s greatest Root. This result is the

same as most results in the analogue univariate tests shown in section 10.2.1 where the feed x time

effect was only found to be statistically significant for one attribute.

Post hoc analysis The univariate tests have already been done in section 10.2.1 to summarize Feed effect is found in

attributes Earthy(o), Earthy(f), Flaky(t), Cooked Potato(o), Sourish(o), Sour(o) and Oiliness(t) in ANOVA.

Furthermore differences were found significant for several pairs of Feed.

| 81

As post hoc analysis to explore the differences in Feed and Time effects respectively CVA’s are carried

out for each case. As mentioned in Chapter 7 Sort does not have a Time and does not take part in the

analysis.

As the CVA from the Feed effect test, seeks the canoncal variates that lead to the largest spread in

products given by the Feed only the plot color coded by the different Feeds is given below.

Figure 34 Canonical score plot maximizing spread of products by their Feed, indicated by Feed

From the canonical score plot above essentially the same conclusions can be drawn as from the similar

score plot in Figure 32 for the CVA from the 3-way mixed model MANOVA. There seem to be grouping of

the products by feed with Gul, Grøn and Jis in one group and Rød and Blå followed by Hvid and Grå in

another.

The grouping however is more distinct here than in Figure 32. This is a result of the different model and

the multivariate test, ensuring maximal spread according to Feed. Another consequence of this can be

read directly from the plot where the separation of Feed is explained by the first canonical variate, and

not the second as in Figure 32.

Again the interest is on the structural interpretation and the structural loading plot is shown below.

82 |

Figure 35 Structural loadings for CVA on the 4-way MANOVA test of Feed

From the structural loading plot it is seen that the leftmost samples in the score plot, that is those with

Feed Gul, Jis and Grøn, are described by high values in the attributes Sour(o), Sweet(f), Mushroom(f),

Muddy(o) and Sourish(f). These attributes are highly correlated and negatively correlated with the

attributes on the other end of the scale: Juicy(t), Earthy(f) and in particular Flaky(t). These attributes are

characterized in high values for the rightmost samples in the score plot Figure 34 which are Hvid and Grå.

Comparing to the univariate results of section 10.2.1 these are not the exact same attributes found

influential in the univariate analysis.

| 83

Figure 36 Score plot for the CVA on the 4-way mixed model MANOVA test of the factor Time

The score plot for the CVA resulting from the multivariate test for Time effect is shown above. This plot

is clearly different than the one in Figure 34 other than just the color indication. The canonical variates are

found to spread the products the most according to the factor Time. As a result the Time effect is seen

along the first canonical variate. Compared to Figure 30 where the Time effect is similarly most evident

along the first canonical variate, the grouping seems more underlined here.

There is not a clear time trend in the sense that the groups are not plotted in a strict descending order.

Time=12 is the leftmost group and 0 is the rightmost group as could be expected. But the groups for

Time=3, 5 and 7 do not express an explicit ranking given by the first canonical variate. Viewing only the

samples from the three intermediate Times an other time trend can be spotted along the second

principal component. Samples with Time=3 in the bottom and samples with Time=7 at the top.

This phenomenon of two time trends along both the first and the second component was also seen in

the PCA of the full samples except Sort. The same pattern in the score plot was found with Time=12 and

Time=0 explained by the first component and the samples with intermediate Times explained by the

second. This is an indication of the Time effect being so dominant that the PCA performs like a CVA from

an multivariate test of the Time effect.

84 |

As before the structural loading plot is shown.

Figure 37 Structural loading plot for the CVA for the mixed model MANOVA test of the factor Time

An interesting pattern is seen from the loading plot for the CVA separated in Time. The attributes Firm(t)

and Flaky(t) both explain the larger Time samples along the second canonical variate, but at the same

time Firm(t) describes the samples with Time=0 along the first canonical variate. The same is the case

with the attribute Muddy(f) the other way around as the attribute is seen to be negatively correlated

with the attribute Flaky(t). The attributes Sour(o) and Fibrousness(t) are also influential in describing the

Time effect.

Compared to the loading plot for the PCA in Figure 14 a similar pattern can be found. Only in the case of

the PCA attributes found less important here (the attributes in the middle of the loading plot) are highly

negatively correlated with the attributes connected with the samples with Time=12. They are grouped

with the attribute Flaky(t) in the PCA. This could be an indication that the Time effect and the Feed x

Time interaction are in fact more evident than the Feed effect in this particular data set.

The validations of the two models follow the outline given in section 11.1.

| 85

11.3 Comparison with PCA In the following the different models of the ANOVAs and Manovas are listed. For simplicity the replicate

effect is omitted. The model only including Product is included

Model 1: Product ANOVA: F=MSProduct/MSE MANOVA test of Product

CVA

Model 2: Product + Assessor + Product*Assessor ANOVA: F=MSproduct/MSproduct*assessor MANOVA test of Product

CVA

Model 3: Feed + Time + Feed*Time + Assessor + Feed*Assessor + Time*Assessor ANOVA: F=MSFeed/MSFeed*Assessor MANOVA test of Feed

CVA

ANOVA:F=MSTime/MSTime*Assessor MANOVA test of Time CVA

Table 18 Different simplified ANOVA and MANOVA models

The 3-way mixed model MANOVA in section 11.1 resembles Model 2 in Table 18. The 4-way mixed model

MANOVA in section 11.2 resembles Model 3 a and b respectively. These are mixed models taking the

variation from the Assessors into account.

Model 1 has not been modeled in this report (not regarding the 1-way ANOVAs done for each assessor

in the panel assessment). The MANOVA with the CVA in this case would be an exact analogue of the PCA

done in Chapter 9, only taking the error structure into account and having p-values for test statistics for

the test of Product differences.

The models in Table 18 model the data set with increasing complexity and detail. The results are similarly

increasing in quality.

In the PCA a Time trend was found along both the first and the second principal components. The Time

effect was found to be explained by attributes Muddy(f), Firm(t), Sour(o) and Fibrousness(t) for large

Times and all other attributes excluding Muddy(o) and those already mentioned explaining samples

with small Times. Interpretation was made difficult resulting from a overshadowing Time effect.

In the PCA the Feed effect was indecisive.

In the 3-way mixed model MANOVA with CVA a Time trend was found along the first canonical variate.

Attributes Sour(o), Muddy(f) and Firm(t) explain the samples with large Time. Attributes Green(f),

Sourish(f), Mushroom(f) and sweet(f) describe samples with small Time.

A feed effect is found along the second canonical variable. Gul, Grøn and Jis described by the attribute

Sourish(f) and Rød and Blå followed by Hvid and Grå described by the attributes Flaky(t) and Juicy(t).

86 |

As in the PCA the interpretation of the results is not completely straight forward resulting from the large

effect of the Time.

The Feed effect and Time effect is modeled directly into the model for the 4-way mixed model

MANOVA.

The results from the CVA, maximising the spread amongst samples by their Feed, leads to a Feed effect

visible along the first canonical variable. The grouping is clear with Gul, Grøn and Jis in one followed by

Rød and Blå and then by Hvid and Grå. The attributes Sweet(f), Sour(o), Mushroom(f), Muddy(o) and

Sourish(f) explains the samples in the end containing Gul. The attributes Earthy(f), Juicy(f) and Flaky(t)

describes the samples in the end containing Hvid.

The result from the CVA, maximizing the spread amongst samples by Time, leads to a Time effect visible

along both the first and the second canonical variable. Samples are clearly separated along the first

canonical variable with samples with Time=12 in one end and samples with time 0 in the other end. The

effect along the second canonical variate is less pronounced than that in the first. It seems to be a time

effect underlying the Time effect in the first. Here the samples with Time=3 is in one end and samples

with time=7 in the other. The attributes Firm(t), Flaky(t), Sour(o), Muddy(f) and Fibrousness(t) are found

to explain the difference in Time. The interpretation is not completely clear as there seem to be a two

way interpretation for the attributes Firm(t) and Muddy(f).

The results from the different analysis’ differ slightly in what attributes are most influential (and also

from the attributes found influential in univariate analysis). Other than that a clear Time effect is found

for all models. No clear Feed effect is found by the PCA. The 3-way mixed model MANOVA with CVA

showed sign of Feed effect but not fully interpretable. Only the 4-way mixed model MANOVA with CVA

gave clear results both on the Feed effect and the Time effect.

| 87

Chapter 12

Multivariate Analysis by 50-50 MANOVA

So far either a PCA have been done or a MANOVA have been done. It is a possibility to do a PCA on the

data first and then do a MANOVA on a number of the principal components found reasonable. This way

the analysis falls into the same box as the PCA in Fejl! Henvisningskilde ikke fundet.. Another way of

doing a combined PCA and MANOVA is the 50-50 MANOVA. This method does not average the data

over assessors and replicates and hence fall into the same box as the MANOVAs in . The 50-50 MANOVA

has been implemented in Matlab and R and is free to download and use. It is however still quite rough

and options of coding variables as random do not exist. In order to test the mixed model MANOVA as in

section 11.1 precautions have to be made.

As no scaling is done in the PCA, no scaling will be done in the 50-50 MANOVA.

First the model resulting in the following table is run.

Source DF exVarSS nPC nBu exVarPC exVarBU p-Value

Code 37 0.159523 5 12 0.784 1.000 0.000000

Assessor 9 0.270473 6 11 0.826 1.000 0.000000

code*assessor 270 0.308476 11 6 0.909 1.000 0.000000

code*replicate 37 0.028832 9 8 0.862 1.000 0.770615

assessor*replicate 9 0.005878 8 9 0.831 1.000 0.836566

Error 269 0.215208 Table 19 Result from 50-50 MANOVA with Replicate in model

As can be seen the two replicate interactions are non significant. The model averaging over replicate,

only containing the factors Product, Assessor and Product*Assessor resembles the mixed Model 2 in

Table 18.

The results from this model is given below

Source DF exVarSS nPC nBu exVarPC exVarBU p-Value

Code 37 0.159560 6 11 0.818 1.000 0.000000

Assessor 9 0.270401 6 11 0.817 1.000 0.000000

code*assessor 270 0.308477 11 6 0.908 1.000 0.000000

Error 315 0.249526 Table 20 Results from the 50-50 MANOVA

88 |

As can be seen the factor Product is tested statistically significant. The 50-50 MANOVA gives the same

result as the corresponding 3-way mixed model MANOVA.

The comparison here is not so much in favor of the 50-50 MANOVA as no indication of multicollinearity

in the data. In this case where the optimal number of principal components were found to be 14 (only a

reduction in dimension of three) the two models are expected to be quite similar. The PCA done in this

section is not the same as the one done in Chapter 9 but the principle and models are still applicable for

visualization and explanation

The validation of the 50-50 MANOVA is done similar as for PCA in Chapter 9 and will not be repeated

here.

| 89

Chapter 13

Conclusion and Discussion

The analysis of a sensory profiling set describing quality of fish on 17 odor, flavor and texture attributes

is carried out by various multivariate methods.

In Chapter 8 initial descriptive analysis of the data showed no indication of multicollinearities between

attributes.

In Chapter 9 a PCA is employed on the data averaged over assessors and replicates. Product difference is

found on the sensory attributes, mainly as a result of the different storage times. Validation and

reliability of the model is assessed by full cross validation with jack-knifing.

As no feed effect is found in the preliminary analysis a further PCA is done on a subset of the data

containing fewer samples limiting the difference of the storage time. Though not reliable this PCA

suggests that the products are differentiated as a result of the different feeds. This remodeling of the

data is recommended in later profiling data sets, if indeed it is the feed effect that is of interest and PCA

is the method that is used for analysis.

In Chapter 10 the data is modeled by mixed models in which only the variation from assessors (and

replicates) are taken into account. The results from 3-way and 4-way mixed model ANOVAs and

ANCOVAs indicate product differences for most attributes and feed effect for less than half of the

attributes. In all analysis the effect of the replicates are seen to be non-significant.

Multivariate mixed models are modeled based on the results from the univariate analysis. In Chapter 10

3-way and 4-way mixed model MANOVAs indicate product difference and feed effect. These differences

are explored by means of CVA.

Finally the results from the 50-50 MANOVA are presented in Chapter 12. The difference in products is

found to be significant.

The multivariate methods MANOVA and 50-50 MANOVA gave similar results. Product difference is

found. Furthermore difference is found as a result of both feed and storage time. The nature of the

differences is not explained by these methods.

90 |

Univariate ANOVAs( and ANCOVAs) give an indication of what attributes show differences in the

products.

Similar information can be found in a truly multivariate setting by means of CVA and PCA. The results

from these differ slightly and overall the CVA results are easiest to interpret.

From analysis of this particular data set it can be concluded that mixed model MANOVA with CVA is the

best means of analysis. In the univariate analysis product difference is not found for more than half the

attributes. The results from the PCA indicate product differences but the systematic variance of the data

is dominated by the storage time effect and the feed effect is not found by analysis of this data set. The

50-50 MANOVA does not really offer anything new in this case. It should be emphasized again though

that this particular data set has no multicollinearities as could be expected for an average sensory

profiling data set.

The results from the mixed model MANOVA with CVA is restated as the results from the final analysis:

Analysis indicate product differences. Fish samples fed by Gul, Grøn and Jis are described by the

attributes Sweet(f), Sour(o), Mushroom(f), Muddy(o) and Sourish(f). Samples fed with Rød and Blå and

even more Hvid and Grå are better explained by attributes Earthy(f), Juicy(f) and Flaky(t).

The attributes Firm(t), Flaky(t), Sour(o), Muddy(f) and Fibrousness(t) are found to explain the difference

in samples with different storage times.

| 91

Appendix A

Characteristic Roots and Vectors

The method of the PC’s is based on a key result from Matrix algebra. Let U be an orthonormal matrix

that is the columns in U; ui and uj holds i and ji

1i

T

i uu (A.1)

0j

T

i uu (A.2)

A pp symmetric, nonsingular matrix (such as the covariance matrix S) may be reduced to a diagonal

matrix L by

LSUU T

(A.3)

Diagonal elements of L ( pll ,...,1 ) are called characteristic roots, latent roots or eigenvalues of S.

Columns of U ( puu ,...,1 ) are called e characteristic vectors or eigenvectors of S.

(Geometrically: elements of e are the direction cosines of the new axes related to the old:)

’s may be obtained from the solution of the characteristic equation:

0 IS l (A.4)

e may then be obtained by solution of the equations

0 IS l (A.5)

And

iTi

i

itt

tu (A.6)

92 |

Appendix B

General Test on a Subset of the X’s

This appendix is for use of derivation of the 50-50 test used in the 50-50 MANOVA. Specific test statistics

will not be discussed. The point of interest is the p-value, as a function of X and Y denoted as YX,pv .

Consider the following model

)()()()( qnqnnnqn E BXY

Where the difference from the former convention is that there is n x’s where n is the number of

observations. It is assumed that 10 nxxX has full rank n, the rows of E are uncorrelated

multinormal ,0qN , 10 n1x which means that the B0 is the intercept terms.

Here the test considered is

010 ,...,: nm BBH against the alternative

0

0

1

11

,...,

,...,:

npm

pmm

ABB

BBH (B1)

The H0 model contains the first m x’s and the testing is of the next p (Note that p in this section is not

the same as in the former) .

A p-value for a test statistic testing H0 in (B1) for the linear combination of for the linear combination of

Y is desired.

Here the details of retrieving the p-value will not be given. For full detail on deduction see Langsrud

(2002).

Consider the linear combination Z of Y.

YXzzzT

Q

T

n 110

where the orthogonal matrix QX is given by the QR decomposition of X.

| 93

RQXXX

Under H0 in (B1) 1,..., nm zz are observations from ,0qN . Under HA it is clear that 1,..., npm zz still

follow this distribution. These vectors are called error observations. Similarly 1,..., pmm zz are referred

to as the hypothesis observations. The hypothesis test can be viewed as a comparison of the hypothesis

observation with the error observations. Hence the p-value can be expressed as

T

nm

ppmn

pppv 1,...,, zz

0

I (B2)

Where it is noted that the dimensions of the input matrices to pv, pmn )( and qmn )( ,

correspond to the number of responses, q, the degrees of freedom for the hypothesis, p, and the

degrees of freedom for the error (n-m-p). It is further noted that the test described by (B1) and (B2)

collapses when the number of responses pmnq .

94 |

Appendix C

Correlations of Response Variables

Pearson Correlation Coefficients, N = 632 Prob > |r| under H0: Rho=0

Earthy_O_

Cooked_potato_O_

Sourish_O_

Sour_O_ Muddy_O_

earthy_f_ Mushroom_F_

Cooked_potato_F_

Sourish_F_

Sweet_F_

Green_F_

Muddy_F_

Flaky_T_ Firm_T_ Juicy_T_ Fibrousness_T_

Oiliness_T_

Earthy_O_

Earthy(O)

1.000

00

0.65338

<.0001

0.500

49

<.000

1

0.068

42

0.085

7

0.206

73

<.000

1

0.421

50

<.000

1

0.3670

5

<.0001

0.22004

<.0001

0.134

95

0.000

7

0.024

31

0.541

9

0.188

81

<.000

1

-

0.040

83

0.305

4

0.085

07

0.032

5

-

0.008

96

0.822

2

0.092

00

0.020

7

-

0.04403

0.2690

0.059

36

0.136

1

Cooked_pota

to_O_

Cooked

potato(O)

0.653

38

<.000

1

1.00000

0.474

03

<.000

1

0.119

85

0.002

5

0.201

44

<.000

1

0.291

37

<.000

1

0.2648

0

<.0001

0.34806

<.0001

0.046

19

0.246

2

0.077

12

0.052

6

0.208

05

<.000

1

0.004

70

0.906

0

0.215

29

<.000

1

0.100

15

0.011

8

0.156

35

<.000

1

0.08397

0.0348

0.154

16

<.000

1

Sourish_O_

Sourish(O

)

0.500

49

<.000

1

0.47403

<.0001

1.000

00

-

0.006

28

0.874

8

0.078

47

0.048

6

0.317

46

<.000

1

0.2675

3

<.0001

0.20909

<.0001

0.337

82

<.000

1

0.179

68

<.000

1

0.128

79

0.001

2

-

0.019

01

0.633

4

0.147

49

0.000

2

0.069

69

0.080

0

0.092

08

0.020

6

0.04068

0.3073

0.095

05

0.016

8

Sour_O_

Sour(O)

0.068

42

0.085

7

0.11985

0.0025

-

0.006

28

0.874

8

1.000

00

0.483

10

<.000

1

-

0.076

34

0.055

1

-

0.0422

7

0.2887

-0.03058

0.4428

-

0.151

72

0.000

1

-

0.104

13

0.008

8

-

0.017

07

0.668

5

0.139

06

0.000

5

0.112

48

0.004

6

0.060

16

0.130

9

0.005

07

0.898

7

0.04237

0.2875

-

0.012

56

0.752

7

Muddy_O_

Muddy(O)

0.206

73

<.000

1

0.20144

<.0001

0.078

47

0.048

6

0.483

10

<.000

1

1.000

00

-

0.107

40

0.006

9

-

0.0388

0

0.3301

-0.01038

0.7946

-

0.152

30

0.000

1

-

0.107

37

0.006

9

0.109

86

0.005

7

0.387

89

<.000

1

0.032

35

0.416

9

-

0.085

03

0.032

6

0.003

64

0.927

2

-

0.11768

0.0030

0.023

87

0.549

2

earthy_f_

Earthy(O)

1

0.421

50

<.000

1

0.29137

<.0001

0.317

46

<.000

1

-

0.076

34

0.055

1

-

0.107

40

0.006

9

1.000

00

0.6779

2

<.0001

0.34530

<.0001

0.285

23

<.000

1

0.034

40

0.387

9

0.057

06

0.151

9

-

0.192

34

<.000

1

0.116

30

0.003

4

0.062

23

0.118

1

0.136

07

0.000

6

0.00437

0.9127

0.063

97

0.108

2

Mushroom_

F_

Mushroom(

F)

0.367

05

<.000

1

0.26480

<.0001

0.267

53

<.000

1

-

0.042

27

0.288

7

-

0.038

80

0.330

1

0.677

92

<.000

1

1.0000

0

0.29445

<.0001

0.232

37

<.000

1

0.150

86

0.000

1

0.282

89

<.000

1

-

0.277

98

<.000

1

0.079

05

0.047

0

0.073

76

0.063

9

0.192

34

<.000

1

-

0.06708

0.0920

0.030

16

0.449

2

Cooked_pota

to_F_

Cooked

potato(F)

0.220

04

<.000

1

0.34806

<.0001

0.209

09

<.000

1

-

0.030

58

0.442

8

-

0.010

38

0.794

6

0.345

30

<.000

1

0.2944

5

<.0001

1.00000

0.236

32

<.000

1

0.127

27

0.001

3

0.163

36

<.000

1

-

0.102

73

0.009

8

0.041

85

0.293

6

0.119

60

0.002

6

0.095

63

0.016

2

0.14112

0.0004

0.136

83

0.000

6

Sourish_F_

Sourish(F

)

0.134

95

0.000

7

0.04619

0.2462

0.337

82

<.000

1

-

0.151

72

0.000

1

-

0.152

30

0.000

1

0.285

23

<.000

1

0.2323

7

<.0001

0.23632

<.0001

1.000

00

0.144

91

0.000

3

0.106

31

0.007

5

-

0.068

30

0.086

2

0.074

26

0.062

1

0.210

89

<.000

1

0.082

56

0.038

0

0.11523

0.0037

0.069

73

0.079

8

Sweet_F_

Sweet(F)

0.024

31

0.541

0.07712

0.0526

0.179

68

<.000

-

0.104

13

-

0.107

37

0.034

40

0.387

0.1508

6

0.0001

0.12727

0.0013

0.144

91

0.000

1.000

00

0.177

77

<.000

-

0.213

34

0.155

78

<.000

0.133

15

0.000

0.015

07

0.705

0.03988

0.3168

0.064

40

0.105

| 95

Pearson Correlation Coefficients, N = 632 Prob > |r| under H0: Rho=0

Earthy_O

_

Cooked_potat

o_O_

Sourish_

O_

Sour_O_ Muddy_O

_

earthy_f_ Mushroom

_F_

Cooked_pota

to_F_

Sourish_

F_

Sweet_F

_

Green_F

_

Muddy_F

_

Flaky_T_ Firm_T_ Juicy_T_ Fibrousnes

s_T_

Oiliness_

T_

9

1

0.008

8

0.006

9

9

3

1

<.000

1

1

8

4

8

Green_F_

Green(F)

0.188

81

<.000

1

0.20805

<.0001

0.128

79

0.001

2

-

0.017

07

0.668

5

0.109

86

0.005

7

0.057

06

0.151

9

0.2828

9

<.0001

0.16336

<.0001

0.106

31

0.007

5

0.177

77

<.000

1

1.000

00

0.044

56

0.263

4

0.115

23

0.003

7

0.082

99

0.037

0

-

0.016

14

0.685

5

-

0.00644

0.8716

-

0.036

64

0.357

8

Muddy_F_

Muddy(F)

-

0.040

83

0.305

4

0.00470

0.9060

-

0.019

01

0.633

4

0.139

06

0.000

5

0.387

89

<.000

1

-

0.192

34

<.000

1

-

0.2779

8

<.0001

-0.10273

0.0098

-

0.068

30

0.086

2

-

0.213

34

<.000

1

0.044

56

0.263

4

1.000

00

-

0.081

07

0.041

6

-

0.019

82

0.618

9

-

0.208

97

<.000

1

0.06834

0.0861

-

0.038

38

0.335

4

Flaky_T_

Flaky(T)

0.085

07

0.032

5

0.21529

<.0001

0.147

49

0.000

2

0.112

48

0.004

6

0.032

35

0.416

9

0.116

30

0.003

4

0.0790

5

0.0470

0.04185

0.2936

0.074

26

0.062

1

0.155

78

<.000

1

0.115

23

0.003

7

-

0.081

07

0.041

6

1.000

00

0.299

51

<.000

1

0.513

14

<.000

1

0.15498

<.0001

0.337

58

<.000

1

Firm_T_

Firm(T)

-

0.008

96

0.822

2

0.10015

0.0118

0.069

69

0.080

0

0.060

16

0.130

9

-

0.085

03

0.032

6

0.062

23

0.118

1

0.0737

6

0.0639

0.11960

0.0026

0.210

89

<.000

1

0.133

15

0.000

8

0.082

99

0.037

0

-

0.019

82

0.618

9

0.299

51

<.000

1

1.000

00

0.105

40

0.008

0

0.52107

<.0001

0.200

16

<.000

1

Juicy_T_

Juicy(T)

0.092

00

0.020

7

0.15635

<.0001

0.092

08

0.020

6

0.005

07

0.898

7

0.003

64

0.927

2

0.136

07

0.000

6

0.1923

4

<.0001

0.09563

0.0162

0.082

56

0.038

0

0.015

07

0.705

4

-

0.016

14

0.685

5

-

0.208

97

<.000

1

0.513

14

<.000

1

0.105

40

0.008

0

1.000

00

-

0.04567

0.2516

0.468

67

<.000

1

Fibrousness_

T_

Fibrousne

ss(T)

-

0.044

03

0.269

0

0.08397

0.0348

0.040

68

0.307

3

0.042

37

0.287

5

-

0.117

68

0.003

0

0.004

37

0.912

7

-

0.0670

8

0.0920

0.14112

0.0004

0.115

23

0.003

7

0.039

88

0.316

8

-

0.006

44

0.871

6

0.068

34

0.086

1

0.154

98

<.000

1

0.521

07

<.000

1

-

0.045

67

0.251

6

1.00000

0.306

56

<.000

1

Oiliness_T_

Oiliness(

T)

0.059

36

0.136

1

0.15416

<.0001

0.095

05

0.016

8

-

0.012

56

0.752

7

0.023

87

0.549

2

0.063

97

0.108

2

0.0301

6

0.4492

0.13683

0.0006

0.069

73

0.079

8

0.064

40

0.105

8

-

0.036

64

0.357

8

-

0.038

38

0.335

4

0.337

58

<.000

1

0.200

16

<.000

1

0.468

67

<.000

1

0.30656

<.0001

1.000

00

| 97

Appendix D

3-way Mixed Model ANOVA results

Least square mean estimates Product LSM Estimate Lower 95% CI Upper 95% CI StdErr

BLÅ_0 4.335 3.683918 4.986082 0.327247

BLÅ_12a 3.52998 2.823776 4.236184 0.356148

BLÅ_12b 3.456858 2.713481 4.200234 0.375478

BLÅ_3a 4.162077 3.485894 4.838259 0.340448

BLÅ_3b 3.381858 2.638481 4.125234 0.375478

BLÅ_5 3.722562 3.016332 4.428792 0.356161

BLÅ_7 3.78 3.128918 4.431082 0.327247

GRÅ_0a 4.071462 3.365289 4.777636 0.356132

GRÅ_0b 4.201233 3.495026 4.90744 0.356149

GRÅ_12 3.919437 3.213207 4.625667 0.356161

GRÅ_5 4.528956 3.822639 5.235272 0.356205

GRÅ_7 4.0275 3.376418 4.678582 0.327247

GRØN_0 4.4775 3.826418 5.128582 0.327247

GRØN_12 3.211094 2.512981 3.909207 0.352031

GRØN_3 3.480837 2.774664 4.187011 0.356132

GRØN_5 3.558105 2.851901 4.264309 0.356148

GRØN_7 3.957483 3.251276 4.66369 0.356149

GUL_0 3.9525 3.301418 4.603582 0.327247

GUL_12a 3.114 2.370624 3.857377 0.375478

GUL_12b 3.282039 2.576028 3.98805 0.356049

GUL_3a 3.490212 2.784039 4.196386 0.356132

GUL_3b 3.306858 2.563481 4.050234 0.375478

GUL_5 3.84873 3.142526 4.554934 0.356148

GUL_7 3.835608 3.129401 4.541815 0.356149

HVID_0a 4.753743 4.077561 5.429926 0.340448

HVID_0b 3.591414 2.885403 4.297425 0.356049

HVID_12 3.750687 3.044457 4.456917 0.356161

HVID_5 4.650831 3.944514 5.357147 0.356205

HVID_7 4.4925 3.841418 5.143582 0.327247

JIS_3 3.357081 2.650764 4.063397 0.356205

RØD_0 4.3125 3.661418 4.963582 0.327247

RØD_12 3.62373 2.917526 4.329934 0.356148

98 |

RØD_3 3.887077 3.210894 4.563259 0.340448

RØD_5 3.806937 3.100707 4.513167 0.356161

RØD_7 3.645 2.993918 4.296082 0.327247

GRÅ_3 4.564076 3.820745 5.307406 0.375455

HVID_3 4.135504 3.392174 4.878835 0.375455

SORT 3.364076 2.620745 4.107406 0.375455

Pair wise comparisons using tukey adjustment Pair wise comparisons between products. In order to prevent inflation of the type 1 error, Tukey

adjustment has been applied i.e. adjusted 95% confidence intervals are given together with the adjusted

p-values. The unadjusted p-values are shown for the complete picture.

Product A Product B Est difference Adj Lower CI

Adj Upper

CI Adj p-value Unadj p-value

BLÅ_0 BLÅ_12a 0.805019939 -0.819249928 2.429289806 0.995758053 0.05379515

BLÅ_0 BLÅ_12b 0.878142419 -0.811314209 2.567599047 0.990698139 0.043197553

BLÅ_0 BLÅ_3a 0.172923287 -1.399085391 1.744931965 1 0.667610278

BLÅ_0 BLÅ_3b 0.953142419 -0.736314209 2.642599047 0.969081612 0.028307491

BLÅ_0 BLÅ_5 0.612437857 -1.011876442 2.236752157 0.9999851 0.1417707

BLÅ_0 BLÅ_7 0.555 -0.973590881 2.083590881 0.99999404 0.157063631

BLÅ_0 GRÅ_0a 0.263537535 -1.360680836 1.887755907 1 0.526542295

BLÅ_0 GRÅ_0b 0.133766531 -1.490508195 1.758041258 1 0.747812603

BLÅ_0 GRÅ_12 0.415562857 -1.208751442 2.039877157 1 0.318276213

BLÅ_0 GRÅ_5 -0.193955607 -1.818417558 1.430506345 1 0.641148132

BLÅ_0 GRÅ_7 0.3075 -1.221090881 1.836090881 1 0.432449694

BLÅ_0 GRØN_0 -0.1425 -1.671090881 1.386090881 1 0.715897501

BLÅ_0 GRØN_12 1.123905898 -0.486599819 2.734411616 0.719203821 0.006777259

BLÅ_0 GRØN_3 0.854162535 -0.770055836 2.478380907 0.98882841 0.040816709

BLÅ_0 GRØN_5 0.776894939 -0.847374928 2.401164806 0.997735197 0.062667066

BLÅ_0 GRØN_7 0.377516531 -1.246758195 2.001791258 1 0.364510635

BLÅ_0 GUL_0 0.3825 -1.146090881 1.911090881 1 0.328982138

BLÅ_0 GUL_12a 1.220999562 -0.468457066 2.91045619 0.642857466 0.005088986

BLÅ_0 GUL_12b 1.05296078 -0.570980201 2.676901762 0.848202642 0.011842139

BLÅ_0 GUL_3a 0.844787535 -0.779430836 2.469005907 0.990601005 0.043061983

BLÅ_0 GUL_3b 1.028142419 -0.661314209 2.717599047 0.921136648 0.018086363

BLÅ_0 GUL_5 0.486269939 -1.137999928 2.110539806 0.999999964 0.243029124

BLÅ_0 GUL_7 0.499391531 -1.124883195 2.123666258 0.999999924 0.230580378

BLÅ_0 HVID_0a -0.418743379 -1.990752057 1.153265298 0.999999999 0.298793364

BLÅ_0 HVID_0b 0.74358578 -0.880355201 2.367526762 0.998998553 0.074656304

BLÅ_0 HVID_12 0.584312857 -1.040001442 2.208627157 0.999995263 0.160912556

BLÅ_0 HVID_5 -0.315830607 -1.940292558 1.308631345 1 0.448020656

BLÅ_0 HVID_7 -0.1575 -1.686090881 1.371090881 1 0.687502719

BLÅ_0 JIS_3 0.977919393 -0.646542558 2.602381345 0.930321789 0.019356859

BLÅ_0 RØD_0 0.0225 -1.506090881 1.551090881 1 0.954169027

BLÅ_0 RØD_12 0.711269939 -0.912999928 2.335539806 0.99958719 0.088160073

BLÅ_0 RØD_3 0.447923287 -1.124085391 2.019931965 0.999999991 0.266453059

| 99

BLÅ_0 RØD_5 0.528062857 -1.096251442 2.152377157 0.999999653 0.2049891

BLÅ_0 RØD_7 0.69 -0.838590881 2.218590881 0.999241643 0.078844094

BLÅ_0 GRÅ_3 -0.229075843 -1.918453632 1.460301946 1 0.596592068

BLÅ_0 HVID_3 0.199495585 -1.489882204 1.888873374 1 0.644807371

BLÅ_0 SORT 0.970924157 -0.718453632 2.660301946 0.960513729 0.025506731

BLÅ_12a BLÅ_12b 0.07312248 -1.706132919 1.852377879 1 0.872517427

BLÅ_12a BLÅ_3a -0.632096652 -2.298061078 1.033867775 0.999982697 0.139288916

BLÅ_12a BLÅ_3b 0.14812248 -1.631132919 1.927377879 1 0.745166458

BLÅ_12a BLÅ_5 -0.192582082 -1.908833232 1.523669069 1 0.661347986

BLÅ_12a BLÅ_7 -0.250019939 -1.874289806 1.374249928 1 0.547964569

BLÅ_12a GRÅ_0a -0.541482404 -2.257517536 1.174552729 0.999999845 0.218578634

BLÅ_12a GRÅ_0b -0.671253408 -2.387444278 1.044937463 0.999964736 0.127536251

BLÅ_12a GRÅ_12 -0.389457082 -2.105708232 1.326794069 1 0.375949548

BLÅ_12a GRÅ_5 -0.998975546 -2.711794584 0.713843492 0.952398352 0.023426423

BLÅ_12a GRÅ_7 -0.497519939 -2.121789806 1.126749928 0.999999932 0.232326389

BLÅ_12a GRØN_0 -0.947519939 -2.571789806 0.676749928 0.952279329 0.023399164

BLÅ_12a GRØN_12 0.318885959 -1.382565384 2.020337302 1 0.464485699

BLÅ_12a GRØN_3 0.049142596 -1.666892536 1.765177729 1 0.910971005

BLÅ_12a GRØN_5 -0.028125 -1.73714156 1.68089156 1 0.948766502

BLÅ_12a GRØN_7 -0.427503408 -2.143694278 1.288687463 1 0.331166377

BLÅ_12a GUL_0 -0.422519939 -2.046789806 1.201749928 0.999999999 0.310243849

BLÅ_12a GUL_12a 0.415979623 -1.363275776 2.195235022 1 0.361688118

BLÅ_12a GUL_12b 0.247940841 -1.46770765 1.963589333 1 0.572682455

BLÅ_12a GUL_3a 0.039767596 -1.676267536 1.755802729 1 0.927903138

BLÅ_12a GUL_3b 0.22312248 -1.556132919 2.002377879 1 0.624467912

BLÅ_12a GUL_5 -0.31875 -2.02776656 1.39026656 1 0.466688611

BLÅ_12a GUL_7 -0.305628408 -2.021819278 1.410562463 1 0.487037792

BLÅ_12a HVID_0a -1.223763318 -2.889727745 0.442201108 0.605004249 0.004418855

BLÅ_12a HVID_0b -0.061434159 -1.77708265 1.654214333 1 0.888809056

BLÅ_12a HVID_12 -0.220707082 -1.936958232 1.495544069 1 0.615668376

BLÅ_12a HVID_5 -1.120850546 -2.833669584 0.591968492 0.834682245 0.011091614

BLÅ_12a HVID_7 -0.962519939 -2.586789806 0.661749928 0.942143337 0.021313099

BLÅ_12a JIS_3 0.172899454 -1.539919584 1.885718492 1 0.693519549

BLÅ_12a RØD_0 -0.782519939 -2.406789806 0.841749928 0.997420564 0.060801349

BLÅ_12a RØD_12 -0.09375 -1.80276656 1.61526656 1 0.830408343

BLÅ_12a RØD_3 -0.357096652 -2.023061078 1.308867775 1 0.402937857

BLÅ_12a RØD_5 -0.276957082 -1.993208232 1.439294069 1 0.52878947

BLÅ_12a RØD_7 -0.115019939 -1.739289806 1.509249928 1 0.782184896

BLÅ_12a GRÅ_3 -1.034095782 -2.805264552 0.737072988 0.951768328 0.023282408

BLÅ_12a HVID_3 -0.605524354 -2.376693124 1.165644416 0.999998697 0.182639323

BLÅ_12a SORT 0.165904218 -1.605264552 1.937072988 1 0.714597172

BLÅ_12b BLÅ_3a -0.705219132 -2.431363795 1.020925531 0.999905599 0.111507903

BLÅ_12b BLÅ_3b 0.075 -1.752015553 1.902015553 1 0.872661256

BLÅ_12b BLÅ_5 -0.265704562 -2.040305007 1.508895884 1 0.558933917

BLÅ_12b BLÅ_7 -0.323142419 -2.012599047 1.366314209 1 0.4554058

BLÅ_12b GRÅ_0a -0.614604884 -2.389227471 1.160017704 0.999998182 0.17702598

BLÅ_12b GRÅ_0b -0.744375888 -2.518915645 1.03016387 0.999832264 0.102301172

BLÅ_12b GRÅ_12 -0.462579562 -2.237180007 1.312020884 0.999999999 0.30924625

100 |

BLÅ_12b GRÅ_5 -1.072098026 -2.847113987 0.702917936 0.92761084 0.018962107

BLÅ_12b GRÅ_7 -0.570642419 -2.260099047 1.118814209 0.999999048 0.187939522

BLÅ_12b GRØN_0 -1.020642419 -2.710099047 0.668814209 0.927430263 0.018936563

BLÅ_12b GRØN_12 0.245763479 -1.518889364 2.010416323 1 0.586669061

BLÅ_12b GRØN_3 -0.023979884 -1.798602471 1.750642704 1 0.957922771

BLÅ_12b GRØN_5 -0.10124748 -1.880502879 1.678007919 1 0.824177704

BLÅ_12b GRØN_7 -0.500625888 -2.275165645 1.27391387 0.999999993 0.271204815

BLÅ_12b GUL_0 -0.495642419 -2.185099047 1.193814209 0.99999998 0.252584042

BLÅ_12b GUL_12a 0.342857143 -1.48415841 2.169872696 1 0.463950882

BLÅ_12b GUL_12b 0.174818362 -1.60017846 1.949815183 1 0.700606472

BLÅ_12b GUL_3a -0.033354884 -1.807977471 1.741267704 1 0.941498069

BLÅ_12b GUL_3b 0.15 -1.677015553 1.977015553 1 0.748562676

BLÅ_12b GUL_5 -0.39187248 -2.171127879 1.387382919 1 0.390138674

BLÅ_12b GUL_7 -0.378750888 -2.153290645 1.39578887 1 0.404940034

BLÅ_12b HVID_0a -1.296885798 -3.023030461 0.429258865 0.55084816 0.003608392

BLÅ_12b HVID_0b -0.134556638 -1.90955346 1.640440183 1 0.76725654

BLÅ_12b HVID_12 -0.293829562 -2.068430007 1.480770884 1 0.518122679

BLÅ_12b HVID_5 -1.193973026 -2.968988987 0.581042936 0.789634863 0.009058973

BLÅ_12b HVID_7 -1.035642419 -2.725099047 0.653814209 0.914481877 0.017269961

BLÅ_12b JIS_3 0.099776974 -1.675238987 1.874792936 1 0.82628311

BLÅ_12b RØD_0 -0.855642419 -2.545099047 0.833814209 0.993890563 0.048796197

BLÅ_12b RØD_12 -0.16687248 -1.946127879 1.612382919 1 0.714251624

BLÅ_12b RØD_3 -0.430219132 -2.156363795 1.295925531 1 0.330901914

BLÅ_12b RØD_5 -0.350079562 -2.124680007 1.424520884 1 0.44139966

BLÅ_12b RØD_7 -0.188142419 -1.877599047 1.501314209 1 0.663749364

BLÅ_12b GRÅ_3 -1.107218262 -2.943227112 0.728790587 0.928890439 0.019145628

BLÅ_12b HVID_3 -0.678646834 -2.514655683 1.157362016 0.999990765 0.149736696

BLÅ_12b SORT 0.092781738 -1.743227112 1.928790587 1 0.843587857

BLÅ_3a BLÅ_3b 0.780219132 -0.945925531 2.506363795 0.99922144 0.078445213

BLÅ_3a BLÅ_5 0.43951457 -1.226549557 2.105578698 0.999999999 0.303470802

BLÅ_3a BLÅ_7 0.382076713 -1.189931965 1.954085391 1 0.343026243

BLÅ_3a GRÅ_0a 0.090614248 -1.571587267 1.752815764 1 0.831448127

BLÅ_3a GRÅ_0b -0.039156756 -1.705169838 1.626856327 1 0.926881994

BLÅ_3a GRÅ_12 0.24263957 -1.423424557 1.908703698 1 0.569715215

BLÅ_3a GRÅ_5 -0.366878894 -2.033132645 1.299374858 1 0.390275484

BLÅ_3a GRÅ_7 0.134576713 -1.437431965 1.706585391 1 0.738210203

BLÅ_3a GRØN_0 -0.315423287 -1.887431965 1.256585391 1 0.433629193

BLÅ_3a GRØN_12 0.950982611 -0.701395143 2.603360365 0.959797588 0.025255986

BLÅ_3a GRØN_3 0.681239248 -0.980962267 2.343440764 0.999898801 0.110388299

BLÅ_3a GRØN_5 0.603971652 -1.061992775 2.269936078 0.999994257 0.157682221

BLÅ_3a GRØN_7 0.204593244 -1.461419838 1.870606327 1 0.631661288

BLÅ_3a GUL_0 0.209576713 -1.362431965 1.781585391 1 0.60277659

BLÅ_3a GUL_12a 1.048076275 -0.678068388 2.774220938 0.923140716 0.018348429

BLÅ_3a GUL_12b 0.880037493 -0.785519783 2.54559477 0.987986245 0.039881706

BLÅ_3a GUL_3a 0.671864248 -0.990337267 2.334065764 0.999925583 0.115352962

BLÅ_3a GUL_3b 0.855219132 -0.870925531 2.581363795 0.995782833 0.053876292

BLÅ_3a GUL_5 0.313346652 -1.352617775 1.979311078 1 0.462933583

BLÅ_3a GUL_7 0.326468244 -1.339544838 1.992481327 1 0.444445638

| 101

BLÅ_3a HVID_0a -0.591666667 -2.202942932 1.019609598 0.999992135 0.152421845

BLÅ_3a HVID_0b 0.570662493 -1.094894783 2.23621977 0.999998621 0.18168754

BLÅ_3a HVID_12 0.41138957 -1.254674557 2.077453698 1 0.335403017

BLÅ_3a HVID_5 -0.488753894 -2.155007645 1.177499858 0.99999998 0.252662699

BLÅ_3a HVID_7 -0.330423287 -1.902431965 1.241585391 1 0.412109776

BLÅ_3a JIS_3 0.804996106 -0.861257645 2.471249858 0.997288086 0.060085955

BLÅ_3a RØD_0 -0.150423287 -1.722431965 1.421585391 1 0.708724168

BLÅ_3a RØD_12 0.538346652 -1.127617775 2.204311078 0.999999705 0.207717089

BLÅ_3a RØD_3 0.275 -1.336276265 1.886276265 1 0.505335094

BLÅ_3a RØD_5 0.35513957 -1.310924557 2.021203698 1 0.405545409

BLÅ_3a RØD_7 0.517076713 -1.054931965 2.089085391 0.999999526 0.199721724

BLÅ_3a GRÅ_3 -0.40199913 -2.128028736 1.324030475 1 0.363517567

BLÅ_3a HVID_3 0.026572298 -1.699457308 1.752601904 1 0.952067903

BLÅ_3a SORT 0.79800087 -0.928028736 2.524030475 0.998795033 0.07189738

BLÅ_3b BLÅ_5 -0.340704562 -2.115305007 1.433895884 1 0.453713907

BLÅ_3b BLÅ_7 -0.398142419 -2.087599047 1.291314209 1 0.357870109

BLÅ_3b GRÅ_0a -0.689604884 -2.464227471 1.085017704 0.99996967 0.13001472

BLÅ_3b GRÅ_0b -0.819375888 -2.593915645 0.95516387 0.998824147 0.072260399

BLÅ_3b GRÅ_12 -0.537579562 -2.312180007 1.237020884 0.99999995 0.237496593

BLÅ_3b GRÅ_5 -1.147098026 -2.922113987 0.627917936 0.852900454 0.012122075

BLÅ_3b GRÅ_7 -0.645642419 -2.335099047 1.043814209 0.999979482 0.136465096

BLÅ_3b GRØN_0 -1.095642419 -2.785099047 0.593814209 0.847939924 0.01182645

BLÅ_3b GRØN_12 0.170763479 -1.593889364 1.935416323 1 0.705573436

BLÅ_3b GRØN_3 -0.098979884 -1.873602471 1.675642704 1 0.827611311

BLÅ_3b GRØN_5 -0.17624748 -1.955502879 1.603007919 1 0.698965623

BLÅ_3b GRØN_7 -0.575625888 -2.350165645 1.19891387 0.999999673 0.205988565

BLÅ_3b GUL_0 -0.570642419 -2.260099047 1.118814209 0.999999048 0.187939522

BLÅ_3b GUL_12a 0.267857143 -1.55915841 2.094872696 1 0.567142646

BLÅ_3b GUL_12b 0.099818362 -1.67517846 1.874815183 1 0.826210369

BLÅ_3b GUL_3a -0.108354884 -1.882977471 1.666267704 1 0.81157857

BLÅ_3b GUL_3b 0.075 -1.752015553 1.902015553 1 0.872661256

BLÅ_3b GUL_5 -0.46687248 -2.246127879 1.312382919 0.999999999 0.306050041

BLÅ_3b GUL_7 -0.453750888 -2.228290645 1.32078887 1 0.318537123

BLÅ_3b HVID_0a -1.371885798 -3.098030461 0.354258865 0.413580281 0.002098105

BLÅ_3b HVID_0b -0.209556638 -1.98455346 1.565440183 1 0.644885456

BLÅ_3b HVID_12 -0.368829562 -2.143430007 1.405770884 1 0.417357386

BLÅ_3b HVID_5 -1.268973026 -3.043988987 0.506042936 0.667385437 0.005579706

BLÅ_3b HVID_7 -1.110642419 -2.800099047 0.578814209 0.82765566 0.010731153

BLÅ_3b JIS_3 0.024776974 -1.750238987 1.799792936 1 0.956535115

BLÅ_3b RØD_0 -0.930642419 -2.620099047 0.758814209 0.977774295 0.032219755

BLÅ_3b RØD_12 -0.24187248 -2.021127879 1.537382919 1 0.595665033

BLÅ_3b RØD_3 -0.505219132 -2.231363795 1.220925531 0.999999981 0.253695035

BLÅ_3b RØD_5 -0.425079562 -2.199680007 1.349520884 1 0.350038427

BLÅ_3b RØD_7 -0.263142419 -1.952599047 1.426314209 1 0.54322476

BLÅ_3b GRÅ_3 -1.182218262 -3.018227112 0.653790587 0.857912675 0.01243328

BLÅ_3b HVID_3 -0.753646834 -2.589655683 1.082362016 0.999895269 0.109834353

BLÅ_3b SORT 0.017781738 -1.818227112 1.853790587 1 0.969834933

BLÅ_5 BLÅ_7 -0.057437857 -1.681752157 1.566876442 1 0.890187721

102 |

BLÅ_5 GRÅ_0a -0.348900322 -2.065065788 1.367265144 1 0.427581873

BLÅ_5 GRÅ_0b -0.478671326 -2.190607363 1.23326471 0.999999995 0.275478811

BLÅ_5 GRÅ_12 -0.196875 -1.90589156 1.51214156 1 0.652920844

BLÅ_5 GRÅ_5 -0.806393464 -2.518710913 0.905923985 0.998296205 0.066794734

BLÅ_5 GRÅ_7 -0.304937857 -1.929252157 1.319376442 1 0.46377535

BLÅ_5 GRØN_0 -0.754937857 -2.379252157 0.869376442 0.998668563 0.070418679

BLÅ_5 GRØN_12 0.511468041 -1.191351089 2.214287171 0.99999996 0.241422077

BLÅ_5 GRØN_3 0.241724678 -1.474440788 1.957890144 1 0.582453041

BLÅ_5 GRØN_5 0.164457082 -1.551794069 1.880708232 1 0.708332219

BLÅ_5 GRØN_7 -0.234921326 -1.946857363 1.47701471 1 0.592195

BLÅ_5 GUL_0 -0.229937857 -1.854252157 1.394376442 1 0.580559289

BLÅ_5 GUL_12a 0.608561704 -1.166038741 2.383162149 0.999998589 0.181302361

BLÅ_5 GUL_12b 0.440522923 -1.275156661 2.156202507 1 0.316534188

BLÅ_5 GUL_3a 0.232349678 -1.483815788 1.948515144 1 0.597157297

BLÅ_5 GUL_3b 0.415704562 -1.358895884 2.190305007 1 0.360750163

BLÅ_5 GUL_5 -0.126167918 -1.842419069 1.590083232 1 0.774101303

BLÅ_5 GUL_7 -0.113046326 -1.824982363 1.59888971 1 0.796548653

BLÅ_5 HVID_0a -1.031181237 -2.697245365 0.634882891 0.905022393 0.016230265

BLÅ_5 HVID_0b 0.131147923 -1.584531661 1.846827507 1 0.765366847

BLÅ_5 HVID_12 -0.028125 -1.73714156 1.68089156 1 0.948766502

BLÅ_5 HVID_5 -0.928268464 -2.640585913 0.784048985 0.982345661 0.035033903

BLÅ_5 HVID_7 -0.769937857 -2.394252157 0.854376442 0.998079034 0.065047497

BLÅ_5 JIS_3 0.365481536 -1.346835913 2.077798985 1 0.404926065

BLÅ_5 RØD_0 -0.589937857 -2.214252157 1.034376442 0.999993994 0.156934877

BLÅ_5 RØD_12 0.098832082 -1.617419069 1.815083232 1 0.822107002

BLÅ_5 RØD_3 -0.16451457 -1.830578698 1.501549557 1 0.699868984

BLÅ_5 RØD_5 -0.084375 -1.79339156 1.62464156 1 0.847145635

BLÅ_5 RØD_7 0.077562143 -1.546752157 1.701876442 1 0.852101404

BLÅ_5 GRÅ_3 -0.841513701 -2.620721446 0.937694045 0.998154899 0.065631513

BLÅ_5 HVID_3 -0.412942272 -2.192150018 1.366265474 1 0.365186167

BLÅ_5 SORT 0.358486299 -1.420721446 2.137694045 1 0.431715401

BLÅ_7 GRÅ_0a -0.291462465 -1.915680836 1.332755907 1 0.483715121

BLÅ_7 GRÅ_0b -0.421233469 -2.045508195 1.203041258 0.999999999 0.311717961

BLÅ_7 GRÅ_12 -0.139437143 -1.763751442 1.484877157 1 0.737516423

BLÅ_7 GRÅ_5 -0.748955607 -2.373417558 0.875506345 0.998857302 0.072686273

BLÅ_7 GRÅ_7 -0.2475 -1.776090881 1.281090881 1 0.527411756

BLÅ_7 GRØN_0 -0.6975 -2.226090881 0.831090881 0.99906355 0.075663026

BLÅ_7 GRØN_12 0.568905898 -1.041599819 2.179411616 0.999996995 0.168492261

BLÅ_7 GRØN_3 0.299162535 -1.325055836 1.923380907 1 0.472249131

BLÅ_7 GRØN_5 0.221894939 -1.402374928 1.846164806 1 0.593850274

BLÅ_7 GRØN_7 -0.177483469 -1.801758195 1.446791258 1 0.66969031

BLÅ_7 GUL_0 -0.1725 -1.701090881 1.356090881 1 0.659544355

BLÅ_7 GUL_12a 0.665999562 -1.023457066 2.35545619 0.999957765 0.124579178

BLÅ_7 GUL_12b 0.49796078 -1.125980201 2.121901762 0.99999993 0.231819811

BLÅ_7 GUL_3a 0.289787535 -1.334430836 1.914005907 1 0.486229181

BLÅ_7 GUL_3b 0.473142419 -1.216314209 2.162599047 0.999999995 0.274711767

BLÅ_7 GUL_5 -0.068730061 -1.692999928 1.555539806 1 0.868775821

BLÅ_7 GUL_7 -0.055608469 -1.679883195 1.568666258 1 0.893661414

| 103

BLÅ_7 HVID_0a -0.973743379 -2.545752057 0.598265298 0.904205034 0.016146788

BLÅ_7 HVID_0b 0.18858578 -1.435355201 1.812526762 1 0.650303841

BLÅ_7 HVID_12 0.029312857 -1.595001442 1.653627157 1 0.943826007

BLÅ_7 HVID_5 -0.870830607 -2.495292558 0.753631345 0.985041008 0.037098893

BLÅ_7 HVID_7 -0.7125 -2.241090881 0.816090881 0.998594164 0.069616693

BLÅ_7 JIS_3 0.422919393 -1.201542558 2.047381345 0.999999999 0.309844563

BLÅ_7 RØD_0 -0.5325 -2.061090881 0.996090881 0.999997889 0.174510897

BLÅ_7 RØD_12 0.156269939 -1.467999928 1.780539806 1 0.707212246

BLÅ_7 RØD_3 -0.107076713 -1.679085391 1.464931965 1 0.790287219

BLÅ_7 RØD_5 -0.026937143 -1.651251442 1.597377157 1 0.948372056

BLÅ_7 RØD_7 0.135 -1.393590881 1.663590881 1 0.730247476

BLÅ_7 GRÅ_3 -0.784075843 -2.473453632 0.905301946 0.998703454 0.070811016

BLÅ_7 HVID_3 -0.355504415 -2.044882204 1.333873374 1 0.411568928

BLÅ_7 SORT 0.415924157 -1.273453632 2.105301946 1 0.336818468

GRÅ_0a GRÅ_0b -0.129771004 -1.845876295 1.586334286 1 0.767814623

GRÅ_0a GRÅ_12 0.152025322 -1.564140144 1.868190788 1 0.729461703

GRÅ_0a GRÅ_5 -0.457493142 -2.173920716 1.258934432 0.999999999 0.298496756

GRÅ_0a GRÅ_7 0.043962465 -1.580255907 1.668180836 1 0.915834573

GRÅ_0a GRØN_0 -0.406037535 -2.030255907 1.218180836 1 0.329446722

GRÅ_0a GRØN_12 0.860368363 -0.842292297 2.563029023 0.994117076 0.049249834

GRÅ_0a GRØN_3 0.590625 -1.11839156 2.29964156 0.999998278 0.177946728

GRÅ_0a GRØN_5 0.513357404 -1.202677729 2.229392536 0.999999965 0.243380699

GRÅ_0a GRØN_7 0.113978996 -1.602126295 1.830084286 1 0.795394558

GRÅ_0a GUL_0 0.118962465 -1.505255907 1.743180836 1 0.77491095

GRÅ_0a GUL_12a 0.957462026 -0.817160562 2.732084614 0.983542071 0.035904122

GRÅ_0a GUL_12b 0.789423245 -0.926114221 2.504960711 0.998898653 0.073234737

GRÅ_0a GUL_3a 0.58125 -1.12776656 2.29026656 0.999998861 0.184908713

GRÅ_0a GUL_3b 0.764604884 -1.010017704 2.539227471 0.999704562 0.093368115

GRÅ_0a GUL_5 0.222732404 -1.493302729 1.938767536 1 0.612388567

GRÅ_0a GUL_7 0.235853996 -1.480251295 1.951959286 1 0.591628094

GRÅ_0a HVID_0a -0.682280915 -2.344482431 0.979920601 0.999895341 0.109847308

GRÅ_0a HVID_0b 0.480048245 -1.235489221 2.195585711 0.999999995 0.275108763

GRÅ_0a HVID_12 0.320775322 -1.395390144 2.036940788 1 0.465725808

GRÅ_0a HVID_5 -0.579368142 -2.295795716 1.137059432 0.999999064 0.188230758

GRÅ_0a HVID_7 -0.421037535 -2.045255907 1.203180836 0.999999999 0.311925928

GRÅ_0a JIS_3 0.714381858 -1.002045716 2.430809432 0.999858333 0.104992597

GRÅ_0a RØD_0 -0.241037535 -1.865255907 1.383180836 1 0.562413933

GRÅ_0a RØD_12 0.447732404 -1.268302729 2.163767536 0.999999999 0.308793944

GRÅ_0a RØD_3 0.184385752 -1.477815764 1.846587267 1 0.664979739

GRÅ_0a RØD_5 0.264525322 -1.451640144 1.980690788 1 0.547419053

GRÅ_0a RØD_7 0.426462465 -1.197755907 2.050680836 0.999999999 0.305744237

GRÅ_0a GRÅ_3 -0.492613379 -2.267056712 1.281829954 0.999999996 0.278906413

GRÅ_0a HVID_3 -0.06404195 -1.838485283 1.710401383 1 0.887935624

GRÅ_0a SORT 0.707386621 -1.067056712 2.481829954 0.999945475 0.120406931

GRÅ_0b GRÅ_12 0.281796326 -1.43013971 1.993732363 1 0.520571137

GRÅ_0b GRÅ_5 -0.327722138 -2.03999187 1.384547594 1 0.455108835

GRÅ_0b GRÅ_7 0.173733469 -1.450541258 1.798008195 1 0.676266595

GRÅ_0b GRØN_0 -0.276266531 -1.900541258 1.348008195 1 0.50679657

104 |

GRÅ_0b GRØN_12 0.990139367 -0.712625061 2.692903795 0.954154516 0.02379523

GRÅ_0b GRØN_3 0.720396004 -0.995709286 2.436501295 0.999829584 0.102048603

GRÅ_0b GRØN_5 0.643128408 -1.073062463 2.359319278 0.999987133 0.144208376

GRÅ_0b GRØN_7 0.24375 -1.46526656 1.95276656 1 0.577714725

GRÅ_0b GUL_0 0.248733469 -1.375541258 1.873008195 1 0.5500256

GRÅ_0b GUL_12a 1.08723303 -0.687306727 2.861772788 0.914971834 0.017327493

GRÅ_0b GUL_12b 0.919194249 -0.796428499 2.634816997 0.985163451 0.037201638

GRÅ_0b GUL_3a 0.711021004 -1.005084286 2.427126295 0.999871598 0.106564767

GRÅ_0b GUL_3b 0.894375888 -0.88016387 2.668915645 0.994364899 0.04989278

GRÅ_0b GUL_5 0.352503408 -1.363687463 2.068694278 1 0.422837838

GRÅ_0b GUL_7 0.365625 -1.34339156 2.07464156 1 0.40383673

GRÅ_0b HVID_0a -0.552509911 -2.218522993 1.113503172 0.999999411 0.196054492

GRÅ_0b HVID_0b 0.609819249 -1.105803499 2.325441997 0.999996486 0.165931123

GRÅ_0b HVID_12 0.450546326 -1.26138971 2.162482363 0.999999999 0.304615676

GRÅ_0b HVID_5 -0.449597138 -2.16186687 1.262672594 0.999999999 0.305727614

GRÅ_0b HVID_7 -0.291266531 -1.915541258 1.333008195 1 0.484023984

GRÅ_0b JIS_3 0.844152862 -0.86811687 2.556422594 0.996127981 0.055064434

GRÅ_0b RØD_0 -0.111266531 -1.735541258 1.513008195 1 0.789121753

GRÅ_0b RØD_12 0.577503408 -1.138687463 2.293694278 0.999999137 0.189591142

GRÅ_0b RØD_3 0.314156756 -1.351856327 1.980169838 1 0.461792042

GRÅ_0b RØD_5 0.394296326 -1.31763971 2.106232363 1 0.368853196

GRÅ_0b RØD_7 0.556233469 -1.068041258 2.180508195 0.999998639 0.181910635

GRÅ_0b GRÅ_3 -0.362842375 -2.141973821 1.416289072 1 0.426127005

GRÅ_0b HVID_3 0.065729054 -1.713402393 1.844860501 1 0.885304599

GRÅ_0b SORT 0.837157625 -0.941973821 2.616289072 0.998322461 0.067020048

GRÅ_12 GRÅ_5 -0.609518464 -2.321835913 1.102798985 0.999996358 0.16532688

GRÅ_12 GRÅ_7 -0.108062857 -1.732377157 1.516251442 1 0.795060304

GRÅ_12 GRØN_0 -0.558062857 -2.182377157 1.066251442 0.99999852 0.180493033

GRÅ_12 GRØN_12 0.708343041 -0.994476089 2.411162171 0.999859948 0.105109665

GRÅ_12 GRØN_3 0.438599678 -1.277565788 2.154765144 1 0.318784322

GRÅ_12 GRØN_5 0.361332082 -1.354919069 2.077583232 1 0.411345419

GRÅ_12 GRØN_7 -0.038046326 -1.749982363 1.67388971 1 0.930851018

GRÅ_12 GUL_0 -0.033062857 -1.657377157 1.591251442 1 0.936653996

GRÅ_12 GUL_12a 0.805436704 -0.969163741 2.580037149 0.999156006 0.077225956

GRÅ_12 GUL_12b 0.637397923 -1.078281661 2.353077507 0.999989555 0.147682761

GRÅ_12 GUL_3a 0.429224678 -1.286940788 2.145390144 1 0.329219431

GRÅ_12 GUL_3b 0.612579562 -1.162020884 2.387180007 0.999998329 0.178447085

GRÅ_12 GUL_5 0.070707082 -1.645544069 1.786958232 1 0.872205902

GRÅ_12 GUL_7 0.083828674 -1.628107363 1.79576471 1 0.848378989

GRÅ_12 HVID_0a -0.834306237 -2.500370365 0.831757891 0.994942987 0.051370994

GRÅ_12 HVID_0b 0.328022923 -1.387656661 2.043702507 1 0.455591126

GRÅ_12 HVID_12 0.16875 -1.54026656 1.87776656 1 0.699879304

GRÅ_12 HVID_5 -0.731393464 -2.443710913 0.980923985 0.999753574 0.096215666

GRÅ_12 HVID_7 -0.573062857 -2.197377157 1.051251442 0.999997089 0.169097015

GRÅ_12 JIS_3 0.562356536 -1.149960913 2.274673985 0.999999545 0.200416012

GRÅ_12 RØD_0 -0.393062857 -2.017377157 1.231251442 1 0.345137531

GRÅ_12 RØD_12 0.295707082 -1.420544069 2.011958232 1 0.501289905

GRÅ_12 RØD_3 0.03236043 -1.633703698 1.698424557 1 0.939547708

| 105

GRÅ_12 RØD_5 0.1125 -1.59651656 1.82151656 1 0.797172103

GRÅ_12 RØD_7 0.274437143 -1.349877157 1.898751442 1 0.509622434

GRÅ_12 GRÅ_3 -0.644638701 -2.423846446 1.134569045 0.999994341 0.157930038

GRÅ_12 HVID_3 -0.216067272 -1.995275018 1.563140474 1 0.635451262

GRÅ_12 SORT 0.555361299 -1.223846446 2.334569045 0.999999885 0.223571417

GRÅ_5 GRÅ_7 0.501455607 -1.123006345 2.125917558 0.999999915 0.228717605

GRÅ_5 GRØN_0 0.051455607 -1.573006345 1.675917558 1 0.90157182

GRÅ_5 GRØN_12 1.317861505 -0.385194502 3.020917511 0.478564236 0.00271895

GRÅ_5 GRØN_3 1.048118142 -0.668309432 2.764545716 0.918070847 0.017700209

GRÅ_5 GRØN_5 0.970850546 -0.741968492 2.683669584 0.967082624 0.027582109

GRÅ_5 GRØN_7 0.571472138 -1.140797594 2.28374187 0.999999304 0.193226608

GRÅ_5 GUL_0 0.576455607 -1.048006345 2.200917558 0.99999663 0.166634699

GRÅ_5 GUL_12a 1.414955168 -0.360060793 3.18997113 0.406406097 0.00203515

GRÅ_5 GUL_12b 1.246916387 -0.46905951 2.962892284 0.630361497 0.004856786

GRÅ_5 GUL_3a 1.038743142 -0.677684432 2.755170716 0.925970202 0.018731471

GRÅ_5 GUL_3b 1.222098026 -0.552917936 2.997113987 0.746342852 0.007573846

GRÅ_5 GUL_5 0.680225546 -1.032593492 2.393044584 0.99994999 0.12181687

GRÅ_5 GUL_7 0.693347138 -1.018922594 2.40561687 0.999922536 0.114702608

GRÅ_5 HVID_0a -0.224787773 -1.891041524 1.441465978 1 0.598465012

GRÅ_5 HVID_0b 0.937541387 -0.77843451 2.653517284 0.980227769 0.033640344

GRÅ_5 HVID_12 0.778268464 -0.934048985 2.490585913 0.999132412 0.076810935

GRÅ_5 HVID_5 -0.121875 -1.83089156 1.58714156 1 0.780687619

GRÅ_5 HVID_7 0.036455607 -1.588006345 1.660917558 1 0.930176002

GRÅ_5 JIS_3 1.171875 -0.53714156 2.88089156 0.754244417 0.007819587

GRÅ_5 RØD_0 0.216455607 -1.408006345 1.840917558 1 0.602967352

GRÅ_5 RØD_12 0.905225546 -0.807593492 2.618044584 0.987942094 0.039834805

GRÅ_5 RØD_3 0.641878894 -1.024374858 2.308132645 0.999975246 0.133363164

GRÅ_5 RØD_5 0.722018464 -0.990298985 2.434335913 0.999812519 0.100534434

GRÅ_5 RØD_7 0.883955607 -0.740506345 2.508417558 0.98135214 0.034358748

GRÅ_5 GRÅ_3 -0.035120237 -1.810579931 1.740339457 1 0.938436773

GRÅ_5 HVID_3 0.393451192 -1.382008502 2.168910886 1 0.387223108

GRÅ_5 SORT 1.164879763 -0.610579931 2.940339457 0.830703344 0.010884952

GRÅ_7 GRØN_0 -0.45 -1.978590881 1.078590881 0.999999978 0.250951621

GRÅ_7 GRØN_12 0.816405898 -0.794099819 2.426911616 0.99379707 0.048527989

GRÅ_7 GRØN_3 0.546662535 -1.077555836 2.170880907 0.999999132 0.189504853

GRÅ_7 GRØN_5 0.469394939 -1.154874928 2.093664806 0.999999987 0.259728907

GRÅ_7 GRØN_7 0.070016531 -1.554258195 1.694291258 1 0.866343018

GRÅ_7 GUL_0 0.075 -1.453590881 1.603590881 1 0.848080489

GRÅ_7 GUL_12a 0.913499562 -0.775957066 2.60295619 0.983005463 0.03550531

GRÅ_7 GUL_12b 0.74546078 -0.878480201 2.369401762 0.99894881 0.073931268

GRÅ_7 GUL_3a 0.537287535 -1.086930836 2.161505907 0.999999449 0.197184613

GRÅ_7 GUL_3b 0.720642419 -0.968814209 2.410099047 0.999760568 0.096668422

GRÅ_7 GUL_5 0.178769939 -1.445499928 1.803039806 1 0.667439145

GRÅ_7 GUL_7 0.191891531 -1.432383195 1.816166258 1 0.644663003

GRÅ_7 HVID_0a -0.726243379 -2.298252057 0.845765298 0.99881223 0.072111301

GRÅ_7 HVID_0b 0.43608578 -1.187855201 2.060026762 0.999999998 0.294898555

GRÅ_7 HVID_12 0.276812857 -1.347501442 1.901127157 1 0.505967677

GRÅ_7 HVID_5 -0.623330607 -2.247792558 1.001131345 0.999977422 0.134883612

106 |

GRÅ_7 HVID_7 -0.465 -1.993590881 1.063590881 0.999999944 0.235539059

GRÅ_7 JIS_3 0.670419393 -0.954042558 2.294881345 0.999882115 0.107935147

GRÅ_7 RØD_0 -0.285 -1.813590881 1.243590881 1 0.46684269

GRÅ_7 RØD_12 0.403769939 -1.220499928 2.028039806 1 0.332165417

GRÅ_7 RØD_3 0.140423287 -1.431585391 1.712431965 1 0.72728351

GRÅ_7 RØD_5 0.220562857 -1.403751442 1.844877157 1 0.596076767

GRÅ_7 RØD_7 0.3825 -1.146090881 1.911090881 1 0.328982138

GRÅ_7 GRÅ_3 -0.536575843 -2.225953632 1.152801946 0.999999815 0.215574544

GRÅ_7 HVID_3 -0.108004415 -1.797382204 1.581373374 1 0.802890657

GRÅ_7 SORT 0.663424157 -1.025953632 2.352801946 0.999961318 0.126018872

GRØN_0 GRØN_12 1.266405898 -0.344099819 2.876911616 0.439348581 0.002321356

GRØN_0 GRØN_3 0.996662535 -0.627555836 2.620880907 0.91351927 0.017158408

GRØN_0 GRØN_5 0.919394939 -0.704874928 2.543664806 0.967681353 0.027794311

GRØN_0 GRØN_7 0.520016531 -1.104258195 2.144291258 0.99999977 0.211943742

GRØN_0 GUL_0 0.525 -1.003590881 2.053590881 0.999998532 0.180638866

GRØN_0 GUL_12a 1.363499562 -0.325957066 3.05295619 0.377121317 0.001791802

GRØN_0 GUL_12b 1.19546078 -0.428480201 2.819401762 0.599915158 0.004336045

GRØN_0 GUL_3a 0.987287535 -0.636930836 2.611505907 0.922167377 0.018220376

GRØN_0 GUL_3b 1.170642419 -0.518814209 2.860099047 0.733684254 0.007201531

GRØN_0 GUL_5 0.628769939 -0.995499928 2.253039806 0.999972252 0.131480965

GRØN_0 GUL_7 0.641891531 -0.982383195 2.266166258 0.999955303 0.123652489

GRØN_0 HVID_0a -0.276243379 -1.848252057 1.295765298 1 0.492821291

GRØN_0 HVID_0b 0.88608578 -0.737855201 2.510026762 0.980600894 0.033874267

GRØN_0 HVID_12 0.726812857 -0.897501442 2.351127157 0.999362224 0.081474819

GRØN_0 HVID_5 -0.173330607 -1.797792558 1.451131345 1 0.677009667

GRØN_0 HVID_7 -0.015 -1.543590881 1.513590881 1 0.969436623

GRØN_0 JIS_3 1.120419393 -0.504042558 2.744881345 0.742873273 0.007469601

GRØN_0 RØD_0 0.165 -1.363590881 1.693590881 1 0.673466826

GRØN_0 RØD_12 0.853769939 -0.770499928 2.478039806 0.98891321 0.040915013

GRØN_0 RØD_3 0.590423287 -0.981585391 2.162431965 0.99998642 0.143312496

GRØN_0 RØD_5 0.670562857 -0.953751442 2.294877157 0.999881327 0.107828335

GRØN_0 RØD_7 0.8325 -0.696090881 2.361090881 0.981118938 0.034206696

GRØN_0 GRÅ_3 -0.086575843 -1.775953632 1.602801946 1 0.84141128

GRØN_0 HVID_3 0.341995585 -1.347382204 2.031373374 1 0.429545154

GRØN_0 SORT 1.113424157 -0.575953632 2.802801946 0.823667515 0.010534748

GRØN_12 GRØN_3 -0.269743363 -1.972404023 1.432917297 1 0.5363123

GRØN_12 GRØN_5 -0.347010959 -2.048462302 1.354440384 1 0.426074656

GRØN_12 GRØN_7 -0.746389367 -2.449153795 0.956375061 0.999578673 0.087777846

GRØN_12 GUL_0 -0.741405898 -2.351911616 0.869099819 0.998889678 0.073044871

GRØN_12 GUL_12a 0.097093663 -1.66755918 1.861746507 1 0.829896514

GRØN_12 GUL_12b -0.070945118 -1.769146397 1.627256161 1 0.870422072

GRØN_12 GUL_3a -0.279118363 -1.981779023 1.423542297 1 0.522252846

GRØN_12 GUL_3b -0.095763479 -1.860416323 1.668889364 1 0.832191841

GRØN_12 GUL_5 -0.637635959 -2.339087302 1.063815384 0.999987118 0.144125473

GRØN_12 GUL_7 -0.624514367 -2.327278795 1.078250061 0.999992362 0.152843279

GRØN_12 HVID_0a -1.542649278 -3.195027032 0.109728477 0.112058368 0.000313107

GRØN_12 HVID_0b -0.380320118 -2.078521397 1.317881161 1 0.382175319

GRØN_12 HVID_12 -0.539593041 -2.242412171 1.163226089 0.999999826 0.216574409

| 107

GRØN_12 HVID_5 -1.439736505 -3.142792511 0.263319502 0.274355273 0.001074827

GRØN_12 HVID_7 -1.281405898 -2.891911616 0.329099819 0.410906959 0.002061727

GRØN_12 JIS_3 -0.145986505 -1.849042511 1.557069502 1 0.737864708

GRØN_12 RØD_0 -1.101405898 -2.711911616 0.509099819 0.759308171 0.007953048

GRØN_12 RØD_12 -0.412635959 -2.114087302 1.288815384 1 0.344030769

GRØN_12 RØD_3 -0.675982611 -2.328360365 0.976395143 0.999902778 0.110963029

GRØN_12 RØD_5 -0.595843041 -2.298662171 1.106976089 0.999997636 0.172534787

GRØN_12 RØD_7 -0.433905898 -2.044411616 1.176599819 0.999999998 0.29325161

GRØN_12 GRÅ_3 -1.352981742 -3.117405425 0.411441942 0.500900823 0.002968607

GRØN_12 HVID_3 -0.924410313 -2.688833997 0.840013371 0.989463391 0.041522548

GRØN_12 SORT -0.152981742 -1.917405425 1.611441942 1 0.73497245

GRØN_3 GRØN_5 -0.077267596 -1.793302729 1.638767536 1 0.860448327

GRØN_3 GRØN_7 -0.476646004 -2.192751295 1.239459286 0.999999996 0.278676289

GRØN_3 GUL_0 -0.471662535 -2.095880907 1.152555836 0.999999985 0.257424488

GRØN_3 GUL_12a 0.366837026 -1.407785562 2.141459614 1 0.419880644

GRØN_3 GUL_12b 0.198798245 -1.516739221 1.914335711 1 0.650998909

GRØN_3 GUL_3a -0.009375 -1.71839156 1.69964156 1 0.982911663

GRØN_3 GUL_3b 0.173979884 -1.600642704 1.948602471 1 0.701913392

GRØN_3 GUL_5 -0.367892596 -2.083927729 1.348142536 1 0.402856694

GRØN_3 GUL_7 -0.354771004 -2.070876295 1.361334286 1 0.419840786

GRØN_3 HVID_0a -1.272905915 -2.935107431 0.389295601 0.504159426 0.003020189

GRØN_3 HVID_0b -0.110576755 -1.826114221 1.604960711 1 0.801307206

GRØN_3 HVID_12 -0.269849678 -1.986015144 1.446315788 1 0.53939154

GRØN_3 HVID_5 -1.169993142 -2.886420716 0.546434432 0.765512194 0.008186975

GRØN_3 HVID_7 -1.011662535 -2.635880907 0.612555836 0.898391312 0.015572623

GRØN_3 JIS_3 0.123756858 -1.592670716 1.840184432 1 0.778325811

GRØN_3 RØD_0 -0.831662535 -2.455880907 0.792555836 0.992687686 0.046381089

GRØN_3 RØD_12 -0.142892596 -1.858927729 1.573142536 1 0.745109511

GRØN_3 RØD_3 -0.406239248 -2.068440764 1.255962267 1 0.340360203

GRØN_3 RØD_5 -0.326099678 -2.042265144 1.390065788 1 0.458362141

GRØN_3 RØD_7 -0.164162535 -1.788380907 1.460055836 1 0.693152765

GRØN_3 GRÅ_3 -1.083238379 -2.857681712 0.691204954 0.918329008 0.017732434

GRØN_3 HVID_3 -0.65466695 -2.429110283 1.119776383 0.999991175 0.150495814

GRØN_3 SORT 0.116761621 -1.657681712 1.891204954 1 0.797248746

GRØN_5 GRØN_7 -0.399378408 -2.115569278 1.316812463 1 0.363911641

GRØN_5 GUL_0 -0.394394939 -2.018664806 1.229874928 1 0.343494699

GRØN_5 GUL_12a 0.444104623 -1.335150776 2.223360022 1 0.330195441

GRØN_5 GUL_12b 0.276065841 -1.43958265 1.991714333 1 0.529971318

GRØN_5 GUL_3a 0.067892596 -1.648142536 1.783927729 1 0.877236022

GRØN_5 GUL_3b 0.25124748 -1.528007919 2.030502879 1 0.581495795

GRØN_5 GUL_5 -0.290625 -1.99964156 1.41839156 1 0.506877195

GRØN_5 GUL_7 -0.277503408 -1.993694278 1.438687463 1 0.527962947

GRØN_5 HVID_0a -1.195638318 -2.861602745 0.470326108 0.658708852 0.005400477

GRØN_5 HVID_0b -0.033309159 -1.74895765 1.682339333 1 0.939573704

GRØN_5 HVID_12 -0.192582082 -1.908833232 1.523669069 1 0.661347986

GRØN_5 HVID_5 -1.092725546 -2.805544584 0.620093492 0.870175721 0.013256676

GRØN_5 HVID_7 -0.934394939 -2.558664806 0.689874928 0.960024868 0.025368481

GRØN_5 JIS_3 0.201024454 -1.511794584 1.913843492 1 0.646834387

108 |

GRØN_5 RØD_0 -0.754394939 -2.378664806 0.869874928 0.998685866 0.070612117

GRØN_5 RØD_12 -0.065625 -1.77464156 1.64339156 1 0.880821597

GRØN_5 RØD_3 -0.328971652 -1.994936078 1.336992775 1 0.440951417

GRØN_5 RØD_5 -0.248832082 -1.965083232 1.467419069 1 0.571439111

GRØN_5 RØD_7 -0.086894939 -1.711164806 1.537374928 1 0.834545583

GRØN_5 GRÅ_3 -1.005970782 -2.777139552 0.765197988 0.966178481 0.027270358

GRØN_5 HVID_3 -0.577399354 -2.348568124 1.193769416 0.999999627 0.203741767

GRØN_5 SORT 0.194029218 -1.577139552 1.965197988 1 0.668896442

GRØN_7 GUL_0 0.004983469 -1.619291258 1.629258195 1 0.990441929

GRØN_7 GUL_12a 0.84348303 -0.931056727 2.618022788 0.99797879 0.064309466

GRØN_7 GUL_12b 0.675444249 -1.040178499 2.391066997 0.999958993 0.125062339

GRØN_7 GUL_3a 0.467271004 -1.248834286 2.183376295 0.999999998 0.288219495

GRØN_7 GUL_3b 0.650625888 -1.12391387 2.425165645 0.999992423 0.153043374

GRØN_7 GUL_5 0.108753408 -1.607437463 1.824944278 1 0.804589116

GRØN_7 GUL_7 0.121875 -1.58714156 1.83089156 1 0.780687619

GRØN_7 HVID_0a -0.796259911 -2.462272993 0.869753172 0.99776598 0.062863552

GRØN_7 HVID_0b 0.366069249 -1.349553499 2.081691997 1 0.40507647

GRØN_7 HVID_12 0.206796326 -1.50513971 1.918732363 1 0.637243228

GRØN_7 HVID_5 -0.693347138 -2.40561687 1.018922594 0.999922536 0.114702608

GRØN_7 HVID_7 -0.535016531 -2.159291258 1.089258195 0.999999508 0.199094473

GRØN_7 JIS_3 0.600402862 -1.11186687 2.312672594 0.999997507 0.171707354

GRØN_7 RØD_0 -0.355016531 -1.979291258 1.269258195 1 0.393751728

GRØN_7 RØD_12 0.333753408 -1.382437463 2.049944278 1 0.447899555

GRØN_7 RØD_3 0.070406756 -1.595606327 1.736419838 1 0.868941108

GRØN_7 RØD_5 0.150546326 -1.56138971 1.862482363 1 0.731355171

GRØN_7 RØD_7 0.312483469 -1.311791258 1.936758195 1 0.452789271

GRØN_7 GRÅ_3 -0.606592375 -2.385723821 1.172539072 0.999998785 0.18382637

GRØN_7 HVID_3 -0.178020946 -1.957152393 1.601110501 1 0.696067408

GRØN_7 SORT 0.593407625 -1.185723821 2.372539072 0.999999315 0.193508935

GUL_0 GUL_12a 0.838499562 -0.850957066 2.52795619 0.995655142 0.053462483

GUL_0 GUL_12b 0.67046078 -0.953480201 2.294401762 0.999881126 0.107801303

GUL_0 GUL_3a 0.462287535 -1.161930836 2.086505907 0.999999991 0.266981217

GUL_0 GUL_3b 0.645642419 -1.043814209 2.335099047 0.999979482 0.136465096

GUL_0 GUL_5 0.103769939 -1.520499928 1.728039806 1 0.803024631

GUL_0 GUL_7 0.116891531 -1.507383195 1.741166258 1 0.778733325

GUL_0 HVID_0a -0.801243379 -2.373252057 0.770765298 0.993215297 0.047383987

GUL_0 HVID_0b 0.36108578 -1.262855201 1.985026762 1 0.385629357

GUL_0 HVID_12 0.201812857 -1.422501442 1.826127157 1 0.627667969

GUL_0 HVID_5 -0.698330607 -2.322792558 0.926131345 0.999718133 0.094105142

GUL_0 HVID_7 -0.54 -2.068590881 0.988590881 0.999996991 0.168540339

GUL_0 JIS_3 0.595419393 -1.029042558 2.219881345 0.999992479 0.153168339

GUL_0 RØD_0 -0.36 -1.888590881 1.168590881 1 0.358180636

GUL_0 RØD_12 0.328769939 -1.295499928 1.953039806 1 0.429608897

GUL_0 RØD_3 0.065423287 -1.506585391 1.637431965 1 0.870917524

GUL_0 RØD_5 0.145562857 -1.478751442 1.769877157 1 0.726440589

GUL_0 RØD_7 0.3075 -1.221090881 1.836090881 1 0.432449694

GUL_0 GRÅ_3 -0.611575843 -2.300953632 1.077801946 0.999994458 0.15828067

GUL_0 HVID_3 -0.183004415 -1.872382204 1.506373374 1 0.672372786

| 109

GUL_0 SORT 0.588424157 -1.100953632 2.277801946 0.999997897 0.174573598

GUL_12a GUL_12b -0.168038781 -1.943035603 1.606958041 1 0.711684636

GUL_12a GUL_3a -0.376212026 -2.150834614 1.398410562 1 0.408114109

GUL_12a GUL_3b -0.192857143 -2.019872696 1.63415841 1 0.680272753

GUL_12a GUL_5 -0.734729623 -2.513985022 1.044525776 0.999880609 0.107729878

GUL_12a GUL_7 -0.72160803 -2.496147788 1.052931727 0.99991489 0.113179632

GUL_12a HVID_0a -1.639742941 -3.365887604 0.086401722 0.09277371 0.000249289

GUL_12a HVID_0b -0.477413781 -2.252410603 1.297583041 0.999999998 0.294126025

GUL_12a HVID_12 -0.636686704 -2.411287149 1.137913741 0.999995565 0.162018215

GUL_12a HVID_5 -1.536830168 -3.31184613 0.238185793 0.227374655 0.000821214

GUL_12a HVID_7 -1.378499562 -3.06795619 0.310957066 0.35175271 0.001596955

GUL_12a JIS_3 -0.243080168 -2.01809613 1.531935793 1 0.592950501

GUL_12a RØD_0 -1.198499562 -2.88795619 0.490957066 0.684418196 0.005951364

GUL_12a RØD_12 -0.509729623 -2.288985022 1.269525776 0.99999999 0.263868205

GUL_12a RØD_3 -0.773076275 -2.499220938 0.953068388 0.999350667 0.081200988

GUL_12a RØD_5 -0.692936704 -2.467537149 1.081663741 0.999966075 0.128172906

GUL_12a RØD_7 -0.530999562 -2.22045619 1.158457066 0.999999861 0.220390233

GUL_12a GRÅ_3 -1.450075405 -3.286084254 0.385933444 0.428709795 0.00223486

GUL_12a HVID_3 -1.021503977 -2.857512826 0.814504873 0.974440247 0.030544628

GUL_12a SORT -0.250075405 -2.086084254 1.585933444 1 0.594946866

GUL_12b GUL_3a -0.208173245 -1.923710711 1.507364221 1 0.635714207

GUL_12b GUL_3b -0.024818362 -1.799815183 1.75017846 1 0.956462116

GUL_12b GUL_5 -0.566690841 -2.282339333 1.14895765 0.999999471 0.19784689

GUL_12b GUL_7 -0.553569249 -2.269191997 1.162053499 0.999999717 0.208391502

GUL_12b HVID_0a -1.47170416 -3.137261436 0.193853117 0.191148807 0.000642856

GUL_12b HVID_0b -0.309375 -2.01839156 1.39964156 1 0.479883826

GUL_12b HVID_12 -0.468647923 -2.184327507 1.247031661 0.999999997 0.286684982

GUL_12b HVID_5 -1.368791387 -3.084767284 0.34718451 0.404833165 0.002021504

GUL_12b HVID_7 -1.21046078 -2.834401762 0.413480201 0.57006578 0.003878629

GUL_12b JIS_3 -0.075041387 -1.791017284 1.64093451 1 0.864424698

GUL_12b RØD_0 -1.03046078 -2.654401762 0.593480201 0.876951295 0.01375335

GUL_12b RØD_12 -0.341690841 -2.057339333 1.37395765 1 0.437043438

GUL_12b RØD_3 -0.605037493 -2.27059477 1.060519783 0.999993965 0.15685219

GUL_12b RØD_5 -0.524897923 -2.240577507 1.190781661 0.999999934 0.232876649

GUL_12b RØD_7 -0.36296078 -1.986901762 1.260980201 1 0.383170499

GUL_12b GRÅ_3 -1.282036624 -3.060456624 0.496383377 0.648647343 0.005200188

GUL_12b HVID_3 -0.853465195 -2.631885196 0.924954806 0.997594506 0.061799715

GUL_12b SORT -0.082036624 -1.860456624 1.696383377 1 0.857069002

GUL_3a GUL_3b 0.183354884 -1.591267704 1.957977471 1 0.686685695

GUL_3a GUL_5 -0.358517596 -2.074552729 1.357517536 1 0.414935219

GUL_3a GUL_7 -0.345396004 -2.061501295 1.370709286 1 0.432216255

GUL_3a HVID_0a -1.263530915 -2.925732431 0.398670601 0.522292732 0.00323809

GUL_3a HVID_0b -0.101201755 -1.816739221 1.614335711 1 0.817841697

GUL_3a HVID_12 -0.260474678 -1.976640144 1.455690788 1 0.553565743

GUL_3a HVID_5 -1.160618142 -2.877045716 0.555809432 0.780312487 0.008707205

GUL_3a HVID_7 -1.002287535 -2.626505907 0.621930836 0.908033696 0.016547772

GUL_3a JIS_3 0.133131858 -1.583295716 1.849559432 1 0.762025026

GUL_3a RØD_0 -0.822287535 -2.446505907 0.801930836 0.993929118 0.048882344

110 |

GUL_3a RØD_12 -0.133517596 -1.849552729 1.582517536 1 0.76130366

GUL_3a RØD_3 -0.396864248 -2.059065764 1.265337267 1 0.351602088

GUL_3a RØD_5 -0.316724678 -2.032890144 1.399440788 1 0.471371892

GUL_3a RØD_7 -0.154787535 -1.779005907 1.469430836 1 0.709853821

GUL_3a GRÅ_3 -1.073863379 -2.848306712 0.700579954 0.925962008 0.018730743

GUL_3a HVID_3 -0.64529195 -2.419735283 1.129151383 0.999993801 0.156403982

GUL_3a SORT 0.126136621 -1.648306712 1.900579954 1 0.781369924

GUL_3b GUL_5 -0.54187248 -2.321127879 1.237382919 0.999999942 0.235002209

GUL_3b GUL_7 -0.528750888 -2.303290645 1.24578887 0.999999968 0.245253439

GUL_3b HVID_0a -1.446885798 -3.173030461 0.279258865 0.292176548 0.001191166

GUL_3b HVID_0b -0.284556638 -2.05955346 1.490440183 1 0.531496774

GUL_3b HVID_12 -0.443829562 -2.218430007 1.330770884 1 0.329230787

GUL_3b HVID_5 -1.343973026 -3.118988987 0.431042936 0.53195369 0.003359239

GUL_3b HVID_7 -1.185642419 -2.875099047 0.503814209 0.707507678 0.006501647

GUL_3b JIS_3 -0.050223026 -1.825238987 1.724792936 1 0.912032613

GUL_3b RØD_0 -1.005642419 -2.695099047 0.683814209 0.938958082 0.020742976

GUL_3b RØD_12 -0.31687248 -2.096127879 1.462382919 1 0.487019271

GUL_3b RØD_3 -0.580219132 -2.306363795 1.145925531 0.999999161 0.190073964

GUL_3b RØD_5 -0.500079562 -2.274680007 1.274520884 0.999999994 0.27174332

GUL_3b RØD_7 -0.338142419 -2.027599047 1.351314209 1 0.434776839

GUL_3b GRÅ_3 -1.257218262 -3.093227112 0.578790587 0.756889945 0.007903141

GUL_3b HVID_3 -0.828646834 -2.664655683 1.007362016 0.999243806 0.078885235

GUL_3b SORT -0.057218262 -1.893227112 1.778790587 1 0.903151653

GUL_5 GUL_7 0.013121592 -1.703069278 1.729312463 1 0.976184215

GUL_5 HVID_0a -0.905013318 -2.570977745 0.760951108 0.981803486 0.034659995

GUL_5 HVID_0b 0.257315841 -1.45833265 1.972964333 1 0.558264237

GUL_5 HVID_12 0.098042918 -1.618208232 1.814294069 1 0.823503835

GUL_5 HVID_5 -0.802100546 -2.514919584 0.910718492 0.998465777 0.068329822

GUL_5 HVID_7 -0.643769939 -2.268039806 0.980499928 0.999952217 0.122561115

GUL_5 JIS_3 0.491649454 -1.221169584 2.204468492 0.999999989 0.262945439

GUL_5 RØD_0 -0.463769939 -2.088039806 1.160499928 0.999999991 0.265469052

GUL_5 RØD_12 0.225 -1.48401656 1.93401656 1 0.607306033

GUL_5 RØD_3 -0.038346652 -1.704311078 1.627617775 1 0.928388495

GUL_5 RØD_5 0.041792918 -1.674458232 1.758044069 1 0.924251666

GUL_5 RØD_7 0.203730061 -1.420539806 1.827999928 1 0.624395238

GUL_5 GRÅ_3 -0.715345782 -2.486514552 1.055822988 0.99992688 0.115637742

GUL_5 HVID_3 -0.286774354 -2.057943124 1.484394416 1 0.527414893

GUL_5 SORT 0.484654218 -1.286514552 2.255822988 0.999999997 0.285842651

GUL_7 HVID_0a -0.918134911 -2.584147993 0.747878172 0.977634522 0.032144032

GUL_7 HVID_0b 0.244194249 -1.471428499 1.959816997 1 0.578488114

GUL_7 HVID_12 0.084921326 -1.62701471 1.796857363 1 0.846427272

GUL_7 HVID_5 -0.815222138 -2.52749187 0.897047594 0.997917218 0.063875758

GUL_7 HVID_7 -0.656891531 -2.281166258 0.967383195 0.999924673 0.11515555

GUL_7 JIS_3 0.478527862 -1.23374187 2.190797594 0.999999995 0.275715663

GUL_7 RØD_0 -0.476891531 -2.101166258 1.147383195 0.999999979 0.252215435

GUL_7 RØD_12 0.211878408 -1.504312463 1.928069278 1 0.629845954

GUL_7 RØD_3 -0.051468244 -1.717481327 1.614544838 1 0.903991068

GUL_7 RØD_5 0.028671326 -1.68326471 1.740607363 1 0.947861627

| 111

GUL_7 RØD_7 0.190608469 -1.433666258 1.814883195 1 0.64687591

GUL_7 GRÅ_3 -0.728467375 -2.507598821 1.050664072 0.999900904 0.110724188

GUL_7 HVID_3 -0.299895946 -2.079027393 1.479235501 1 0.510604939

GUL_7 SORT 0.471532625 -1.307598821 2.250664072 0.999999999 0.301223267

HVID_0a HVID_0b 1.16232916 -0.503228117 2.827886436 0.719194887 0.006803302

HVID_0a HVID_12 1.003056237 -0.663007891 2.669120365 0.930247404 0.019345461

HVID_0a HVID_5 0.102912773 -1.563340978 1.769166524 1 0.809444853

HVID_0a HVID_7 0.261243379 -1.310765298 1.833252057 1 0.516585368

HVID_0a JIS_3 1.396662773 -0.269590978 3.062916524 0.292209061 0.0011913

HVID_0a RØD_0 0.441243379 -1.130765298 2.013252057 0.999999994 0.273633809

HVID_0a RØD_12 1.130013318 -0.535951108 2.795977745 0.774642963 0.008502896

HVID_0a RØD_3 0.866666667 -0.744609598 2.477942932 0.984273656 0.036472519

HVID_0a RØD_5 0.946806237 -0.719257891 2.612870365 0.965925479 0.027184106

HVID_0a RØD_7 1.108743379 -0.463265298 2.680752057 0.696913626 0.006242339

HVID_0a GRÅ_3 0.189667536 -1.53636207 1.915697142 1 0.667936384

HVID_0a HVID_3 0.618238965 -1.107790641 2.344268571 0.999995744 0.162709676

HVID_0a SORT 1.389667536 -0.33636207 3.115697142 0.382776413 0.001837277

HVID_0b HVID_12 -0.159272923 -1.874952507 1.556406661 1 0.717032363

HVID_0b HVID_5 -1.059416387 -2.775392284 0.65655951 0.907556004 0.016496094

HVID_0b HVID_7 -0.90108578 -2.525026762 0.722855201 0.975380184 0.030992123

HVID_0b JIS_3 0.234333613 -1.481642284 1.95030951 1 0.59399074

HVID_0b RØD_0 -0.72108578 -2.345026762 0.902855201 0.999452931 0.083820796

HVID_0b RØD_12 -0.032315841 -1.747964333 1.68333265 1 0.94137237

HVID_0b RØD_3 -0.295662493 -1.96121977 1.369894783 1 0.488431234

HVID_0b RØD_5 -0.215522923 -1.931202507 1.500156661 1 0.623867972

HVID_0b RØD_7 -0.05358578 -1.677526762 1.570355201 1 0.897486543

HVID_0b GRÅ_3 -0.972661624 -2.751081624 0.805758377 0.979932018 0.033458649

HVID_0b HVID_3 -0.544090195 -2.322510196 1.234329806 0.999999934 0.232878366

HVID_0b SORT 0.227338376 -1.551081624 2.005758377 1 0.617774147

HVID_12 HVID_5 -0.900143464 -2.612460913 0.812173985 0.988895466 0.040894392

HVID_12 HVID_7 -0.741812857 -2.366127157 0.882501442 0.999047918 0.075413785

HVID_12 JIS_3 0.393606536 -1.318710913 2.105923985 1 0.369796534

HVID_12 RØD_0 -0.561812857 -2.186127157 1.062501442 0.999998242 0.17759163

HVID_12 RØD_12 0.126957082 -1.589294069 1.843208232 1 0.772727278

HVID_12 RØD_3 -0.13638957 -1.802453698 1.529674557 1 0.749265928

HVID_12 RØD_5 -0.05625 -1.76526656 1.65276656 1 0.8977452

HVID_12 RØD_7 0.105687143 -1.518627157 1.730001442 1 0.799468108

HVID_12 GRÅ_3 -0.813388701 -2.592596446 0.965819045 0.999028772 0.075113505

HVID_12 HVID_3 -0.384817272 -2.164025018 1.394390474 1 0.398704531

HVID_12 SORT 0.386611299 -1.392596446 2.165819045 1 0.396512395

HVID_5 HVID_7 0.158330607 -1.466131345 1.782792558 1 0.703567329

HVID_5 JIS_3 1.29375 -0.41526656 3.00276656 0.532427801 0.003365652

HVID_5 RØD_0 0.338330607 -1.286131345 1.962792558 1 0.41638625

HVID_5 RØD_12 1.027100546 -0.685718492 2.739919584 0.933407853 0.019828043

HVID_5 RØD_3 0.763753894 -0.902499858 2.430007645 0.998978292 0.074355715

HVID_5 RØD_5 0.843893464 -0.868423985 2.556210913 0.996150383 0.055145378

HVID_5 RØD_7 1.005830607 -0.618631345 2.630292558 0.904617036 0.016188803

HVID_5 GRÅ_3 0.086754763 -1.688704931 1.862214457 1 0.848696681

112 |

HVID_5 HVID_3 0.515326192 -1.260133502 2.290785886 0.999999985 0.257659311

HVID_5 SORT 1.286754763 -0.488704931 3.062214457 0.636428581 0.004968045

HVID_7 JIS_3 1.135419393 -0.489042558 2.759881345 0.715970601 0.006718536

HVID_7 RØD_0 0.18 -1.348590881 1.708590881 1 0.645739472

HVID_7 RØD_12 0.868769939 -0.755499928 2.493039806 0.985537051 0.037523348

HVID_7 RØD_3 0.605423287 -0.966585391 2.177431965 0.99997539 0.133460155

HVID_7 RØD_5 0.685562857 -0.938751442 2.309877157 0.999808674 0.100212601

HVID_7 RØD_7 0.8475 -0.681090881 2.376090881 0.975653729 0.031126051

HVID_7 GRÅ_3 -0.071575843 -1.760953632 1.617801946 1 0.868610654

HVID_7 HVID_3 0.356995585 -1.332382204 2.046373374 1 0.409612508

HVID_7 SORT 1.128424157 -0.560953632 2.817801946 0.801759739 0.009547552

JIS_3 RØD_0 -0.955419393 -2.579881345 0.669042558 0.947193984 0.022294992

JIS_3 RØD_12 -0.266649454 -1.979468492 1.446169584 1 0.543423951

JIS_3 RØD_3 -0.529996106 -2.196249858 1.136257645 0.999999807 0.214913638

JIS_3 RØD_5 -0.449856536 -2.162173985 1.262460913 0.999999999 0.305462627

JIS_3 RØD_7 -0.287919393 -1.912381345 1.336542558 1 0.489106654

JIS_3 GRÅ_3 -1.206995237 -2.982454931 0.568464457 0.770500458 0.008357498

JIS_3 HVID_3 -0.778423808 -2.553883502 0.997035886 0.999576801 0.087774423

JIS_3 SORT -0.006995237 -1.782454931 1.768464457 1 0.987726084

RØD_0 RØD_12 0.688769939 -0.935499928 2.313039806 0.999788586 0.09863248

RØD_0 RØD_3 0.425423287 -1.146585391 1.997431965 0.999999998 0.291166876

RØD_0 RØD_5 0.505562857 -1.118751442 2.129877157 0.999999894 0.224896408

RØD_0 RØD_7 0.6675 -0.861090881 2.196090881 0.999610173 0.089048771

RØD_0 GRÅ_3 -0.251575843 -1.940953632 1.437801946 1 0.561061542

RØD_0 HVID_3 0.176995585 -1.512382204 1.866373374 1 0.682530347

RØD_0 SORT 0.948424157 -0.740953632 2.637801946 0.971072846 0.029084538

RØD_12 RØD_3 -0.263346652 -1.929311078 1.402617775 1 0.537242142

RØD_12 RØD_5 -0.183207082 -1.899458232 1.533044069 1 0.676872085

RØD_12 RØD_7 -0.021269939 -1.645539806 1.602999928 1 0.959221968

RØD_12 GRÅ_3 -0.940345782 -2.711514552 0.830822988 0.987070408 0.0389435

RØD_12 HVID_3 -0.511774354 -2.282943124 1.259394416 0.999999987 0.259796562

RØD_12 SORT 0.259654218 -1.511514552 2.030822988 1 0.567165208

RØD_3 RØD_5 0.08013957 -1.585924557 1.746203698 1 0.851028655

RØD_3 RØD_7 0.242076713 -1.329931965 1.814085391 1 0.547797434

RØD_3 GRÅ_3 -0.67699913 -2.403028736 1.049030475 0.999962378 0.126475149

RØD_3 HVID_3 -0.248427702 -1.974457308 1.477601904 1 0.574242416

RØD_3 SORT 0.52300087 -1.203028736 2.249030475 0.999999949 0.237376308

RØD_5 RØD_7 0.161937143 -1.462377157 1.786251442 1 0.697120983

RØD_5 GRÅ_3 -0.757138701 -2.536346446 1.022069045 0.999772187 0.097451176

RØD_5 HVID_3 -0.328567272 -2.107775018 1.450640474 1 0.471091073

RØD_5 SORT 0.442861299 -1.336346446 2.222069045 1 0.331535875

RØD_7 GRÅ_3 -0.919075843 -2.608453632 0.770301946 0.981412117 0.034398055

RØD_7 HVID_3 -0.490504415 -2.179882204 1.198873374 0.999999985 0.257501465

RØD_7 SORT 0.280924157 -1.408453632 1.970301946 1 0.516322554

GRÅ_3 HVID_3 0.428571429 -1.398444124 2.255586982 1 0.360092088

GRÅ_3 SORT 1.2 -0.627015553 3.027015553 0.829040716 0.010801313

HVID_3 SORT 0.771428571 -1.055586982 2.598444124 0.999807012 0.10007628

| 113

Appendix E

Program Code

In this appendix some of the code used in the analysis is given.

114 |

Mixed model ANOVA and ANCOVA from Chapter 10 Here on attribute Earthy(o)

3-way mixed model ANOVA proc mixed data=sasuser.fisk2 order=internal method=reml cl nobound;

class code assessor replicate;

model earthy_o_=code/ ddfm=satterth intercept;

random assessor code*assessor replicate(code);

lsmeans code;

run;

4-way mixed model ANOVA proc mixed data=sasuser.fisk2 order=internal method=reml cl nobound;

class foder istid assessor replicate;

model earthy_o_=foder istid foder*istid/ ddfm=satterth intercept;

random assessor foder*assessor istid*assessor istid*foder*assessor

replicate(foder*istid);

*lsmeans foder;

run;

3-way mixed model ANCOVA proc mixed data=sasuser.fisk2 order=internal method=reml cl nobound;

class foder assessor replicate;

model earthy_o_=istid foder foder*istid/ddfm=satterth;

random assessor assessor*foder replicate(foder);

run;

quit;

| 115

3-way mixed model MANOVA with CVA *1-way Full model for hver assessor, TODO: så brug andre datasæt et for hver assessor;

data udensort;

set sasuser.fisk2;

if foder ne 'sort';

run;

*Full model uden assessor*replicate(code;

proc glm data=udensort order=internal outstat=fiskstat;

class code assessor replicate;

model earthy_o_ cooked_potato_o_ sourish_o_ sour_o_ muddy_o_

earthy_f_ mushroom_f_ cooked_potato_f_ sourish_f_ sweet_f_ green_f_ muddy_f_

flaky_t_ firm_t_ juicy_t_ fibrousness_t_ oiliness_t_

= code assessor code*assessor replicate(code)/nouni;

*means code/lsd e=code*assessor;

manova h=code e=code*assessor/canonical;

*random assessor code*assessor replicate(code);

run;

116 |

4-way mixed model MANOVA with CVA data udensort;

set sasuser.fisk2;

if foder ne 'sort';

run;

*Full model uden assessor*replicate(foder*istid);

proc glm data=udensort order=internal outstat=fiskstat4;

class foder istid assessor replicate;

model earthy_o_ cooked_potato_o_ sourish_o_ sour_o_ muddy_o_

earthy_f_ mushroom_f_ cooked_potato_f_ sourish_f_ sweet_f_ green_f_ muddy_f_

flaky_t_ firm_t_ juicy_t_ fibrousness_t_ oiliness_t_

= foder istid assessor

foder*istid assessor*foder assessor*istid

assessor*foder*istid replicate(foder*istid);

*assessor*replicate(foder*istid);

*manova h = foder e=foder*assessor/canonical;

manova h = istid e=istid*assessor/canonical;

*manova h = foder*istid e=foder*assessor*istid;

random assessor

assessor*foder assessor*istid

assessor*foder*istid replicate(foder*istid);

run;

quit;

| 117

References

Borga, M., & Landelius, T., Knutsson, H (1997). A Unified Approach to PCA, PLS, MLR and CCA. Brockhoff, P.B. (2000). Mixed Linear Models. Online Lecture Notes for course on Mixed LinearModels

from an applied perspective. (URL: http://www.imm.dtu.dk/˜pbb/st113).

Brockhoff, P.B. (2007) The design and univariate analysis of sensory profile data

Campbell, N.A., & Atchley, W.R. (1981). The geometry of canonical variate analysis. Systematic Zoology, 30(3), 268-280.

Esbensen, K.H. (2002) Multivariate Data Analysis -in practice (5th Edition).

Everitt, B.G. (2001), A Handbook of Statistical Analyses Using SAS (2nd Edition). Taylor & Franciss LTD

Gittins, R. (1985). Canonical analysis: a review with applications in ecology. Berlin: Springer Verlag.

Hartley, H.O. and Rao, J.N.K. (1967), "Maximum-Likelihood Estimation for the Mixed Analysis of Variance

Model," Biometrika, 54, 93 -108.

Henderson, C.R. (1984), Applications of Linear Models in Animal Breeding, University of Guelph.

Jackson, J. E. (2003). A User's Guide to Principal Components. Wiley-Interscience.

Jolliffe, I.T. (2002) Principal Component Analysis. Springer.

Krzanowski, W.J. (2000). Principles of Multivariate Analysis: A users perspective, Oxford statistical

science.

Langsrud, Ø. (2002). 50-50 multivariate analysis of variance for collinear responses

Little, R.J.A. (1995), "Modeling the Drop-Out Mechanism in Repeated-Measures Studies," Journal of the

American Statistical Association, 90, 1112 -1121.

http://www.imt.liu.se/~knutte/

http://www.amazon.com/exec/obidos/search-handle-url/002-2002045-2788850?%5Fencoding=UTF8&search-type=ss&index=books&field-author=J.%20Edward%20Jackson

http://www.amazon.com/exec/obidos/search-handle-url/002-2002045-2788850?%5Fencoding=UTF8&search-type=ss&index=books&field-author=I.T.%20Jolliffe

118 |

Martens, H., Martens, M. (2001) Multivariate Analysis of Quality : An Introduction. Wiley.

Rencher, A.C. (2002) Methods of Multivariate Analysis. Wiley.

Tinsley, E.A. (2000) Handbook of Applied Multivariate Statistics and Mathematical Modeling. Acedemic

Press.

http://www.amazon.com/exec/obidos/search-handle-url/002-2002045-2788850?%5Fencoding=UTF8&search-type=ss&index=books&field-author=Harald%20Martens

http://www.amazon.com/exec/obidos/search-handle-url/002-2002045-2788850?%5Fencoding=UTF8&search-type=ss&index=books&field-author=M.%20Martens

multivariate analysis of sensory dataetd.dtu.dk/thesis/222770/ep08_84.pdf · as evaluated by ten...

Documents