lc quality assurance services provided by lcbo quality assurance leading sensory evaluation services

LCLC Quality AssuranceQuality Assurance

Services Provided by LCBO Quality Assurance

Leading Sensory Evaluation

Services


VQA Mandate

Appellation of Origin SystemQuality StandardControl of use of specified terms, descriptions and designations associated with the VQA appellation system

VQA Appellation System


Sensory Quality Chemical Composition Microbiological Stability Packaging and Labelling

Standards

Role of Sensory Evaluation Testing

Quality of Beverage Alcohol Products


Free of technical faults / defects

Typicity of varietal character – for varietal wines “A wine bearing varietal designation shall be

assessed to determine if the varietal designation for the wine exhibits the predominant character of a wine produced from the designated grape variety/varieties” (VQA Rules)

Typicity of the wine category: Late Harvest, Icewine, Nouveau, Sparkling – Traditional Method, Icewine Dosage, Botrytized Wine (VQA Rules)

Role of VQA Sensory Evaluation Testing

Scope


“A wine shall be deemed to have passed the taste test if a majority of the members of the Tasting Panel determine: That the wine’s attributes fairly reflect the

viticultural and oenological quality standards established in O.Reg. 406/00 (Rules) without defects or flaws; and

That the wine is representative of quality wines of the stated category” (VQA Rules)

Role of VQA Sensory Evaluation Testing


Grading Panel – 4 panel groups of 5 panellists each

28 Members (20 regular panellists, 8 alternate tasters)

Sensory Panels


Panel members are LCBO Product Consultants who work in retail stores within metropolitan Toronto area

Continuous training/ development opportunitiesLCBO Product Knowledge I, I, IIICertification from the Wine and Spirit Education TrustMaster of Wine Certification

Panel Member Selection Criteria


Panel members are subjected to annual :

Sensory evaluation testing Product knowledge testing

25 % of the questions are VQA specific

Demonstrated professionalism – Sensory Evaluations Code of Conduct

Panel Member Selection Criteria


Consistent Performance

Balanced Panels

Reproducible Results

Panel Groups Assembly Criteria:

Results from the annual testing Testing performance historyPrevious experience (as a Grading Panel member, from the industry)

Performance monitoring dataSensory sensitivities


Training Elements

Aroma, flavour recognitionProduct category recognitionVarietal character recognitionRegional character recognitionWinemaking techniquesTechnological influencesDefect identification

Panel Training


Blind sample presentation:

Samples are presented without identifying markingsUniform sample presentation

Sample Presentation

Sample information: Varietal composition, vintage year, wine category, method of production (where applicable: sparkling wines, icewine dosage, etc)


Sensory Evaluation Grading System

Sensory Evaluation Method


Superior objectivity – system design eliminates bias

Wider system applicability– system design equally applies to all wine categories

Efficient and effective calculation of results – time savings, accuracy

Data Analysis Tools & Traceability of results – enabled by automated data collection and quantification of the results

Objectivity

Ap

plic

abili

ty

Efficiency

Client Satisfaction



Sensory evaluation grading system elements: New sensory evaluation

grading method


Integrating technology:• Automated data

collection• Automated data

quantification, analysis and reporting


Appearance and Colour Aroma (primary and secondary) and Bouquet Taste Harmony

Sensory characteristics applicable to all types of beverage alcohol products.

Appearance

ColourHarmony

BouquetTaste

Aroma



Sensory characteristics and attributes are evaluated using five grading categories.


Category

Excellent Very Good

Good Satisfactory

Unsatisfactory

Quality Level

Outstanding

/Exceptional

Superior/ Very

Correct

Typical Weak/Not at full

potential

Faulty / Defective


Characteristics & Attributes

Grading Categories Comments

Excellent

Very Good

Good Satisfactory

Unsatisfactory

Appearance & Colour

x

Aroma&Bouquet

Correctness

x

Intensity

x

Quality x

Taste Correctness

x

Intensity

x

Finish x

Quality x

Harmony x

Grading Form


Data collectionData quantificationData analysisData reporting

Innovative Technology


AutomatedInteractiveFlexible-WirelessPen or/and keyboardHandwritten notes

Data Collection – Tablet PCs


5.7

-3.2 -3.0

-11.4

5.94.5

-4.0

3.50.3 1.6

-15

-10

-5

0

5

10

15

20

25

30

35

40

45

<12.0 12.0 - 12.4 12.5 - 12.9 13.0 - 13.4 13.5 - 13.9 14.0 - 14.4 14.5 - 14.9 15.0 - 15.4 15.5 - 15.9 >16.0

Scoring Intervals (0-20 Scoring Scale)

Scores Frequency, %

Scores Frequency, 2003

Scores Frequency, 2004

Difference 04-03

2004

2003

New system implemented on June 1 2004; Comparison for periods June - December 2003 and 2004


Quality Sensory Evaluation Result

System Controls

Proficiency Programs

Verification Tasting

Performance Monitoring

Quality Assurance

QA of Sensory Evaluation Results



VQA Tasting protocol (VQA Rules)• 2nd Bottle tasting

Grading System Controls:• Security of panellist registration • Forced completeness of the assessment• Ratings cannot be changed, once finalized• Sample evaluation cannot be redone, once

completed• Validation questions in the grading questionnaire• Calculation validation based on statistical

measures

Automated calculation and reporting



Verification Tasting - an independent sensory evaluation of the products in conditions identical to the tasting panel. The verification process has a quality assurance role and provides a reference value that is then compared with the panel results to identify any discrepancies in the assessment.Verifiers – qualified Quality Assurance tasters. A tasting session may require up 2 Verifiers.Verifiers’ results are typically not included in the calculation, unless discrepancy situation.



Panellists’ performance is monitored for each tasting sessionMeasures:• Outliers Frequency, % - measure of rating

accuracy• Rating Rank – measure of rating bias (high,

low, trends)• Sensitivity data - missed defects, good

detection, hyper-sensitivityFeedback on performance – Feedback ReportProvides tools to help identify training opportunities


Panellist Ratings vs. Panel Median

6

7

8

9

10

11

12

13

14

15

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Samples

Gra

din

g S

co

res

Panelist Values

Product Result(Median)

Panellist FeedbackReport

Tasting Session: 4-Apr-2005 2:23 PM

Panellist:

Total Number of Samples: 26

Total Number of Outliers: 4

Outliers Frequency, % 15.4Rank, Descending 2.9

Panellist Name




Internally designed to measure quality and consistency of the sensory evaluation assessmentsDesigned to measure:• System performance• Panel Performance• Panellist Performance

Typical measures: repeatability, reproducibility, bias, defects identification, etc.

Panellist

Panel

System


Internal Proficiency Testing

Objective: Assessment of results repeatability & reproducibility

Product Categ.: Ontario Wines

Methodology: Assessment of panel results for three Ontario wines presented as blind duplicates.Wines are of sound quality and similar quality levels.

Replicate 1 Replicate 2 Replicate 1 Replicate 2 Replicate 1 Replicate 2

Average 13.76 13.80 13.86 13.56 13.98 13.64SD 1.24 0.57 0.36 1.02 0.69 1.30

Sample 1 Sample 2 Sample 3Sensory Results


Repeatability


Repeatability Results: p-Values * (2-tailed test, 95 % Confidence Level)

Sample 1 Sample 2 Sample 31. Panel level No significant difference between replicate results

0.8711 0.1895 0.30442. Panel groups level: No significant difference between replicatesGroup A: 0.4382 0.2809 0.0759Group B: 0.4814 0.5232 0.5746Group C: 0.7688 0.5137 0.2235Group D: 0.9653 0.5291 0.08233. Panellist level: Only one taster exibited significant variability of the results.

Source of variation: Sample position in the line-up in reference with the sensory profile of adjacent samples.* Interpretation of results:: If the p-Value is lower than 0.05, then there is a signifficant difference at a 95% CL



Reproducibility

Sample

Replicate 1 Replicate 2 Replicate 1 Replicate 2 Replicate 1 Replicate 2 Replicate 1 Replicate 2

Group ANov. 8/04

Group ANov. 8/04

Group BNov. 10/04

Group BNov. 10/04

Group CNov.29/04

Group CNov.29/04

Group DNov.17/04

Group DNov.17/04

Panel Ave.

Panel SD

Sample 1 13.2 13.6 14.6 14 13.4 13.7 13.8 13.8 13.8 0.39370Sample 2 13.8 13.8 13.6 13.6 13.7 13.7 14 14.4 13.6 0.25981Sample 3 14.4 13.6 13.8 14.4 13.7 14.2 13.2 14 13.6 0.40552

Average SD: 0.35301

Results, Median Value (reporting value)

No significant difference between reported results, i.e. median values (p-Value: 0.649671) at a 95% confidence level


Grading Panel

Average*

QA PanelSept. 16/04(Ref. Panel)

Group ASept.13/04

Group BSept.15/04

Group CSept.20/04

Group DSept.22/04

Grading Panel SD

Sample 1 12.9 13.8 13.8 13.5 13.4 13.2 0.25000Sample 2 14.4 14.2 14.2 14.4 14.8 14.2 0.28284Sample 3 10.7 10.6 10.2 9.4 9.8 12 1.14746

Average: 0.56010

* Average of individual panellists' score.

Sample

Results, Median Value (reporting value)

Reproducibility


No significant difference between panels' (i.e. all 5 panels) reported results (i.e. median value), at a 95% confidence level (p-value = 0.99605)



Summary A superior method for performing sensory evaluations. Eliminates many sources of bias. Customized for use with all beverage alcohol products. Flexibility (data collection, quantification & reporting). Provides significant time savings in data management. Provides tools to analyze panellist results to help identify

training opportunities.


Leading the Sensory Evaluation of

Beverage Alcohol

lc quality assurance services provided by lcbo quality assurance leading sensory evaluation services

Documents

lc quality assurance

lcbo quality assurance

oenological quality

wine category

tasting panel

grading panel member

varietal wines

varietal composition