diploma in cosmetic science module 4 unit 25 …...iso 5496:2006 sensory analysis iso 11132:2012...

DIPLOMA IN COSMETIC SCIENCE

MODULE 4 UNIT 25 PRODUCT EVALUATION

PRODUCT EVALUATION – WITH EXAMPLES

FROM SKIN CARE

Paul Cornwell, SCS - Summer School 5th July 2020

Key points

Definition of product evaluation

Objective vs subjective

To design a good test

Have a clear objective and hypothesis

Define study outline

Have a tested & validated protocol

Get the operational details right

Define your statistics and success criteria

Understand the range of tests used for skin and

hair

To write a good protocol

Always follow the five steps above!

What is product evaluation? (section 1)

The term product evaluation is used to describe the process of

assessing one or more characteristics or attributes of a

product

The unit focuses on evaluation tests that assess how well

products perform when they come in contact with their target

body sites (skin, hair, teeth etc)

Includes instrumental and sensory tests

Formulation tests and product safety tests are also classed as

product evaluation but are the focus of other modules

Contact ATR-FTIR being used at TRI to test

scalp condition

Who uses skin product evaluation (section 1.3)

Discovery

Post-launch

evaluation

LaunchTesting and

validationDevelopment

Build Business

CaseScoping Gate 2

Idea

Screen

Second screen

Go to development Go to testing Go to launch

Gate 5Gate 4Gate 3

Gate 1

Gate 1

Gate 3

Gate 4

Gate 5

Gate 2

Skin

clinical Consumer

test

Expert

panel


Quiz time! Match the evaluation test with the business need

Test comparing consumer preference for

liquid soap and synthetic detergent

based body wash products in India


Question 1. Which type of test is

recommended in this case:

(a) Expert sensory panel

(b) Clinical test

(c) Consumer test

Performance test on a new cream formulation

for sensitive skin using TEWL (improved barrier

function) and SLS occlusive patch test (reduced

sensitivity to cosmetic products)





(b) Clinical test

(c) Consumer test


Rapid testing of multiple laboratory

prototype formulations for a new, fast

absorbing body lotion




(b) Clinical test

(c) Consumer test

Different categories of evaluation methods (section 2)

Product evaluation

Objective* Subjective+

Instrumental

e.g. skin clinical

Sensory

e.g. expert

sensory panel or

skin clinical with

expert grading

Sensory

e.g. consumer test

Adapted from: IFSCC Monograph Number 1. Principles of Product Evaluation: Objective Sensory Methods

* Procedures carried out by

trained personnel using either

instrumental or sensory methods

which produce reproducible and

quantifiable results

+ Procedures involving sensory

test methods carried out by

untrained people ..(and which

are)..influenced by personal

preference

Designing a product evaluation test (section 3)

1. Have a clear objective and technical hypothesis

2. Consider the study outline

3. Have a robust and validated protocol

4. Get the operational details right

5. Define your statistical tests and success criteria

1. Have a clear objective (section 3.2.2)

In product testing you should always ask: “why am I doing this?”, and “what, exactly, will the information I

collect be used for?”

A crystal clear objective will help you build a strong experimental plan

In businesses this usually means working closely with your team and key stakeholders to define what the

project needs really are

1. Have a clear objective (section 3.2.2)

Having a Hypothesis

Definition of hypothesis: A supposition or proposed explanation made on the basis of limited evidence as a starting point for further investigation (Oxford Dictionary)

In product testing you should always ask: “why should this work?”, ”is there any technical reason why this product should do what we want it to do?”

Tests based on strong hypotheses are more likely to succeed

Be ready not to perform a test if there is no hypothesis as to why it should succeed (You will be surprised how often this happens!)

2. Study outline – (a) Safety & Regulations (section

3.2.3)

International Ethical Guidelines (2002)1 based on 1964

Declaration of Helsinki

Ethical and justifiable experiment

Reviewed by an ethical review body

Informed consent

Suitable reimbursement or payment

Safety and wellbeing of panellists protected – safety

review

Protection of vulnerable panellists

Adverse reaction processes in place

Confidentiality

1. International Ethical Guidelines for Biomedical Research Involving Human Subjects. Prepared by the Council for International

Organizations of Medical Sciences (CIOMS) in collaboration with the World Health Organization (WHO), Geneva (2002)

2. Study Outline – (a) Safety & Regulations (section

3.2.3)

Guideline for Good Clinical Practice (GCP) (1996) ICH

– covers European Union, US and Japan

In the US you also follow either US FDA regulations or

‘Common Rule’

2. Study Outline – (b) Panellist recruitment (section

3.2.3)

Target population - right distribution of sex, age, ethnicity, socio-economic status etc

Skin health - various expert grading systems

Exclusion criteria – medical/skin conditions, pregnancy, breast feeding etc

(time of year)

2. Study Outline – (b) Panellist recruitment – dryness

(section 3.2.3)

1. Simion F.A., Abrutyn E.S., Draelos Z.D. Ability of moisturizers to reduce dry skin and irritation and to prevent their return. J Cosmet Sci.

56(6):427-44 (2005)

2. Yoshida-Amano Y., Nomura T., Sugiyama Y., Iwata K., Higaki Y., Tanahashi M. Dry skin conditions are related to the recovery rate of

skin temperature after cold stress rather than to blood flow Int J Dermatol. 56(2):176-183 (2017)

Images illustrating dryness grading system2

Words used for a dryness grading system1

Dryness

score

Description of skin

0 No dryness

1 Slight flaking

2 Moderate flaking/scaling

3 Marked scaling/slight fissuring

4 Severe scaling/fissuring

2. Study Outline – (b) Panellist recruitment – wrinkling

(section 3.2.3)

1. Dayan, N. Selecting the Population for Human Clinical Studies Part 1. HAPPI Magazine, 31st January (2018)

Glogau Scale for severity of photoaging and especially wrinkling1

Group Classification Typical age Description Skin Characteristics

1 Mild Late 20s to

mid 30s

No wrinkles Early photo-ageing; mild

pigment changes; no

keratosis, minimal wrinkles

II Moderate Mid 30s to

early 50s

Wrinkles in facial

expression

Early to moderate

photoageing; early brown

spots; visible keratosis; smile

lines begin to appear

III Advanced Early 50s to

mid 60s

Wrinkles at rest Advanced photo-ageing;

obvious discolorations; visible

capillaries; visible keratosis

2. Study Outline – (b) Panellist recruitment – UV

response (section 3.2.3)

1. Dayan, N. Selecting the Population for Human Clinical Studies Part 1. HAPPI Magazine, 31st January (2018)

Fitzpatrick scale for skin colour and response to UV light1

Skin type Skin response Skin Characteristics

I Skin that always burns, never tans Pale white; blond or red hair; blue eyes; freckles

II Usually burns, tans minimally White; fair; blond or red hair; blue, green or hazel eyes

III Sometimes mild burn, tans uniformly Cream white; fair with any hair or eye colour

IV Burns minimally, always tans well Moderate brown

V Very rarely burns, tans very easily Moderate to darker brown

VI Never burns, never tans Deep pigmented dark brown to darkest brown

2. Study Outline - (c) Method of assessment (section

3.2.3)

Skin hydration

Skin colour

Skin topography

In vivo tensile properties of the skin

Trans-epidermal water loss

In vivo skin greasiness

Skin surface pH

etc

Modified from – Nobile, V. Guidelines on Cosmetic Efficacy Testing on Humans. Ethical, Technical, and Regulatory Requirements in the Main

Cosmetics Markets J. Cosmo. Trichol. 2:1 (2016)

EEMCO guidance notes for skin measurements

2. Study Outline - (c) Method of assessment (section

3.2.3)

ISO 5496:2006 sensory analysis

ISO 11132:2012 expert sensory panels

ASTM E-1958, 2018 edition. sensory claim substantiation

ISO/TR 26369:2009 sun protection test methods

ISO 24444:2010 in vivo SPF

ISO 24442:2011 UVA protection

Final Rule 29: sunscreen drug products. FDA regulations

Modified from – Nobile, V. Guidelines on Cosmetic Efficacy Testing on Humans. Ethical, Technical, and Regulatory Requirements in the Main

Cosmetics Markets J. Cosmo. Trichol. 2:1 (2016)

Useful standards and regulations

2. Study Outline – (c) Method of assessment – sensory

(section 3.2.3)

1. Classification

2. Grading

3. Ranking

4. Scaling (e.g. category scaling, line scaling)

Chapter 5. Measuring Responses Meilgaard, M.C., Civille, G.V., Carr, B.T. Sensory Evaluation Techniques, Fifth Edition, CRC Press, pp 51-

70 (2016)


(section 3.2.3)

1. (Objective) Difference test – Does a sensory difference exist between

samples? e.g. triangle test, duo-trio test

2. (Objective) Attribute difference test – How does an attribute X differ

between samples? e.g. ranking test, rating test

3. (Subjective) Affective test – Which sample do people prefer? e.g. rank

preference, hedonic rating

Chapter 6. Guidelines for Choice of Technique Meilgaard, M.C., Civille, G.V., Carr, B.T. Sensory Evaluation Techniques, Fifth Edition, CRC

Press, pp 71-78 (2016)


(section 3.2.3)

Quantitative Descriptive Analysis (QDA) – expert panel

Use carefully selected panellists + use panel to define sensory

attributes + use line scales to score the products

Data limited to each test

SpectrumTM Descriptive Analysis – expert panel

Use carefully selected panellists + train panel to score reproducibly

+ use category scaling

Time consuming to perform, but data can be compared across studies

and time


(section 3.2.3)

Quiz time! Question 4. The on-line triangle test. Perhaps one

picture is slightly different to the other two? Which one could it

be? You have to pick one, even if you think they are all the

same.


(section 3.2.3)

A B C


(section 3.2.3)

n a 0.04 0.30 0.20 0.10 0.05 0.01 0.001

3 2 2 3 3 3 - -

4 3 3 3 4 4 - -

5 3 3 4 4 4 5 -

6 3 4 4 5 5 6 -

7 4 4 4 5 5 6 7

8 4 4 5 5 6 7 8

9 4 5 5 6 6 7 8

10 5 5 6 6 7 8 9

11 5 5 6 7 7 8 10

12 5 6 6 7 8 9 10

13 6 6 7 8 8 9 11

14 6 7 7 8 9 10 11

15 6 7 8 8 9 10 12

16 7 7 8 9 9 11 12

17 7 8 8 9 10 11 13

18 7 8 9 10 10 12 13

19 8 8 9 10 11 12 14

20 8 9 9 10 11 13 14

21 8 9 10 11 12 13 15

22 9 9 10 11 12 14 15

23 9 10 11 12 12 14 16

24 10 10 11 12 13 15 16

25 10 11 11 12 13 15 17

26 10 11 12 13 14 15 17

27 11 11 12 13 14 16 18

28 11 12 12 14 15 16 18

29 11 12 13 14 15 17 19

30 12 12 13 14 15 17 19

Critical number of correct responses in a triangle test. Taken from: Statistical tables in

Meilgaard, M.C., Civille G.V. and Carr, B.T Sensory Evaluation Techniques, 5th Edition,

CRC Press, page 554 (2016)

2. Study Outline (d) – other considerations (section 3.2.3

- 3.2.8)

Study duration. How much time would be needed to study

product performance?

Application regime. Realistic or exaggerated?

Avoiding bias. Single or double blind

Controls. Are the right controls in place?

3. Have a tested & validated protocol (section 3.2.4)

For both objective and subjective sensory tests it is advisable to do 1-2 dry-runs of the protocol to ensure

the instructions are clear and the questionnaires work properly

For expert sensory panels and skin clinicals with expert grading it will be necessary to train the expert

evaluators to perform the scoring, and then to check for accuracy, precision, discrimination and

reproducibility. Ineffective evaluators will need to be removed from the test

For objective instrumental tests equipment needs to be calibrated and validated

4. Get the operational details right (section 3.2.11)

Right study location. Humidity & temperature control?

Right amounts of product, packaged and labelled/coded in the correct way

Good GLP - all formulation and ingredient details recorded & available

Avoid unnecessary biases, e.g. colour, fragrance, packaging, knowledge etc

Right consumables available for evaluators

Panellists recruited and booked-in

Safety assessments done, and consent forms completed

5. Define your statistical tests and success criteria

(section 3.2.12)

Define your key measurements; and, if possible, have several measurements for robustness and

secondary analysis

Select the appropriate statistical tests and success criteria ahead of testing

Do power calculations before starting to get the right sample size. Type I and Type II errors are

usually set at 0.05 (5% chance of a “false positive”) and ≤0.2 (20% chance of a “false

negative”; at least 80% statistical power), respectively. Pilot studies might be necessary to

determine the expected differences and standard deviations

Understand the difference between statistically significant and consumer meaningful data

Back in the real world: SOPs and protocols

SOP – Detailed instructions for performing a task (usually a routine, validated method)

Protocol – Experimental plan specific to your study

NOTE- sometimes both are combined into one and just called a ‘protocol’

SOP

Scope

Roles &Responsibilities

Safety & ethics

Panellist recruitment

Location, materials & equipment

Experimental procedure

Data analysis

Appendices

Consent form

Protocol sheet

Assessment form

SOP – Assessment Sheet

SOP - Protocol sheet

Test Number: XXXXX

Project Name: XXXXX

Test objective: To compare the performance of lotion 345262 and 755465 using the lotion

testing SOP. Follow instructions for lotion applications.

Recruitment

criteria:

Females, 18-45 yrs, dry skin

Minimum replicates:

N=25

Success criteria: Paired t-tests should show p<0.05. Key attributes are absorbance and stickiness

Product details

Product code: 118

Treatment: Current lotion batch 345262

Product code 345

Treatment: Prototype lotion batch 755465

Panellist Name Arrangement Left Right

1 Mamma Mia AB 118 345

2 BA 345 118

3 BA 345 118

4 AB 118 345

5 BA 345 118

6 AB 118 345

7 AB 118 345

8 AB 118 345

9 BA 345 118

10 BA 345 118

11 AB 118 345

12 BA 345 118

13 BA 345 118

14 AB 118 345

15 BA 345 118

16 AB 118 345

17 AB 118 345

18 AB 118 345

19 BA 345 118

20 BA 345 118

21 AB 118 345

22 BA 345 118

23 BA 345 118

24 AB 118 345

25 BA 345 118

Key points

Definition of product evaluation

Objective vs subjective

To design a good test

Have a clear objective and hypothesis

Define study outline

Have a tested & validated protocol

Get the operational details right

Define your statistics and success criteria

Understand the range of tests used for skin and

hair

To write a good protocol

Always follow the five steps above!

Book list

IFSCC Monograph Number 1. Principles of Product Evaluation: Objective Sensory Methods

Kemp, S.E., Hollowood, T. and Hort, J. Sensory Evaluation: A Practical Handbook. Wiley-Blackwell,

2009

Meilgaard, M.C., Civille, G.V., Carr, B.T. Sensory Evaluation Techniques, Fifth Edition, CRC Press, 2016

Fluhr, J. (Ed.) Practical Aspects of Cosmetic Testing: How to Set up a Scientific Study in Skin

Physiology, Springer 2011

Glantz, S.A. Primer of Biostatistics, Seventh Edition, McGraw-Hill, 2012

diploma in cosmetic science module 4 unit 25 …...iso 5496:2006 sensory analysis iso 11132:2012...

Documents