examination of the validity of nutrient profiling models for …€¦ · to dr. marie-Ève...

Examination of the Validity of Nutrient Profiling Models for Assessing the Nutritional Quality of Foods

by

Theresa H. K. Poon

A thesis submitted in conformity with the requirements for the degree of Master of Science

Department of Nutritional Sciences University of Toronto

© Copyright by Theresa H. K. Poon 2018

ii

Examination of the Validity of Nutrient Profiling Models for

Assessing the Nutritional Quality of Foods

Theresa H. K. Poon

Master of Science

Department of Nutritional Sciences University of Toronto

2018

Abstract

Nutrient profiling (NP) is a method for evaluating the healthfulness of foods. Although

many NP models exist, most have not been validated. This thesis aimed to examine

various aspects of validity of four models developed by authoritative bodies. All four

models demonstrated moderate content validity, which was assessed by examining the

nutrients/components considered in the models. However, different models exhibited

varying levels of construct or convergent validity with Ofcom, a previously validated

model which served as the reference, in classifying over 15,000 Canadian pre-

packaged foods from a branded, national database. Numerous incongruencies were

identified for certain models, which highlights the importance of examining

classifications across food categories, the level at which differences between models

become apparent. The challenges associated with validating models are discussed.

Overall, this thesis provides data that can inform regulators seeking to adapt existing NP

models and validate them for use in country-specific applications for nutrition policies.

iii

Acknowledgments I would like to express my deepest appreciation and sincerest gratitude to the following

individuals who made this experience more rewarding than I could have ever imagined.

To my supervisor, Dr. Mary L’Abbé, thank you for your generous mentorship and

unwavering support. Your energy, dedication, and work ethic is inspiring. Thank you

for giving me the opportunity to realize my long-time goal of undertaking graduate

studies and helping me grow as a researcher.

To Dr. Marie-Ève Labonté, thank you for your always helpful guidance, our excellent

collaboration, and your friendship.

To my committee members, Dr. JoAnne Arcand and Dr. Erin Hobin, thank you for your

insightful advice and encouragement on both academic and professional matters. Your

generosity in sharing your time is deeply appreciated.

To my appraiser, Dr. David Jenkins, thank you for your thoughtful questions and sharing

your valuable time.

To Dr. Kathy Musa-Veloso, thank you for your patience during this undertaking and for

my previous training, which has allowed me to truly appreciate and enjoy this learning

experience.

To the past and current members of the L’Abbé lab, thank you for our ever interesting

discussions that have enlightened me and made me laugh. Thank you for sharing in

this experience with me and making it that much more enjoyable.

Lastly, to my family, thank you for your unconditional support and encouragement.

Specifically to my partner, Chris Kwong, thank you for your selflessness, infinite

patience, and guidance. This milestone is as much yours as it is mine. I am forever

grateful.

iv

Table of Contents Acknowledgments ........................................................................................................... iii

Table of Contents ............................................................................................................ iv

List of Tables.................................................................................................................. vii

List of Figures ............................................................................................................... viii

List of Appendices ........................................................................................................... ix

List of Abbreviations ......................................................................................................... x

Chapter 1 ......................................................................................................................... 1

1 Introduction .................................................................................................................. 1

Chapter 2 ......................................................................................................................... 3

2 Background and Literature Review .............................................................................. 3

2.1 Overview of Nutrient Profiling (NP) ....................................................................... 3

2.1.1 Brief History of NP ...................................................................................... 3

2.1.2 Applications of NP ...................................................................................... 3

2.2 Key Characteristics of NP Models ......................................................................... 4

2.2.1 Intended Objectives and Scope of Models ................................................. 5

2.2.2 Exclusions and Exemptions ........................................................................ 6

2.2.3 Food Categories ......................................................................................... 7

2.2.4 Nutrients/components for Inclusion and Corresponding Nutrient Criteria ... 7

2.2.5 Reference Amounts .................................................................................. 10

2.2.6 Output of Models ...................................................................................... 11

2.2.7 Example NP Models ................................................................................. 12

2.3 Validation of NP Models ...................................................................................... 15

2.3.1 Types of Validity ....................................................................................... 15

2.3.2 Methods of Validity Testing ...................................................................... 18

v

2.4 Current Status of NP and Models ........................................................................ 21

2.4.1 Canadian Context ..................................................................................... 21

2.4.2 Global Context .......................................................................................... 22

2.4.3 Knowledge Gaps ...................................................................................... 26

Chapter 3 ....................................................................................................................... 31

3 Study on Validation of NP Models ............................................................................. 31

3.1 Manuscript ........................................................................................................... 31

3.1.1 Abstract .................................................................................................... 31

3.1.2 Introduction ............................................................................................... 32

3.1.3 Methods .................................................................................................... 34

3.1.4 Results ..................................................................................................... 41

3.1.5 Discussion ................................................................................................ 53

3.1.6 Supplementary Material ............................................................................ 59

3.2 Additional Analyses Not Included in Manuscript .................................................. 65

3.2.1 Summary of Results on Construct/Convergent Validity ............................ 65

3.2.2 Data Not Shown as Reported in Manuscript ............................................. 65

3.2.3 Modified-PAHO......................................................................................... 67

Chapter 4 ....................................................................................................................... 72

4 Overall Discussion ..................................................................................................... 72

4.1 Challenges Associated with Validating NP Models ............................................. 72

4.1.1 Ambiguity in the Definitions and Methods Used in Validating NP Models ...................................................................................................... 72

4.1.2 Lack of a Definition of a Validated Model ................................................. 75

4.2 Other Considerations .......................................................................................... 76

4.3 Significance and Implications of Research .......................................................... 77

Chapter 5 ....................................................................................................................... 80

vi

5 Conclusions ............................................................................................................... 80

References..................................................................................................................... 82

Appendix A .................................................................................................................... 90

Study on NP Models for Marketing Restrictions ............................................................. 90

Appendix B .................................................................................................................. 102

Ethics Review Application for Study on Proposed NP Model for Marketing Restrictions .............................................................................................................. 102

Appendix C .................................................................................................................. 133

Systematic Review of NP Models ................................................................................ 133

Copyright Acknowledgements...................................................................................... 134

vii

List of Tables Table 2-1 Food categories, nutrients/components, and nutrient criteria used in

the WHO EURO model .............................................................................. 9

Table 2-2 Summary of the key characteristics of several NP models used in this research ....................................................................................................13

Table 2-3 Definitions of different types of validity used by different researchers ......16

Table 2-4 Methods of testing different types of validity used by different researchers ...............................................................................................19

Table 2-5 Studies in which construct/convergent validity was assessed based on agreement between NP models (n=19) ...............................................28

Table 3-1 Authors’ contributions to study on validation of NP models ......................31

Table 3-2 Summary of NP models examined ...........................................................39

Table 3-3 Associations between the proportions (%, 95% CI) of foods classified as “less healthy” by the models and quartiles of Ofcom scores (n=15,227) ................................................................................................42

Table 3-4 Method based on the ingredient list used to estimate the FVN points of foods in the Ofcom model .....................................................................60

Table 3-5 Proportion (%) of “healthier” and “less healthy” foods classified by models compared to the Ofcom model for all foods (n=15,227) and by food category .......................................................................................62

Table 3-6 Results of the assessment of construct/convergent validity ......................65

Table 3-7 Proportion (%) of “healthier” and “less healthy” foods classified by the modified-PAHO model compared to the Ofcom model for all foods (n=15,227) and by food category ..............................................................70

Table 4-1 Descriptions for criterion, concurrent, convergent, and construct validity used by different researchers........................................................74

Table A-1 Authors’ contributions to study on NP models for marketing restrictions to children ...............................................................................90

Table B-1 Authors’ contributions to ethics review application for study on proposed NP model for marketing restrictions to children .......................102

Table C-1 Authors’ contributions to systematic review of NP models ......................133

viii

List of Figures Figure 2-1 Flow diagram of the publication and NP model selection. .........................24

Figure 3-1 Overall proportions (%, 95% CI) of “healthier” and “less healthy” foods and agreement (κ) between each model and the Ofcom model (n=15,227). .............................................................................................44

Figure 3-2 Agreement (κ, 95% CI) and discordance (%) between the Food Standards Australia New Zealand (FSANZ) and Ofcom model for all foods (n=15,183; data missing for n=44) and 22 food categories. ............45

Figure 3-3 Agreement (κ, 95% CI) and discordance (%) between the Health Canada Surveillance Tool (HCST) and Ofcom model for all foods (n=15,165; data missing for n=62) and 22 food categories. ......................47

Figure 3-4 Cross-classification analyses between four Health Canada Surveillance Tool (HCST) tiers versus quartiles of Ofcom scores for all foods (n=15,165; data missing for n=62) and 22 food categories. .......48

Figure 3-5 Agreement (κ, 95% CI) and discordance (%) between the World Health Organization Regional Office for Europe (EURO) and Ofcom model for all foods (n=15,182; data missing for n=45) and 22 food categories. ................................................................................................50

Figure 3-6 Agreement (κ, 95% CI) and discordance (%) between the World Health Organization Regional Office for the Americas/Pan American Health Organization (PAHO) and Ofcom model for all foods (n=15,182; data missing for n=45) and 22 food categories. ......................52

Figure 3-7 Cross-classification analyses between quartiles of Food Standards Australia New Zealand (FSANZ) and Ofcom scores for all foods (n=15,183; data missing for n=44) and 22 food categories. ......................66

Figure 3-8 Agreement (κ, 95% CI) and discordance (%) between the modified World Health Organization Regional Office for the Americas/Pan American Health Organization (modified-PAHO) and Ofcom model for all foods (n=15,177; data missing for n=50) and 22 food categories. ................................................................................................69

ix

List of Appendices Appendix A Study on NP Models for Marketing Restrictions to Children

Appendix B Ethics Review Application for Study on Proposed NP Model for Marketing Restrictions to Children

Appendix C Systematic Review of NP Models

x

List of Abbreviations CFG Canada’s Food Guide

CI confidence interval

EURO World Health Organization Regional Office for Europe

FDA Food and Drug Administration

FSANZ Food Standards Australia New Zealand

FVN/L fruits, vegetables, nuts, and/or legumes

HCST Health Canada Surveillance Tool

NP nutrient profiling

ONQI Overall Nutritional Quality Index

PAHO World Health Organization Regional Office for the Americas/Pan American Health Organization

PNIG Population Nutrient Intake Goals to Prevent Obesity and Related Non-communicable Diseases, World Health Organization

RR relative risk

U.S. United States

UK United Kingdom

WHO World Health Organization

1

Chapter 1

1 Introduction Nutrient profiling (NP) is defined as the science of classifying foods according to their

nutritional composition for the purpose of promoting health and preventing disease(1).

NP models have numerous applications in government and industry, for example, the

regulation of health claims, front-of-pack nutrition labelling, restrictions on marketing of

less healthy foods to children, and guidelines on product reformulation, amongst other

applications(1-3). Overall, NP models are important for ensuring that regulations and

health promotion strategies are aligned with dietary guidance and public health

objectives in the reduction of diet-related non-communicable diseases.

Based on a series of systematic reviews on the use of NP for various applications that

have been updated over the years(4-8), the number of potential NP models globally has

increased dramatically. Given the proliferation of NP models and the extensive

resources required to develop and validate a new model, the adaptation of an existing

NP model is preferred by the World Health Organization (WHO) and is becoming an

increasingly common practice for government and industry(1,2,9). However, although

many NP models exist, most models have not been fully evaluated for validity(2,7,8,10).

Even amongst NP models that have been evaluated for validity, most have been

validated only to a limited extent(8). Moreover, in the context of adaptation, NP models

need to be validated when used for different applications and in different countries, as it

is unknown whether the criteria that underpins a model developed to assess foods for

one application or from one population are appropriate for another(1,9). Furthermore, the

methods of validating NP models are in their infancy(2). As such, several researchers

have suggested that validity testing of NP models should be given the highest priority in

this field of research(3,10,11). Thus, the objective of this thesis was to examine the

content and construct/convergent validity of several NP models developed by

authoritative bodies for assessing the nutritional quality of foods in a Canadian context.

2

In Chapter 2, background on NP models and a review of the literature on the validation

of models is provided. In Chapter 3, the manuscript of the validation study is presented.

In Chapter 4, the challenges associated with validating NP models along with other

considerations are discussed. Overall, this thesis aims to explore whether four NP

models are valid tools to assess the nutritional quality of foods in Canada. This

research will contribute to the emerging science of NP and provide regulators with

pertinent data that will facilitate the selection and validation of NP models for public

health initiatives and nutrition policy.

3

Chapter 2

2 Background and Literature Review

2.1 Overview of NP

2.1.1 Brief History of NP

Although NP is a relatively new term in the field of nutrition research, the concept of NP

itself has been in use for some time(1,9). Several of the earliest forms of NP were

introduced by government bodies in the 1980s and 1990s(9), including the:

• minimum nutritional requirements for foods as part of the United States Department of Agriculture’s Special Supplemental Nutrition Program for Women, Infants, and Children in 1980(12);

• Swedish Keyhole by the Swedish National Food Administration in 1989(13); • disqualifying nutrient levels for health claims by the United States (U.S.) Food

and Drug Administration (FDA) in 1993(14); and • minimum nutrient criteria for the “healthy” implied nutrient content claim by the

U.S. FDA in 1993(15).

The term NP gained ground and began to be referred to as such following the

development of the Ofcom model by the United Kingdom (UK) Food Standards Agency

in 2004 and 2005(4,5) and the mention of nutrient profiles in Regulation (EC) No

1924/2006 on nutrition and health claims by the European Commission in 2006(16). In

2010, NP became even more widely known when the WHO provided to its Member

States a set of recommendations on the marketing of foods to children, one of which

advocated the use of NP models in defining the foods to be covered by the marketing

restrictions(17).

2.1.2 Applications of NP

As NP is increasingly recognized by authoritative bodies and industry as a transparent

and reproducible method for evaluating the healthfulness of foods, NP models are now

having numerous applications in government and industry, including for example(1-3,8):

• regulations of nutrition and health claims;

4

• guidelines for front-of-pack nutrition labelling; • restrictions on the marketing of foods to childrena; • food procurement policies for public institutions (e.g., schools, hospitals); • economic tools to orient food consumption (e.g., food subsidies or taxes); • standards for the composition of foods; • food fortification policies; and • product reformulation.

Overall, NP is important in ensuring that regulations and health promotion strategies are

aligned with dietary guidance and public health objectives in the reduction of diet-related

non-communicable diseases. NP should be thought of as a tool to be used in

conjunction with interventions aimed at improving diets within a jurisdiction(1).

It is important to differentiate between the NP model itself and the application of that

model(9). Ideally, a single model should be used across policies within a jurisdiction,

while the threshold criteria or cut-off scores of the model are adjusted for different

applications(9). For example, the Ofcom model may be used across multiple policies

within the UK; however, less stringent threshold criteria may be chosen to underpin the

regulations on front-of-pack labelling which is applicable to the general population,

whereas more stringent threshold criteria may be chosen to underpin the restrictions on

marketing to children in the context of protecting vulnerable populations.

2.2 Key Characteristics of NP Models This section provides an overview of NP models by summarizing the key characteristics

of these models, including:

• intended objectives and scope; • exclusions and exemptions; • food categories; • nutrients/components for consideration and corresponding nutrient criteria;

a We conducted a study on the application of NP models for the restriction of marketing unhealthy foods to children. This publication is presented in Appendix A (Labonté MÈ, Poon T, Mulligan C, Bernstein JT, Franco-Arellano B, L’Abbé MR. Comparison of global nutrient profiling systems for restricting the commercial marketing of foods and beverages of low nutritional quality to children in Canada. Am J Clin Nutr 106, 1471-1481.).

5

• reference amounts; and • output.

2.2.1 Intended Objectives and Scope of Models

The intended objectives of NP models may vary with regards to(1,9):

1) whether the model is used for a single application or multiple applications; 2) the end-user of the model; 3) whether the model aims to identify foods with a favourable profile (i.e., “healthier”

foods), an unfavourable profile (i.e., “less healthy” foods), or both; and 4) whether the model aims to shift consumption to “healthier” foods between or

within food categories.

The objectives of the model are crucial in determining the type and characteristics of the

model needed for a particular application. For example, a relatively simple model may

be preferable when the end-user of the model includes the general public viewing the

front-of-pack rating system on nutrition labels, whereas a relatively complex model may

be used when the end-user includes the regulatory officers who apply the model to

determine the eligibility of products for subsidies under food assistance programs.

Moreover, the inclusion of nutrients to limit (e.g., sodium) in a model may be sufficient

when it primarily aims to identify “less healthy” foods, whereas the inclusion of both

nutrients to limit and nutrients to encourage (e.g., fibre) may be preferable when the

model aims to identify foods across the full spectrum of healthfulness. Lastly, a model

with an “across the board” design, such that the same nutrient criteria are applied to all

foods, may be preferable when the model aims to encourage healthier choices between

categories (e.g., from ready-to-eat breakfast cereals high in sugar versus fruit)(1). In

contrast, a model with a “food category specific” design, such that the nutrient criteria

are applied at the food category level, may be preferable when the model aims to

encourage healthier choices within categories (e.g., from ready-to-eat breakfast cereals

high in sugar versus ready-to-eat breakfast cereals high in fibre)(1).

In addition, the scope of models may vary with regards to the form of food that is

specifically subjected or not subjected to the model(9). For example, models typically

cover either foods as sold or as consumed(9). Although the intended objectives and

scope of the models should be dictated by the aforementioned practical aspects

6

associated with the use of the models, often the objectives and scope are influenced by

the legislative frameworks and political context in which the models must thrive(9).

2.2.2 Exclusions and Exemptions

NP models may incorporate exclusions and exemptions, which refer to foods or food

categories that are explicitly allowed or not allowed as they relate to the application of

the model. Exclusions refer to foods that are never allowed by a model, while

exemptions refer to foods that are always allowed by a model. It is important to

differentiate between exclusions versus foods that are outside the scope of the model,

the latter which often includes foods for special medical purposes (e.g., termed “medical

foods” in the U.S.) and meal replacements because their nutritional composition is

usually defined in regulation. For example, in the case of school food models, soft

drinks are often exclusions because the models explicitly aim to prohibit soft drink

consumption in schools, whereas infant and follow-on formula are not explicitly

prohibited and would be considered outside the scope of these models with school-

aged children as the target population.

It should be noted that there is inconsistency with regards to this differentiation across

NP models, as illustrated in Table 2-2 in Section 2.2.7. According to the model

developed by the Food Standards Australia New Zealand (FSANZ) for the regulation of

claims(18), infant formula is explicitly prohibited from bearing nutrition content or health

claims; as such, infant formula should be considered as an exclusion. In contrast,

according to the model developed by the WHO Regional Office for Europe (EURO) for

the restrictions on the marketing of foods to children(19), infant formula is reported to be

“not covered by this model”; given that the decision on whether infant formula is

prohibited for marketing to children was not explicitly stated, infant formula should be

considered to be outside the scope of this model. Instead, seven food categories are

7

explicitly stated as not permitted for marketing to children and are considered as

exclusionsb.

2.2.3 Food Categories

The number and type of food categories included in a NP model vary widely across

different models(8). A model may have many, several, or even no food categories. The

most common method of defining food categories within a model is to align with existing

definitions in food regulations or categories in food composition databases(1). The food

categories may be organized simply as foods versus beverages, or by main categories

(e.g., bakery versus dairy products), sub-categories (e.g., milks versus yogurts), and

even sub-sub-categories (e.g., almond versus soy beverages). There are advantages

and disadvantages associated with various options(1). For example, while a model with

many food categories of different levels is more complex and may hinder the usability of

the model, it may be more effective in encouraging product reformulation(1);

alternatively, the application of a model with few food categories across different food

supplies is much simpler. Often the decisions related to food categories are made in

the context of balancing comprehensiveness of the model, feasibility of use of the

model, and consistency with other regulatory or policy tools in place(1).

2.2.4 Nutrients/components for Inclusion and Corresponding Nutrient Criteria

The number and type of nutrients/components included in a NP model vary widely

across different models(8). A model may have many or only several

nutrients/components with nutrient criteria. A model may include nutrients/components

to limit (e.g., energy, sodium, sugars), nutrients/components to encourage (e.g., protein,

fibre, fruits and vegetables), or both. There are advantages and disadvantages

associated with various models(1); for example, while a model with many

nutrients/components is more complex and may hinder usability of the model, it is more

b The exclusions included: 1) chocolate and sugar confectionery, energy bars, and sweet toppings and desserts; 2) cakes, sweet biscuits, and pastries; other sweet bakery wares, and dry mixes for making such; 3) beverages: juices; 4) beverages: energy drinks; 5) edible ices; 6) foods containing >1 g of industrially-produced trans fat per 100 g total fat; and 7) foods with alcohol content ≥0.5% of total energy.

8

comprehensive and may be more effective in differentiating the healthfulness of different

foods.

The consideration of which nutrients/components to include in a model is largely

dependent on the availability of nutrient composition data(1,9). For example, although

the WHO guidelines on sugars are for free sugarsc, added sugars were included in the

WHO EURO model due to the availability of added sugard data within the food

composition tables of European countries(19). In addition, the consideration of

nutrients/components to include is influenced by the co-linearity amongst nutrients such

that the inclusion of additional correlated nutrients lends itself to redundancy(9). For

example, according to the model adapted by the Queensland government in Australia

under their Food for Sport Guidelines, total fat and added sugars were intentionally

excluded from consideration; rather, the model opted to include a nutrient criterion for

total energy, which was said to have accounted for restricting the amount of fat or

sugars from a food(20).

As discussed in Section 2.2.1, the nutrient criteria are applied across all foods in “across

the board” models, whereas the nutrient criteria are applied at the food category level in

“food category specific” models(1). It should be noted that the way in which the nutrient

criteria are applied in “food category specific” models varies widely across different

models(8). The nutrient criteria may be applied at one or more food category levels

(e.g., for main categories only or for both main and sub-categories). In addition, the

nutrient criteria may differ for various food categories, as certain nutrients/components

may be more pertinent for specific food categories (e.g., the consideration of sodium

content, as opposed to sugar content, may be more pertinent for cheese products).

Thus, the number of food categories and the level at which the nutrient criteria are

applied may not always align and adds to the complexity to the model, as illustrated by

the WHO EURO model in Table 2-1.

c Free sugars are defined as “monosaccharides and disaccharides added to foods and beverages by the manufacturer, cook, or consumer, and sugars naturally present in honey, syrups, fruit juices, and fruit juice concentrates (WHO, 2015)(19). d Added sugars are defined as “all monosaccharides and disaccharides added to foods and beverages by the manufacturer, cook, or consumer during processing or preparation” (WHO, 2015)(20).

9

Table 2-1 Food categories, nutrients/components, and nutrient criteria used in the WHO EURO modela

Main and Sub-Categories

Nutrients/Components and Corresponding Criteriab Total Fat (g)

Saturated Fat (g)

Total Sugars (g)

Added Sugars (g)

Non-sugar Sweeteners (g)

Salt (g) Energy (kcal)

Savoury snacks – – – 0 – 0.1 – Beverages

Milk drinks 2.5 – – 0 0 – – Other beverages – – – 0 0 – –

Breakfast cereals 10 – 15 – – 1.6 – Yoghurts, sour milk, cream and other similar foods

2.5 2.0 10 – – 0.2 –

Cheese 20 – – – – 1.3 – Ready-made and convenience foods and composite dishes

10 4 10 – – 1 225

Butter and other fats and oils

– 20 – – – 1.3 –

Bread, bread products and crisp breads

10 – 10 – – 1.2 –

Fresh or dried pasta, rice and grains

10 – 10 – – 1.2 –

Processed meat, poultry, fish and similar

20 – – – – 1.7 –

Processed fruit, vegetables and legumes

5 – 10 0 – 1 –

Sauces, dips and dressings

10 – – 0 – 1 –

Abbreviation: WHO EURO, World Health Organization Regional Office for Europe. a Data are from WHO, 2015(19). The model consists of seven exclusions: 1) chocolate and sugar confectionery, energy bars, and sweet toppings and desserts; 2) cakes, sweet biscuits, and pastries; other sweet bakery wares, and dry mixes for making such; 3) beverages: juices; 4) beverages: energy drinks; 5) edible ices; 6) foods containing >1 g of industrially-produced trans fat per 100 g total fat; and 7) foods with alcohol content ≥0.5% of total energy. It also consists of 2 exemptions: 1) fresh and frozen meat, poultry, fish, and similar; and 2) fresh and frozen fruit, vegetables, and legumes. b If a food exceeds on a per 100 g basis any of relevant nutrient criteria for that food category, the food is classified as not permitted for marketing to children.

In addition to the exemptions for foods or food categories within models such that they

are not subject to any nutrient criteria (as discussed in Section 2.2.2), models may also

include exemptions to specific nutrient criteria for certain foods or food categories. For

10

example, this is commonly done for nuts. As certain types of nuts contain high levels of

mono- or polyunsaturated fat, their total fat content is correspondingly high. However,

given that salted nuts with high sodium content are widely available and consumed, it is

imperative that the food category for nuts not be treated as an exemption without

subjection to any nutrient criteria. Thus, in order to account for their high unsaturated

fat content, nuts are often exempted from only the total fat criterion, but not other

criteria.

Lastly, the basis of the nutrient criteria applied in a model varies widely across different

models(8). This is due to the fact it is advantageous to have coherence between the

nutrient criteria that underpin the model and nutrient criteria used in other regulatory or

policy tools within the same jurisdiction(1). For example, the nutrients and their criteria

included in the model developed by the WHO Regional Office for the Americas/Pan

American Health Organization (PAHO) were based on the WHO Population Nutrient

Intake Goals to Prevent Obesity and Related Non-communicable Diseases (PNIGs) and

their updates(21,22), including the WHO guidelines on sugars(23) and sodium(24), and the

expert consultation on fats(25). The PNIGs, defined as the average dietary intake

recommended for maintenance of good health in a population(22), were formulated after

comprehensive reviews of all the updated evidence related to specific nutrient intakes

and public health outcomes(21). According to the PAHO model, a food is classified as

“excessive” in one or more of the critical nutrients (i.e., total fat, saturated fat, trans fat,

sodium, and free sugars) if it contains more than the corresponding maximum level

recommended in the PNIGs(21,22). Although the PNIGs relate to the overall diet and not

to individual foods, the rationale for their use as the basis of the nutrient criteria of the

model was that the consumption of individual foods classified as “excessive” increases

the likelihood that the diet also will exceed the recommended nutrient goals(21).

2.2.5 Reference Amounts

The reference amount of a NP model refers to the standard amount of a food that

serves as the basis for the calculations of the nutrients/components present in the

food(1). The most commonly used reference amounts include per 100 g and/or 100 mL,

per 100 kcal, or per serving of the food(1,9). The reference amount used by a model

varies across different models, and there are advantages and disadvantages associated

11

with various options(1,8). For example, the reference amount per 100 g and/or 100 mL is

frequently used because it often corresponds to the data from food composition

databases or the basis for other nutrition-related tools in regulations (e.g., nutrition

labels and nutrition and health claims in the European Union are determined on a per

100 g or 100 mL basis(16)). However, this reference amount is problematic for foods

consumed in small amounts because the level of certain nutrients may appear to be

high when calculated on a per 100 g basis (e.g., sodium in yeast extracts)(1). While a

per 100 kcal basis may be advantageous because it can give an indication of the

nutrient density of a food, it is problematic for low or high energy foods(1). For example,

the WHO PAHO model utilizes a derivative of the per 100 kcal basis, such that the

sodium criterion (i.e., ≥1 mg/kcal per serving) resulted in beverages with very low

sodium and energy content (e.g., 5 mg sodium and 1 kcal per serving) to be classified

as “excessive” in sodium, when in fact, these beverages were not significant sources of

sodium(26)e. Furthermore, a combination of reference amounts may be used within a

model(1,9). For example, according to the WHO PAHO model, the reference amount

used for several of the nutrient thresholds is the contribution of those nutrients to the

total energy of the food provided in a single serving (e.g., ≥30% of total energy from

total fat in any given quantity of the food)(21).

2.2.6 Output of Models

Depending on the output generated by NP models, they can be categorized as scoring

models, threshold models, or both(1). Scoring models generate a score for each food

and allow for foods to be ranked (e.g., from “healthiest” to “least healthy”)(1). In contrast,

threshold models typically generate binary classifications of the foods (e.g., “healthier”

or “less healthy”) and do not allow foods to be ranked(1). However, threshold models

that generate more than two classifications (e.g., the red, amber, and green

classifications by the UK traffic light model(27)) allow foods to be ranked into categories

e In our study on the application of NP models for the restriction of marketing unhealthy foods to children, we accounted for this discrepancy by adjusting the sodium criterion in these beverages. This adjustment is described in our publication presented in Appendix A (Labonté MÈ, Poon T, Mulligan C, Bernstein JT, Franco-Arellano B, L’Abbé MR. Comparison of global nutrient profiling systems for restricting the commercial marketing of foods and beverages of low nutritional quality to children in Canada. Am J Clin Nutr 106, 1471-1481.).

12

of increasing or decreasing healthfulness. Models that generate both types of output

allow for ranking and classification by specifying pre-determined threshold criteria or

cut-off scores. For example, the Ofcom model generates a total score between -15 to

40 (from “healthiest” to “least healthy”), and if this total score is greater than or equal to

the cut-off score of 4 for foods or 1 for beverages, then the food is classified as “less

healthy”(28).

As discussed in Section 2.1.2, it is ideal to use a single model to underpin different

policies within a jurisdiction, with the threshold criteria or cut-off scores adjusted for

different applications of the model(9). For example in the case of the Ofcom model, the

cut-off scores (i.e., 4 for foods, 1 for beverages) underpinning the restrictions on

marketing to children, in the context of protecting vulnerable populations, could be

shifted higher along the scale from -15 to 40 (from “healthiest” to “least healthy”) to less

stringent cut-off scores for the regulations on front-of-pack labelling, which are

applicable to the general population.

2.2.7 Example NP Models

As discussed above, the characteristics of a NP model vary widely across different

models. This is illustrated in Table 2-2, which provides a summary of the key

characteristics of several NP models developed by authoritative bodies that are

evaluated in this thesis.

13

Table 2-2 Summary of the key characteristics of several NP models used in this research

Characteristic of NP Model Ofcom(28) FSANZ(18) HCST(29) EURO(19) PAHO(21) Intended objective/ application

Restrictions on marketing foods to children

Regulation of health claims

Surveillance of dietary intakes of Canadians relative to CFG



Scope of model • Within: as sold or as consumed

• Outside: not specified

• Within: as sold or as consumed

• Outside: 4 categoriesa

• Within: as consumed

• Outside: 3 categoriesb

• Within: as sold or as consumed

• Outside: follow-up formula and growing-up milks

• Within: not specified • Outside: foods for

special uses (breast milk substitutes), food supplements, alcoholic beverages

Exclusions (n) Not specified 3c Not specified 7d Not specified Exemptions (n) 0 Not specified 0 2e 3f Food categories (n) 2g 3h 4i 20j 5k Nutrients/components (n) 7 7 4 8 6 Nutrients to limit

Energy Total fat

Saturated fat

Trans fat (industrial sources) (total) Sodium

Total sugars Free/added sugars (added) (free) Sweeteners

Nutrients to encourage Protein Fibre Fruits/vegetables (FVN) (FVNL)

14

Table 2-2 Summary of the key characteristics of several NP models used in this research

Characteristic of NP Model Ofcom(28) FSANZ(18) HCST(29) EURO(19) PAHO(21) Level at which nutrient criteria applied

Main categories; across the board


Main categories; food category specific

Main and subcategories; food category specific


Reference amount 100 g 100 g or mL Serving 100 g % energy of foodl Output Score and

classification Score and classification

Classification Classification Classification

Abbreviations: CFG, Canada’s Food Guide; EURO, World Health Organization Regional Office for Europe; FSANZ, Food Standards Australia New Zealand; FVN/L, fruits, vegetables, nuts, and/or legumes; HCST, Health Canada’s Surveillance Tool; NP, nutrient profiling; PAHO, World Health Organization Regional Office for the Americas/Pan American Health Organization. a Foods that are 1) intended for further processing, packaging, or labelling prior to retail sale; 2) delivered to a vulnerable person by a delivered meal organization; 3) provided to a patient in a hospital or medical institution, other than food in a package; and 4) special purpose foods. b 1) "Other" foods and beverages recommended in CFG (e.g., unsaturated fats and oils, water); 2) "other" foods and beverages not in the food groups of CFG (e.g., saturated and/or trans fats and oils; beverages; uncategorized; alcoholic beverages; high fat and/or sugar foods); and 3) foods and beverages that are not classified (i.e., recipes; foods with missing nutrient data in the Canadian Nutrient File; infant formula). c 1) Kava; 2) foods with alcohol content >1.15% by volume; and 3) infant formula. d 1) Chocolate and sugar confectionery, energy bars, and sweet toppings and desserts; 2) cakes, sweet biscuits, pastries, other sweet bakery wares, and dry mixes for making such; 3) beverages: juices; 4) beverages: energy drinks; 5) edible ices; 6) foods containing >1 g of industrially-produced trans fat per 100 g total fat; and 7) foods with alcohol content ≥0.5% of total energy. e 1) Fresh and frozen meat, poultry, fish, and similar; and 2) fresh and frozen fruit, vegetables, and legumes. f 1) Unprocessed or minimally processed foods; 2) culinary ingredients; and 3) freshly prepared dishes. g 1) Beverages; and 2) foods. h 1) Beverages; 2) any food not in category 1 or 3; and 3) cheese high in calcium (i.e., >320 mg/100 g of food) and fats. i 1) Vegetables and fruits; 2) grain products; 3) milk and alternatives; and 4) meat and alternatives. j The model consists of 17 main categories, with the beverage category containing 4 sub-categories, for a total of 20 unique categories: 1) chocolate and sugar confectionery, energy bars, and sweet toppings and desserts; 2) cakes, sweet biscuits, pastries, other sweet bakery wares, and dry mixes for making such; 3) savoury snacks; 4) beverages: juices; 5) beverages: milk drinks; 6) beverages: energy drinks; 7) beverages: other beverages; 8) edible ices; 9) breakfast cereals; 10) yogurts, sour milk, cream, and other similar foods; 11) cheese; 12) ready-made and convenience foods and composite dishes; 13) butter and other fats and oils; 14) bread, bread products, and crisp breads; 15) fresh or dried pasta, rice, and grains; 16) fresh and frozen meat, poultry, fish, and similar; 17) processed meat, poultry, fish, and similar; 18) fresh and frozen fruit, vegetables, and legumes; 19) processed fruit, vegetables, and legumes; and 20) sauces, dips, and dressings. k 1) Ultra-processed foods; 2) processed foods; 3) unprocessed or minimally processed foods; 4) culinary ingredients; and 5) freshly prepared dishes. l While sweeteners were evaluated based on their absence or presence in the ingredient list and sodium was evaluated on a per kcal basis, the thresholds for the other nutrients were presented on as a % energy of the food (e.g., an excess of total fat is ≥30% of total energy of the food).

15

2.3 Validation of NP Models

2.3.1 Types of Validity

In general, validity is defined as the adequacy with which a measurement reflects what

is intended to be measured(30). Correspondingly, validation refers to the process of

establishing that a method is sound or that data are correctly measured(31). It should be

noted that validity is distinct from other related concepts, including accuracyf, precisiong,

and reliability or reproducibilityh. There are two fundamental categories for examining

validity: internal and external(30,31). Internal validity is the adequacy of the measurement

for the specific population being studied, whereas external validity (also referred to as

generalizability) is the adequacy of the measurement when applied to wider populations

not under study(30,31).

In alignment with the definition of NP, the validity of a NP model refers to the adequacy

with which the model classifies the healthfulness of foods for the purpose of promoting

health and preventing disease. There are different types of validity, including criterion

(further classified into predictive and concurrent), convergent, discriminant, construct,

content, and face validity(1,2,10,32-34). The different types of validity vary with respect to

their robustness in contributing to the validation of a model. The classic definitions of

the various types of validity according to Porta (2014)(31) and Cronbach and Meehl

(1955)(35) are summarized in Table 2-3. In addition to the classic definitions, the

definitions of validity used in the context of NP are provided to allow for comparisons of

the variations in definitions used by different experts on NP validation, namely:

Scarborough et al. (2007)(33), Arambepola et al. (2008)(32), Townsend (2010)(34), Rayner

(who prepared the WHO manual on NP models (2011, in press(1))), and Cooper et al.

(2016(10), 2017(36)).

f Accuracy is the degree to which a measurement represents the true value of the attribute that is being measured(27). g Precision is the quality of being sharply defined or stated through exact detail(27). h Reliability or reproducibility is the degree of stability or capacity to yield the same result when a measurement is repeated under identical conditions(2,27).

16

Table 2-3 Definitions of different types of validity used by different researchers (shaded cells indicate those upon which analyses in Chapter 3 were baseda)

Type Definitions Reference Criterion Extent to which the measurement correlates with an external criterion of the phenomenon under

study; ideally, a gold standard(31) (32), but it need not(33). • Porta, 2014(31) • Arambepola et al., 2008(32) • Scarborough et al., 2007(33)

Accuracy of the NP model scores based on an externally derived objective measure of healthfulness.

• Cooper et al., 2016(10)

Extent to which the method is accurately based on an externally derived gold standard; examines whether method correlates in a predicted manner with variables with which, theoretically, it should correlate.

• Townsend, 2010(34)

Predictive (type of criterion)

Extent to which the measurement is able to predict an external criterion of the phenomenon under study.b

• Porta, 2014(31) • WHO, 2011 (in press)(1)

Independent criterion measure is obtained after the test score. • Cronbach and Meehl, 1955(35) Extent to which the system reflects a population’s or an individual’s diet change over time; extent to which the system reflects a change in nutritional and health status.


Ability of the tool to predict future outcomes, in which case the tests are performed at different times and then the correlation between the result of the tool and the secondary outcome is determined.


Concurrent (type of criterion)

Measurement and criterion refer to the same point in time.c • Porta, 2014(31) • Cronbach and Meehl, 1955(35)

Requires that the tool be tested against some other method acknowledged as a gold standard for assessing the same variable, and the two tests are conducted at the same time.


Convergent Extent to which the system is accurately based on self-report (internally derived) data with hypothesized relation; examines whether method correlates in a predicted manner with variables with which, theoretically, it should correlate.


Extent to which the measurement correlates with an external criterion of the phenomenon under study at the same point in time.

• WHO, 2011 (in press)(1)

Comparison with other measures, not necessarily better measures, of the same variable or a closely related variable.

• Arambepola et al., 2008(32)

17

Table 2-3 Definitions of different types of validity used by different researchers (shaded cells indicate those upon which analyses in Chapter 3 were baseda)

Type Definitions Reference Discriminant Extent to which the system discriminates between groups expected to be different. • Townsend, 2010(34)

Comparison with other measures, not necessarily better measures, of variables that are not closely related.

• Arambepola et al., 2008(32)

Construct Extent to which the measurement corresponds to theoretical concepts (constructs) concerning the phenomenon under study.d

• Porta, 2014(31) • Arambepola et al., 2008(32) • WHO, 2011 (in press)(1)

Correlation between how the NP model ranks the healthfulness of foods in comparison to other measures that determine the healthfulness of food.


Comparison with other measures of the same variable, or closely related ones. • Cooper et al., 2017(36) Content Extent to which the measurement incorporates the domain of the phenomenon under study.e • Porta, 2014(31)

• WHO, 2011 (in press)(1) Extent to which the system covers the full range of meaning for the concept being measured; assessment of the consistency between the science underlying the algorithms and the science published in the peer-reviewed literature.


Consideration and analysis of the components that make up the NP model with reference to current scientific literature.


Face Extent to which a measurement or a measurement instrument appears reasonable on superficial inspection.

• Porta, 2014(31)

Extent to which the system is a useful tool to the consumer making food-purchase decisions in the marketplace; determined by end users of the system (i.e., the consumer).

• Townsend, 2010(34) • Cooper et al., 2016(10)

Abbreviation: NP, nutrient profiling. a The shaded cells indicate the definitions upon which the analyses in the validation study (Chapter 3) were based. Refer to Section 4.1.1 for the full discussion. b An example is an academic aptitude test that is validated against subsequent academic performance(31). c An example is a visual inspection of a wound for evidence of infection validated against bacteriological examination of a specimen taken at the same time(31). d For example, if on theoretical grounds the phenomenon should change with age, a measurement with construct validity would reflect such a change(31). e For example, a measurement of functional health status should embrace activities of daily living (e.g., occupational, family, and social functioning)(31).

18

2.3.2 Methods of Validity Testing

The validity of NP models can be tested using a variety of methods(2,9,37). These

methods are summarized in detail in Table 2-4. In addition, these methods are cross-

referenced to the type of validity that can be inferred, which often differs between

various experts on validation(1,2,10,32-34).

Briefly, one can assess whether the classifications of foods determined using the model

are aligned with, for example(1,2,9,36):

1) food-based dietary guidelines; 2) views of nutrition professionals; 3) classifications determined by other NP models; 4) foods included as part of either self-reported or theoretically or modelled “healthy”

and “less healthy” diets; or 5) foods for which their consumption has been associated with various health

outcomes.

The methods listed above are associated with their respective strengths and

limitations(2,9,37). The first several methods (i.e., methods 1 to 4) can be relatively

simpler, but may be less robust methodologically. For example, the views of nutrition

professionals and dietary patterns self-reported by “healthy” eaters are subject to bias

by individual, socioeconomic, and cultural factors(38). While theoretically or modelled

“healthy” and “less healthy” dietary patterns are relatively more objective, these diets

simulate food choices based on a set of pre-defined and purposefully selected

constraints (e.g., for protein, >65 g/day for models of “healthy” diets and <65 g/day for

models of “less healthy” diets)(38), which limit their real-world relevance compared to

observed dietary patterns of free-living individuals. Furthermore, while biomarkers and

medical records are objective indicators of health status, the methods that incorporate

them are complex, costly, and time-consuming as they are typically collected from

prospective cohort studies or randomized clinical trials(2). As such, the WHO

recommends that simpler validation methods should be applied first, typically during

early development of the model, to ensure robust classifications of the foods(9). More

complex validation methods should be applied thereafter to increase the evidence-base

and confidence in support of the use of the model(9).

19

Table 2-4 Methods of testing different types of validity used by different researchers (shaded cells indicate those upon which analyses in Chapter 3 were baseda)

Methods of Validity Testing that have been used in NP

Validity Type and Reference Criterion Predictive

(type of criterion)


Convergent Discriminant Construct Content Face

Outcome is an observed change in health status or disease risk using biomarkers/medical records:

(34)

1) Measured at different time points (i.e., based on longitudinal data)

(1,2,10)

2) Measured at the same time (i.e., based on cross-sectional data)

(10)

Outcome is an observed change in dietary intake of individuals:

1) Measured at different time points (i.e., based on longitudinal data)

(10,34)

2) Measured at the same time (i.e., based on cross-sectional data)

(34)

3) Used to predict change in health status

(10)

Outcome is an observed change in gross food sales receipts

(10,34)

Outcome is a predicted change in dietary intake of individuals using theoretically or modelled “healthy” or “less healthy” diets or dietary patterns

(34) (1,10,32,37)

Determined by convening a panel of a group of experts in relevant fields (e.g., comparison to views of nutrition professionals)b

(33) (1,34) (10,36) (1,34)

20

Table 2-4 Methods of testing different types of validity used by different researchers (shaded cells indicate those upon which analyses in Chapter 3 were baseda)

Methods of Validity Testing that have been used in NP

Validity Type and Reference Criterion Predictive

(type of criterion)


Convergent Discriminant Construct Content Face

Comparison of the usage of the system between nutrition professionals and high school graduates

(34)

Comparison to other NP models (1) (10,36)

Comparison to food-based dietary guidelines

c (32) d (32) (10,36) (1)

Outcome is the agreement with nutritional endpoints (e.g., energy density of foods)

(10)

Outcome is the consistency between scientific underpinnings of algorithm of model and current scientific literature

(10,34)

Consideration of nutrients that are of major public health concern

(32)

Outcome is the consumer groups’ use and understanding (e.g., FOP systems)

(10,34)

Abbreviations: BGH, Balance of Good Health; FOP, front-of-pack; NP, nutrient profiling. a The shaded cells indicate the definitions and methods upon which the analyses in the validation study (Chapter 3) were based. Refer to Section 4.1.1 for the full discussion. b Although not explicitly part of my MSc thesis, we are currently conducting a study to examine the validity of the NP model proposed for use in Canada against the views of nutrition professionals (refer to Appendix B). c The UK’s national food guide, BGH, indicates that individuals should consume many of foods from ‘fruit and vegetables’, ‘bread, other cereals and potatoes’; intermediate amounts of foods from ‘milk and dairy foods’, ‘meat, fish and alternatives’; and ‘fatty and sugary foods’ sparingly. Therefore, testing for convergent validity should indicate that 1) foods classified as “healthier” by the NP model should be found more frequently within foods categorized by the BGH as ‘fruit and vegetables’ and ‘bread, other cereals and potatoes’, and 2) foods categorized by the model as “less healthy” should be found more frequently within foods categorized by the BGH as ‘fatty and sugary foods’(32). d Testing for discriminant validity should indicate no relationship between the way a NP model categorizes foods and the way ‘milk and dairy foods’ and ‘meat, fish and alternatives’ are categorized by the BGH(32).

21

2.4 Current Status of NP and Models

2.4.1 Canadian Context

In Canada, NP has been used primarily at the provincial level for school foods(8). Only

two models developed for use at the national level were identified, namely the Canada

Provincial and Territorial Nutrient Criteria for Foods and Beverages in Schools in

2013(39) and the Health Canada’s Surveillance Tool (HCST) tier system developed in

2014 for assessing the adherence of the dietary intakes of Canadians to Eating Well

with Canada’s Food Guide(8). More recently on October 24, 2016, the Minister of Health

announced the launch of Health Canada’s Health Eating Strategy, which aims to

improve healthy eating information, strengthen labelling and claims, improve the

nutritional quality of foods, protect vulnerable populations, and support increased

access to and availability of nutritious foods(40). As part of this initiative, the Canadian

government has proposed to use NP in their policies on front-of-pack labelling and

restrictions on the marketing of foods to children(40).

In order to inform the use of NP in the proposed policy related to the restrictions on

marketing to children, we conducted a study that compared, in the Canadian context,

the degree of strictness and agreement between several NP models developed by

authoritative bodies for application in the marketing restrictions of foods to children(26).

This publication, along with author contributions, is presented in Appendix Ai.

In June 2017, Health Canada published the discussion paper for the public consultation

on restricting unhealthy food and beverage marketing to children, detailing the NP

model proposed for use in this policy(26). We are currently conducting a study to

examine the construct and convergent/concurrent validity of this proposed NP model

against views of nutrition professionals. The Ethics Review Application that was

i Labonté MÈ, Poon T, Mulligan C, Bernstein JT, Franco-Arellano B, L’Abbé MR. Comparison of global nutrient profiling systems for restricting the commercial marketing of foods and beverages of low nutritional quality to children in Canada. Am J Clin Nutr 106, 1471-1481.

22

submitted to and subsequently approved by the Review Ethics Board at the University

of Toronto, along with author contributions, is presented in Appendix Bj.

2.4.2 Global Context

A series of systematic reviews on the use of NP for various applications have been

updated over the years, namely by Rayner and colleagues at the University of Oxford

and other global collaborators. As part of the work conducted in developing the Ofcom

model for the restrictions on marketing to children, a review was published in 2004k, and

very limited data on NP directly relevant to marketing to children were identified(4-6). The

2004 review was subsequently updated in 2008, and 39 NP models that met the

inclusion criteria were identified in the updated review(6)l. As part of the work conducted

in building an unpublished catalogue of NP models for the WHO, the 2008 review was

updated again in 2013, and a total of 119 models (both included and excluded) were

identified(7)m. This 2013 unpublished catalogue was subsequently updated by our

research group in 2017, and a total of 387 models (both included and excluded) were

identified from this latest systematic review(8,41)n. The manuscript of this systematic

review, along with author contributions, is presented in Appendix Co. Results pertaining

to the current status of the validity of NP models from this systematic review are

described below.

The objective of our systematic review was to develop an up-to-date, accessible, and

global resource which summarizes the key characteristics of NP models with

j L’Abbé MR, Poon T, Labonté MÈ, Bernstein JT (2017). Ethics Review Application Form: Validation of a nutrient profile model proposed for restricting the marketing of unhealthy foods to children by nutrition professionals (protocol ID 34962). k The systematic literature searches were executed in 2003. l The systematic literature searches identified studies published between 2004 and 2007. m The systematic literature searches identified studies published between 2008 and 2012. n The systematic literature searches identified studies published between 2012 and 2016. o Labonté MÈ, Poon T, Gladanac B, Ahmed M, Franco-Arellano B, Rayner M, L’Abbé MR (unpublished). Nutrient profile models with applications in government-led nutrition policies aimed at health promotion and noncommunicable disease prevention: a systematic review. (To be submitted to Adv Nutr in December 2017).

23

applications in government-led nutrition policies and programs in order to support the

adoption or adaptation of existing NP models(8). NP models were identified from three

sources: 1) the unpublished catalogue completed in 2013 by Rayner et al.(7); 2) search

of the peer-reviewed literature using 3 databases; and 3) search of the grey literature

using 15 databases(8). Models included in the systematic review must have been

developed or endorsed by government or intergovernmental organizations, allowed for

the evaluation of individual food items, had publicly available nutrient criteria, amongst

other criteria(8). As illustrated in Figure 2-1, 387p potentially relevant NP models were

initially identified, with 119 of the models identified from the 2013 unpublished

catalogue(7) and 268q additional models from the full text assessment of more than 600

publications(8). A total of 78 models were included(8).

p A total of 387 models include 368 potential models first assessed for eligibility and 19 additional models identified during the eligibility assessment stage and evaluated for inclusion/exclusion. q A total of 268 models include 249 potential models from full-text review and 19 additional models identified during the eligibility assessment stage and evaluated for inclusion/exclusion.

24

Figure 2-1 Flow diagram of the publication and NP model selection. Data are reproduced from the manuscript of the systematic review presented in Appendix C (Labonté MÈ, Poon T, Gladanac B, Ahmed M, Franco-Arellano B, Rayner M, L’Abbé MR (unpublished). Nutrient profile models with applications in government-led nutrition policies aimed at health promotion and noncommunicable disease prevention: a systematic review. (To be submitted to Adv Nutr in December 2017)). Data are current as of December 22, 2016. 1 Of these 19 models, 15 were specifically identified during the process of assessing the eligibility of the first 368 potential models (e.g., through additional documentation reviewed), and 4 were identified from other sources (e.g., personal communication or email newsletter received during the weeks that the eligibility assessment process occurred). Abbreviations: lit., literature; NP, nutrient profiling; WHO, World Health Organization.

25

Data on any form of validity testing were not identified specifically for 58% (n=45 of 78)

of these models developed or endorsed by government organizations(8). However, 6 of

these 45 models were considered to be indirectly validated to a minor extent because

either 1) data on validity testing were identified for a previous version of the specific

model which had been updatedr, or 2) the specific model was adapted from another

model for which some validity testing had been identifieds. In contrast, data on some

form of validity testing were identified for 42% (n=33 of 78) of models(8). Across these

33 models, data on:

face validity were identified for 5 models; content validity were identified for 8 models; construct/convergent validity were identified for 24 models; and criterion/predictive validity, the most robust assessment of validity, were identified

for 8 models.

Furthermore, adequate validity testing, with regards to both the variety in type and

robustness of validity testing (e.g., construct/convergent and criterion/predictive), was

identified for only 15% (n=12 of 78) of modelst.

The Ofcom model remained the most frequently validated model(8), a finding which was

also reported by Rayner et al. in 2013(7) and again by Cooper et al. in 2016(10). Given

that face validity has limited relevance for models with non-consumer end-users(34), the

Ofcom model has demonstrated all other types of validity, including content(4,5,42),

construct(10,32), convergent(32), discriminant(32), concurrent(33), and predictive(43-47) validity.

r This pertained to two models: 1) England Requirements for School Food Regulations; and 2) Minimum requirements and specifications for food items allowed in the Women, Infants and Children food packages (supplemental foods). s This pertained to four models: 1) Australian National Healthy School Canteens Project; 2) Government of Victoria (Australian state) Go For Your Life Healthy Canteen Kit; 3) Ireland broadcasting authority model for restricting the marketing of food and drink to children; and 4) Georgia (U.S. state) Women, Infants and Children (WIC) approved food list. t 1) International Choices model; 2) Keyhole model from Sweden, Denmark, Norway, and Iceland; 3) U.S. requirements for health claims for foods; 4) U.S. definition of a “healthy” food; 5) Health Canada’s Surveillance Tool tier system; 6) UK Ofcom model; 7) UK Traffic Light Labelling; 8) France Nutri-Score (or Five-Colour Nutrition Label); 9) France SAIN, LIM system; 10) Danish Code of responsible food marketing communication to children; 11) South Africa NP model; and 12) Food Standards Australia New Zealand’s Nutrient Profiling Scoring Criterion.

26

For the assessments of predictive validity, the Ofcom model was adapted into a dietary

index to represent the overall quality of an individual’s diet(43-47). This Ofcom-based

dietary index was validated based on data from a prospective cohort studyu conducted

in a UK population (n=10,308 adults)(43) and a longitudinal cohort nested in a

randomized controlled trialv conducted in a French population (n=13,017 adults)(44-47).

Diets that consisted of foods with lower Ofcom scores (i.e., “healthier” foods) were

significantly associated with decreases in the risk of all-cause mortality(43), cancer

mortality(43), long-term weight gain(44), metabolic syndrome(45), overall cancer(46), and

cardiovascular disease(47).

A detailed summary of the validity testing identified for all 78 included models is

presented in Table 6 in the manuscript of the systematic review (Appendix C).

2.4.3 Knowledge Gaps

Based on the series of systematic reviews discussed previously(4-8), the number of

potential NP models globally has increased dramatically, particularly within the last 5

years. Given this proliferation of models, there is less need for the development of new

models, which requires extensive resources and expertise(2). As such, the adaptation of

an existing model is preferred by the WHO and is becoming increasingly common

practice for government and industry(1,2,9). Specifically, the WHO provides guidance on

the process of adapting a model, and at the outset, there are two key considerations(9).

The first is the organization that developed the model(1,9), as data have shown that

industry-led models were less stringent with regards to the advertising of energy-dense

foods compared to government-led models(48). The second is the data on the validity of

the model(1,9). In fact, there is consensus amongst the scientific community that it is

prudent to adapt a model that has been developed by an authoritative body, and more

importantly, validated(1,2,9,10,34,37,48). Thus, in the context of adaptation, models need to

be validated when used for different applications and in different countries, as it is

u Whitehall II study. v The Supplémentation en Vitamines et Minéraux Antioxydants (SU.VI.MAX) cohort. The intervention study lasted up to eight years from 1994 to 2002, followed by the observational follow-up of health events until 2007.

27

unknown whether the criteria that underpins a model developed to assess foods for one

application or from one population are appropriate for another(1,9).

However, although many models exist, the majority of NP models have not been fully

evaluated, compared, or validated prior to their adaptation or implementation(1,2,8,10).

Even amongst models that have been evaluated for validity, most have been validated

only to a limited extent(8). In order to identify published studies with a design similar to

the study presented in Chapter 3 (i.e., assessment of validity based on agreement

between different NP models), references were reviewed from three data sources:

1) the systematic review on the construct and criterion-related validation of NP

models conducted by Cooper et al. (2016)(10), in which the literature searches

identified studies published up to June 30, 2015;

2) our aforementioned systematic review on the application of NP models(8), in

which the searches identified studies published between 2008 and 2016; and

3) additional PubMed literature searches conducted in 2017w to identify validation

studies published in 2016 and 2017 after the searches from the two other

systematic reviews were done.

A review of these data sources resulted in the identification of 19 studies in which

construct/convergent validity was assessed using the method of comparing agreement

between different NP models (Table 2-5). Studies in which different NP models were

compared but did not include discussions on the validity of the models, such as the

study by Eržen et al. (2015)(49)x, were not included in this table. None of the 19 studies

examined the construct/convergent validity of any NP model when applied to the

Canadian food supply.

w Three searches were conducted using PubMed in January, June, and December 2017 using the following search terms in any field of the records: (“nutrient profiling” or “nutrient profile” or “nutritional quality”) and (validity or validation). x Although the agreement between the classifications determined using the Ofcom and FSANZ models for 125 foods available in the Slovenian marketplace was assessed, the validation of the models was not an objective of the study, nor was it discussed within the study.

28

Table 2-5 Studies in which construct/convergent validity was assessed based on agreement between NP models (n=19)a

Reference Food Supply (n=Foods Examined)

NP Models Compared

Wicks et al., 2017(50)

South Africa (n=197) 1) South African health claims model 2) UK Ofcom model 3) WHO EURO model 4) WHO Eastern Mediterranean Regional Office model 5) South African Department of Health marketing to

children model Faulkner et al., 2014(51)

UK (n=32) 1) UK Traffic Light 2) UK Ofcom

Clerfeuille et al., 2013(37)

France (n=597) 1) Choices International 2) Green Keyhole 3) Agence Française de Sécurité Sanitaire des Aliments 4) European Commission draft model 5) Food Profiler

Rosentreter et al., 2013(52)

New Zealand (n=407) 1) UK Traffic Light 2) FSANZ

Scarborough et al., 2013(53)

UK (n=336) 1) Brazilian model 2) Danish Forum Code 3) Disney model 4) PepsiCo model 5) UK Ofcom 6) U.S. CSPI 7) U.S. Interagency 8) EU Pledge

Trichterborn et al., 2012(54)

Germany (n=307) 1) Swedish Keyhole 2) Choices International 3) UK Ofcom 4) SAIN/LIM score 5) U.S. FDA health claims criteria

Drewnowski and Fulgoni, 2011(55)

U.S. (n=1,045) 1) Nutrient Rich Food Index 2) U.S. National Heart, Lung, and Blood Institute “Go”,

“Slow”, “Whoa” Roberto et al., 2011(56)

U.S. (n=100) 1) U.S. Smart Choices 2) UK Ofcom


France, Germany, Spain, Sweden, UK (n=238)

1) Choices International 2) U.S. Smart Choices 3) UK Ofcom 4) LIM score 5) U.S. FDA health claims criteria


France, Germany, UK (n=242)

1) Swedish Keyhole 2) Choices International 3) U.S. Smart Choices 4) UK Ofcom 5) SAIN/LIM score 6) U.S. FDA health claims criteria

Eyles et al., 2010(59)

New Zealand (n=550) 1) Modified Heart Foundation tick 2) Modified FSANZ

Hebden et al., Australia (n not 1) Nestle nutritional profiling system

29

Table 2-5 Studies in which construct/convergent validity was assessed based on agreement between NP models (n=19)a

Reference Food Supply (n=Foods Examined)

NP Models Compared

2010(60) reported) 2) Kraft Sensible Solutions 3) Cereal Partners Worldwide Nutrition Foundation criteria 4) Kellogg’s global nutrient criteria 5) Fonterra Good Choice 6) FSANZ

Walker et al., 2010(61)

Australia (n=388) 1) UK Traffic Light 2) UK Ofcom

Drewnowski et al., 2009(62)

U.S. (n=378) 1) Nutrient Adequacy Score 2) Nutrient Density Score 3) Nutrient Rich Food models 4) LIM score 5) UK Ofcom

Walker et al., 2008(63)

Australia (n=1,933) 1) New South Wales School Canteens criteria 2) Uncle Toby's nutrition criteria 3) UK Traffic Light

Garsetti et al., 2007(64)

Netherlands, UK (n=62)

1) A Little A Lot (UK Ministry of Agriculture) 2) U.S. FDA health claims criteria 3) Netherlands Tripartite 4) UK Ofcom 5) U.S. CSPI

Labouze et al., 2007(65)

France, UK (n=98) 1) Nutrimap 2) UK Ofcom 3) Netherlands Tripartite

Quinio et al., 2007(66)

Belgium, Denmark, France, Ireland, Italy (n not reported)

1) UK Ofcom 2) Netherlands Tripartite 3) U.S. FDA health claims criteria

Azais-Braesco et al., 2006(67)

Europe/UK (n=125) 1) Calorie For Nutrient Index 2) Nutritious Food Index 3) Ratio of Recommended to Restricted Foods 4) UK Ofcom

Abbreviations: CSPI, Center for Science and the Public Interest; EU, European Union; FDA, Food and Drug Administration; FSANZ, Food Standards Australia New Zealand; UK, United Kingdom, U.S., United States. a The data for all studies, except the study by Wicks et al. (2017)(50), were adapted from Cooper et al. (2016)(10).

Moreover, as discussed in Section 2.2, different NP models can vary substantially in

their design and characteristics, and this variation ultimately gives rise to inherent

differences in the manner in which these models classify the healthfulness of foods.

Furthermore, as discussed in Section 2.3, the extent of the documentation and reporting

of validity testing varies widely across different NP models. This is due, in part, to the

infancy of the methods of validation(2). As such, several researchers have suggested

30

that validity testing of NP models should be given the highest priority in this field of

research(3,10,11).

Based on the data presented in this literature review, the need for validity testing of NP

models is a main knowledge gap in NP. Additionally, amongst models for which

construct/convergent validity testing has been conducted, the testing was typically done

in a small sample of indicator foods for countries other than Canada. Thus, the

objective of this thesis was to examine the content and construct/convergent validity of

several NP models developed by authoritative bodies in assessing the nutritional quality

of foods from a large, branded, Canadian database.

31

Chapter 3

3 Study on Validation of NP Models

3.1 Manuscript This manuscript has been submitted to the British Journal of Nutrition for publication:

Poon T, Labonté MÈ, L’Abbé MR. Comparison of nutrient profiling models for assessing

the nutritional quality of foods: a validation study (submitted October 12, 2017; revisions

to be submitted by January 29, 2018).

The contributions of the authors to the manuscript are summarized in Table 3-1.

Table 3-1 Authors’ contributions to study on validation of NP models

Task Author Designed research study Poon, Labonté, L’Abbé Generated data for nutrient profile models Poon, Labonté Analyzed data, ran statistical analyses Poon Wrote manuscript Poon Provided comments/revisions to draft and final manuscripts

Labonté, L’Abbé

Submitted manuscript to journal Poon

3.1.1 Abstract

Nutrient profiling (NP) is a method for evaluating the healthfulness of foods. Although

many NP models exist, most have not been validated. This study aimed to examine the

content and construct/convergent validity of four models from different regions:

Australia/New Zealand (FSANZ), Canada (HCST), Europe (EURO), and Americas

(PAHO). Content validity was assessed by examining the nutrients/components

considered by each model. Using data from the 2013 University of Toronto Food Label

Information Program (n=15,342 foods/beverages), construct/convergent validity was

assessed by comparing the classifications of foods determined by each model to a

previously validated model, which served as the reference (Ofcom). The parameters

assessed included agreement (kappa statistic (κ)), discordant classifications

(McNemar’s test), and associations (Cochran-Armitage trend test). Analyses were

32

conducted across all foods and by food category. All models exhibited moderate

content validity. Although positive associations were observed between each model

and Ofcom (all P<0.001 for trend), agreement with Ofcom was “fair” for HCST (κ=0.26)

and PAHO (κ=0.28), “moderate” for EURO (κ=0.54), and “near perfect” for FSANZ

(κ=0.89). There were discordant classifications with Ofcom for 5.3% (FSANZ), 22.0%

(EURO), 33.4% (PAHO), and 37.0% (HCST) of foods (all P<0.001).

Construct/convergent validity was confirmed between FSANZ and Ofcom, and to a

lesser extent between EURO and Ofcom. Numerous incongruencies with Ofcom were

identified for HCST and PAHO, which highlights the importance of examining

classifications across food categories, the level at which differences between models

become apparent. These results may be informative for regulators seeking to adapt

existing models for use in country-specific applications.

3.1.2 Introduction

Nutrient profiling (NP), defined as the science of classifying foods according to their

nutritional composition for the purpose of promoting health and preventing disease, is a

relatively new term in the field of nutrition research(1,9). Several of the earliest forms of

NP were introduced by government bodies in the 1980s and 1990s, including the United

States (U.S.) Special Supplemental Nutrition Program for Women, Infants, and Children

in 1980(12), Swedish Keyhole in 1989(13), and disqualifying nutrient levels for U.S. health

claims in 1993(14). The term NP gained ground following the development of the Ofcom

model by the United Kingdom (UK) Food Standards Agency in 2004 to 2005(4,5) and the

mention of nutrient profiles in Regulation (EC) No 1924/2006 on nutrition and health

claims by the European Commission in 2006(16). In 2010, NP became even more widely

known when the World Health Organization (WHO) provided to its Member States a set

of recommendations on the marketing of foods and beverages to children, one of which

advocated the use of NP models in defining the products to be covered by the

marketing restrictions(17). Globally, NP is recognised now as a transparent and

reproducible method of evaluating the healthfulness of foods, and for its use in

numerous applications in government and industry (e.g., front-of-pack food labelling,

food taxes, reformulation)(2,3).

33

Based on a series of systematic reviews that have been updated over the years, the

number of potential NP models identified globally was as follows: 39 (included models

only) in 2008(6), 119 in 2013(7), and 387 in 2016 (Labonté ME, Poon T, Gladanac B et

al., unpublished results). Given this recent proliferation of NP models and the extensive

resources required to develop and validate a new model, the adaptation of an existing

model is preferred by the WHO and is becoming increasingly common practice for

government agencies(1,2,9). It is prudent to adapt a model that has been developed by

an authoritative body, and more importantly, validated(1,2,9,48). However, although many

models exist, the majority of NP models have not been fully evaluated or validated prior

to their implementation(2,10). This is due, in part, to the infancy of the methods used to

validate NP models(2). Several researchers have suggested that validity testing of NP

models should be given the highest priority in this field of research(3,10,11).

There are several types of validity (e.g., content, construct, convergent), and there are

different methods of testing for validity(1,9,34). Content validity is defined as the extent to

which the model encompasses the full range of meaning for the concept being

measured(34). One method of testing for content validity is to assess the consistency

between the algorithmic underpinnings of a model and the current scientific literature(34),

such as whether a model considers nutrients of public health concern. Construct

validity is the extent to which the model corresponds to theoretical constructs under

study(1), and convergent validity is the extent to which the model correlates with

variables that theoretically should correlate at the same point in time(1,34). The

comparison of one NP model to another has been used as a method of testing either

construct(10) or convergent validity(1). In particular, the comparison to a model that has

been previously validated is considered by the WHO to be a reasonably strong form of

validation(1). Thus, the objective of this study was to examine the content and

construct/convergent validity of several NP models developed by authoritative bodies for

the assessment of the nutritional quality of pre-packaged foods from a large, national,

branded database.

34

3.1.3 Methods

NP models

The key characteristics of the models examined in this study are summarised in Table

3-2 and described below.

The Ofcom model was developed for the regulation of television advertising to

children(28). The model consists of two food categories: 1) beverages; and 2) foods. It

takes into consideration a total of seven nutrients to limit and nutrients/food components

to encourage, the latter including fruit, vegetable, and nut (FVN) content. In order to

estimate the FVN content of a food without quantitative declarations in the ingredient

list, which are not required in Canada, the presence and positions of the FVN

ingredients within the ingredient list were used (refer to Table 3.4 in Section 3.1.6)y.

Based on the level of nutrients/components present per 100 g of the food, the model

generates a summary score in which a lower score represents a food with a more

favourable nutritional profile. The model also classifies the food as “permitted” or “not

permitted” for advertising to children based on pre-determined cut-off scores for foods

and beverages.

The Food Standards Australia New Zealand (FSANZ) model was developed for the

regulation of health claims on foods(18). As a derivative of the Ofcom model, the FSANZ

model is similar, except for the following characteristics: it consists of an additional food

category of cheeses high in calcium (i.e., >320 mg/100 g) and fats; it considers nutrients

on a per 100 mL basis in addition to 100 g; and it considers legumes in addition to FVN

(i.e., FVNL). The method used to estimate the FVNL content for the FSANZ model was

similar to that used to estimate the FVN content for the Ofcom model, and is described

in detail elsewhere(68). The model classifies the food as “permitted” or “not permitted” to

carry health claims based on pre-determined cut-off scores for each food category.

The Health Canada Surveillance Tool (HCST) tier system was developed to assess the

adherence of the dietary intakes of Canadians to the dietary guidance provided by

y Table 3.4 is presented as a supplementary table in the manuscript.

35

Canada’s Food Guide (CFG)(29). The model consists of four food categories: 1)

vegetables and fruits; 2) grain products; 3) milk and alternatives; and 4) meat and

alternatives. It takes into consideration four nutrients to limit (i.e., total fat, saturated fat,

sodium, and total sugars). Based on the level of nutrients present per serving of the

food, the model classifies the food into one of four tiers as follows: tier 1 and 2 foods are

in line with CFG and should be consumed often; tier 3 foods are partially in line with

CFG and should be consumed less often; and tier 4 foods are not in line with CFG and

their consumption should be limited.

The WHO Regional Office for Europe (EURO) model was developed for the restriction

of marketing less healthy foods to children(19). The model consists of 20 food categories

(listed in a footnote to Table 3-2). It takes into consideration eight nutrients to limit,

including industrially-produced trans fat and added sugars. Given that quantitative

declarations of industrially-produced trans fat are not required in Canada, the amount

was estimated based on the presence of hydrogenated or partially hydrogenated oils in

the ingredient list and the total trans fat level declared in the Nutrition Facts table.

Instead of added sugars, free sugar levels were used in this study because the WHO

considers free sugars as part of their guidelines on sugars(23). In order to estimate the

free sugar content of a food without a quantitative declaration in the Nutrition Facts

table, which is not required in Canada, the University of Toronto’s free sugar algorithm

was used; this algorithm is described elsewhere(69). Based on the level of nutrients

present per 100 g of the food, the model classifies the food as “permitted” or “not

permitted” for marketing to children. It should be noted that seven food categories are

not subject to the nutrient criteria and are automatically classified as “permitted” (i.e.,

fresh and frozen meat/poultry/fish; fresh and frozen fruits/vegetables/legumes) or “not

permitted” for marketing (i.e., confectionery; sweet bakery products; juices; energy

drinks; edible ices).

The WHO Regional Office for the Americas/Pan American Health Organization (PAHO)

model was developed for a variety of regulatory purposes aimed at addressing the

obesity epidemic(21). The model consists of five food categories: 1) ultra-processed

foods; 2) processed foods; 3) unprocessed or minimally processed foods; 4) culinary

ingredients; and 5) freshly prepared dishes. It takes into consideration six nutrients to

36

limit. Based on the level of nutrients present per % energy of the food, the model

classifies the food as “not excessive” in any nutrient or “excessive” in one or more of the

nutrients. It should be noted that three food categories (i.e., unprocessed or minimally

processed foods; culinary ingredients; freshly prepared dishes) are not subject to the

nutrient criteria and are automatically classified as “not excessive” in any nutrient.

In this study, “healthier” foods were defined as those permitted for marketing to children

as per the Ofcom and EURO models, permitted to carry health claims as per FSANZ,

classified as tier 1 or 2 foods as per HCST, and not excessive in any nutrient as per

PAHO (Table 3-2). Correspondingly, “less healthy” foods were defined as those not

permitted for marketing to children as per the Ofcom and EURO models, not permitted

to carry health claims as per FSANZ, classified as tier 3 or 4 foods as per HCST, and

excessive in one or more nutrients as per PAHO.

Content validity

For the FSANZ, HCST, EURO, and PAHO models, content validity was assessed by

examining the consistency between the nutrients/food components included in the

models versus those considered by the WHO to be of immediate importance in

promoting health and preventing disease. These included the nutrients/food

components explicitly stated within objective three of the WHO’s Global action plan for

the prevention and control of non-communicable disease 2013-2020 as those to

encourage (i.e., unsaturated fat and fruits and vegetables) and those to limit (i.e.,

energy, total fat, saturated fat, trans fat, sodium, and sugars (type not specified))(70,71).

Construct/convergent validity

Construct/convergent validity was assessed by comparing the classifications of foods

determined by each of the models (FSANZ, HCST, EURO, and PAHO) versus those

determined by a reference model using several parameters. The Ofcom model was

chosen as the reference model because it has been extensively validated using various

methods for different applications in multiple countries(3). Specifically, the Ofcom model

has been demonstrated to have content(4,5,42), construct(10,32), convergent(32),

discriminant(32), and concurrent validity(33). In addition, the adaptation of the Ofcom

37

model into a dietary index to represent the overall quality of an individual’s diet has been

demonstrated to have predictive validity(43-47). Details of the food database and

parameters used in this assessment are provided below.

Food database

A total of 15,342 pre-packaged foods and beverages from the University of Toronto’s

Food Label Information Program (FLIP) 2013 database were examined. Data were

collected in 2013 across the four largest grocery chains in Canada, which represented

75.4% of the grocery retail market share(69). The product information collected included

the Nutrition Facts table and ingredient list, amongst other data. Details of FLIP 2013

are provided elsewhere(69). Foods were classified into the 22 food categories as per

Schedule M of the Food and Drug Regulations (version in force between March 2012

and December 2016)(72). A total of 115 products were excluded from the analyses: 55

products due to manufacturer errors in the nutrient declarations in the Nutrition Facts

table (i.e., >20% difference between the calories declared and calories calculated using

Atwater factors for macronutrients) and 60 products that did not align with any Schedule

M category (i.e., 55 meal replacements, 4 instant or dry yeast products, and 1 natural

health product). Thus, 15,227 unique products were available for analyses. In order to

generate the classifications of the foods for each NP model, the 15,227 foods in FLIP

2013 were first classified independently by two authors (T.P. and M.E.L.) into the food

categories specific to the NP models using information from the ingredient lists and/or

pre-classifications from Schedule M food categories and subcategories and sugar-

focused categories from Bernstein et al.(69). For all models, nutrient data for products in

their “as consumed” form were used. Subsequently, the nutrient criteria for each model

were applied to the foods.

Statistical analyses

Construct/convergent validity was assessed using four parameters. First, pairwise

agreement between each model and the Ofcom model in the proportions of foods

classified as “healthier” or “less healthy” was assessed across all foods (n=15,227) and

by the 22 Schedule M food categories using the kappa statistic (κ, 95% CI), as follows:

0.01 to 0.20 “slight”; 0.21 to 0.40 “fair”; 0.41 to 0.60 “moderate”; 0.61 to 0.80

38

“substantial”; 0.81 to 0.99 “near perfect”(73). Second, discordant classifications

(hereafter referred to as “discordance”) between each model and Ofcom was defined as

the sum of the percentage of foods classified as “healthier” by a model but “less healthy”

by Ofcom and the percentage of foods classified as “less healthy” by a model but

“healthier” by Ofcom. Discordance was assessed across all foods and by food category

using McNemar’s test for paired data. Third, the association between the proportion

(and 95% CI) of foods classified as “less healthy” by each model and quartiles of Ofcom

scores was assessed across all foods using the Cochran-Armitage trend test.

Fourth, because the FSANZ and HCST models also generated ordinal outcomes (Table

3-2), cross-classification analyses between quartiles of FSANZ scores and the four

HCST tiers versus quartiles of Ofcom scores were conducted. Exact agreement was

defined as the classification of a food in the same quartiles/tiers using different models

(e.g., FSANZ Q1 and Ofcom Q1; HCST tier 4 and Ofcom Q4). In addition, agreement

within an adjacent (i.e., ±1) quartile/tier (e.g., FSANZ Q1 and Ofcom Q2; HCST tier 2

and Ofcom Q1) and disagreement (e.g., FSANZ Q1 and Ofcom Q3; HCST tier 2 and

Ofcom Q4) were assessed. Lastly, gross misclassification was defined as the

classification of a food in opposite quartiles/tiers (e.g., FSANZ Q1 and Ofcom Q4; HCST

tier 4 and Ofcom Q1). A P-value <0.05 was considered statistically significant.

Statistical analyses were conducted using SAS 9.4 (SAS Institute Inc., Cary, NC, USA).

39

Table 3-2 Summary of NP models examined

Ofcom (referencea) FSANZ HCST EURO PAHO Food categories (n) 2 3 4 20b 5 Reference amount 100 gc 100 g or mL Serving 100 g % energy of foodd Nutrients/food components (n)

7 7 4 8 6

Energye Total fate

Saturated fate

Unsaturated fatf Trans fate (industrially-

produced)g (total)

Sodiume

Total sugarse Free/added sugarse (added)h (free) Sweeteners

Protein Fibre Fruits/vegetablesf (FVN)i (FVNL)i

Continuous outcome (Ofcom score) (FSANZ score) Ordinal outcome (Ofcom quartile) (FSANZ quartile) (HCST tiers 1 to 4) Binary outcome

“Healthier” foods Permitted for marketing to children

Permitted to carry health claims

Tier 1 and 2 foods in line with CFG

Permitted for marketing to children

Not excessive in any nutrient

“Less healthy” foods Not permitted for marketing to children

Not permitted to carry health claims

Tier 3 foods partially in line with CFG and tier 4 foods not in line with CFG

Not permitted for marketing to children

Excessive in ≥1 nutrient(s)

Abbreviations: CFG, Canada’s Food Guide; EURO, World Health Organization Regional Office for Europe; FSANZ, Food Standards Australia New Zealand; FVN/L, fruits, vegetables, nuts, and/or legumes; HCST, Health Canada Surveillance Tool; NP, nutrient profiling; PAHO, World Health Organization Regional Office for the Americas/Pan American Health Organization.

40

a The Ofcom model was chosen as the reference model for the assessment of construct/convergent validity in this study. b The model consists of 17 main categories, with the beverage category containing 4 sub-categories, for a total of 20 unique categories: 1) chocolate and sugar confectionery, energy bars, and sweet toppings and desserts; 2) cakes, sweet biscuits, pastries, other sweet bakery wares, and dry mixes for making such; 3) savoury snacks; 4) beverages: juices; 5) beverages: milk drinks; 6) beverages: energy drinks; 7) beverages: other beverages; 8) edible ices; 9) breakfast cereals; 10) yogurts, sour milk, cream, and other similar foods; 11) cheese; 12) ready-made and convenience foods and composite dishes; 13) butter and other fats and oils; 14) bread, bread products, and crisp breads; 15) fresh or dried pasta, rice, and grains; 16) fresh and frozen meat, poultry, fish, and similar; 17) processed meat, poultry, fish, and similar; 18) fresh and frozen fruit, vegetables, and legumes; 19) processed fruit, vegetables, and legumes; and 20) sauces, dips, and dressings(19). c Although a reference amount of 100 g for foods and beverages was specified as part of the model, a reference amount of 100 mL for beverages and other products in liquid form was used for the assessment of construct/convergent validity in this study. d While sweeteners were evaluated based on their absence or presence in the ingredient list and sodium was evaluated on a per kcal basis, the thresholds for the other nutrients were presented on as a % energy of the food (e.g., an excess of total fat is ≥30% of total energy of the food)(21). e For the assessment of content validity in this study, the nutrients to limit according to the WHO include energy, total fat, saturated fat, trans fat, sodium, and sugars (type not specified)(70,71). f For the assessment of content validity in this study, the nutrients to encourage according to the WHO include unsaturated fat and fruits and vegetables(70,71). g For the assessment of construct/convergent validity in this study, the industrially-produced trans fat content of a food was estimated based on the presence of hydrogenated or partially hydrogenated oils in the ingredient list and the total trans fat level declared in the Nutrition Facts table. h Although added sugars were specified as part of the model, free sugar levels estimated using the University of Toronto’s free sugar algorithm(69) were used for the assessment of construct/convergent validity in this study. i For the assessment of construct/convergent validity in this study, the FVN/L content of a food was estimated based on the presence and positions of the FVN/L ingredients within the ingredient list.

41

3.1.4 Results

Content validity

Of the eight nutrients/food components considered by the WHO to be of immediate

importance in promoting health and preventing disease (i.e., energy, total fat, saturated

fat, unsaturated fat, trans fat, sodium, sugars (type not specified), and fruits and

vegetables)(70,71), each of the NP models considered at least four of these nutrients/food

components (Table 3-2). With respect to the nutrients to limit, saturated fat, sodium,

and some form of sugar (e.g., total, free, or added) all were considered in each of the

four models. Total fat was considered in the HCST, EURO, and PAHO models, energy

was considered in FSANZ and EURO, and trans fat was considered in EURO and

PAHO. With respect to the nutrients/food components to encourage, the fruit and

vegetable content of foods was considered in only the FSANZ model, and unsaturated

fat was not considered in any model.

Construct/convergent validity

Across all foods, a positive association was observed between each model and the

Ofcom model, such that the proportion of foods classified as “less healthy” by FSANZ,

HCST, EURO, or PAHO increased across quartiles of Ofcom scores, with the highest

quartile representing the “least healthy” foods (all P<0.001 for trend) (Table 3-3).

However, varying levels of agreement and discordance were observed between each

model and Ofcom, as described below.

42

Table 3-3 Associations between the proportions (%, 95% CI) of foods classified as “less healthy” by the models and quartiles of Ofcom scores (n=15,227)

NP model Foods (n) Quartiles of Ofcom (reference) scores P for trenda Q1 (“healthier”) Q2 Q3 Q4 (“less healthy”)

% 95% CI % 95% CI % 95% CI % 95% CI FSANZ 15,183b 0.7 0.5, 1.0 29.5 27.9, 31.1 92.3 91.5, 93.2 90.5 89.5, 91.4 <0.001 HCST 15,165c 22.4 21.2, 23.6 61.8 60.1, 63.5 49.1 47.5, 50.7 73.8 72.4, 75.3 <0.001 EURO 15,182d 36.2 34.9, 37.6 63.7 62, 65.4 94.4 93.6, 95.1 94.4 93.7, 95.1 <0.001 PAHO 15,182d 58.9 57.5, 60.4 93.8 92.9, 94.6 96.3 95.7, 96.9 95.7 95.1, 96.4 <0.001 Abbreviations: EURO, World Health Organization Regional Office for Europe; FSANZ, Food Standards Australia New Zealand; HCST, Health Canada Surveillance Tool; NP, nutrient profiling; PAHO, World Health Organization Regional Office for the Americas/Pan American Health Organization; Q, quartile. a Assessed using the Cochran-Armitage trend test. b Data were missing for 0.29% (n=44) of foods. c Data were missing for 0.41% (n=62) of foods. d Data were missing for 0.30% (n=45) of foods.

43

FSANZ model

Across all foods, there was “near perfect” agreement (κ=0.89; 95% CI 0.89, 0.90)

between the FSANZ and Ofcom model. Although the overall proportions of foods

classified as “healthier” by the two models were similar in magnitude (49.0% by FSANZ;

44.3% by Ofcom), significant discordance in the classifications between the models was

observed for 5.3% of the large sample of n=15,183 foods analyzed (P<0.001) (Figures

3-1 and 3-2). Specifically, significant discordance was observed for 11 of the 22 food

categories analyzed (all P<0.05); however, agreement was “near perfect” or

“substantial” for nine of these categories (κ=0.66 to 0.97; 1.2 to 17.6% discordance).

Notably, amongst the remaining two categories, “slight” agreement and higher

proportions of discordance were observed: potatoes/sweet potatoes/yams (κ=0.17;

27.9% discordance) and fats/oils (κ=0.07; 33.8% discordance). Across all foods, cross-

classification analyses using quartiles of FSANZ and Ofcom scores indicated that there

was exact agreement for 95.2% of foods and no gross misclassification of foods (data

across food categories not shown).

44

Figure 3-1 Overall proportions (%, 95% CI) of “healthier” and “less healthy” foods and agreement (κ) between each model and the Ofcom model (n=15,227). Data were missing for 0.01 to 0.34% (n=1 to 52) of foods within a model. Agreement was assessed using the kappa statistic (κ) as follows: 0.01 to 0.20 “slight”; 0.21 to 0.40 “fair”; 0.41 to 0.60 “moderate”; 0.61 to 0.80 “substantial”; 0.81 to 0.99 “near perfect”(73). Abbreviations: EURO, World Health Organization Regional Office for Europe; FSANZ, Food Standards Australia New Zealand; HCST, Health Canada Surveillance Tool; PAHO, World Health Organization Regional Office for the Americas/Pan American Health Organization.

45

Figure 3-2 Agreement (κ, 95% CI) and discordance (%, indicated above each line) between the Food Standards Australia New Zealand (FSANZ) and Ofcom model for all foods (n=15,183; data missing for n=44) and 22 food categories from Schedule M of the Food and Drug Regulations(72). Agreement was assessed using the kappa statistic (κ) as follows: 0.01 to 0.20 “slight”; 0.21 to 0.40 “fair”; 0.41 to 0.60 “moderate”; 0.61 to 0.80 “substantial”; 0.81 to 0.99 “near perfect”(73). Significant discordance in classifications between models using McNemar’s test (*P<0.05, **P<0.001).

46

HCST tier system

Across all foods, there was “fair” agreement (κ=0.26; 95% CI 0.25, 0.28) between the

HCST and Ofcom model. Although the overall proportions of foods classified as

“healthier” by the two models were similar in magnitude (50.4% by HCST; 44.3% by

Ofcom), significant discordance in the classifications between the models was observed

for 37.0% of the foods (P<0.001) (Figures 3-1 and 3-3). With the exception of dairy

products/substitutes (P=0.38) and snacks (P=0.08), significant discordance was

observed for 20 of the 22 food categories analyzed (all P<0.05). However, agreement

was “substantial” or “moderate” for five of these categories (κ=0.53 to 0.77; 11.4 to

22.9% discordance). Amongst the remaining 15 categories, less agreement and higher

proportions of discordance were observed. Agreement was “fair” or “slight” (κ=0.02 for

fats/oils to κ=0.36 for marine products), and discordance ranged from 17.8% for

legumes to 81.5% for soups. Notably, there was disagreement more than expected by

chance, as indicated by a negative kappa statistic (κ=-0.07), and 56.4% discordance for

sauces/dips/gravies/condiments.

Across all foods, cross-classification analyses conducted using the HCST tiers and

quartiles of Ofcom scores indicated that the classifications were in exact agreement for

32.7%, within an adjacent tier/quartile for 49.0%, in disagreement for 15.9%, or grossly

misclassified for 2.4% of foods (Figure 3-4). Specifically, the three food categories with

the highest proportions of classifications that disagreed and/or were grossly

misclassified included eggs (75.0%), fats/oils (55.1%), and combination dishes (47.5%).

A total of 15 categories included grossly misclassified foods, with the highest

proportions amongst miscellaneous items (10.6%), combination dishes (9.2%), and

sauces/dips/gravies/condiments (6.2%) (the remaining categories had <5.0%).

47

Figure 3-3 Agreement (κ, 95% CI) and discordance (%, indicated above each line) between the Health Canada Surveillance Tool (HCST) and Ofcom model for all foods (n=15,165; data missing for n=62) and 22 food categories from Schedule M of the Food and Drug Regulations(72). Agreement was assessed using the kappa statistic (κ) as follows: 0.01 to 0.20 “slight”; 0.21 to 0.40 “fair”; 0.41 to 0.60 “moderate”; 0.61 to 0.80 “substantial”; 0.81 to 0.99 “near perfect”(73). Significant discordance in classifications between models using McNemar’s test (*P<0.05, **P<0.001).

48

Figure 3-4 Cross-classification analyses between four Health Canada Surveillance Tool (HCST) tiers versus quartiles of Ofcom scores for all foods (n=15,165; data missing for n=62) and 22 food categories from Schedule M of the Food and Drug Regulations(72). Exact agreement occurs when a food is classified in the same tiers/quartiles (e.g., HCST tier 1 and Ofcom quartile 1). Agreement within an adjacent (±1) tier/quartile (e.g., HCST tier 1 and Ofcom quartile 2) and disagreement (e.g., HCST tier 1 and Ofcom quartile 3) also were assessed. Gross misclassification occurs when a food is classified in opposing tiers/quartiles (e.g., HCST tier 1 and Ofcom quartile 4).

49

EURO model

Across all foods, there was “moderate” agreement (κ=0.54; 95% CI 0.53, 0.55) between

the EURO and Ofcom model. While the overall proportions of foods classified as

“healthier” by the two models differed (29.8% by EURO; 44.3% by Ofcom), significant

discordance in the classifications between the models was observed for 22.0% of the

foods (P<0.001) (Figures 3-1 and 3-5). According to the EURO model, none of the

desserts or dessert toppings/fillings were classified as “healthier”, and none of the eggs

were classified as “less healthy” (refer to Table 3.5 in Section 3.1.6)z; thus, the kappa

statistic and McNemar’s test for significance in discordance, which required 2 by 2

tables to be generated, could not be conducted for these three food categories.

Significant discordance was observed for all 19 food categories analyzed using

McNemar’s test (all P<0.01); however, agreement was “substantial” or “moderate” for

nine of these categories (κ=0.45 to 0.80; 7.8 to 23.6% discordance). Amongst the

remaining 10 categories, less agreement and higher proportions of discordance were

observed. Agreement was “fair” or “slight” (κ=0.06 for legumes to κ=0.39 for dairy

products/substitutes), and discordance ranged from 3.0% for sugars/sweets to 62.7%

for fruit/fruit juices. Notably, there was disagreement more than expected by chance

(κ=-0.01) and 29.7% discordance for fats/oils.

z Table 3.5 is presented as a supplementary table in the manuscript.

50

Figure 3-5 Agreement (κ, 95% CI) and discordance (%, indicated above each line) between the World Health Organization Regional Office for Europe (EURO) and Ofcom model for all foods (n=15,182; data missing for n=45) and 22 food categories from Schedule M of the Food and Drug Regulations(72). Agreement was assessed using the kappa statistic (κ) as follows: 0.01 to 0.20 “slight”; 0.21 to 0.40 “fair”; 0.41 to 0.60 “moderate”; 0.61 to 0.80 “substantial”; 0.81 to 0.99 “near perfect”(73). Significant discordance in classifications between models using McNemar’s test (*P<0.01, **P<0.001). The “X” symbol represents a food category for which the kappa statistic and McNemar’s test could not be conducted because 2 by 2 tables could not be generated (i.e., none of the desserts or dessert toppings/fillings were classified as “healthier”, and none of the eggs were classified as “less healthy” by the EURO model).

51

PAHO model


PAHO and Ofcom model. While the overall proportions of foods classified as “healthier”

by the two models differed (15.9% by PAHO; 44.3% by Ofcom), significant discordance

in the classifications between the models was observed for 33.4% of the foods

(P<0.001) (Figures 3-1 and 3-6). According to the PAHO model, none of the packaged

salads (e.g., pasta or potato salads) were classified as “healthier” (refer to Table 3.5 in

Section 3.1.6)aa; thus, the kappa statistic and McNemar’s test for significance in

discordance, which required 2 by 2 tables to be generated, could not be conducted for

this food category. Significant discordance was observed for all 21 food categories

analyzed using McNemar’s test (all P<0.01); however, agreement was “substantial” or

“moderate” for four of these categories (κ=0.41 to 0.77; 9.4 to 15.0% discordance).

Amongst the remaining 17 categories, less agreement and higher proportions of

discordance were observed. Agreement was “fair” or “slight” (κ=0.01 for legumes to

κ=0.32 for vegetables), and discordance ranged from 16.5% for snacks to 69.6% for

combination dishes. Notably, there was no agreement (κ=0) or there was disagreement

more than expected by chance (κ=-0.05 to -0.01) and a range of 12.3 to 87.9%

discordance for desserts, soups, fats/oils, dessert toppings/fillings, and sugars/sweets.

aa Table 3.5 is presented as a supplementary table in the manuscript.

52

Figure 3-6 Agreement (κ, 95% CI) and discordance (%, indicated above each line) between the World Health Organization Regional Office for the Americas/Pan American Health Organization (PAHO) and Ofcom model for all foods (n=15,182; data missing for n=45) and 22 food categories from Schedule M of the Food and Drug Regulations(72). Agreement was assessed using the kappa statistic (κ) as follows: 0.01 to 0.20 “slight”; 0.21 to 0.40 “fair”; 0.41 to 0.60 “moderate”; 0.61 to 0.80 “substantial”; 0.81 to 0.99 “near perfect”(73). Significant discordance in classifications between models using McNemar’s test (*P<0.01, **P<0.001). The “X” symbol represents a food category for which the kappa statistic and McNemar’s test could not be conducted because 2 by 2 tables could not be generated (i.e., none of the packaged salads were classified as “healthier” by the PAHO model).

53

3.1.5 Discussion

In this study, the four NP models were considered to have moderate content validity

because each model accounted for at least half of the nutrients/food components that

characterise “healthy” or “less healthy” diets according to the WHO(70,71). The WHO’s

Global action plan for the prevention and control of non-communicable disease 2013-

2020, which was updated recently in May 2017, was used because it represents

consensus in the current scientific literature with regards to the nutrients/food

components of immediate importance in promoting health and preventing disease(70,71).

Other studies that employed this method of testing for content validity (i.e., assessing

consistency between the underpinnings of a model and the current scientific

literature(34)) were not identified in the published literature, likely because content validity

is typically assessed during the early phases of NP model development(34) for which the

data are rarely published.

Several considerations should be noted. First, the WHO used the umbrella terms total

fat and total sugars as part of the recommendations for avoiding a “less healthy” diet.

Although recommendations specific to saturated, unsaturated, and trans fat also were

provided, sugars were not further differentiated, despite the WHO’s consideration of free

sugars as part of their sugar guidelines(23). Second, a model’s inclusion of

nutrients/food components for which their roles in health are still being debated (e.g.,

sweeteners, saturated fat) was not considered to detract from its content validity. Third,

a model’s lack of inclusion of nutrients/food components may not necessarily reflect a

lack of content validity. Although the fruit and vegetable content of foods was not

included as a food component in the EURO model, the model indirectly accounted for it

by treating the food category for fresh and frozen fruits/vegetables/legumes as an

exemption, such that these foods were automatically considered as “healthier”

irrespective of their nutrient content. Nevertheless, the high fruit or vegetable content of

foods belonging to other food categories (e.g., vegetable lasagne as combination

dishes) would remain unrecognised in this case. Furthermore, trans fat was not

considered in the FSANZ and HCST models, likely because several voluntary initiatives

introduced in Australia/New Zealand and Canada during the early 2000s were

successful in reducing trans fat levels in the food supply(74-76). Thus, the lack of

54

inclusion of nutrients/food components in a model should be considered in the context

of other policy frameworks.

With respect to construct/convergent validity, our findings related to the FSANZ and

Ofcom models were consistent with another study in that there was overall “near

perfect” agreement between the models, other than for fats/oils and dairy products

which were classified differently by the models(49). Although the overall discordance

between FSANZ and Ofcom observed for 5.3% of the foods was statistically significant,

this was likely due to the large sample of n=15,183 foods analyzed. Such a low

proportion of discordance would not be considered of practical significance; rather, this

result is meaningful in the context of establishing construct/convergent validity between

FSANZ and Ofcom when compared to the overall discordance observed for the other

models, which ranged from 22.0 to 37.0%. The “slight” agreement and high proportions

of discordance observed for fats/oils and potatoes/sweet potatoes/yams can be

explained by the differences in the characteristics of the two models, such as the

inclusion of fats in a third food category with a different pre-determined cut-off score and

the inclusion of potatoes and tubers as part of FVNL in the FSANZ model but not FVN in

the Ofcom model. Although our findings were expected given that FSANZ is a

derivative of the Ofcom model, the results confirmed construct/convergent validity

between these models when used to assess the nutritional quality of pre-packaged

foods available in the Canadian marketplace.

The case for construct/convergent validity with the Ofcom model was less convincing for

the EURO, PAHO, and HCST models. Limited published data on the validity of the

WHO EURO and PAHO models were identified. The data were primarily related to the

pilot testing conducted during the models’ development process. The EURO model

underwent expert consultation and field-testing using 10 country-specific food

composition databases, each of which contained approximately 200 foods that were

commonly consumed and/or advertised to children(3,19). To date, the EURO model has

not been tested for construct/convergent validity with healthful diets or predictive validity

with health outcomes(3). Nevertheless, the EURO model has been adapted for use by

other WHO Regional Offices, including the office for the Western Pacific in 2016(77),

South-East Asia in 2017(78), and the Eastern Mediterranean in 2017(79). The

55

development of these other WHO models underwent similar testing using country-

specific food databases in their respective regions(77-79). Although the development of

the PAHO model consisted of a comparison to three other NP models (i.e., Ofcom,

EURO, and a draft of the model for the Eastern Mediterranean region) for the

classification of 1,992 pre-packaged foods from five European countries, parameters

related to agreement or discordance between the models were not assessed, and

validity was not discussed(21).

Our findings related to the EURO and Ofcom models were consistent with other studies

in that there was overall “moderate” agreement(50), and that fats/oils and fruit juices were

often classified differently by the models(3). For example, there was only “slight”

agreement (κ=0.08) between the EURO and Ofcom model with a high level of

discordance for 62.7% (n=682 of 1,088) of the fruit/fruit juices, the majority of which

were fruit juices specifically (63.0%, n=430 of 682). This is likely because the EURO

model considers juices as exclusions, such that they are automatically considered as

“less healthy” irrespective of their nutrient content. The healthfulness of juices is viewed

differently in various jurisdictions; for example, the UK Eatwell Guide recommends that

consumption of fruit juices and/or smoothies should be limited to ≤150 mL/day as they

are a source of free sugars(80), while the 2015-2020 Dietary Guidelines for Americans

states that 100% fruit juices without added sugars and vegetable juices can be part of

healthy eating patterns(81). As the healthfulness of juices is currently under debate, it

remains to be seen whether the consideration of juices in NP models will change.

When PAHO was compared to the Ofcom model, there was only “fair” agreement

(κ=0.28) and discordance for 33.4% of the foods overall and across all 21 food

categories analyzed. For example, agreement was negligible (κ=0.01) with a high level

of discordance for 44.4% (n=80 of 180) of packaged legumes, all of which were

classified as “less healthy” by PAHO and “healthier” by Ofcom. For the majority of these

legume products (86.3%, n=69 of 80), their “less healthy” status was triggered because

the sodium threshold was exceeded. This discrepancy in classifying legumes is likely

because the sodium criterion is calculated per kcal of the food according to the PAHO

model, whereas it is calculated per 100 g of the food according to the Ofcom model.

56

Only a single study on the validity of the HCST model was identified, and only face

validity (i.e., extent to which the system is a useful tool for end users, such as

consumers making food-purchase decisions in the marketplace(34)) was discussed(82).

Based on the cross-classification analyses conducted to assess construct/convergent

validity between the HCST and Ofcom model in our study, adjustments to the nutrient

criteria may be prudent for the food categories with the highest proportions of food

classifications that disagreed and/or were grossly misclassified (i.e., eggs, fats/oils,

combination dishes, miscellaneous items, and sauces/dips/gravies/condiments). For

example, 75.0% (n=42 of 56) of the classifications for eggs disagreed because they

were classified as HCST tier 3 foods (i.e., partially in line with CFG such that fewer

choices should come from this tier), but had the lowest quartile of Ofcom scores (i.e.,

“healthiest” foods). For the majority of these egg products (73.8%, n=31 of 42), their tier

3 status was triggered only because the saturated fat threshold was exceeded, and not

because the thresholds for sodium or sugars were exceeded. In fact, although the

saturated fat content ranged from 2.8 to 3.3 g per CFG serving, these 31 egg products

were protein-rich sources that provided 10.9 to 13.2 g of protein per CFG serving. This

discrepancy is likely explained by the consideration of positive nutrients, such as

protein, by the Ofcom model, but not the HCST model. Although it should be noted that

NP was not an intended purpose of this model, adjustments to the HCST model may be

warranted before its continued use in public health policies and may in fact coincide well

with the revision of CFG, which is currently underway(83).

There are several strengths and limitations associated with this study. First, we

recognize that there are many other NP models developed by authoritative bodies that

could have been examined for content and construct/convergent validity in this study,

such as the Chilean “stop sign” warning labels and Nutri-Score, the latter which was

recently adopted by the French government as the voluntary front-of-pack nutrition

labelling scheme on 31 October 2017(84). In fact, based on the aforementioned

systematic review that was recently completed by our research group (Labonté ME,

Poon T, Gladanac B et al., unpublished results), we identified 387 potentially relevant

NP models, 78 of which met the inclusion criteria and were developed or endorsed by

government organizations. Given that it is neither feasible nor practical to examine the

57

validity of all models within a single research paper, our intent for the current study was

two-fold. We aimed to examine models developed by authoritative bodies, as data have

shown that industry-led models were less stringent with regards to the advertising of

energy-dense foods compared to government-led models(48). In addition, we aimed to

examine models with potentially wide applicability and/or relevance to the North

American context (e.g., WHO EURO and PAHO models), and so models from regions

with substantial differences in the food supply (e.g., Asia, Africa) were not examined.

Moreover, an important limitation of assessing validity by comparing NP models is the

lack of a gold standard measure for defining a “healthier” food, which was first

discussed by Arambepola et al.(32) nearly 10 years ago. Although validity testing ideally

should involve a gold standard comparator, some have reported that it is not a necessity

in the context of NP(32,33), given that all methods used in NP validity testing including the

most robust measurements are imperfect and subject to random and systematic

error(34). Indeed, depending on the research group, criterion validity has been described

as the extent to which the model is accurate based on “an externally derived gold

standard”(34), “an externally derived objective measure of healthfulness”(10), or “a

comparison of one measure with another(33). The method involving the comparison of

several models to the Ofcom model, a validated model which served as an objective

measure of healthfulness, is recognized by the WHO(1,9) and others(50,52,59) to be a valid

approach. It should be emphasised that we did not consider the Ofcom model to be

gold standard, as one does not exist in NP, and we did not consider models which

differed from the Ofcom model to be invalid. Rather, the Ofcom model was selected as

the reference because it is one of the most extensively studied NP models currently in

use, as discussed in the methods section. Our intent was to use the most extensively

validated NP model currently available as an objective measure of healthfulness and as

one method of identifying components of other models that may warrant adjustment or

further investigation. Nevertheless, a limitation of this method is that the scientific

underpinnings of the Ofcom model were established more than a decade ago, and the

science of NP has evolved since then. In fact, the Ofcom model is currently being

reviewed by Public Health England to ensure that the model reflects the latest dietary

guidance(85).

58

Furthermore, Townsend(34) aptly noted that a single validation study does not make a

NP model valid. Different methods of validity testing are associated with various

strengths and limitations(1,9,34). For example, the method of comparing the

classifications of foods determined using the NP model against food-based dietary

guidelines or views of nutrition experts typically involve a small sample of several

hundred indicator foods, are relatively simpler, and have been used more often(1,9,34). In

contrast, the method of assessing the association between the healthfulness of the

foods consumed by individuals, as determined using the NP model, and observed

changes in health outcomes is considered the most robust assessment of validity (i.e.,

predictive). However, this method is complex, costly in time and resources because it

requires data from prospective cohort studies, remains susceptible to recall bias from

self-reported diet recalls and food records, and has rarely been used(1,9,34). As such,

multiple validation studies using different comparison measures are required to validate

one model(10,34).

To our knowledge, this is the first study to assess the validity of several models

developed by authoritative bodies by using them to classify over 15,000 foods across

numerous food categories, in comparison to other validation studies which typically

used a small sample of several hundred indicator foods. Moreover, while data from

food composition databases are useful for NP purposes and have been used in most

validation studies, often the food categories are highly aggregated and the nutrient data

are presented as averages, and so variability across similar foods or different brands

cannot be ascertained(9). The use of foods from a large, national, branded database,

which is more reflective of the foods available in the marketplace, adds to the

robustness of the current analyses. However, the use of this database of pre-packaged

foods does not provide information with regards to how the NP models classify fresh

foods without Nutrition Facts tables on the labelling.

While the overall proportions of “healthier” and “less healthy” foods were similar in

magnitude between the FSANZ and HCST models versus the Ofcom model (all

approximately 50%), the kappa statistic indicated vast differences in agreement with

Ofcom (“near perfect” for FSANZ, but only “fair” for HCST); likewise, the discordance

parameter indicated vast differences in the classifications between each model and

59

Ofcom (only 5.3% for FSANZ, but 37.0% for HCST). This highlights the importance of

examining classifications across food categories, the level at which differences between

models become apparent. Because this study used a combination of parameters to

assess construct/convergent validity and consisted of detailed analyses across food

categories, it allowed for a comprehensive examination of the differences in

classifications between models.

In this study, all four NP models (FSANZ, HCST, EURO, and PAHO) demonstrated

moderate content validity. In contrast, the different models exhibited varying levels of

construct/convergent validity with the Ofcom model in classifying over 15,000 pre-

packaged foods. Construct/convergent validity was confirmed between the FSANZ and

Ofcom model, and to a lesser extent between the EURO and Ofcom model. Numerous

incongruencies with Ofcom were identified for the HCST and PAHO models, which

suggest that the classifications of several food categories may warrant further

investigation and adjustment. The results of this study may be informative for regulators

who wish to adapt existing models for use in country-specific applications. This

research is conducted as the WHO and many regulatory agencies are working to

establish transparent and reproducible methods to underpin their policies aimed at

curbing the obesity epidemic and preventing non-communicable diseases. In the span

of a year, the Canadian government has proposed to use NP in their policies on front-of-

pack labelling and restrictions on marketing to children(40). Research specifically on

validity testing is timely and globally relevant, as the number of NP models has

proliferated, but evidence on the adequacy of these models has lagged behind. As

several researchers have suggested that validity testing should be given the highest

priority in this field of research(3,10,11), this study contributes to addressing this need.

3.1.6 Supplementary Material

Tables 3-4 and 3-5, presented as supplementary material in the manuscript, are

provided here.

60

Table 3-4 Method based on the ingredient list used to estimate the FVN points of foods in the Ofcom modela

Ofcom criteria In-house criteria based on the ingredient list used to estimate FVN contentb Point(s) FVN content

of food (%) Non-concentrated FVN ingredients 0 ≤40 FVN is not one of the first two ingredients. 1 >40 FVN is the second ingredient. 2 >60 FVN is the first ingredient, but non-FVN ingredients appear to contribute

substantially to the product's weightc. 5 >80 FVN is the first ingredient, and only FVN ingredients contribute substantially

to the product's weight. Concentrated FVN ingredientsd 0 ≤40 FVN is not one of the first three ingredients. 1 >40 FVN is the second ingredient, but the amounts of the first and second

ingredients are not similar (see corresponding criteria for 2 pointse). FVN is the third ingredient and is estimated to represent ≥25% of the product’s weightf.

2 >60 FVN is the first ingredient, but non-FVN ingredients appear to contribute substantially to the product's weightc. FVN is the second ingredient, and the amounts of the first and second ingredients are similare.

5 >80 FVN is the first ingredient, and only FVN ingredients contribute substantially to the product's weight.

Abbreviation: FVN, fruits, vegetables, and nuts. a In the absence of quantitative declarations in the ingredient list, which are not required for food labelling in Canada, a method was developed by the research group at the University of Toronto in order to estimate the FVN content and corresponding FVN points of the foods for the Ofcom model. Given that the ingredients are listed in descending order by weight, the presence and positions of the FVN ingredients within the ingredient list were used to estimate the FVN content. b If sub-ingredients in brackets were presented for an ingredient, the ingredient was considered a FVN if one of the first two sub-ingredients was a FVN. If a food consisted of more than one component, and therefore more than one ingredient list (e.g., tuna kit with crackers), FVN points were calculated for the individual components, and the average of the FVN points was used to represent the food as a whole. If the ingredient list of a food was missing (<2% of foods), FVN points were not assigned unless it was evident from the product name that FVN contributed substantially to the product’s weight. c Non-FVN ingredients that were considered to contribute minimally to the product's weight included: salt, preservatives, colour, vitamins, minerals, oils, flavour extracts, antioxidants, and food additives. Other non-FVN ingredients were considered to contribute minimally if they appeared after salt or preservatives in the ingredient list. Non-FVN ingredients that were considered to contribute substantially to the product’s weight were other than those previously listed (e.g., sugar and water were considered to contribute substantially to the product’s weight). d Concentrated FVN ingredients are those in concentrated form (i.e., dried, evaporated, as pastes). According to the Ofcom model, concentrated FVN contribute less than non-concentrated FVN to the weight of the food; thus, the weight of concentrated FVN should be multiplied by two when calculating the FVN content of a food. Given that this criterion cannot be applied directly in the absence of quantitative declarations, the criteria were adjusted for concentrated FVN ingredients.

61

e A food may score two points if the total number of ingredients present in the ingredient list is low (e.g., ≤3), and if it is estimated that the amount of the first ingredient is only slightly higher than the amount of FVN in the food (e.g., estimated proportion of 55 versus 45%, respectively). f A food may score one point because at least 25% of the weight represented by a concentrated FVN equals to at least 40% FVN when multiplying the amount by two according to the Ofcom model.

62

Table 3-5 Proportion (%) of “healthier” and “less healthy” foods classified by models compared to the Ofcom model for all foods (n=15,227) and by food category

Schedule M category and description

Foods (n)

Ofcom FSANZ HCST EURO PAHO “Healthier”

“Less healthy”

“Healthier” “Less healthy”

“Healthier”

“Less healthy”


All 15,156 to 15,183a

“Healthier” 44.1 0.3 29.0 15.4 26.1 18.3 13.4 31.0 “Less healthy” 5.0 50.6 21.6 34.0 3.7 52.0 2.4 53.2

1. Bakery products

2082 to 2083


2. Beverages 481 to 482


3. Cereals, other grains

978 to 981


4. Dairy products, substitutes

1,237 “Healthier” 37.1 0.1 27.8 9.4 16.3 20.9 6.1 31.1 “Less healthy” 17.5 45.3 8.3 54.5 5.2 57.6 2.4 60.4

5. Desserts 820 to 827


6. Dessert toppings, fillings

115 “Healthier” 11.3 1.7 8.7 4.4 0.0 13.0 0.0 13.0 “Less healthy” 0.0 87.0 37.4 49.6 0.0 87.0 0.9 86.1

7. Eggs 56 “Healthier” 94.6 0.0 19.6 75.0 94.6 0.0 82.1 12.5 “Less healthy” 0.0 5.4 0.0 5.4 5.4 0.0 0.0 5.4

8. Fats, oils 535 “Healthier” 1.9 0.0 1.9 0.0 0.4 1.5 0.4 1.5 “Less healthy” 33.8 64.3 66.7 31.4 28.2 69.9 27.3 70.8

9. Marine products


10. Fruit, fruit juices

1,088 “Healthier” 69.8 2.7 59.6 12.9 9.7 62.7 38.8 33.6 “Less healthy” 1.3 26.3 16.4 11.2 0.0 27.6 3.1 24.5

63



Foods (n)


“Less healthy”


“Healthier”

“Less healthy”


11. Legumes 180 “Healthier” 99.4 0.0 81.7 17.8 85.0 14.4 55.0 44.4 “Less healthy” 0.0 0.6 0.0 0.6 0.0 0.6 0.0 0.6

12. Meat, poultry, their products, substitutes


13. Miscellaneous

444 to 445


14. Combination dishes

1,347 to 1,348


15. Nuts, seeds 220 “Healthier” 65.5 0.0 47.7 17.7 47.7 17.7 62.7 2.7 “Less healthy” 10.5 24.1 4.1 30.5 5.9 28.6 12.3 22.3

16. Potatoes, sweet potatoes, yams


17. Packaged salads


18. Sauces, dips, gravies, condiments

1,219 to 1,224


19. Snacks 794 “Healthier” 17.6 0.1 4.8 13.0 5.5 12.2 2.6 15.1 “Less healthy” 10.6 71.7 16.4 65.9 1.5 80.7 1.4 80.9

20. Soups 454 to 456


21. Sugars, 739 “Healthier” 3.5 0.0 3.5 0.0 0.5 3.0 0.0 3.5

64



Foods (n)


“Less healthy”


“Healthier”

“Less healthy”


sweets “Less healthy” 0.1 96.4 37.4 59.1 0.0 96.5 8.8 87.7 22. Vegetables 827 to


Abbreviations: EURO, World Health Organization Regional Office for Europe; FSANZ, Food Standards Australia New Zealand; HCST, Health Canada Surveillance Tool; PAHO, World Health Organization Regional Office for the Americas/Pan American Health Organization. a Across the models, data were missing for 0.29 to 0.41% (n=44 to 62) of foods.

65

3.2 Additional Analyses Not Included in Manuscript

3.2.1 Summary of Results on Construct/Convergent Validity

The results of the four parameters used to assess construct/convergent validity are

summarized in Table 3-6.

Table 3-6 Results of the assessment of construct/convergent validity

Parameter Statistical Test or Analysis

NP Model FSANZ HCST EURO PAHO

Association between the proportion of “less healthy” foods classified by model and quartiles of Ofcom scores

Cochran-Armitage trend test

Positive association (P<0.001)




Agreement with Ofcom

Kappa statistica

Near perfect Fair Moderate Fair

Discordance in classifications with Ofcom (% of foods)

McNemar’s test

Significant (5.3) Significant (37.0) Significant (22.0)

Significant (33.4)

Agreement and misclassification between quartiles of FSANZ scores and 4 HCST tiers versus quartiles of Ofcom scores (% of foods)b

Cross-classification analyses

Exact agreement: 95.2

Exact agreement: 32.6

NA NA

Agreement ±1 Q: 4.2

Agreement ±1 Q: 48.8

Disagreement: 0.3

Disagreement: 15.9

Gross misclassification: 0

Gross misclassification: 2.3

Overall convergence with Ofcom Convergent Not convergent Moderately convergent

Not convergent

Abbreviation: NA, not applicable. a Agreement was assessed using the kappa statistic (κ) as follows: 0.01 to 0.20 “slight”; 0.21 to 0.40 “fair”; 0.41 to 0.60 “moderate”; 0.61 to 0.80 “substantial”; 0.81 to 0.99 “near perfect”(73). b Data were missing for 0.3 and 0.4% of foods for the comparisons between FSANZ and HCST versus Ofcom, respectively.

3.2.2 Data Not Shown as Reported in Manuscript

As discussed in Section 3.1.4, “across all foods, cross-classification analyses using

quartiles of FSANZ and Ofcom scores indicated that there was exact agreement for

95.2% of foods and no gross misclassification of foods (data across food categories not

shown)”. These data are presented in Figure 3-7.

66

Figure 3-7 Cross-classification analyses between quartiles of Food Standards Australia New Zealand (FSANZ) and Ofcom scores for all foods (n=15,183; data missing for n=44) and 22 food categories from Schedule M of the Food and Drug Regulations(72). Exact agreement occurs when a food is classified in the same quartiles (e.g., FSANZ quartile 1 and Ofcom quartile 1). Agreement within an adjacent (±1) quartile (e.g., FSANZ quartile 1 and Ofcom quartile 2) and disagreement (e.g., FSANZ quartile 1 and Ofcom quartile 3) also were assessed. There was no gross misclassification, which occurs when a food is classified in opposing quartiles (e.g., FSANZ quartile 1 and Ofcom quartile 4).

67

3.2.3 Modified-PAHO

As discussed in Section 3.1.3, the PAHO model was intended for the assessment of

processed or ultra-processed foods only; that is, unprocessed or minimally processed

foods, culinary ingredients, and freshly prepared dishes were not subject to the nutrient

criteria and were automatically classified as “not excessive” in any nutrient. Due to the

subjectivity associated with classifying foods according to their level of processing for

the PAHO model, our research group decided to evaluate the construct/convergent

validity of a modified version of the PAHO model (modified-PAHO), which consisted of

the following two adjustments: 1) the assessment of all foods irrespective of the level of

processing; and 2) a revised sodium criterion applied specifically to beverages with very

low sodium and energy content. These adjustments are described in detail in our

publication on the application of NP models for the restrictions on marketing to children

(Appendix A)(26).

Across all foods (n=15,177), a positive association was observed between the modified-

PAHO and Ofcom model, such that the proportion of foods classified as “less healthy”

by the modified-PAHO model increased across quartiles of Ofcom scores, with the

highest quartile representing the “least healthy” foods. Specifically, the proportions (%,

95% confidence intervals (CI)) of foods classified as “less healthy” were 71.3 (70.0,

72.6), 96.0 (95.3, 96.7), 99.2 (98.9, 99.5), and 100 (100, 100) in quartiles 1, 2, 3, and 4,

respectively (P<0.001 for trend).


modified-PAHO and Ofcom model. While the overall proportions (%, 95% CI) of foods

classified as “healthier” by the two models differed (9.8 (9.4, 10.3) and 44.3 (43.5, 45.1),

respectively), significant discordance in the classifications between the models was

observed for 35.4% of the foods (P<0.001) (Figure 3-8). According to the modified-

PAHO model, none of the eggs, packaged salads (e.g., pasta or potato salads), or

sugars/sweets were classified as “healthier” (Table 3-7); thus, the kappa statistic and

McNemar’s test for significance in discordance, which required 2 by 2 tables to be

generated, could not be conducted for these three food categories. Significant

discordance was observed for all 19 food categories analyzed using McNemar’s test (all

68

P<0.01); however, agreement was “substantial” or “moderate” for two of these

categories (κ=0.56 or 0.76; 9.8 or 11.7% discordance). Amongst the remaining 17

categories, less agreement and higher proportions of discordance were observed.

Agreement was “fair” or “slight” (κ=0.01 for meat/poultry to 0.33 for fats/oils), and

discordance ranged from 1.5% for fats/oils to 73.4% for marine products. Notably, there

was no agreement (κ=0) or there was disagreement more than expected by chance (κ=-

0.02) and a range of 13.9 to 87.9% discordance for desserts, nuts/seeds, soups, and

dessert toppings/fillings.

Overall, the results pertaining to construct/convergent validity between the modified-

PAHO and Ofcom model were similar to those observed between the PAHO and Ofcom

model. Numerous incongruencies were identified between the modified-PAHO and

Ofcom model, which suggest that the classifications of several food categories may

warrant further investigation and adjustment.

69

Figure 3-8 Agreement (κ, 95% CI) and discordance (%, indicated above each line) between the modified World Health Organization Regional Office for the Americas/Pan American Health Organization (modified-PAHO) and Ofcom model for all foods (n=15,177; data missing for n=50) and 22 food categories from Schedule M of the Food and Drug Regulations(72). Agreement was assessed using the kappa statistic (κ) as follows: 0.01 to 0.20 “slight”; 0.21 to 0.40 “fair”; 0.41 to 0.60 “moderate”; 0.61 to 0.80 “substantial”; 0.81 to 0.99 “near perfect”(73). Significant discordance in classifications between models using McNemar’s test (*P<0.01, **P<0.001). The “X” symbol represents a food category for which the kappa statistic and McNemar’s test could not be conducted because 2 by 2 tables could not be generated (i.e., none of the eggs, packaged salads, or sugars/sweets were classified as “healthier” by the modified-PAHO model).

70

Table 3-7 Proportion (%) of “healthier” and “less healthy” foods classified by the modified-PAHO model compared to the Ofcom model for all foods (n=15,227) and by food category


Foods (n) Ofcom Modified-PAHO “Healthier” “Less healthy”

All 15,177a “Healthier” 9.4 35.0 “Less healthy” 0.5 55.2

1. Bakery products 2,082 “Healthier” 3.1 21.0 “Less healthy” 0.2 75.7

2. Beverages 481 “Healthier” 12.3 38.1 “Less healthy” 0.0 49.7

3. Cereals, other grains 978 “Healthier” 66.7 8.8 “Less healthy” 1.0 23.5

4. Dairy products, substitutes

1,237 “Healthier” 1.5 35.7 “Less healthy” 0.4 62.4

5. Desserts 827 “Healthier” 0.1 32.9 “Less healthy” 0.0 67.0


115 “Healthier” 0.0 13.0 “Less healthy” 0.9 86.1

7. Eggs 56 “Healthier” 0.0 94.6 “Less healthy” 0.0 5.4

8. Fats, oils 535 “Healthier” 0.4 1.5 “Less healthy” 0.0 98.1

9. Marine products 440 “Healthier” 3.9 73.4 “Less healthy” 0.0 22.7

10. Fruit, fruit juices 1,088 “Healthier” 17.5 55.0 “Less healthy” 1.7 25.9

11. Legumes 180 “Healthier” 51.7 47.8 “Less healthy” 0.0 0.6

12. Meat, poultry, their products, substitutes


13. Miscellaneous 445 “Healthier” 9.4 11.5 “Less healthy” 0.2 78.9

14. Combination dishes 1,348 “Healthier” 2.0 69.6 “Less healthy” 0.0 28.4

15. Nuts, seeds 220 “Healthier” 0.5 65.0 “Less healthy” 0.0 34.6



71

Table 3-7 Proportion (%) of “healthier” and “less healthy” foods classified by the modified-PAHO model compared to the Ofcom model for all foods (n=15,227) and by food category


Foods (n) Ofcom Modified-PAHO “Healthier” “Less healthy”

17. Packaged salads 70 “Healthier” 0.0 75.7 “Less healthy” 0.0 24.3

18. Sauces, dips, gravies, condiments

1,224 “Healthier” 2.5 28.6 “Less healthy” 1.5 67.4

19. Snacks 794 “Healthier” 2.4 15.4 “Less healthy” 1.4 80.9

20. Soups 456 “Healthier” 1.3 87.9 “Less healthy” 0.0 10.8

21. Sugars, sweets 739 “Healthier” 0.0 3.5 “Less healthy” 0.0 96.5

22. Vegetables 827 “Healthier” 22.4 48.4 “Less healthy” 0.0 29.3

Abbreviation: PAHO, World Health Organization Regional Office for the Americas/Pan American Health Organization. a Data were missing for 0.33% (n=50) of foods.

72

Chapter 4

4 Overall Discussion

4.1 Challenges Associated with Validating NP Models As the methods of validation are in their infancy(2), there are several main challenges

associated with validating NP models, namely the ambiguity in definitions and methods

used in validation studies, lack of a definition of a validated model, and lack of a gold

standard for defining a “healthier” food. A comprehensive discussion on the lack of a

gold standard in NP and the rationale for using the extensively validated Ofcom model

as the reference model in the validation study is provided in Section 3.1.5; thus, this

discussion is not be repeated in this section.

4.1.1 Ambiguity in the Definitions and Methods Used in Validating NP Models

As illustrated in Tables 2-2 and 2-3, although there is some commonality across the

definitions of validity used by different researchers, it is evident that there is variability in

the definitions used even amongst experts of NP, including Scarborough et al.

(2007)(33), Arambepola et al. (2008)(32), Townsend (2010)(34), Rayner (who prepared the

WHO manual on NP models (2011, in press(1))), and Cooper et al. (2016(10), 2017(36))bb.

This is particularly true for the definitions of criterion, concurrent, convergent, and

construct validity, as illustrated in Table 4-1. For example, the description:

• “correlates with an external criterion” is used to describe criterion validity

according to Scarborough et al. (2007)(33), Arambepola et al. (2008)(32),

Townsend (2010)(34), and Cooper et al. (2016)(10), but convergent validity

according to Rayner (WHO, 2011, in press)(1);

bb The references to A Dictionary of Epidemiology by Porta (2014)(30) and the seminal publication on validity by Cronbach and Meehl (1955)(35) discuss validity in general terms and not in relation to NP specifically.

73

• “correlates... with variables with which, theoretically, it should correlate” is used

to describe criterion and convergent validity according to Townsend (2010)(34),

but construct validity according to Arambepola et al. (2008)(32) and Rayner

(WHO, 2011, in press)(1); and

• “comparison with other measures of the same variable or a closely related

variable” is used to describe convergent validity according to Arambepola et al.

(2008)(32), but construct validity according to Cooper et al. (2017)(36).

With respect to criterion validity, while the majority of researchers agree that the

external criterion to which the NP model is compared should ideally be a gold standard,

both definitions used by Scarborough et al. (2007)(33) and Cooper et al. (2016)(10) allow

for comparisons to non-gold standard objective measures of healthfulness. In addition,

there appears to be consensus on the definition of concurrent validity, such that it is a

type of criterion validity and requires the measurement and criterion to refer to the same

time point. However, concurrent validity is not widely used in the literature on NP, as it

was explicitly defined only by Cooper et al., 2016(10). In contrast, the term convergent

validity was used by Arambepola et al. (2008)(32), Townsend (2010)(34), and Rayner

(WHO, 2011, in press)(1). Thus, based on the similarities between the definitions for

convergent versus concurrent and criterion, it appears that convergent validity is the

equivalent term for concurrent validity, particularly when a non-gold standard

comparison is used. Lastly, it is curious that there is no mention of construct validity by

Townsend (2010)(34). In fact, Townsend (2010)(34) describes criterion and convergent

validity in terms of theoretical correlations, which is typically used to describe construct

validity.

74

Table 4-1 Descriptions for criterion, concurrent, convergent, and construct validity used by different researchersa

Description of Type of Validity

Scarborough et al., 2007(33)

Arambepola et al., 2008(32)

Townsend, 2010(34)

Rayner (WHO, 2011, in press)(1)

Cooper et al., 2016(10), 2017(36)

Extent to which the measurement correlates with an external criterion of the phenomenon under study

• Criterion (GS or non-GS)

• Criterion (GS)

• Criterion (GS)

• Convergent (non-GS; same time point)b

• Criterion (GS or non-GS)

• Concurrent (GS; same time point)

Examines whether the method correlates in a predicted manner with variables with which, theoretically, it should correlate

NA • Construct • Criterion (GS)

• Convergent (time point not specified)

• Construct NA

Comparison with other measures of the same variable, or closely related ones

NA • Convergent NA NA • Constructb

Abbreviations: GS, gold standard, NA, not applicable. a The shaded cells represent the definitions and corresponding method used to determine the type of validity assessed in the validation study presented in Chapter 3. b This type of validity was inferred based on the method involving the comparison of NP models to one another.

Due to this ambiguity in the definitions of the different types of validity, there is

inconsistency with regards the type of validity that can be inferred from using certain

methods. While the use of biomarkers or medical records as indicators of health status

is always presented as a method of assessing criterion-related validity (either predictive

or concurrent), it is less clear for other methods. For example, convening a panel of

experts in relevant fields has been reported as a method of establishing content validity

according to Townsend (2010)(34) and Rayner (WHO, 2011, in press(1)); specifically, the

comparison of a NP model to the views of nutrition professionals has been reported as a

method to assess criterion validity according to Scarborough et al. (2007)(33),

convergent validity according to Townsend (2010)(34) and Rayner (WHO, 2011, in

press(1)), or construct validity according to Cooper et al. (2016(10), 2017(36)).

As discussed in Chapter 3, the validation study conducted involved the comparison

between several NP models to the extensively validated Ofcom model, which served as

75

an external objective measure of healthfulness. This validation study can be considered

to have assessed criterion, convergent, and construct validity given that, as indicated by

the shaded cells in Table 4-1:

• Scarborough et al. (2007)(33) and Cooper et al. (2016)(10) do not consider a gold

standard comparator a necessity for the assessment of criterion validity; and

• the comparison of NP models to one another has been reported as a method to

assess convergent validity by Rayner (WHO, 2011, in press(1)) or construct

validity by Cooper et al. (2016(10), 2017(36)).

Overall, based on the data presented, there appears to be a need for improved

reporting on the types of validity in relation to the methods used in future validation

studies of NP models.

4.1.2 Lack of a Definition of a Validated Model

In light of the evidence presented for the Ofcom model in Sections 2.4.2 and 3.1.5,

several questions remain: at what point is a NP model considered valid? Does every

type of validity need to be demonstrated, or would demonstration of predictive validity

suffice? Is there a minimum number of validation studies that need to be conducted?

There is currently no known definition or consensus on a definition with regards to when

a model is considered valid.

While a validation study contributes evidence towards establishing validity of a model for

a specific application in a specific population, a single study does not make a model

valid, as all methods of validation remain susceptible to error(1,9,34). In discussing the

validation of models in the U.S., Townsend(34) has emphasized that different types of

validation studies and multiple studies with various sub-population groups should be

conducted, given the diversity of the population. Specifically, Townsend(34) called for a

gold standard that requires “many studies of convergent and/or criterion validity” to

validate a model that aims to help consumers with food selection in the U.S.

marketplace(34). Cooper et al. (2016)(10) echoed this sentiment and stated that “a large

number of carefully designed studies to establish both construct and criterion-related

validity” is required to validate one model. However, “many studies” and “a large

76

number of studies” was not further defined, and the scientific standard required to

establish that a NP model is sufficiently valid remains unclear.

In contrast, the WHO suggested that consensus regarding the validity of a NP should be

developed through active engagement with nutrition experts and stakeholders with an

interest in the application of the model(1). In the case of the Ofcom model, after its

adoption by the media and communication regulator Ofcom in 2007, an independent

review panel endorsed the model and the Food Standards Agency board confirmed that

the model was “generally scientifically robust and fit for purpose” in 2009(3). Given the

updated recommendations on sugars and fibres from the government’s Scientific

Advisory Committee on Nutrition, Public Health England initiated a review of the Ofcom

model in 2016(3,85)cc. As such, even if validity was established previously, it is prudent

for NP models to be re-assessed periodically for validity and to ensure consistency with

the latest dietary guidance.

4.2 Other Considerations While the nutrient content of a food is crucial to NP, bioavailability of the nutrients

provided by a food, which is modified by numerous factors, is not captured by NP(86). In

addition, while NP is primarily concerned with the energy and nutrient content of a food,

other characteristics of foods are typically not considered, for example, beneficial

components such as phytochemicals, or detrimental components such as food toxins,

pathogens, or contaminants(1,9). Moreover, NP does not encompass, nor is there an

avenue to incorporate, factors related to the environmental sustainability of foods or

ethical or religious concerns regarding certain foods(9).

Furthermore, the focus of this thesis has been on NP models either developed or

endorsed by authoritative bodies. While outside the scope of this thesis, it should be

recognized that there is extensive literature on non-government-led NP models, some of

which have been validated using robust methods. Specifically, of the 309dd models that

cc The outcome of this review is anticipated to be announced in June 2018. dd Out of a total of 378 models identified.

77

did not meet the inclusion criteria of the systematic review, 53% (n=164 of 309) of the

models were excluded because they were not developed or endorsed by a government

body(8). For example, the Overall Nutritional Quality Index (ONQI) was excluded

because it was developed by an academic institution (Yale University), was not

subsequently endorsed by a government organization, and the details of the model

were not publicly available due to the propriety nature of the algorithm(8). While the

ONQI is an algorithm that incorporates more than 30 dietary components and ranks

individual foods by relative healthfulness, it also generates an average ONQI score for

the diet consumed by individuals(87). This NP model was demonstrated to have

predictive validity based on longitudinal data from the U.S. Health Professionals Follow-

up Study and Nurses’ Health Study (n=42,382 and 62,284 healthy men and women,

respectively)(87). Men in the highest versus lowest quintile of ONQI scores had relative

risks (RR, 95% CI) of 0.77 (0.70, 0.85) for cardiovascular disease, 0.84 (0.73, 0.96) for

diabetes, and 0.89 (0.83, 0.97) for all-cause mortality(87). Similarly, women in the

highest versus lowest quintile of ONQI scores had RR (95% CI) of 0.79 (0.71, 0.88) for

cardiovascular disease, 0.86 (0.78, 0.96) for diabetes, and 0.90 (0.84, 0.97) for all-

cause mortality(87).

4.3 Significance and Implications of Research The research presented in this thesis has implications in both a global context and a

Canadian context.

There has been increasing emphasis that one should be more concerned with the

healthfulness of diets, as opposed to the healthfulness foods(3). This is primarily

because the healthfulness of diets is not solely determined by the nutrient composition

of individual foods, but numerous other factors, including the portion sizes of the

individual foods, frequency of consumption of foods, variety of foods that contribute to

the diet, and the combinations in which they are consumed(1,9,32). Although NP itself is

used to evaluate individual foods, as opposed to diets, and considers the food in

isolation such that the amount or frequency of consumption is not taken into

account(32,88), NP can be used as a tool to improve the nutritional quality of diets(88).

Certain techniques may be used, including the weighting of analyses according to the

78

amount of energy the food contributed to the diet(32). Alternatively, serving size or food

category (e.g., main dish or snack) may be incorporated as factors within the model(2,32).

Over the last decade, substantial strides have been made in the field of NP research,

and several NP models have been studied extensively, especially the Ofcom model. As

discussed in Section 2.4.2, the Ofcom model was successfully adapted into a dietary

index, which demonstrated predictive validity between overall diet quality and several

health outcomes(43-47). Thus, NP models have the potential to substantially contribute to

research on dietary patterns and disease risk.

It should be noted that the systematic review was not designed a priori to identify validity

testing of NP models specifically; therefore, the results pertaining to the validity of the

models from the systematic review should be interpreted with caution(8). For example, it

should be emphasised that the 58% (n=45 of 78) of models for which data on validity

testing were not identified are not necessarily invalid; rather, validity testing may not

have been identified because, as an example, content validity is typically assessed

during the early phases of NP model development(34) for which the data are rarely

disclosed, either in the published or grey literature. Nevertheless, data on some form of

validity testing were identified for 42% (n=33 of 78) of the models(8), an increase from

approximately 33% of models for which validity testing was identified by Rayner et al. in

the 2013 unpublished WHO catalogue(7). These results, albeit promising, indicate that

there is still substantial improvement to be made in the validation of NP models,

particularly with regards to the reporting and documentation of validity testing.

Nevertheless, as Townsend(34) noted, a single validation study contributes evidence

towards establishing the validity of that model for a specific application in a specific

population. Thus, the validation study presented in Chapter 3 contributes to the

scientific evidence in establishing the overall validity of the FSANZ, HCST, EURO, and

PAHO models for assessing the nutritional quality of foods.

Based on the literature review presented in Chapter 2, none of identified studies

examined the construct/convergent validity of any NP model when applied to the

Canadian food supply. To our knowledge, the validation study presented in Chapter 3 is

the first study to assess the validity of several models developed by authoritative bodies

by using them to classify over 15,000 foods from a branded, Canadian database.

79

Moreover, as Health Canada has proposed to use a newly developed NP model in their

policies on front-of-pack labelling and restrictions on marketing to children(40), this

research can inform the use of this proposed NP model for these applications. First, the

validation study highlighted the importance of examining classifications across food

categories and identified specific categories that are prone to misclassification when

assessed by different models, including combination dishes, soups, fats/oils, and eggs.

Regulators will be able to use these data to prioritize the classifications of these

“problem” food categories in order to ensure that the proposed NP model generates

accurate classifications.

Second, this thesis provided an understanding of the different characteristics of NP

models, and the validation study postulated reasons for the discrepancies in the

classifications determined using several models which differed in their characteristics.

For example, it was observed that the HCST model, which includes only four negative

nutrients, fared poorly in identifying “healthier” foods when compared to the Ofcom

model, which takes into account the positive attributes of foods. Thus, similar results to

the HCST will likely be observed for the proposed NP model, which includes only three

negative nutrients (i.e., saturated fat, sugars, and sodium). As such, regulators will be

able to use these data to inform the decisions on whether any exemptions or

adjustments to the nutrient criteria should be made, such as including the protein

content for eggs, mono- or polyunsaturated fat content for fats/oils, or

fruit/vegetable/legume content for combination dishes and soups.

Lastly, this thesis provided an understanding of the definitions and methods for NP

model validation. Regulators will be able to use these data to prioritize the different

validation studies, based on their strengths and limitations, that will need to be

conducted in order to validate their newly developed NP model. For example, studies

involving consumers will be important for establishing face validity of the proposed NP

model for use in front-of-pack labelling; however, studies involving nutrition

professionals, who may be more discerning towards the healthfulness of foods, will be

important for establishing criterion/convergent/construct validity of the model for use in

restrictions on marketing to children.

80

Chapter 5

5 Conclusions Based on the data presented in this thesis, it is evident that different NP models can

vary substantially in their design and characteristics, and this variation ultimately gives

rise to inherent differences in the manner in which these models classify the

healthfulness of foods. As such, validity testing is important to ensure that these NP

models are consistent with and complement other nutrition regulations or policies

already in place within a jurisdiction. In addition, NP is a rapidly evolving field with a

proliferation in the number of NP models globally. As the evidence on the adequacy of

these models has lagged behind, particularly in the Canadian context, validity testing of

NP models is of utmost importance in this field of research.

Furthermore, the data presented in this thesis illustrate the challenges associated with

validating NP models in a global context. Due to the ambiguity in the definitions of the

different types of validity, there is inconsistency with regards the type of validity that can

be inferred from using certain methods. Although there are promising results to indicate

that the identification of validity testing amongst NP models has increased in the last

several years, there is still substantial improvement to be made in the validation of NP

models, particularly with regards to the reporting and documentation of validity testing.

In addition, the scientific standard required to establish that a NP model is sufficiently

valid remains unclear.

In this thesis, a comprehensive examination of the validity of the FSANZ, HCST, EURO,

and PAHO models was conducted by classifying over 15,000 foods, using a

combination of parameters for assessment, and carrying out detailed analyses across

food categories. This research provides regulators with pertinent data that will facilitate

the selection and validation of NP models for public health initiatives and nutrition policy.

Specifically in a Canadian context, this research provides valuable information that will

help inform the application and validity testing of the new NP model proposed by Health

Canada to underpin the regulations on front-of-pack nutrition labelling and restrictions

81

on marketing to children. Overall, this research contributes to the scientific evidence in

establishing the validity of the four models for assessing the nutritional quality foods,

globally and in Canada, and to the emerging science of NP.

82

References 1. World Health Organization (in press) Guiding Principles and Framework Manual for the Development or Adaptation of Nutrient Profile Models (First Edition). Geneva: WHO.

2. Rayner M, Scarborough P, Kaur A (2013) Nutrient profiling and the regulation of marketing to children. Possibilities and pitfalls. Appetite 62, 232-235.

3. Rayner M (2017) Nutrient profiling for regulatory purposes. Proc Nutr Soc 76, 230-236.

4. Rayner M, Scarborough P, Stockley L (2004) Nutrient profiles: options for definitions for use in relation to food promotion and children’s diets (final report). London: Food Standards Agency. http://webarchive.nationalarchives.gov.uk/20100403014958/http://www.food.gov.uk/multimedia/pdfs/nutrientprofilingfullreport.pdf.

5. Rayner M, Scarborough P, Stockley L et al. (2005) Nutrient profiles: further refinement and testing of Model SSCg3d (final report). London: Food Standards Agency. http://www.food.gov.uk/multimedia/pdfs/npreportsept05.pdf.

6. Stockley L, Rayner M, Kaur A (2008) Nutrient profiles for use in relation to food promotion and children’s diet: update of 2004 literature review. Oxford: University of Oxford. http://webarchive.nationalarchives.gov.uk/20090808161247/http://text.food.gov.uk/cgi-bin/parser.pl/0005/www.food.gov.uk/multimedia/pdfs/publication/npliteraturereviewupdate.pdf.

7. World Health Organization (in press) Nutrient Profiling. Catalogue of Nutrient Profile Models: Summary Report. Geneva: WHO.

8. Labonte ME, Poon T, Gladanac B et al. (Unpublished) Nutrient profile models with applications in government-led nutrition policies aimed at health promotion and noncommunicable disease prevention: a systematic review. (To be submitted to Adv Nutr in December 2017).

9. World Health Organization (2010) Nutrient Profiling. Report of a WHO/IASO Technical Meeting, London, United Kingdom 4–6 October 2010. Geneva: WHO. http://www.who.int/nutrition/publications/profiling/WHO_IASO_report2010.pdf.

10. Cooper SL, Pelly FE, Lowe JB (2016) Construct and criterion-related validation of nutrient profiling models: a systematic review of the literature. Appetite 100, 26-40.

11. Drewnowski A, Fulgoni V (2008) Nutrient profiling of foods: creating a nutrient-rich food index. Nutr Rev 66, 23-39.

12. U.S. Department of Agriculture (2014) Federal Register Part II Vol. 79. No. 42. 7 CFR §246: Special Supplemental Nutrition Program for Women, Infants and Children

http://webarchive.nationalarchives.gov.uk/20100403014958/http:/www.food.gov.uk/multimedia/pdfs/nutrientprofilingfullreport.pdf

http://webarchive.nationalarchives.gov.uk/20100403014958/http:/www.food.gov.uk/multimedia/pdfs/nutrientprofilingfullreport.pdf

http://www.food.gov.uk/multimedia/pdfs/npreportsept05.pdf

http://webarchive.nationalarchives.gov.uk/20090808161247/http:/text.food.gov.uk/cgi-bin/parser.pl/0005/www.food.gov.uk/multimedia/pdfs/publication/npliteraturereviewupdate.pdf



http://www.who.int/nutrition/publications/profiling/WHO_IASO_report2010.pdf

83

(WIC): Revisions in the WIC Food Packages – Final Rule. http://www.fns.usda.gov/sites/default/files/03-04-14_WIC-Food-Packages-Final-Rule.pdf (accessed June 2017).

13. National Food Agency of Sweden (2015) Regulations amending the National Food Agency's regulations (SLVFS 2005:9) on the use of a particular symbol. http://www.livsmedelsverket.se/globalassets/om-oss/lagstiftning/livsmedelsinfo-till-konsum---markning/livsfs-2015-1-particular-symbol-eng.pdf (accessed June 2017).

14. U.S. Food and Drug Administration (2017) 21 CFR §101.14. Health claims: general requirements. http://www.ecfr.gov/cgi-bin/text-idx?SID=c77c3ef247de8778a86e598ef4c2dcdc&mc=true&node=pt21.2.101&rgn=div5#se21.2.101_114 (accessed June 2017).

15. U.S. Food and Drug Administration (2017) 21 CFR §101.65. Implied nutrient content claims and related label statements. https://www.ecfr.gov/cgi-bin/text-idx?SID=c77c3ef247de8778a86e598ef4c2dcdc&mc=true&node=pt21.2.101&rgn=div5#se21.2.101_165 (accessed June 2017).

16. European Commission (2012) Regulation (EC) No 1924/2006 of the European parliament and of the council of 20 December 2006 on nutrition and health claims made on foods. http://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX:02006R1924-20121129 (accessed June 2017).

17. World Health Organization (2010) Set of recommendations on the marketing of foods and non-alcoholic beverages to children. Geneva: WHO. http://apps.who.int/iris/bitstream/10665/44416/1/9789241500210_eng.pdf.

18. Australian Government (2015) Australia New Zealand Food Standards Code - Standard 1.2.7 - Nutrition, Health and Related Claims (F2015C00967). https://www.comlaw.gov.au/Details/F2015C00967 (accessed June 2017).

19. World Health Organization Regional Office for Europe (2015) WHO Regional Office for Europe nutrient profile model. Copenhagen: WHO Regional Office for Europe. http://www.euro.who.int/__data/assets/pdf_file/0005/270716/Nutrient-children_web-new.pdf?ua=1.

20. Queensland Government (2015) Food for Sport. Resource 4: red criteria and label reading. http://www.nprsr.qld.gov.au/industry-information/pdf/clubs/food-for-sport-resource-4.pdf (accessed August 2016).

21. Pan American Health Organization (2016) Pan American Health Organization Nutrient Profile Model. Washington, D.C.: PAHO. http://iris.paho.org/xmlui/bitstream/handle/123456789/18621/9789275118733_eng.pdf?sequence=8&isAllowed=y.

22. World Health Organization and Food and Agricultural Organization (2003) Diet, nutrition and the prevention of chronic diseases. Report of the joint WHO/FAO expert

http://www.fns.usda.gov/sites/default/files/03-04-14_WIC-Food-Packages-Final-Rule.pdf

http://www.livsmedelsverket.se/globalassets/om-oss/lagstiftning/livsmedelsinfo-till-konsum---markning/livsfs-2015-1-particular-symbol-eng.pdf

http://www.livsmedelsverket.se/globalassets/om-oss/lagstiftning/livsmedelsinfo-till-konsum---markning/livsfs-2015-1-particular-symbol-eng.pdf

http://www.ecfr.gov/cgi-bin/text-idx?SID=c77c3ef247de8778a86e598ef4c2dcdc&mc=true&node=pt21.2.101&rgn=div5#se21.2.101_114



https://www.ecfr.gov/cgi-bin/text-idx?SID=c77c3ef247de8778a86e598ef4c2dcdc&mc=true&node=pt21.2.101&rgn=div5#se21.2.101_165



http://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX:02006R1924-20121129

http://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX:02006R1924-20121129

http://apps.who.int/iris/bitstream/10665/44416/1/9789241500210_eng.pdf

https://www.comlaw.gov.au/Details/F2015C00967

http://www.euro.who.int/__data/assets/pdf_file/0005/270716/Nutrient-children_web-new.pdf?ua=1

http://www.euro.who.int/__data/assets/pdf_file/0005/270716/Nutrient-children_web-new.pdf?ua=1

http://www.nprsr.qld.gov.au/industry-information/pdf/clubs/food-for-sport-resource-4.pdf

http://www.nprsr.qld.gov.au/industry-information/pdf/clubs/food-for-sport-resource-4.pdf

http://iris.paho.org/xmlui/bitstream/handle/123456789/18621/9789275118733_eng.pdf?sequence=8&isAllowed=y

http://iris.paho.org/xmlui/bitstream/handle/123456789/18621/9789275118733_eng.pdf?sequence=8&isAllowed=y

84

consultation (WHO Technical Report Series No. 916). Geneva: WHO. http://www.who.int/dietphysicalactivity/publications/trs916/en/.

23. World Health Organization (2015) Guideline: sugars intake for adults and children. Geneva: WHO. http://apps.who.int/iris/bitstream/10665/149782/1/9789241549028_eng.pdf?ua=1.

24. World Health Organization (2012) Guideline: sodium intake for adults and children. Geneva: WHO. http://apps.who.int/iris/bitstream/10665/77985/1/9789241504836_eng.pdf.

25. Food and Agricultural Organization (2010) Fats and fatty acids in human nutrition. Report of an expert consultation (Food and Nutrition Paper No. 91). Rome: FAO. http://www.who.int/nutrition/publications/nutrientrequirements/fatsandfattyacids_humannutrition/en/.

26. Labonte ME, Poon T, Mulligan C et al. (2017) Comparison of global nutrient profiling systems for restricting the commercial marketing of foods and beverages of low nutritional quality to children in Canada. Am J Clin Nutr 106, 1471-1481.

27. UK Department of Health (2013) Guide to creating a front of pack (FoP) nutrition label for pre-packaged products sold through retail outlets. London: Department of Health. https://www.gov.uk/government/publications/front-of-pack-nutrition-labelling-guidance.

28. United Kingdom Department of Health (2011) Nutrient Profiling Technical Guidance. London: Department of Health. https://www.gov.uk/government/publications/the-nutrient-profiling-model.

29. Health Canada (2014) The Development and Use of a Surveillance Tool: the Classification of Foods in the Canadian Nutrient File According to Eating Well with Canada’s Food Guide. Ottawa: Ministry of Health. http://publications.gc.ca/collections/collection_2014/sc-hc/H164-158-2-2014-eng.pdf.

30. Gibson RS (2005) The design of nutritional assessment systems. In Principles of Nutritional Assessment, 2nd ed., pp. 11-15. New York: Oxford University Press.

31. Porta M (2014) A Dictionary of Epidemiology, 6th ed., [S Greenland, M Hernán, I dos Santos Silva and JM Last, editors]. New York: Oxford University Press.

32. Arambepola C, Scarborough P, Rayner M (2008) Validating a nutrient profile model. Public Health Nutr 11, 371-378.

33. Scarborough P, Boxer A, Rayner M et al. (2007) Testing nutrient profile models using data from a survey of nutrition professionals. Public Health Nutr 10, 337-345.

34. Townsend MS (2010) Where is the science? What will it take to show that nutrient profiling systems work? Am J Clin Nutr 91, 1109S-1115S.

http://www.who.int/dietphysicalactivity/publications/trs916/en/

http://apps.who.int/iris/bitstream/10665/149782/1/9789241549028_eng.pdf?ua=1


http://www.who.int/nutrition/publications/nutrientrequirements/fatsandfattyacids_humannutrition/en/

http://www.who.int/nutrition/publications/nutrientrequirements/fatsandfattyacids_humannutrition/en/

https://www.gov.uk/government/publications/front-of-pack-nutrition-labelling-guidance

https://www.gov.uk/government/publications/front-of-pack-nutrition-labelling-guidance

https://www.gov.uk/government/publications/the-nutrient-profiling-model


http://publications.gc.ca/collections/collection_2014/sc-hc/H164-158-2-2014-eng.pdf

85

35. Cronbach LJ, Meehl PE (1955) Construct validity in psychological tests. Psychol Bull 52, 281-302.

36. Cooper SL, Pelly FE, Lowe JB (2017) Assessment of the construct validity of the Australian Health Star Rating: a nutrient profiling diagnostic accuracy study. Eur J Clin Nutr, 1-7.

37. Clerfeuille E, Vieux F, Lluch A et al. (2013) Assessing the construct validity of five nutrient profiling systems using diet modeling with linear programming. Eur J Clin Nutr 67, 1003-1005.

38. Darmon N, Vieux F, Maillot M et al. (2009) Nutrient profiles discriminate between foods according to their contribution to nutritionally adequate diets: a validation study using linear programming and the SAIN,LIM system. Am J Clin Nutr 89, 1227-1236.

39. Federal Provincial Territorial Group on Nutrition Working Group on Improving the Consistency of School Food and Beverage Criteria (2013) Provincial and Territorial Guidance Document for the Development of Nutrient Criteria for Foods and Beverages in Schools 2013. http://foodsecurecanada.org/resources-news/resources-research/provincial-and-territorial-guidance-document-development-nutrient.

40. Health Canada (2016) Healthy Eating Strategy. Ottawa: Ministry of Health. https://www.canada.ca/content/dam/canada/health-canada/migration/publications/eating-nutrition/healthy-eating-strategy-canada-strategie-saine-alimentation/alt/pub-eng.pdf.

41. Labonte ME, Poon T, Gladanac B et al. (2015) Systematic review of nutrient profile models developed for nutrition-related policies and regulations aimed at noncommunicable disease prevention. http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42015024750 (accessed January 2017).

42. Rayner M, Scarborough P, Boxer A et al. (2005) Nutrient profiles: development of final model (final report). London: Food Standards Agency.

43. Masset G, Scarborough P, Rayner M et al. (2015) Can nutrient profiling help to identify foods which diet variety should be encouraged? Results from the Whitehall II cohort. Br J Nutr 113, 1800-1809.

44. Julia C, Ducrot P, Lassale C et al. (2015) Prospective associations between a dietary index based on the British Food Standard Agency nutrient profiling system and 13-year weight gain in the SU.VI.MAX cohort. Prev Med 81, 189-194.

45. Julia C, Fezeu LK, Ducrot P et al. (2015) The nutrient profile of foods consumed using the British Food Standards Agency nutrient profiling system is associated with metabolic syndrome in the SU.VI.MAX cohort. J Nutr 145, 2355-2361.

http://foodsecurecanada.org/resources-news/resources-research/provincial-and-territorial-guidance-document-development-nutrient

http://foodsecurecanada.org/resources-news/resources-research/provincial-and-territorial-guidance-document-development-nutrient

https://www.canada.ca/content/dam/canada/health-canada/migration/publications/eating-nutrition/healthy-eating-strategy-canada-strategie-saine-alimentation/alt/pub-eng.pdf



http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42015024750

86

46. Donnenfeld M, Julia C, Kesse-Guyot E et al. (2015) Prospective association between cancer risk and an individual dietary index based on the British Food Standards Agency nutrient profiling system. Br J Nutr 114, 1702-1710.

47. Adriouch S, Julia C, Kesse-Guyot E et al. (2016) Prospective association between a dietary quality index based on a nutrient profiling system and cardiovascular disease risk. Eur J Prev Cardiol, 1-8.

48. Brinsden H, Lobstein T (2013) Comparison of nutrient profiling schemes for restricting the marketing of food and drink to children. Pediatr Obes 8, 325-337.

49. Eržen N, Rayner M, Pravst I (2015) A comparative evaluation of the use of a food composition database and nutrition declarations for nutrient profiling. J Food Nutr Res 54, 93-100.

50. Wicks M, Wright H, Wentzel-Viljoen E (2017) Restricting the marketing of foods and non-alcoholic beverages to children in South Africa: are all nutrient profiling models the same? Br J Nutr 116, 2150-2159.

51. Faulkner GP, Livingstone MB, McCaffrey TA et al. (2014) Supermarket own brand foods: lower in energy cost but similar in nutritional quality to their market brand alternatives. J Hum Nutr Diet 27, 617-625.

52. Rosentreter SC, Eyles H, Ni Mhurchu C (2013) Traffic lights and health claims: a comparative analysis of the nutrient profile of packaged foods available for sale in New Zealand supermarkets. Aust N Z J Public Health 37, 278-283.

53. Scarborough P, Payne C, Agu CG et al. (2013) How important is the choice of the nutrient profile model used to regulate broadcast advertising of foods to children? A comparison using a targeted data set. Eur J Clin Nutr 67, 815-820.

54. Trichterborn J, Drossard C, Kersting M et al. (2012) The potential impact of nutrient profiles on dairy-related energy and nutrient intake in German children and adolescents. Eur J Clin Nutr 66, 466-473.

55. Drewnowski A, Fulgoni V (2011) Comparing the nutrient rich foods index with "Go," "Slow," and "Whoa," foods. J Am Diet Assoc 111, 280-284.

56. Roberto CA, Bragg MA, Livingston KA et al. (2012) Choosing front-of-package food labelling nutritional criteria: how smart were 'Smart Choices'? Public Health Nutr 15, 262-267.

57. Trichterborn J, Harzer G, Kunz C (2011) Fine bakery wares with label claims in Europe and their categorisation by nutrient profiling models. Eur J Clin Nutr 65, 307-312.

58. Trichterborn J, Harzer G, Kunz C (2011) Nutrient profiling and food label claims: evaluation of dairy products in three major European countries. Eur J Clin Nutr 65, 1032-1038.

87

59. Eyles H, Gorton D, Ni Mhurchu C (2010) Classification of 'healthier' and 'less healthy' supermarket foods by two Australasian nutrient profiling models. N Z Med J 123, 8-20.

60. Hebden L, King L, Kelly B et al. (2010) Regulating the types of foods and beverages marketed to Australian children: how useful are food industry commitments? Nutr and Diet 67, 258-266.

61. Walker KZ, Woods J, Ross J et al. (2010) Yoghurt and dairy snacks presented for sale to an Australian consumer: are they becoming less healthy? Public Health Nutr 13, 1036-1041.

62. Drewnowski A, Maillot M, Darmon N (2009) Testing nutrient profile models in relation to energy density and energy cost. Eur J Clin Nutr 63, 674-683.

63. Walker KZ, Woods JL, Rickard CA et al. (2008) Product variety in Australian snacks and drinks: how can the consumer make a healthy choice? Public Health Nutr 11, 1046-1053.

64. Garsetti M, de Vries J, Smith M et al. (2007) Nutrient profiling schemes: overview and comparative analysis. Eur J Nutr 46 Suppl 2, 15-28.

65. Labouze E, Goffi C, Moulay L et al. (2007) A multipurpose tool to evaluate the nutritional quality of individual foods: Nutrimap. Public Health Nutr 10, 690-700.

66. Quinio C, Biltoft-Jensen A, De Henauw S et al. (2007) Comparison of different nutrient profiling schemes to a new reference method using dietary surveys. Eur J Nutr 46 Suppl 2, 37-46.

67. Azais-Braesco V, Goffi C, Labouze E (2006) Nutrient profiling: comparison and critical analysis of existing systems. Public Health Nutr 9, 613-622.

68. Bernstein JT, Franco-Arellano B, Schermel A et al. (2017) Healthfulness and nutritional composition of Canadian prepackaged foods and with and without sugar claims. Appl Phys Nutr Metab 42, 1217-1224.

69. Bernstein JT, Schermel A, Mills CM et al. (2016) Total and free sugar content of Canadian prepackaged foods and beverages. Nutrients 8, 582.

70. World Health Organization (2013) Global action plan for the prevention and control of noncommunicable diseases 2013-2020. Geneva: WHO. http://apps.who.int/iris/bitstream/10665/94384/1/9789241506236_eng.pdf?ua=1.

71. World Health Organization Regional Office for Europe (2017) Seventieth World Health Assembly (18 May 2017): Preparation for the third High-level Meeting of the General Assembly on the Prevention and Control of Non-communicable Diseases, to be held in 2018 – Report by the Director-General. Copenhagen: WHO Regional Office for Europe. http://www.euro.who.int/__data/assets/pdf_file/0010/340786/A70_27-en-cover.pdf?ua=1.


http://www.euro.who.int/__data/assets/pdf_file/0010/340786/A70_27-en-cover.pdf?ua=1

http://www.euro.who.int/__data/assets/pdf_file/0010/340786/A70_27-en-cover.pdf?ua=1

88

72. Government of Canada (2016) Food and Drug Regulations (C.R.C., c. 870), Sections B.01.001, B.01.002A and D.01.001. http://laws-lois.justice.gc.ca/eng/regulations/c.r.c.,_c._870/page-165.html#h-353 (accessed December 2016).

73. Viera AJ, Garrett JM (2005) Understanding interobserver agreement: the kappa statistic. Fam Med 37, 360-363.

74. Food Standards Australia New Zealand (2015) Monitoring of trans fatty acids. http://www.foodstandards.gov.au/science/monitoringnutrients/Pages/Monitoring-of-trans-fatty-acids.aspx (accessed September 2017).

75. Government of Canada (2016) Fats. http://healthycanadians.gc.ca/eating-nutrition/healthy-eating-saine-alimentation/nutrients-nutriments/fats-lipides-eng.php (accessed September 2017).

76. Arcand J, Scourboutakos MJ, Au JT et al. (2014) Trans fatty acids in the Canadian food supply: an updated analysis. Am J Clin Nutr 100, 1116-1123.

77. World Health Organization Regional Office for the Western Pacific (2016) WHO nutrient profile model for the Western Pacific Region: a tool to protect children from food marketing. Manila: WHO Regional Office for the Western Pacific. http://iris.wpro.who.int/bitstream/handle/10665.1/13525/9789290617853-eng.pdf.

78. World Health Organization Regional Office for South-East Asia (2017) WHO Nutrient Profile Model for South-East Asia Region. To implement the set of recommendations on the marketing of foods and non-alcoholic beverages to children. New Delhi: WHO Regional Office for South-East Asia. http://apps.who.int/iris/bitstream/10665/253459/1/9789290225447-eng.pdf.

79. World Health Organization Regional Office for the Eastern Mediterranean (2017) Nutrient profile model for the marketing of food and non-alcoholic beverages to children in the WHO Eastern Mediterranean Region. Cairo: WHO Regional Office for the Eastern Mediterranean. http://applications.emro.who.int/dsaf/EMROPUB_2017_en_19632.pdf?ua=1.

80. Public Health England (2016) The Eatwell Guide. Helping you eat a healthy, balanced diet. London: Public Health England. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/551502/Eatwell_Guide_booklet.pdf.

81. U.S. Department of Health and Human Services and U.S. Department of Agriculture (2015) 2015-2020 Dietary Guidelines for Americans (Eighth Edition). https://health.gov/dietaryguidelines/2015/resources/2015-2020_Dietary_Guidelines.pdf.

82. Jessri M, Nishi SK, L'Abbe MR (2015) Assessing the nutritional quality of diets of Canadian adults using the 2014 Health Canada Surveillance Tool tier system. Nutrients 7, 10447-10468.

http://laws-lois.justice.gc.ca/eng/regulations/c.r.c.,_c._870/page-165.html#h-353


http://www.foodstandards.gov.au/science/monitoringnutrients/Pages/Monitoring-of-trans-fatty-acids.aspx

http://www.foodstandards.gov.au/science/monitoringnutrients/Pages/Monitoring-of-trans-fatty-acids.aspx

http://healthycanadians.gc.ca/eating-nutrition/healthy-eating-saine-alimentation/nutrients-nutriments/fats-lipides-eng.php

http://healthycanadians.gc.ca/eating-nutrition/healthy-eating-saine-alimentation/nutrients-nutriments/fats-lipides-eng.php

http://iris.wpro.who.int/bitstream/handle/10665.1/13525/9789290617853-eng.pdf

http://apps.who.int/iris/bitstream/10665/253459/1/9789290225447-eng.pdf

http://applications.emro.who.int/dsaf/EMROPUB_2017_en_19632.pdf?ua=1

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/551502/Eatwell_Guide_booklet.pdf

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/551502/Eatwell_Guide_booklet.pdf

https://health.gov/dietaryguidelines/2015/resources/2015-2020_Dietary_Guidelines.pdf

89

83. Government of Canada (2017) Revision process for Canada's Food Guide. https://www.canada.ca/en/health-canada/services/canada-food-guides/revision-process.html (accessed September 2017).

84. Santé publique France (2017) Nutri-Score. https://www.santepubliquefrance.fr/Sante-publique-France/Nutri-Score (accessed November 2017).

85. Public Health England (2016) Review of the nutrient profiling model. https://www.gov.uk/government/collections/review-of-the-nutrient-profiling-model (accessed March 2017).

86. Ryckembusch D, Frega R, Silva MG et al. (2013) Enhancing nutrition: a new tool for ex-ante comparison of commodity-based vouchers and food transfers. World Development 49, 58-67.

87. Chiuve SE, Sampson L, Willett WC (2011) The association between a nutritional quality index and risk of chronic disease. Am J Prev Med 40, 505-513.

88. Breda J (2014) A WHO nutrient profile model: the European perspective. Platform for Action on Diet, Physical Activity and Health Brussels, 24 November 2014. http://ec.europa.eu/health/nutrition_physical_activity/docs/ev_20141124_co06_en.pdf.

https://www.canada.ca/en/health-canada/services/canada-food-guides/revision-process.html

https://www.canada.ca/en/health-canada/services/canada-food-guides/revision-process.html

https://www.santepubliquefrance.fr/Sante-publique-France/Nutri-Score

https://www.gov.uk/government/collections/review-of-the-nutrient-profiling-model

http://ec.europa.eu/health/nutrition_physical_activity/docs/ev_20141124_co06_en.pdf

90

Appendix A

Study on NP Models for Marketing Restrictions to Children This study has been published: Labonté MÈ, Poon T, Mulligan C, Bernstein JT, Franco-

Arellano B, L’Abbé MR. Comparison of global nutrient profiling systems for restricting

the commercial marketing of foods and beverages of low nutritional quality to children in

Canada. Am J Clin Nutr 106, 1471-1481.

Although this research was not explicitly part of my MSc thesis, I had a significant role in

this research, as indicated in Table A-1.

Table A-1 Authors’ contributions to study on NP models for marketing restrictions to children

Task Author Prior to January 2016 when TP started MSc Calculated FSANZ scores Labonté, Bernstein, Franco-Arellano After January 2016 when TP started MSc Designed research study Labonté, Poon, L’Abbé Assessed on-package marketing to children Mulligan Generated data for NP models Labonté, Poon Analyzed data, ran statistical analyses Poon Wrote manuscript Labonté Provided comments/revisions to draft manuscripts Poon, L’Abbé Provided comments/revisions to final manuscript Poon, Mulligan, Bernstein, Franco-Arellano, L’Abbé Submitted manuscript to journal Labonté Provided comments/revisions to post-submission responses

Poon, L’Abbé

Comparison of global nutrient profiling systems for restricting thecommercial marketing of foods and beverages of low nutritionalquality to children in Canada

Marie-Eve Labonte, Theresa Poon, Christine Mulligan, Jodi T Bernstein, Beatriz Franco-Arellano, and Mary R L’Abbe

Department of Nutritional Sciences, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada

ABSTRACTBackground: The Canadian government recently committed to in-troduce legislation to restrict the commercial marketing of un-healthy foods and beverages to children.Objective: We compared the degree of strictness and agreementbetween nutrient profile (NP) models relevant to marketing restric-tions by applying them in the Canadian context.Design: With the use of data from the University of Toronto 2013Food Label Information Program (n = 15,342 prepackaged foods), 4 NPmodels were evaluated: the Food Standards Australia New Zealand-Nutrient Profiling Scoring Criterion (FSANZ-NPSC), theWHORegionalOffice for Europe (EURO) model, the Pan American Health Organization(PAHO) model, and a modified version of the PAHO model (Modified-PAHO), which did not consider the extent of food processing because theapplication of this characteristic was prone to ambiguity. The number andproportion of foods that would be eligible for marketing to children wascalculated with the use of each model, overall and by food category.Results: The Modified-PAHO and PAHO models would permit only9.8% (95% CI: 9.4%, 10.3%) and 15.8% (95% CI: 15.3%, 16.4%) offoods, respectively, followed by the EURO model [29.8% (95% CI:29.0%, 30.5%)]. In contrast, the FSANZ-NPSC would consider al-most half of prepackaged foods as eligible for marketing to children[49.0% (95% CI: 48.2%, 49.8%)]. Cross-classification analysesshowed that only 8.1% of foods would be eligible based on all models(e.g., most pastas without sauce). Subanalyses showed that eachmodel would be more stringent when evaluating food items thatspecifically target children on their package (n = 747; from 1.9% offoods eligible under Modified-PAHO to 24.2% under FSANZ-NPSC).Conclusions: The degree of strictness and agreement vary greatlybetween NP models applicable to marketing restrictions. The dis-crepancies between models highlight the importance for policymakers to carefully evaluate the characteristics underlying suchmodels when trying to identify a suitable model to underpin regu-lations restricting the marketing of unhealthy foods to children. Am JClin Nutr 2017;106:1471–81.

Keywords: food processing, healthfulness, marketing restrictions,children, nutrition policy

INTRODUCTION

Recent trends show that the prevalence of obesity in Canadianchildren and adolescents has reached a plateau ofw13% over the

last decade (1). Although the leveling off is promising, thoserates are still 3 times higher than those observed in the early1980s (2). The prevalence of both overweight and obesity inCanadian children and adolescents, estimated at 27.0%, alsoremains relatively high at the present time (1).

A strong evidence base now supports the role of unhealthyfood and beverage marketing as one of the key drivers of theglobal childhood obesity epidemic (3). Food promotion has in-deed been associated with increased food intakes in children (4–6). Marketing practices are also known to shape children’spreferences toward low–nutrient-dense food products high in fat,sugar, and salt (6, 7). Additionally, “less healthy” products, asdefined by various nutritional criteria, represent the ones that aremost heavily marketed to young individuals in Canada and inseveral other countries (8–10).

In 2016, the WHO Commission on Ending Childhood Obesityestablished a set of recommended actions for its Member States toeffectively combat childhood obesity (11). One of them is to

Supported by Canadian Institutes of Health Research (CIHR) Postdoctoral

Fellowship MFE-140953 (M-EL); Burroughs Wellcome Fund, Innovation in

Regulatory Science (TP, CM, MRL; 1014187); a CIHR Strategic Training

Grant in Population Intervention for Chronic Disease Prevention (JTB; TGF-

53893); the CIHR Collaborative Training Program in Public Health Policy

(JTB); a Graduate Student Fellowship of the Department of Nutritional

Sciences, University of Toronto (BF-A); a CIHR Strategic Operating Grant

(MRL; 201103SOK2118150); Canadian Stroke Network (MRL;

201103SOK201194-000); and an Earle W McHenry Research Chair un-

restricted research grant from the University of Toronto (MRL).

Supplemental Figure 1 and Supplemental Table 1 are available from the

“Online Supporting Material” link in the online posting of the article and

from the same link in the online table of contents at http://ajcn.nutrition.org.

Present address for M-EL: Institute of Nutrition and Functional Foods,

Laval University, Quebec City, Quebec, Canada.

Address correspondence to MRL (e-mail: [email protected]).

Abbreviations used: EURO, WHO Regional Office for Europe; FLIP,

Food Label Information Program; FSANZ-NPSC, Food Standards Australia

New Zealand-Nutrient Profiling Scoring Criterion; FVNL, fruit, vegetable,

nut, and legume; Modified-PAHO, modified version of the Pan American

Health Organization model; NFt, Nutrition Facts table; NP, nutrient profile;

PAHO, Pan American Health Organization.

Received May 25, 2017. Accepted for publication October 2, 2017.

First published online October 25, 2017; doi: https://doi.org/10.3945/ajcn.

117.161356.

Am J Clin Nutr 2017;106:1471–81. Printed in USA. � 2017 American Society for Nutrition 1471

at UN

IVE

RS

ITY

OF

TO

RO

NT

O G

ER

ST

EIN

SC

IEN

CE

INF

OR

MA

TIO

N C

TR

on Decem

ber 13, 2017ajcn.nutrition.org

Dow

nloaded from

61356.DCSupplemental.html http://ajcn.nutrition.org/content/suppl/2017/10/25/ajcn.117.1Supplemental Material can be found at:91

mailto:[email protected]

https://doi.org/10.3945/ajcn.117.161356

https://doi.org/10.3945/ajcn.117.161356

http://ajcn.nutrition.org/

http://ajcn.nutrition.org/content/suppl/2017/10/25/ajcn.117.161356.DCSupplemental.html

http://ajcn.nutrition.org/content/suppl/2017/10/25/ajcn.117.161356.DCSupplemental.html

implement the WHO 2010 recommendations to reduce the ex-posure to, and the power of, the marketing of foods and beverageshigh in SFAs, trans fatty acids, free sugars, or salt to childrenand adolescents (11, 12). Another recommended action is todevelop nutrient profile (NP) models, which are based on ob-jective, transparent, and reproducible nutritional criteria, to de-termine whether a food product is eligible or not eligible to bemarketed to children (11). A number of countries, includingChile, Denmark, Ireland, Mexico, New Zealand, Norway,Singapore, South Korea, and the United Kingdom have alreadytaken steps in that direction and use NP models to regulatedifferent forms of marketing to children (13, 14). Other coun-tries, such as Finland and Sweden, instead have implemented atotal ban on the marketing of food products to children, there-fore not requiring the adoption of an NP model (13, 14).

In October 2016, Health Canada committed to introducingrestrictions on the marketing of foods and beverages to childrenas part of its Healthy Eating Strategy (15). In the event that thecomplete ban advocated for by various stakeholders in thecountry is not implemented (16–18), an NP model will need tounderpin the proposed restrictions. The main objective of thepresent study was therefore to compare, in the Canadian context,the degree of strictness and agreement between NP models de-veloped by authoritative organizations for application in re-stricting marketing to children. A secondary objective was toperform these comparisons on the subset of food products cur-rently carrying on-package marketing to children. Based onprevious studies that compared classifications made by NPmodels meant for a variety of purposes (9, 10, 19–23), we hy-pothesized that the degree of strictness and agreement wouldvary substantially between the studied models.

METHODS

Study design

This was a cross-sectional analysis of the Canadian pre-packaged food supply with the use of the University of TorontoFood Label Information Program (FLIP) 2013 database, de-scribed in detail elsewhere (24). Briefly, FLIP 2013 containsnutritional information on 15,342 unique food products with aNutrition Facts table (NFt) from the 4 largest grocery chains inCanada (Loblaws, Metro, Sobeys, and Safeway), representing75.4% of the grocery retail market share (25). Data were collectedbetween May and September 2013 by systematically scanninggrocery store shelves with the use of a smartphone application.The information collected includes product information(e.g., company, brand, Universal Product Code), container size,price, NFt information, ingredient list, package marketing(e.g., nutrient content claims, front-of-package labeling, andmarketing to children), and photos of all sides of the packages.Only 1 package size/food was captured, but all flavors and va-rieties were collected. Nutritional information was recordedfor products in their “as sold” form and, if necessary(e.g., condensed soup), values for the “as consumed” form werecalculated according to package instructions with the use ofESHA Food Processor software (version 10.13.1; ESHA Re-search) and food composition data from the Canadian NutrientFile, version 2010b (26).

Foods in FLIP 2013 were classified into 22 distinct foodcategories and 153 subcategories as defined in Schedule M of theFood and Drug Regulations (version in force between 15 March2012 and 13 December 2016) (27). Specific types of products notclassified in any Schedule M category were excluded from thepresent analyses as follows: meal replacements (n = 55), instantor dry yeast (n = 4), and a natural health product (n = 1). A totalof 55 products were further excluded because of errors in nu-trient declarations in the NFt, as determined by Atwater calcu-lations that were .20% from the declared caloric values. Thus,15,227 products were included in the analyses.

Selected NP systems

Table 1 summarizes the key characteristics of each NP modelselected for the present study, consisting of 2 internationalsystems recently developed by regional offices of the WHO[Europe (28) and the Americas (29)] and an NP system from theAustralia and New Zealand governments (30). These NP modelsbuilt by authoritative sources were specifically retained for theirpotential wide applicability, meaning that they have been de-veloped or tested for use in several countries. Further details oneach model and their application to the FLIP 2013 databasefollow.

Food Standards Australia New Zealand-Nutrient ProfilingScoring Criterion

Although primarily designed to assess the eligibility of a foodproduct to carry a health claim, the Food Standards Australia NewZealand-Nutrient Profiling Scoring Criterion (FSANZ-NPSC) (30)was retained because it represents a modified version of thewell-established Ofcom model used for marketing restrictionsin the United Kingdom (31). The potential effectiveness ofthe FSANZ-NPSC as an NP model to restrict marketing tochildren has also previously been investigated (10, 32). Ad-ditionally, the original Ofcom model is currently undergoingrevision (33).

Briefly, foods were first classified into 1 of 3 possible cate-gories as follows: 1) beverages, 2) any food item not in category1 or 3, and 3) cheese with a high calcium content (.320 mgCa/100 g) and fats (e.g., oil and butter). The third categoryrepresents the main difference between the FSANZ-NPSCand the Ofcom model, which includes only 2 categories.That third category takes into account the higher fat and so-dium content that can be found in fats and cheese products ascompared with other food items that fall into category 2. Asummary score was then calculated for each food productbased on points for both nutrients to limit (energy, saturatedfat, total sugars, and sodium) and nutrients or food compo-nents to encourage [protein, fiber, and percent composition offruits, vegetables, nuts, and legumes (FVNLs) in a product].Points for each nutrient to limit and each nutrient to en-courage were assigned based on nutrient values per 100 g/mL.Because of the absence of quantitative ingredient declarationsin Canada, a method was established by our group, based onthe presence and position of FVNL ingredients within theingredient list, to estimate FVNL points for each food prod-uct. The method is detailed elsewhere (34). Predefined cutoffscores, which vary depending on the FSANZ-NPSC category,were used to classify food products either as eligible or not

1472 LABONTE ET AL.

at UN

IVE

RS

ITY

OF

TO

RO

NT

O G

ER

ST

EIN

SC

IEN

CE

INF

OR

MA

TIO

N C

TR

on Decem


Dow

nloaded from

92


eligible to be marketed to children. The version of theFSANZ-NPSC in force in Australia and New Zealand before1 March 2016 was used in the present analyses (30).

Pan American Health Organization NP model

The Pan American Health Organization (PAHO), representingthe WHO Regional Office for the Americas, published an NPmodel in 2016 designed for applications in a wide range ofnutrition policies, including, among others, restrictions on themarketing of unhealthy food and beverages to children, theregulation of school food environments, the establishment offront-of-package warning labels, and the establishment of tax-ation policies (29).

The application of the PAHO NP model first required fooditems to be classified into 1 of 5 possible categories, based ontheir extent of processing: 1) ultra-processed products, 2) pro-cessed products, 3) unprocessed or minimally processed prod-ucts, 4) culinary ingredients, or 5) freshly prepared products. AsFLIP is a database of packaged foods, no product in FLIP 2013was a freshly prepared product. Processed or ultra-processedproducts were thereafter classified as containing “excessive” or“not excessive” amounts of critical nutrients (sodium, freesugars, other sweeteners, total fat, saturated fat, and trans fat),based on predetermined thresholds in the model. Other sweet-eners were specifically evaluated based on their presence (yesor no) in the ingredient list and sodium was evaluated on a

TABLE 1

Summary of the 4 governmental or intergovernmental nutrient profile models evaluated1

Model and

application(s)

Food

categories, n

Nutrients

n Energy Total fat

Saturated

fat trans Fat

Total

sugars

Free/added

sugars Sweeteners Sodium/salt Protein Fiber FVNL

FSANZ-NPSC

Nutrition content

and health claims233 7 U U U U U U U

PAHO

Marketing of foods

to children

54 7 U5 U U U U (free) U U

School food environments

FOP labeling

Taxation policies

Agricultural subsidies

Food provision guidelines

for social programs

Modified-PAHO

Same as PAHO 16 7 U5 U U U U (free) U U

EURO

Marketing of foods

to children

177 88 U U U U U U (added)9 U U

1 EURO, WHO Regional Office for Europe; FOP, front-of-package; FSANZ-NPSC, Food Standards Australia New Zealand-Nutrient Profiling Scoring

Criterion; FVNL, fruits, vegetables, nuts, and legumes; Modified-PAHO, modified version of Pan American Health Organization model; PAHO, Pan American

Health Organization.2 The FSANZ-NPSC was retained because it consists of an updated version of the Ofcom model used for marketing restrictions in the United Kingdom.3 The 3 food categories are as follows: 1) beverages, 2) any food item not in category 1 or 3, and 3) cheese with a calcium content.320 mg/100 g, edible

oil spreads, margarine, and butter.4 The 5 food categories are as follows: 1) processed products, 2) ultra-processed products, 3) unprocessed or minimally processed products, 4) culinary

ingredients, and 5) freshly prepared dishes. The same nutrient criteria are applied only to processed and ultra-processed products. Nutrient criteria are not

applied to unprocessed or minimally processed products, culinary ingredients, and freshly prepared dishes; that is, the PAHO model considers that these

categories are always eligible to be marketed to children.5 Total energy provided by the food is not a criterion; however, the criteria for the other nutrients are presented on a per-total-energy basis (e.g., the

threshold to indicate an excess of total fat is $30% of total energy).6 All foods are evaluated with the use of the nutrient profile criteria, irrespective of the extent of food processing.7 There are 17 food categories, but the beverages food category contains 4 subcategories. These categories are as follows: 1) chocolate and sugar

confectionery, energy bars, and sweet toppings and desserts; 2) cakes, sweet biscuits, pastries, other sweet bakery wares, and dry mixes for making such; 3)

savory snacks; 4) beverages: a) juices, b) milk drinks, c) energy drinks, and d) other beverages; 5) edible ices; 6) breakfast cereals; 7) yogurts, sour milk,

cream, and other similar foods; 8) cheese; 9) ready-made and convenience foods and composite dishes; 10) butter and other fats and oils; 11) bread, bread

products, and crisp breads; 12) fresh or dried pasta, rice, and grains; 13) fresh and frozen meat, poultry, fish, and similar; 14) processed meat, poultry, fish, and

similar; 15) fresh and frozen fruit, vegetables, and legumes; 16) processed fruit, vegetables, and legumes; and 17) sauces, dips, and dressings.8 The number and types of nutrients or food components considered varies depending on the category, except for trans fat. Indeed, according to the WHO,

“marketing is prohibited if the product contains .1 g per 100 g total fat in the form of industrially-produced trans fatty acids,” irrespective of the food

category. Because the exact contribution of ruminant compared with industrially produced trans fat is not known for products from the Food Label Information

Program, this criterion was evaluated based on the presence of hydrogenated oils or partially hydrogenated oils in the ingredient list of a product and the

amount of total trans fat per 100 g total fat; refer to the Methods section for further details.9 According to the WHO, added sugars are used as a criterion in the EURO model because available data in food composition tables refer to added sugars.

In the current analyses, data on free sugars were used instead.

NUTRIENT PROFILING FOR MARKETING RESTRICTIONS 1473

at UN

IVE

RS

ITY

OF

TO

RO

NT

O G

ER

ST

EIN

SC

IEN

CE

INF

OR

MA

TIO

N C

TR

on Decem


Dow

nloaded from

93


per-kilocalorie basis, whereas all other nutrients were evaluatedas a percentage of energy. The free sugar content of products inFLIP 2013 was calculated with the use of the University ofToronto’s free sugar algorithm, which is based on the WHOdefinitions, as described by Bernstein et al. (24). Food productsthat exceeded $1 of the predetermined thresholds for criticalnutrients were considered not eligible to be marketed to chil-dren, whereas food products that did not exceed any thresholdwere considered eligible. Unprocessed or minimally processedproducts and culinary ingredients were not subject to the ap-plication of the thresholds and therefore were all consideredeligible in the present study. The PAHO model specifies thatthese types of products usually form part of a healthy diet or, inthe case of culinary ingredients (e.g., olive oil), are used inconjunction with unprocessed or minimally processed foods forproducing freshly prepared dishes.

Modified-PAHO NP model

It was decided to test a modified version of the PAHO NPmodel (Modified-PAHO), which took into account 2 modifica-tions. First, an across-the-board approach was used, in which allfoods in FLIP 2013 (i.e., not only processed or ultra-processedproducts, but also the minimally processed and unprocessedfoods) were evaluated against the thresholds and classified asexcessive or not in critical nutrients. This decision was made tosimplify the application of the PAHONPmodel by not taking intoaccount the extent of food processing, because the delimitationbetween processed or ultra-processed products as compared withunprocessed products, minimally processed products, or culinaryingredients was ambiguous for certain food items. An across-the-board approach therefore eliminated any possible subjectivity inthe initial categorization of foods.

Second, an adjustment to the PAHO sodium criterion wasapplied to beverages with a zero- or low-calorie content. Somebeverages with a very low sodium content (e.g., 5 mg Na/serving),but also a very low energy content (e.g., 0 calories), were initiallyclassified as being in excess for sodium (i.e., sodium was wellabove the threshold of$1 mg Na/kcal), however, those beverageswere clearly not a significant source of that nutrient. Based on theFood and Drug Regulations (35), the “low in sodium” nutrientcontent claim cutoff of #140 mg Na per reference amount andper serving of stated size was used as the threshold in these cases[beverages with both a sodium content #140 mg/serving andan energy content #140 kcal/serving were considered to have,1 mg Na/kcal (i.e., “not in excess”)]. A total of 134 beverageswere affected by this change. The “low in sodium” cutoff waschosen considering that most of the non–calorie-reduced ver-sions of the beverages had an energy content close to or ,140kcal/serving, therefore allowing for a 1:1 ratio. Also, beverageswith an energy content #140 kcal, but a sodium content.140 mg/serving were considered as having a ratio .1 (i.e., inexcess).

WHO Regional Office for Europe NP model

The WHO Regional Office for Europe (EURO) model, in-troduced in 2015, was specifically designed for restricting themarketing of unhealthy foods and beverages to children (28). Itwas built based primarily on the government-developed andgovernment-endorsed models from Norway and Denmark.

Foods from FLIP 2013 were first classified into 1 of 17 possiblecategories, among which the beverages category was furtherdivided into 4 subcategories (juices, milk drinks, energy drinks,and other beverages). As specified by the model, foods classifiedin 7 of the food categories had no nutritional criteria and wereautomatically considered either not eligible to be marketed tochildren [categories 1, 2, 4a, 4c, and 5 (e.g., juices)] or alwayseligible [categories 13 and 15 (e.g., frozen fruits)]. Foods in othercategories were evaluated against predetermined thresholds per100 g/mL for the following nutrients or foods components, whichvaried depending on the category: total fat, saturated fat, totalsugar, added sugar, nonsugar sweetener, salt, and energy(e.g., category 6, breakfast cereals, included limits for total fat,total sugar, and salt). The source document for the model pro-vides additional details on the nutrients considered in eachcategory and on how their thresholds were determined (28).Where applicable, free sugars were taken into account instead ofadded sugars because their amounts could be calculated for foodsin FLIP 2013 (see the PAHO NP model), and they are the type ofsugars specifically considered as part of the current WHOguidelines on sugars (36). Additionally, the model specifies that“marketing is prohibited if the product contains .1 g per 100 gtotal fat in the form of industrially-produced trans fatty acids”(28), irrespective of the food category. A total of 55 productsfrom different categories, which were initially classified as eli-gible to be marketed to children based on the criteria for theabove nutrients or food components, were reclassified as noteligible because hydrogenated or partially hydrogenated oilswere present in their ingredient list, and their total trans fattyacids content was .1 g/100 g of total fat. The total trans fattyacid content was used because the exact proportion of in-dustrially produced trans fatty acids in a product could not bedetermined. Thus, the use of total trans fatty acids represented amore conservative approach.

Considerations for all selected models

Under each NP model, food products from FLIP 2013 werefirst classified into their appropriate category independently by 2authors (M-EL and TP), and any discrepancy was resolved byconsensus. The classification of FLIP products into eachmodel’s categories was completed with the use of a combinationof information from Schedule M categories and subcate-gories (described above), sugar-focused categories detailed inBernstein et al. (24), and the ingredient list. For consistency, andbecause the FSANZ-NPSC and EURO models specify that foodproducts should be evaluated with the use of nutritional com-position data in the “as consumed” form (if necessary), this typeof data was used for all models. Nevertheless, the “as sold” datacorrespond to the “as consumed” data for $92.0% of products(n = 14,115/15,342) in the database.

On-package marketing to children

For the purpose of the subanalyses, the presence of on-packagemarketing to children was determined with the use of the ele-ments previously established and tested in our group (37), basedon information from Colby et al. (38) and Elliot (39) as follows: 1)children’s product lines, 2) child-focused lettering or graphics, 3)allusions to fun or play, 4) unconventional flavors, colors, orshapes, 5) toys, coupons, prizes, or contests, 6) games, and 7)

1474 LABONTE ET AL.

at UN

IVE

RS

ITY

OF

TO

RO

NT

O G

ER

ST

EIN

SC

IEN

CE

INF

OR

MA

TIO

N C

TR

on Decem


Dow

nloaded from

94


characters appealing to children. A product had to meet $1 ofthe previous criteria to be considered as marketed to children.The classification of all products in FLIP 2013 as having on-package marketing to children or not was performed by one ofthe authors (CM) and verified by a team member not involved inthe present project. Any uncertainty was resolved by consensuswith the other authors.

Statistical analyses

The degree of strictness was determined by the number andproportion (percentage with accompanying 95% CIs) of foodproducts considered eligible to be marketed to children in FLIP2013 and was reported overall and by Schedule M category foreach of the selected NP models. Agreement between the variousmodels was determined by cross-classification analysis(i.e., number and proportion of food products classified similarlyor differently between any 2 models). Cohen’s k statistic wasalso used, and agreement was interpreted as follows: slight,

0.01–0.20; fair, 0.21–0.40; moderate, 0.41–0.60; substantial,0.61–0.80; and almost perfect, 0.81–0.99 (40). The number andproportion of foods classified as eligible by all models or noteligible by any model was also determined. Subanalyses con-sisted of repeating the above analyses with the use of only thesubset of foods with on-package marketing to children. Allstatistical analyses were carried out in SAS (version 9.4; SASInstitute Inc.).

RESULTS

The degree of strictness, as shown by the proportion of Ca-nadian prepackaged foods classified as eligible to be marketed tochildren, varied considerably between the different NP models(Table 2). The Modified-PAHO and PAHO models were thestrictest overall by allowing only 9.8% (95% CI: 9.4%, 10.3%)and 15.8% (95% CI: 15.3%, 16.4%) of foods, respectively,followed by the EURO model [29.8% (95% CI: 29.0%, 30.5%)].In contrast, almost half of the Canadian prepackaged foodproducts [49.0% (95% CI: 48.2%, 49.8%)] were considered as

TABLE 2

Number and proportion (%) of Canadian prepackaged foods that would be eligible to be marketed to children according to 4 nutrient profile models, overall

and by food category (n = 15,227)1

Schedule M category

number and description2Foods

analyzed, n

Foods eligible to be

marketed to children

FSANZ-NPSC PAHO Modified-PAHO EURO

n % (95% CI) n % (95% CI) n % (95% CI) n % (95% CI)

All 15,227 74553 49.0 (48.2, 49.8) 24133 15.8 (15.3, 16.4) 14973 9.8 (9.4, 10.3) 45343 29.8 (29.0, 30.5)

1. Bakery products 2084 5753 27.6 (25.7, 29.5) 693 3.3 (2.5, 4.1) 693 3.3 (2.5, 4.1) 307 14.7 (13.2, 16.3)

2. Beverages 482 264 54.8 (50.3, 59.2) 46 9.5 (6.9, 12.2) 593 12.2 (9.3, 15.2) 553 11.4 (8.6, 14.3)

3. Cereals, other grain products 988 8014 81.1 (78.6, 83.5) 6753 68.3 (65.4, 71.2) 6625 67.0 (64.1, 69.9) 707 71.6 (68.7, 74.4)

4. Dairy products and substitutes 1240 6763 54.5 (51.7, 57.3) 105 8.5 (6.9, 10.0) 24 1.9 (1.2, 2.7) 265 21.4 (19.1, 23.7)

5. Desserts 827 272 32.9 (29.7, 36.1) 1 0.1 (0, 0.4) 1 0.1 (0, 0.4) 0 0 (0, 0)

6. Dessert toppings, fillings 116 134 11.2 (5.4, 17.0) 2 1.7 (0, 4.1) 2 1.7 (0, 4.1) 0 0 (0, 0)

7. Eggs and substitutes 56 53 94.6 (88.6, 100) 46 82.1 (71.8, 92.5) 0 0 (0, 0) 56 100 (100, 100)

8. Fats, oils 535 191 35.7 (31.6, 39.8) 148 27.7 (23.9, 31.5) 2 0.4 (0, 0.9) 153 28.6 (24.8, 32.4)

9. Marine, fresh water animals 440 340 77.3 (73.3, 81.2) 65 14.8 (11.4, 18.1) 17 3.9 (2.1, 5.7) 374 85.0 (81.7, 88.3)

10. Fruit, fruit juices 1089 7733 71.0 (68.3, 73.7) 4563 41.9 (38.9, 44.8) 2083 19.1 (16.8, 21.4) 106 9.7 (8.0, 11.5)

11. Legumes 180 179 99.4 (98.3, 100) 99 55.0 (47.7, 62.3) 93 51.7 (44.3, 59.0) 153 85.0 (79.7, 90.3)

12. Meat, poultry, their

products, substitutes

895 249 27.8 (24.9, 30.8) 12 1.3 (0.6, 2.1) 1 0.1 (0, 0.3) 378 42.2 (39.0, 45.5)

13. Miscellaneous 446 1003 22.4 (18.5, 26.3) 60 13.5 (10.3, 16.6) 43 9.6 (6.9, 12.4) 35 7.8 (5.3, 10.4)

14. Combination dishes 1357 9814 72.3 (69.9, 74.7) 27 2.0 (1.2, 2.7) 27 2.0 (1.2, 2.7) 708 52.2 (49.5, 54.8)

15. Nuts, seeds 220 167 75.9 (70.2, 81.6) 165 75.0 (69.2, 80.8) 1 0.5 (0, 1.4) 118 53.6 (47.0, 60.3)

16. Potatoes, sweet potatoes,

yams

140 134 95.7 (92.3, 99.1) 18 12.9 (7.2, 18.5) 18 12.9 (7.2, 18.5) 55 39.3 (31.1, 47.5)

17. Salads 70 57 81.4 (72.1, 90.8) 0 0 (0, 0) 0 0 (0, 0) 30 42.9 (31.0, 54.7)

18. Sauces, dips, gravies,

condiments

1229 3853 31.3 (28.7, 33.9) 52 4.2 (3.1, 5.4) 493 4.0 (2.9, 5.1) 144 11.7 (9.9, 13.5)

19. Snacks 794 224 28.2 (25.1, 31.3) 32 4.0 (2.7, 5.4) 30 3.8 (2.4, 5.1) 56 7.1 (5.3, 8.8)

20. Soups 456 408 89.5 (86.6, 92.3) 6 1.3 (0.3, 2.4) 6 1.3 (0.3, 2.4) 425 93.2 (90.9, 95.5)

21. Sugars, sweets 749 275 3.6 (2.3, 4.9) 655 8.7 (6.7, 10.7) 05 0 (0, 0) 4 0.5 (0, 1.1)

22. Vegetables 834 5864 70.3 (67.2, 73.4) 2643 31.7 (28.5, 34.8) 1853 22.2 (19.4, 25.0) 405 48.6 (45.2, 52.0)

1 EURO, WHO Regional Office for Europe; FSANZ-NPSC, Food Standards Australia New Zealand-Nutrient Profiling Scoring Criterion;

Modified-PAHO, modified version of Pan American Health Organization model; PAHO, Pan American Health Organization.2 Schedule M categories are defined in the Canadian Food and Drug Regulations (27). Detailed descriptions for each category and subcategory can also be

found in Supplemental Table 1.3Missing data for ,0.5% of food items.4Missing data for 0.5% to ,1.0% of food items.5Missing data for 1.0% to ,1.5% of food items.


at UN

IVE

RS

ITY

OF

TO

RO

NT

O G

ER

ST

EIN

SC

IEN

CE

INF

OR

MA

TIO

N C

TR

on Decem


Dow

nloaded from

95


eligible to be marketed to children according to the FSANZ-NPSCmodel. Differences in the overall strictness of the various NPmodels were also observed quite consistently across different foodcategories, with the Modified-PAHO model being the strictestmodel and the FSANZ-NPSC being the most permissive model inthe majority of categories (Table 2).

The Modified-PAHO model eliminated large proportions ofsome core foods that contained high amounts of a negativenutrient. For example, only #0.5% of nuts and seeds and eggsand egg substitutes were allowed by the Modified-PAHO model,whereas .53% of foods in those categories were allowed underthe other models (Table 2). Products in both food categorieswere essentially eliminated because of their high content in totalfat and saturated fat (Supplemental Figure 1).

In contrast, substantial proportions of food products notconsistent with dietary guidelines were classified as eligible formarketing to children under the FSANZ-NPSC model, whereasthose same products were permitted in very low proportions(#8%) or not permitted at all under the other models(i.e., carbonated and noncarbonated beverages; most types offrozen desserts; custard, gelatin, and pudding; hors d’oeuvres;sauces for dipping; legume- or dairy-based dips; minor mainentree sauces, such as gravy; and snacks, such as chips andpretzels; Supplemental Table 1).

Cross-classification analyses showed that about half of thefoods identified as not eligible under the FSANZ-NPSC modelwere also not eligible under the other models, whereas thisproportion was .65% for the comparison between the EUROmodel and each of the PAHO models and at 84% between theoriginal PAHO model and its modified version (Table 3). Theseresults are consistent with the observed degrees of agreement

between the NP models as assessed by the k statistic. Indeed,agreement was considered slight to fair between each of thePAHO models and the FSANZ-NPSC model (Modified-PAHO:k = 0.19; PAHO: k = 0.27), whereas it was substantial (k = 0.73)between the PAHO and Modified-PAHO models.

Another way to evaluate agreement between models was bydetermining the number and proportion of foods that would beclassified as eligible by all 4 models or not eligible by any model(Table 4). Overall, 8.1% of foods would be eligible based on allmodels. The top 5 types of such foods, based on Schedule Msubcategories, were as follows: pastas without sauce (n = 414);vegetables without sauce (n = 165); grains, such as rice andbarley (n = 107); fresh, canned, or frozen fruit (n = 90); andbeans, peas, and lentils (n = 89) (data not shown). Additionally,47.4% of foods would not be allowed by any model. The pro-portion of noneligible foods varied considerably between foodcategories, with none of the food items in eggs and egg sub-stitutes not allowed by any model, and $86.5% of products insugars and sweets and dessert toppings and fillings not allowedby any model. Supplemental Table 1 also shows subcategories inwhich 100% of products would not be allowed by any model(e.g., brownies; toaster pastries; and candies, including choco-late bars).

Tables 5 and 6 present the results of subanalyses based onlyon food products that had on-package marketing tochildren, representing 4.9% (n = 747/15,227) of all productsanalyzed. The Modified-PAHO model was still the strictest byallowing only 1.9% (0.9%, 2.8%) of products that were targetingchildren, whereas the most permissive model remained theFSANZ-NPSC, with 24.2% (95% CI: 21.2%, 27.3%) of suchproducts targeting children classified as being eligible to be

TABLE 3

Agreement between classifications made by 4 nutrient profile models applied to Canadian prepackaged foods (n = 15,227)1

Foods eligible or not eligible to be marketed to children, %

PAHO Modified-PAHO EURO

Eligible

(n = 2413)

Not eligible

(n = 12,802)

Eligible

(n = 1497)

Not eligible

(n = 13,705)

Eligible

(n = 4534)

Not eligible

(n = 10,692)

FSANZ-NPSC2

Eligible (n = 7455) 14.5 34.5 9.5 39.5 27.4 21.6

Not eligible (n = 7728) 1.3 49.5 0.4 50.4 2.3 48.5

k (95% CI)3 0.27 (0.26, 0.29) 0.19 (0.18, 0.20) 0.52 (0.51, 0.53)

PAHO4

Eligible 9.7 6.1 11.5 4.4

Not eligible 0.1 83.9 18.3 65.8

k (95% CI)3 0.73 (0.71, 0.74) 0.37 (0.36, 0.39)

Modified-PAHO5

Eligible 8.4 1.4

Not eligible 21.3 68.7

k (95% CI)3 0.33 (0.31, 0.34)


Modified-PAHO, modified version of Pan American Health Organization model; PAHO, Pan American Health Organization.2 There were missing data for 0.3% of the food items for each of the comparisons between the FSANZ-NPSC model and the PAHO, Modified-PAHO, and

EURO models.3Missing data were excluded from the calculation of the simple k statistic. Agreement between the models was assessed as follows: slight, 0.01–0.20;

fair, 0.21–0.40; moderate, 0.41–0.60; substantial, 0.61–0.80; and almost perfect, 0.81–0.99 (40).4 There were missing data for 0.2% of the food items for the comparison between the PAHO model and the Modified-PAHO model and missing data for

0.1% of the food items for the comparison between the PAHO model and the EURO model.5 There were missing data for 0.2% of the food items for the comparison between the Modified-PAHO model and the EURO model.

1476 LABONTE ET AL.

at UN

IVE

RS

ITY

OF

TO

RO

NT

O G

ER

ST

EIN

SC

IEN

CE

INF

OR

MA

TIO

N C

TR

on Decem


Dow

nloaded from

96


marketed to them (Table 5). The overall proportion of foods thatwould be eligible for marketing to children under each NPmodel in the present subanalyses represented, at most, half ofthe one observed in the main analyses when all foods from FLIP2013 were analyzed [e.g., 24.2% (Table 5) compared with49.0% (Table 2) of foods considered eligible for marketing,respectively, under the FSANZ-NPSC]. Agreement between thevarious models as assessed by the k statistic was also lower thanthat observed in the main analyses (Table 6). Cross-classificationanalyses showed that .73% of foods with on-package market-ing to children not permitted by the FSANZ-NPSC model werealso not permitted by the PAHO, Modified-PAHO, and EUROmodels, whereas this proportion was .90% for all other pos-sible comparisons. In the categories of dessert toppings andfillings, miscellaneous, and nuts and seeds, none of the productstargeting children were allowed by any model examined (Table5).

DISCUSSION

This study showed that the proportion and types of foods andbeverages in the Canadian prepackaged food supply that wouldbe permitted to be marketed to children based on governmental orintergovernmental NP models that have been developed formarketing restrictions vary greatly (i.e., from less than one-sixth

to almost half of Canadian prepackaged foods) depending on theselected model. Subanalyses also showed that each of the modelswould be more stringent when evaluating only food items car-rying attributes that are appealing to children. This suggests thatfood items specifically targeting children essentially representproducts for which consumption should be limited to an occa-sional basis, and that the adoption of an NP model for marketingrestrictions could largely reduce children’s exposure to thosefoods.

Consistent with our results, studies in other countries haveshown that the selection of an NP model has a large impact on theproportion and types of either packaged food products or food-and drink-related media advertisements considered eligible formarketing to children (9, 10, 21–23). Combined with these otherstudies, our study highlights the importance of carefully exam-ining the underlying characteristics of NP models that could beadapted or developed for use as part of a specific public healthpolicy. Those characteristics include the types and number ofnutrients or food components considered, the definition of foodcategories, the selected reference amounts, and the establishedthresholds, among others (41, 42).

Similar to observations by Scarborough et al. (21) and NiMhurchu et al. (22), the various models evaluated were moreconsistent in identifying foods that would not be eligible formarketing to children than in identifying eligible foods. This

TABLE 4

Canadian prepackaged foods that would be eligible to be marketed to children by all 4 nutrient profile models and not

eligible by any model, overall and by food category1

Schedule M category

number and description2Foods

analyzed, n

Foods eligible by all

models, n (%)

Foods not eligible by

any model, n (%)

All 15,2273 1234 (8.1) 7220 (47.4)

1. Bakery products 20843 43 (2.1) 1490 (71.5)

2. Beverages 4823 35 (7.3) 217 (45.0)

3. Cereals, other grain products 9884 648 (65.6) 169 (17.1)

4. Dairy products and substitutes 12403 21 (1.7) 494 (39.8)

5. Desserts 827 0 (0.0) 555 (67.1)

6. Dessert toppings, fillings 1163 0 (0.0) 101 (87.1)

7. Eggs and substitutes 56 0 (0.0) 0 (0.0)

8. Fats, oils 535 2 (0.4) 327 (61.1)

9. Marine, fresh water animals 440 17 (3.9) 58 (13.2)

10. Fruit, fruit juices 10893 93 (8.5) 284 (26.1)

11. Legumes 180 91 (50.6) 1 (0.6)

12. Meat, poultry, their products, substitutes 895 1 (0.1) 504 (56.3)

13. Miscellaneous 4463 30 (6.7) 329 (73.8)

14. Combination dishes 13575 17 (1.3) 346 (25.5)

15. Nuts, seeds 220 1 (0.5) 49 (22.3)

16. Potatoes, sweet potatoes, yams 140 8 (5.7) 4 (2.9)

17. Salads 70 0 (0.0) 12 (17.1)

18. Sauces, dips, gravies, condiments 12293 31 (2.5) 817 (66.5)

19. Snacks 794 6 (0.8) 554 (69.8)

20. Soups 456 6 (1.3) 19 (4.2)

21. Sugars, sweets 7494 0 (0.0) 648 (86.5)

22. Vegetables 8345 184 (22.1) 242 (29.0)

1 The 4 nutrient profile models evaluated were the Food Standards Australia New Zealand-Nutrient Profiling Scoring

Criterion, the Pan American Health Organization nutrient profile model, a modified version of the Pan American Health

Organization model, and the WHO Regional Office for Europe nutrient profile model.2 Schedule M categories are defined in the Canadian Food and Drug Regulations (27).3Missing data for ,0.5% of food items.4Missing data for 1.0% to ,1.5% of food items.5Missing data for 0.5% to ,1.0% of food items.


at UN

IVE

RS

ITY

OF

TO

RO

NT

O G

ER

ST

EIN

SC

IEN

CE

INF

OR

MA

TIO

N C

TR

on Decem


Dow

nloaded from

97


could be expected considering that all the models primarily tookinto account nutrients to limit, such as sodium and sugars. Onlyone of the models, the FSANZ-NPSC, considered the contri-bution of nutrients or food components to encourage, and thismay, at least in part, explain the higher percentages of foodsallowed by the FSANZ-NPSC overall and in the majority of foodcategories as compared with the other models. A limited numberof studies that assessed New Zealand or European packagedfoods have similarly shown that the UK Ofcom model or itsderivatives (e.g., the FSANZ-NPSC and Health Star Ratingsystem) are 7–15% and 25% more permissive than the EUROand PAHO models, respectively (22, 29).

A permissive model, such as the FSANZ-NPSC model, alsoclassified a substantial number of items that were not consistentwith dietary guidelines as eligible for marketing to children,whereas those same products were permitted in very low pro-portions or not permitted at all under more restrictive models,such as the EURO and PAHO models. From the perspective ofprotecting vulnerable populations (3), opting for a mandatory andfairly stringent model would be the approach most consistentwith the intent of restricting the exposure to, and the power of, thepromotion of unhealthy foods to children in various mediachannels. Questionable or undesirable food items would be lesslikely to pass the criteria of the system. This is particularlyimportant considering that a complete ban of food and beveragemarketing to children would likely represent the most effective

policy to implement. For example, a total ban in place since 1980in the province of Quebec, Canada (43) has been linked to lowerobesity rates in children aged 6–11 y than in children in otherprovinces (17). However, such bans are not always feasible,depending on the legislation system in place.

To limit subjectivity in the application of an NP model to afood supply, clear, nonambiguous definitions for food categoriesneed to be established. Although we acknowledge that consid-ering the extent of food processing (NOVA system) is a novel andinteresting avenue given that it has been shown to predict dietquality among Canadians (44), the categorization of foodsaccording to this method in the original PAHO model resulted inmuch higher discrepancies between team members than thecategorization of foods in the other NP models. For example,PAHO’s definition of minimally processed foods includes“combinations of 2 or more unprocessed or minimally processedfoods” (29). Because fruit juices without added sugars fall intothis category, a food product, such as sliced pineapples with aningredient list that indicates “pineapple, pineapple juice,” wouldalso, by definition, fall in that same category. In contrast, itappears more accurate to consider this product as processedgiven the substantial amount of sugars added by the inclusion ofpineapple juice. The idea of exempting fruit juice without addedsugars from being assessed against the criteria is also concerningbecause juice itself is a source of free sugars (36) and thereforenot considered in line with many dietary guidelines. As

TABLE 5

Number and proportion (%) of Canadian prepackaged foods specifically targeting children that would be eligible for marketing according to 4 nutrient

profile models, overall and by food category (n = 747)1

Schedule M category

number and description2

Foods with on-package

marketing to children

analyzed, n

Foods eligible to be

marketed to children

FSANZ-NPSC PAHO Modified-PAHO EURO

n % (95% CI) n % (95% CI) n % (95% CI) n % (95% CI)

All 747 1813 24.2 (21.2, 27.3) 26 3.5 (2.2, 4.8) 14 1.9 (0.9, 2.8) 46 6.2 (4.4, 7.9)

1. Bakery products 173 4 2.3 (0.1, 4.6) 1 0.6 (0, 1.7) 1 0.6 (0, 1.7) 3 1.7 (0, 3.7)

2. Beverages 11 2 18.2 (0, 45.4) 0 0 (0, 0) 0 0 (0, 0) 0 0 (0, 0)

3. Cereals and other grain products 51 11 21.6 (9.9, 33.3) 1 2.0 (0, 5.9) 1 2.0 (0, 5.9) 2 3.9 (0, 9.4)

4. Dairy products and substitutes 74 38 51.4 (39.7, 63.0) 0 0 (0, 0) 0 0 (0, 0) 5 6.8 (0.9, 12.6)

5. Desserts 144 39 27.1 (19.7, 34.4) 0 0 (0, 0) 0 0 (0, 0) 0 0 (0, 0)

6. Dessert toppings and fillings 7 0 0 (0, 0) 0 0 (0, 0) 0 0 (0, 0) 0 0 (0, 0)

9. Marine and fresh-water animals 2 2 100 (100, 100) 0 0 (0, 0) 0 0 (0, 0) 2 100 (100, 100)

10. Fruit and fruit juices 58 33 56.9 (43.8, 70.0) 8 13.8 (4.6, 22.9) 5 8.6 (1.2, 16.1) 4 6.9 (0.2, 13.6)

12. Meat, poultry, their products,

and substitutes

4 4 100 (100, 100) 0 0 (0, 0) 0 0 (0, 0) 4 100 (100, 100)

13. Miscellaneous 14 0 0 (0, 0) 0 0 (0, 0) 0 0 (0, 0) 0 0 (0, 0)

14. Combination dishes 69 39 56.5 (44.5, 68.5) 0 0 (0, 0) 0 0 (0, 0) 24 34.8 (23.3, 46.3)

15. Nuts and seeds 9 0 0 (0, 0) 0 0 (0, 0) 0 0 (0, 0) 0 0 (0, 0)

16. Potatoes, sweet potatoes, and yams 4 4 100 (100, 100) 0 0 (0, 0) 0 0 (0, 0) 0 0 (0, 0)

19. Snacks 54 3 5.6 (0, 11.9) 7 13.0 (3.7, 22.2) 7 13.0 (3.7, 22.2) 0 0 (0, 0)

20. Soups 1 1 100 (100, 100) 0 0 (0, 0) 0 0 (0, 0) 1 100 (100, 100)

21. Sugars and sweets 72 14 1.4 (0, 4.2) 9 12.5 (4.7, 20.3) 0 0 (0, 0) 1 1.4 (0, 4.2)


Modified-PAHO, modified version of Pan American Health Organization model; PAHO, Pan American Health Organization.2 None of the foods in Schedule M categories 7 (egg and egg substitutes), 8 (fats and oils), 11 (legumes), 17 (salads), 18 (sauces, dips, gravies, and

condiments), and 22 (vegetables) were specifically marketed to children on their package. Schedule M categories are defined in the Canadian Food and Drug

Regulations (27).3Missing data for 0.1% of food items.4Missing data for 1% of food items.

1478 LABONTE ET AL.

at UN

IVE

RS

ITY

OF

TO

RO

NT

O G

ER

ST

EIN

SC

IEN

CE

INF

OR

MA

TIO

N C

TR

on Decem


Dow

nloaded from

98


described previously, such an ambiguous situation explains whywe also tested a modified, across-the-board version of the PAHOmodel that did not exempt any product from being evaluatedagainst the nutritional criteria. It was also observed that 86.5%of foods in FLIP 2013 (n = 13,166/15,227; data not shown) wereclassified as processed or ultra-processed, supporting the ideathat the use of a model applicable to the entire packaged foodsupply might be the best approach in the Canadian context. Still,the use of the Modified-PAHO model was not without concerns.Our results highlight that applying a model to the entire foodsupply may require clear exemptions to be established for foodsin line with dietary guidelines that naturally contain highamounts of a negative nutrient included in the model’s algorithm(e.g., nuts, which have a high total fat content even in the ab-sence of added fat). With prespecified exemptions, core foods,such as nuts, without any added ingredients would not be in-appropriately ruled out.

A number of limitations and strengths need to be pointed out.First, the present analyses did not weight products by marketshare and did not specifically consider the foods most commonlyconsumed by children, nor the foods most heavily marketed tochildren in various media channels, such as television or theInternet. However, our analyses did provide a comprehensiveevaluation of a large sample (.15,000 items) of foods that areavailable to children and their parents in Canadian grocerystores. Our analyses also provided specific data on foods thatcurrently carry on-package marketing to children, representing amarketing channel that has continued to remain outside currentindustry- or government-based regulatory frameworks, such asthe Canadian Children’s Food and Beverage Advertising Ini-tiative (45) or the Consumer Protection Act in Quebec (43).Second, we recognize that several other NP models meant formarketing restrictions exist worldwide (13, 14), but we decidednot to consider models from regions (e.g., Asia) that were

unlikely to be relevant in a North American context due todifferences in food supplies. Additionally, we decided not toconsider proposed models that are not currently in use [e.g., thenutrition standards of the Interagency Working Group on FoodMarketing to Children, United States (46)]. We also only optedfor models developed by authoritative bodies, which are morelikely to be used by other government bodies and to be trustedand supported by consumers than industry-based models. This isimportant given that industry-based models have been shown tobe less stringent than government-based models (23).

In conclusion, the present study showed wide variations in thedegree of strictness and agreement between NP models havingapplications in restricting the commercial marketing of foods andbeverages of low nutritional quality to children. This highlightsthe importance of carefully evaluating the characteristicsunderlying a model that is being developed or adapted for use in aspecific public health policy. Where total bans of the commercialmarketing of foods and beverages to children are notintroduced, a relatively stringent and mandatory NP model thatuses clearly defined categories and exemptions, and that isconsistent with other nutrition-related policies in the jurisdiction(e.g., front-of-package labeling system), should be considered.Such a model would more closely align with the public healthobjective of protecting vulnerable populations and would ensureconsistency between the country’s policies and national dietaryguidelines.

We thank Mavra Ahmed and Sheida Norsen, who assisted in preliminary

calculations of FVNL points that supported the determination of FSANZ-

NPSC scores used in the analyses of the present study. We also thank Laura

Vergeer, who assisted in evaluating the presence of on-package marketing

to children.

The authors’ responsibilities were as follows—M-EL, TP, and MRL:

designed the research; M-EL, JTB, and BF-A: calculated the FSANZ-NPSC

scores; CM: assessed the presence of marketing to children on food

TABLE 6

Agreement between classifications made by 4 nutrient profile models applied to Canadian prepackaged foods with on-package marketing to children

(n = 747)1

Foods eligible or not eligible to be marketed to children, %

PAHO Modified-PAHO EURO

Eligible (n = 26) Not eligible (n = 721) Eligible (n = 14) Not eligible (n = 733) Eligible (n = 46) Not eligible (n = 701)

FSANZ-NPSC2

Eligible (n = 181) 1.6 22.7 1.2 23.1 5.9 18.4

Not eligible (n = 565) 1.9 73.9 0.7 75.1 0.3 75.5

k (95% CI)3 0.06 (0.00, 0.11) 0.06 (0.01, 0.11) 0.32 (0.25, 0.40)

PAHO

Eligible 1.9 1.6 0.8 2.7

Not eligible 0.0 96.5 5.4 91.2

k (95% CI)3 0.69 (0.53, 0.86) 0.13 (0.01, 0.25)

Modified-PAHO

Eligible 0.8 1.1

Not eligible 5.4 92.8

k (95% CI)3 0.18 (0.04, 0.31)


Modified-PAHO, modified version of Pan American Health Organization model; PAHO, Pan American Health Organization.2 There were missing data for 0.1% of the food items for each of the comparisons between the FSANZ-NPSC model and the PAHO, Modified-PAHO, and

EURO models.3Missing data were excluded from the calculation of the simple k statistic. Agreement between the models was assessed as follows: slight, 0.01–0.20;

fair, 0.21–0.40; moderate, 0.41–0.60; substantial, 0.61–0.80; and almost perfect, 0.81–0.99 (40).


at UN

IVE

RS

ITY

OF

TO

RO

NT

O G

ER

ST

EIN

SC

IEN

CE

INF

OR

MA

TIO

N C

TR

on Decem


Dow

nloaded from

99


packages; M-EL and TP: conducted the research; TP: analyzed the data; M-EL:

assisted in data analysis and wrote the manuscript; MRL: had primary respon-

sibility for the final content; and all authors: critically reviewed and read and

approved the final manuscript. TP is a graduate student and is employed part

time by Intertek Scientific & Regulatory Consultancy. BF-A was a PepsiCo

employee (2009–2015). These companies were not involved in any way in the

present research. None of the remaining authors reported a conflict of interest

related to the study.

REFERENCES1. Rodd C, Sharma AK. Recent trends in the prevalence of overweight

and obesity among Canadian children. CMAJ 2016;188:E313–20.2. Janssen I. The public health burden of obesity in Canada. Can J Di-

abetes 2013;37:90–6.3. WHO. Consideration of the evidence on childhood obesity for the

Commission on Ending Childhood Obesity. Report of the Ad hocWorking Group on Science and Evidence for Ending ChildhoodObesity [Internet]. Geneva (Switzerland); WHO Press. 2016 [cited2017 Jan 30]. Available from: http://apps.who.int/iris/bitstream/10665/206549/1/9789241565332_eng.pdf?ua=1).

4. Boyland EJ, Nolan S, Kelly B, Tudur-Smith C, Jones A, Halford JC,Robinson E. Advertising as a cue to consume: a systematic review andmeta-analysis of the effects of acute exposure to unhealthy food andnonalcoholic beverage advertising on intake in children and adults. AmJ Clin Nutr 2016;103:519–33.

5. Boyland EJ, Whalen R. Food advertising to children and its effects ondiet: review of recent prevalence and impact data. Pediatr Diabetes2015;16:331–7.

6. Sadeghirad B, Duhaney T, Motaghipisheh S, Campbell NR,Johnston BC. Influence of unhealthy food and beverage marketing onchildren’s dietary intake and preference: a systematic review and meta-analysis of randomized trials. Obes Rev 2016;17:945–59.

7. Cairns G, Angus K, Hastings G, Caraher M. Systematic reviews of theevidence on the nature, extent and effects of food marketing to chil-dren. A retrospective summary. Appetite 2013;62:209–15.

8. Potvin Kent M, Dubois L, Wanless A. A nutritional comparison offoods and beverages marketed to children in two advertising policyenvironments. Obesity (Silver Spring) 2012;20:1829–37.

9. Rincon-Gallardo Pati~no S, Tolentino-Mayo L, Flores Monterrubio EA,Harris JL, Vandevijvere S, Rivera JA, Barquera S. Nutritional quality offoods and non-alcoholic beverages advertised on Mexican television ac-cording to three nutrient profile models. BMC Public Health 2016;16:733.

10. Watson WL, Johnston A, Hughes C, Chapman K. Determining the‘healthiness’ of foods marketed to children on television using theFood Standards Australia New Zealand nutrient profiling criteria. NutrDiet 2014;71:178–83.

11. WHO. Report of the Commission on Ending Childhood Obesity [In-ternet]. Geneva (Switzerland); WHO Press. 2016 [cited 2017 Feb 1].Available from: http://apps.who.int/iris/bitstream/10665/204176/1/9789241510066_eng.pdf.

12. WHO. Set of recommendations on the marketing of foods and non-alcoholic beverages to children. Resolution of the Sixty-third WorldHealth Assembly Adopted 21 May 2010. WHA63.14 Marketing of foodand non-alcoholic beverages to children [Internet]. Geneva (Switzerland);WHO Press. 2010 [cited 2017 Jan 30]. Available from: http://apps.who.int/iris/bitstream/10665/44416/1/9789241500210_eng.pdf.

13. Labonte ME, Poon T, Gladanac B, Ahmed M, Franco Arellano B,Rayner M, L’Abbe MR. Systematic review of nutrient profile modelsdeveloped for nutrition-related policies and regulations aimed at non-communicable disease prevention [Internet]. PROSPERO #CRD42015024750.2015 [cited 2017 Jan 24]. Available from: http://www.crd.york.ac.uk/PROS-PERO/display_record.asp?ID=CRD42015024750.

14. Labonte ME, Poon T, Gladanac B, Ahmed M, Franco Arellano B,Rayner M, L’Abbe MR. Systematic review of nutrient profile modelsdeveloped for nutrition-related policies and regulations aimed at non-communicable disease prevention. Appl Physiol Nutr Metab 2017;42:S23 (abstr).

15. Health Canada. Healthy eating strategy [Internet]. 2016 [cited 2017 Jan30]. Available from: https://www.canada.ca/content/dam/canada/health-canada/migration/publications/eating-nutrition/healthy-eating-strategy-canada-strategie-saine-alimentation/alt/pub-eng.pdf.

16. Senate of Canada. Child Health Protection Act. An act to amend theFood and Drugs Act (prohibiting food and beverage marketing directedat children). Bill S-228. 1st Sess., 42nd Parliament, 64-65 ElizabethII, 2015–2016 [Internet]. [cited 2017 Jan 30]. Available from: http://www.parl.gc.ca/content/hoc/Bills/421/Private/S-228/S-228_1/S-228_1.PDF.

17. Heart and Stroke Foundation of Canada. The kids are not alright. Howthe food and beverage industry is marketing our children and youth todeath. 2017 Report on the health of Canadians [Internet]. 2017 [cited2017 Feb 1]. Available from: http://www.heartandstroke.ca/what-we-do/media-centre/report-on-health.

18. Stop Marketing to Kids Coalition. The Ottawa principles [Internet].2016 [cited 2017 Mar 13]. Available from: http://stopmarketingtokids.ca/the-ottawa-principles/.

19. Azaıs-Braesco V, Goffi C, Labouze E. Nutrient profiling: comparisonand critical analysis of existing systems. Public Health Nutr 2006;9:613–22.

20. Clerfeuille E, Vieux F, Lluch A, Darmon N, Rolf-Pedersen N. Assessingthe construct validity of five nutrient profiling systems using diet modelingwith linear programming. Eur J Clin Nutr 2013;67:1003–5.

21. Scarborough P, Payne C, Agu CG, Kaur A, Mizdrak A, Rayner M,Halford JC, Boyland E. How important is the choice of the nutrientprofile model used to regulate broadcast advertising of foods to children?A comparison using a targeted data set. Eur J Clin Nutr 2013;67:815–20.

22. Ni Mhurchu C, Mackenzie T, Vandevijvere S. Protecting New Zealandchildren from exposure to the marketing of unhealthy foods anddrinks: a comparison of three nutrient profiling systems to classifyfoods. N Z Med J 2016;129:41–53.

23. Brinsden H, Lobstein T. Comparison of nutrient profiling schemes forrestricting the marketing of food and drink to children. Pediatr Obes2013;8:325–37.

24. Bernstein JT, Schermel A, Mills CM, L’Abbe MR. Total and free sugarcontent of Canadian prepackaged foods and beverages. Nutrients 2016;8.

25. Canadian Grocer. Executive report, Canadian grocery industry 2012–2013. Toronto (Canada): Rogers Publishing Limited; 2012.

26. Health Canada. Food and nutrition. Nutrient data. Canadian nutrient file,version 2015 [Internet]. 2015. [updated 2016 Jun 3; cited 2017 Feb 3].Available from: http://www.hc-sc.gc.ca/fn-an/nutrition/fiche-nutri-data/index-eng.php.

27. Government of Canada. Food and drug regulations (C.R.C., c. 870),sections B.01.001, B.01.002A and D.01.001 [Internet]. Ottawa (Canada):Minister of Justice. 2016 [cited 2016 Mar 16]. Available from: http://laws-lois.justice.gc.ca/eng/regulations/c.r.c.,_c._870/page-165.html#h-353.

28. WHO Regional Office for Europe. WHO Regional Office for Europenutrient profile model [Internet]. 2015 [cited 2016 Nov 1]. Availablefrom: http://www.euro.who.int/en/health-topics/disease-prevention/nutrition/publications/2015/who-regional-office-for-europe-nutrient-profile-model-2015.

29. Pan American Health Organization. Pan American Health Organizationnutrient profile model [Internet]. 2016 [cited 2016 Oct 24]. Avail-able from: http://www.paho.org/hq/index.php?option=com_content&-view=article&id=11662%3Apaho-nutrient-profile-model&catid=1370%3Amicronutrients&Itemid=41739&lang=fr.

30. Australian Government. ComLaw. Legislative instrument compilation.Australia New Zealand Food Standards Code - Standard 1.2.7 - nu-trition, health and related claims - F2015C00967 [Internet]. 2015 [cited2016 Jan 17]. Available from: https://www.comlaw.gov.au/Details/F2015C00967.

31. Department of Health (UK). Nutrient profiling technical guidance[Internet]. 2011 [cited 2016 Nov 15]. Available from: https://www.gov.uk/government/publications/the-nutrient-profiling-model.

32. Hebden L, King L, Kelly B, Chapman K, Innes-Hughes C,Gunatillaka N. Regulating the types of foods and beverages marketedto Australian children: how useful are food industry commitments?Nutr Diet 2010;67:258–66.

33. Government of UK, Public Health England. Review of the nutrientprofiling model [Internet]. 2016 [cited 2017 Mar 13]. Available from:https://www.gov.uk/government/collections/review-of-the-nutrient-profiling-model.

34. Bernstein JT, Franco Arellano B, Schermel A, Labonte ME,L’Abbe MR. Healthfulness and nutritional composition of Canadianprepackaged foods with and without sugar claims. Appl Physiol NutrMetab 2017. In press.

1480 LABONTE ET AL.

at UN

IVE

RS

ITY

OF

TO

RO

NT

O G

ER

ST

EIN

SC

IEN

CE

INF

OR

MA

TIO

N C

TR

on Decem


Dow

nloaded from

100












http://www.parl.gc.ca/content/hoc/Bills/421/Private/S-228/S-228_1/S-228_1.PDF



http://www.heartandstroke.ca/what-we-do/media-centre/report-on-health

http://www.heartandstroke.ca/what-we-do/media-centre/report-on-health

http://stopmarketingtokids.ca/the-ottawa-principles/

http://stopmarketingtokids.ca/the-ottawa-principles/

http://www.hc-sc.gc.ca/fn-an/nutrition/fiche-nutri-data/index-eng.php

http://www.hc-sc.gc.ca/fn-an/nutrition/fiche-nutri-data/index-eng.php




http://www.euro.who.int/en/health-topics/disease-prevention/nutrition/publications/2015/who-regional-office-for-europe-nutrient-profile-model-2015



http://www.paho.org/hq/index.php?option=com_content&view=article&id=11662%3Apaho-nutrient-profile-model&catid=1370%3Amicronutrients&Itemid=41739&lang=fr










35. Government of Canada, Canadian Food Inspection Agency. Specificnutrient content claim requirements. Sodium (salt) claims [Internet].2016 [cited 2017 Mar 13]. Available from: http://www.inspection.gc.ca/food/labelling/food-labelling-for-industry/nutrient-content/specific-claim-requirements/eng/1389907770176/1389907817577?chap=9.

36. WHO. Guideline: sugars intake for adults and children [Internet].Geneva (Switzerland): WHO. 2015 [cited 2017 Mar 13]. Available from:http://apps.who.int/iris/bitstream/10665/149782/1/9789241549028_eng.pdf?ua=1.

37. Murray C. Examining the nutritional content of prepackaged foods andbeverages marketed to children in Canada. Canada: University ofToronto; 2014.

38. Colby SE, Johnson L, Scheett A, Hoverson B. Nutrition marketing onfood labels. J Nutr Educ Behav 2010;42:92–8.

39. Elliott C. Marketing fun foods: a profile and analysis of supermarketfood messages targeted at children. Can Public Policy 2008;34:259–73.

40. Viera AJ, Garrett JM. Understanding interobserver agreement: thekappa statistic. Fam Med 2005;37:360–3.

41. Sacks G, Rayner M, Stockley L, Scarborough P, Snowdon W,Swinburn B. Applications of nutrient profiling: potential role in diet-related chronic disease prevention and the feasibility of a core nutrient-profiling system. Eur J Clin Nutr 2011;65:298–306.

42. Scarborough P, Rayner M, Stockley L. Developing nutrient profilemodels: a systematic approach. Public Health Nutr 2007;10:330–6.

43. Gouvernement du Quebec. P-40.1 - Consumer Protection Act [Inter-net]. 2016 [cited 2016 Oct 31]. Available from: http://legisquebec.gouv.qc.ca/en/ShowDoc/cs/P-40.1.

44. Moubarac JC, Batal M, Louzada ML, Martinez Steele E, Monteiro CA.Consumption of ultra-processed foods predicts diet quality in Canada.Appetite 2017;108:512–20.

45. Advertising Standards Canada. Canadian Children’s Food & BeverageAdvertising Initiative [Internet]. 2016 [cited 2017 May 16]. Availablefrom: http://www.adstandards.com/en/childrensinitiative/default.htm.

46. Federal Trade Commission, CDC, FDA, USDA. Interagency WorkingGroup on Food Marketed to Children tentative proposed nutritionstandards [Internet]. 2009 [cited 2016 Dec 2]. Available from: https://cspinet.org/sites/default/files/attachment/ftcnewstandards.pdf.


at UN

IVE

RS

ITY

OF

TO

RO

NT

O G

ER

ST

EIN

SC

IEN

CE

INF

OR

MA

TIO

N C

TR

on Decem


Dow

nloaded from

101

http://www.inspection.gc.ca/food/labelling/food-labelling-for-industry/nutrient-content/specific-claim-requirements/eng/1389907770176/1389907817577?chap=9





http://legisquebec.gouv.qc.ca/en/ShowDoc/cs/P-40.1

http://legisquebec.gouv.qc.ca/en/ShowDoc/cs/P-40.1

http://www.adstandards.com/en/childrensinitiative/default.htm

https://cspinet.org/sites/default/files/attachment/ftcnewstandards.pdf

https://cspinet.org/sites/default/files/attachment/ftcnewstandards.pdf


102

Appendix B

Ethics Review Application for Study on Proposed NP Model for Marketing Restrictions to Children We are currently conducting a study to examine the construct and

convergent/concurrent validity of the proposed NP model for use in Canada against

views of nutrition professionals. The Ethics Review Application was approved by the

Review Ethics Board (REB) at the University of Toronto. The pilot study was conducted

between November 3 to 10, 2017, and an amendment to the approval was submitted to

the REB on December 11, 2017: L’Abbé MR, Poon T, Labonté MÈ, Bernstein JT

(2017). Ethics Review Application Form: Validation of a nutrient profile model proposed

for restricting the marketing of unhealthy foods to children by nutrition professionals.


this research to date, as indicated in Table B-1. In addition, I plan to submit the results

of this study for publication in 2018.

Table B-1 Authors’ contributions to ethics review application for study on proposed NP model for marketing restrictions to children

Task Author

Designed research study Poon, L’Abbé Wrote research proposal Poon

Provided comments/revisions to draft proposals L’Abbé Provided comments/revisions to final proposal Labonté, Bernstein, L’Abbé Wrote REB application Poon

Provided comments/revisions to draft REB application

Labonté, Bernstein, L’Abbé

Generated master list of foods Bernstein Generated survey Poon

Conducted pilot survey testing and analyzed feedback

Poon

Wrote amendment to REB application Poon, Bernstein Provided comments/revisions to amendment to REB application

L’Abbé

Office of the Vice-President, Research and Innovation Human Research Ethics Program

UT HREP – Application Form for Faculty Researchers 1 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor

Version Date: December 8, 2017

Office Use Only Application Number:

ETHICS REVIEW APPLICATION FORM – FACULTY

(For use by University of Toronto Faculty Researchers only)

SECTION A – GENERAL INFORMATION 1. TITLE OF RESEARCH PROJECT

Validation of a nutrient profile model proposed for restricting the marketing of unhealthy foods to children by nutrition professionals 2. INVESTIGATOR INFORMATION Principal Investigator (must be a UofT faculty member with research privileges): Title (e.g., Dr., Ms., etc.): Dr.

Name: Mary R. L’Abbé

Department: Nutritional Sciences Mailing address: Phone: Institutional e-mail: Alternate Contact (e.g., Research Coordinator): Title: Ms. Name: Alyssa Schermel Phone: Institutional e-mail: Co-Investigators: Are co-investigators involved? Yes No Title: Ms. Name: Theresa Poon Department (or organization if not affiliated with U of T): Nutritional Sciences Mailing address: Phone: Institutional e-mail: Title: Dr. Name: Marie-Eve Labonté Department (or organization if not affiliated with U of T): Institute of Nutrition and Functional Foods (INAF), Laval University Mailing address:

Before you start, familiarize yourself with:

TCPS2 Application instructions

Office FAQs

103

http://www.pre.ethics.gc.ca/eng/policy-politique/initiatives/tcps2-eptc2/Default/

http://www.research.utoronto.ca/wp-content/uploads/documents/2015/02/Instructions-for-Ethics-Review-Application-Form-Version-March-2015.pdf

http://www.research.utoronto.ca/wp-content/uploads/documents/2015/02/FAQs-for-Site-March-20151.pdf

UT - Office of Research Ethics – Application Form for Faculty Researchers 2 of 30 12 Queen’s Park Crescent West – McMurrich Building, 2nd floor


Phone:

Institutional e-mail:

Title: Ms. Name: Jodi Bernstein Department (or organization if not affiliated with U of T): Nutritional Sciences Mailing address: Phone: Institutional e-mail: Please append additional pages with co-investigators’ names if necessary. 3. UNIVERSITY OF TORONTO RESEARCH ETHICS BOARD:

Social Sciences, Humanities and Education Health Sciences HIV/AIDS To determine which Research Ethics Board (REB) your application should be submitted, please consult: http://www.research.utoronto.ca/about/boards-and-committees/research-ethics-boards-reb/ 4. LOCATION(S) WHERE THE RESEARCH WILL BE CONDUCTED:

(a) If the research is to be conducted at a site requiring administrative approval/consent (e.g., in a school), please include all administrative consent letters. It is the responsibility of the researcher to determine what other means of approval are required, and to obtain approval prior to starting the project.

University of Toronto Hospital specify site(s) School board or community agency specify site(s) Community within the GTA specify site(s) International specify site(s) Other specify site(s) (b) For all off-campus research, whether in the local community or internationally, the researcher should consult with the Framework on Off-Campus Safety, Guidelines on Off-Campus Safety, and Guidelines on Safety in Field for institutional requirements. (c) The University of Toronto has an agreement with the Toronto Academic Health Sciences Network (TAHSN) hospitals regarding ethics review of hospital-based research where the University plays a peripheral role. Based on this agreement, certain hospital-based research may not require ethics review at the University of Toronto. If your research is based at a TAHSN hospital, please consult the following document to determine whether or not your research requires review at the University of Toronto. http://www.research.utoronto.ca/faculty-and-staff/research-ethics-and-protections/humans-in-research/ - “Administrative review” heading toward the bottom of the page. 5. OTHER RESEARCH ETHICS BOARD APPROVAL(S) (a) Does the research involve another institution or site? Yes No (b) Has any other REB approved this project? Yes No

If Yes, please provide a copy of the approval letter upon submission of this application. If No, will any other REB be asked for approval?

104

http://www.research.utoronto.ca/about/boards-and-committees/research-ethics-boards-reb/

http://www.cie.utoronto.ca/safety-abroad/Framework-on-Off-Campus-Safety.aspx

http://www.cie.utoronto.ca/safety-abroad/Off-Campus-Safety-Guidelines.aspx

http://www.ehs.utoronto.ca/Assets/ehs+Digital+Assets/Guidelines+on+Safety+in+Field+Research.pdf

http://www.ehs.utoronto.ca/Assets/ehs+Digital+Assets/Guidelines+on+Safety+in+Field+Research.pdf

http://www.research.utoronto.ca/faculty-and-staff/research-ethics-and-protections/humans-in-research/




Yes (please specify which REB) No 6. FUNDING OF THIS PROJECT

(a) Funding Status Source and Type Details Funded Agency: Canadian Institutes of Health

Research Fund #: (6 digits)

Agency: Burroughs Wellcome Fund #: (6 digits) Applied for funding

Agency: Submission date: Agency: Submission date:

Unfunded If unfunded, please explain why no funding is needed: 7. CONTRACTS AND AGREEMENTS (a) Is this research to be carried out as a contract or under a research agreement? Yes No If yes, is there a University of Toronto funding or non-funded agreement associated with the research? Yes No

If Yes, please append a copy of the agreement with of this application.

Is there any aspect of the contract that could put any member of the research team in a potential conflict of interest? Yes No If yes, please elaborate under #10. (b) Is this a Division 5, Health Canada regulated clinical trial that involves drugs, devices or natural health products? Yes No (if so, the application must be reviewed by the full board) 8. PROJECT START AND END DATES Estimated start date for the component of this project that involves human participants or data: September 2017 Estimated completion date of involvement of human participants or data for this project: September 2018 9. SCHOLARLY REVIEW:

(a) Please check one:

I. The research has undergone scholarly review by thesis committee, departmental review committee, peer review committee or some other equivalent (Specify review type – e.g., departmental research committee, supervisor, CIHR, SSHRC, OHTN, etc.): CIHR, Health Canada

II. The research will undergo scholarly review prior to funding (Specify review committee – e.g., departmental research committee, SSHRC, CIHR peer-review committee, etc.):

III. The research will not undergo scholarly review (Please note that all research greater than minimal risk requires scholarly review)

(b) If box I or II above was checked, please specify if:

The review was/will be specific to this application

105



The review was/will be part of a larger grant 10. CONFLICTS OF INTEREST (a) Will the researcher(s), members of the research team, and/or their partners or immediate family members: (i) Receive any personal benefits (e.g., financial benefit such as remuneration, intellectual property rights, rights of employment, consultancies, board membership, share ownership, stock options, etc.) as a result of or in connection with this study? Yes No (ii) If Yes, please provide further details and discuss how any real, potential or perceived conflicts of interest will be managed during the project. (Do not include conference and travel expense coverage, or other benefits which are considered standard for the conduct of research.) Not applicable (b) Describe any restrictions regarding access to or disclosure of information (during or at the end of the study) that have been placed on the investigator(s). These restrictions include controls placed by the sponsor, funding body, advisory or steering committee. Not applicable (c) Where relevant, please explain any pre-existing relationship between the researcher(s) and the researched (e.g., instructor-student; manager-employee; clinician-patient; minister-congregant). Please pay special attention to relationships in which there may be a power differential – actual or perceived. There is no expected pre-existing relationship between the participants and the research team members in contact with them. SECTION B – SUMMARY OF THE PROPOSED RESEARCH 11. RATIONALE Describe the purpose and scholarly rationale for the proposed project. State the hypotheses/research questions to be examined. The rationale for doing the study must be clear. Please include references in this section. Background: In Canada, the Child Health Protection Act (Bill S-228) was introduced in September 2016 by Senator Greene Raine to amend the current Food and Drugs Act to prohibit food and beverage marketing directed at children under 13 years of age1. Shortly after, Health Canada also committed to introducing marketing restrictions as part of its Healthy Eating Strategy2. In response to this commitment, Health Canada has proposed the use of a new nutrient profile (NP) model specific to Canada for the identification of foods that would be subject to these marketing restrictions3. According to the World Health Organization’s (WHO’s) Guiding Principles and Framework Manual for the Development or Adaptation of NP Models, validation of a model is recommended for all newly developed or adapted models because various approaches of validation aid in the development, consistency, and credibility of the NP model4. One method of ensuring the credibility of the classifications of the foods generated a NP model is to assess convergent validity with food-based dietary guidelines4. Convergent validity is defined as “the extent to which the measurement correlates with an external criterion of the phenomenon under study at the same point in time” 4. The comparison to food-based dietary guidelines can be conducted using two approaches, involving either 1) a set of “indicator” foods that are the focus of food-based dietary guidelines (e.g., fruits, vegetables, confectionery, savoury snacks) and a small number of nutrition professionals as part

106



of an expert panel involved in the development of the model, or 2) a representative sample of foods and a larger number of nutrition experts4. The latter approach is relatively more robust than the former because the representative sample of foods is intended to cover the entire spectrum of foods, and the classifications by the nutrition experts, albeit subjective, are determined independently without influence of the NP model under investigation4. Objective: The objective of this study is to assess convergent validity by comparing the marketing classifications of a representative sample of foods determined using Health Canada’s proposed NP model against the classifications determined by a sample of Canadian nutrition professionals. The marketing classifications refer to whether each food will be classified as “allowed” or “not allowed” to be marketed to children. Hypothesis: It is hypothesized that the marketing classifications of foods determined using the proposed NP model will align with the classifications determined by Canadian nutrition professionals. Significance: This research will be conducted as part of the Canadian Institutes of Health Research (CIHR) Project Grant, entitled “the IMPACT of food environment policies on the Canadian food supply, dietary intakes and health: evidence to inform policy action”. The research team will share the results of this study with Health Canada to inform the use of the proposed NP model in the policy on restricting the marketing of unhealthy foods and beverages to children. References: 1. Senate of Canada. Bill S-228. An Act to amend the Food and Drugs Act (prohibiting food and beverage

marketing directed at children). First reading, September 27, 2016. First Session, Forty-second Parliament, 64-65 Elizabeth II, 2015-2016. Internet: http://www.parl.gc.ca/content/hoc/Bills/421/Private/S-228/S-228_1/S-228_1.PDF (Accessed: 30 January 2017).

2. Health Canada. Healthy Eating Strategy. October 2016. Internet: https://www.canada.ca/content/dam/canada/health-canada/migration/publications/eating-nutrition/healthy-eating-strategy-canada-strategie-saine-alimentation/alt/pub-eng.pdf (Accessed: 30 January 2017).

3. Canada H. Toward Restricting Unhealthy Food and Beverage Marketing to Children: Discussion paper for public consultation. June 2017. Internet: https://s3.ca-central-1.amazonaws.com/ehq-production-canada/documents/attachments/9bced5c3821050c708407be04b299ac6ad286e47/000/006/633/original/Restricting_Marketing_to_Children.pdf (Accessed: 17 June 2017).

4. Rayner M. WHO Guiding Principles and Framework Manual for the development or adaptation of nutrient profile models (first edition). UNPUBLISHED 2011.

12. METHODS (a) Please describe all formal and informal procedures to be used. Describe the data to be collected, where and how they will be obtained and how they will be analyzed. Minor revisions to the study are bolded and are based on the results of the pilot study (section

12.2.4.6) 12.1 OVERVIEW The validation study will compare the marketing classifications of a representative sample of foods determined using the proposed NP model against the classifications determined by Canadian nutrition professionals. This work will be done through an online survey. The marketing classifications refer to whether each food will be classified as “allowed” or “not allowed” to be marketed to children.

107



12.2 DATA COLLECTION 12.2.1 Representative Food Sample The representative food sample will consist of a master list of 200 branded foods from two databases (described in Section 12.2.2) across a variety of food categories pertinent to children’s marketing and Canadian diets. The pertinent food categories will be determined using four sources (refer to Appendix A): 1) 21 food categories across the 220 indicator foods, as per Health Canada’s discussion paper for public

consultation1; 2) 16 food categories across foods specifically marketed to children on food and beverage packaging, as

per Labonté et al. (2017 [submitted])2; 3) eight food categories according to the Canadian Children’s Food and Beverage Advertising Initiative

(CAI) Uniform Criteria3; and 4) 19 food categories across the foods specifically marketed to children on television programs, as per

Potvin Kent et al. (2017 [submitted])4. Specific food items will be selected from these categories for the master list, including a variety of foods that range in healthfulness. Specifically, a widely used NP model (e.g., Foods Standards Australia New Zealand5) will be applied to the foods in the master list to ensure that a range in healthfulness is achieved and that foods contain a range of both nutrients to limit and nutrients to encourage. In addition, the proposed NP model will be applied to the foods in the master list to ensure that both foods that will be permitted or not permitted by the proposed model will be represented. Participants in this study will be randomized into two groups and assigned to either be presented with foods without a Principal Display Panel (PDP) (“without PDP” group) or with a PDP (“with PDP” group). For participants in the “without PDP” group, 40 unique foods will be randomly selected from various food categories from the 200 foods in the master list. For participants in the “with PDP” group, 40 unique foods will be randomly selected from various food categories from a subset of 100 of the 200 foods in the master list. The subset of 100 foods will enable more classifications for each product, but from relatively fewer participants. Food items in the subset will be selected based on the availability of high-quality images of the PDP for each food. One hundred foods with high-quality images will be randomly selected if high-quality images for more than 100 foods are available. For participants in both groups, the random selection of foods will be done at the food category level to ensure that participants are exposed to a variety of food categories. The nutrition professionals will classify these 40 foods for which the Nutrition Facts table (NFt), ingredient list and either a PDP (“with PDP” group) or the generic product name (e.g., chocolate covered granola bar) (“without PDP” group), will be presented. The provision of a generic product name versus a PDP will enable the researchers to differentiate between classifications for 100 foods made solely on nutrition and ingredient information versus classifications that may have been influenced by a brand or symbols/logos. Refer to Section 12.2.4 for details. 12.2.2 Source of Nutrient Data of Representative Food Sample The nutrient data for each of the foods in the master list will be obtained from the following databases: Food Label Information Program (FLIP) and Menu-FLIP 2016. FLIP is a branded food composition database that has been developed and maintained since 2010 by the Principal Investigator (L’Abbé) with peer‐reviewed funding from the Canadian Institutes of Health Research (CIHR), the Canadian Stroke Network, Heart and Stroke Foundation, UofT McHenry Chair, and IT support from Dietitians of Canada (DC)6. This database of packaged foods is updated every 3-4 years for testing hypotheses and monitoring the Canadian food supply. FLIP 2013, contains nutritional information on n=15,342 unique food products with a NFt from the four largest national grocery retailers (Loblaw’s, Metro, Sobeys, and Safeway), representing approximately 75% of the Canadian retail food market share. In-store data collection occurred between May and September 2013 by systematically scanning grocery store shelves using a smartphone application, and web and mobile database software have been developed for efficient database management, processing and analyses. Information collected include product information (e.g.,

108



company, brand, Universal Product Code), container size, price, NFt information, ingredient list, package marketing [e.g., nutrient content claims, front-of-pack (FOP) labelling, marketing to children], and photos of all sides of the packages. Only one package size per food was captured, but all flavours and varieties were collected. Nutritional information was recorded for products in their form “as sold” and “as consumed”, if necessary (e.g., condensed soup). Foods in FLIP 2013 were classified into 22 distinct food categories and 153 subcategories as defined in Schedule M of the Food and Drug Regulations (version current between March 15, 2012 and December 13, 2016)7. A 2017 collection is currently underway with a grant from the Centre for Child Nutrition & Health (now renamed the Lawson Centre for Child Nutrition), UofT, and from CIHR, and the complete data set is anticipated to be ready for analyses in late 2017 or early 2018. Currently, FLIP 2017 contains nutrition information for approximately 11,000 foods collected from the Loblaw’s website, which will be used for this study. The Menu-FLIP database is a unique UofT database containing nutrient composition information (serving size plus up to 13 nutrients commonly found on food labels) from over 80 sit-down and takeout restaurant chains in Canada, and includes data on ~9,000 foods and beverages per cycle. Data collections occurred in 2010, 2013, and 2016. One additional Ontario specific Menu-FLIP data collection supported by Public Health Ontario occurred in January-March 2017. For each collection, inclusion criteria included all food service establishments with ≥ 20 retail locations in Canada and with ≥1 outlet in at least three provinces. Nutrition information has been collected from publicly available resources such as company Canadian websites, and companies were contacted to clarify data if necessary. Establishments were classified according to type (fast-food, full service), and foods by type (meal, entrée, side dish etc). For this study, Menu-FLIP 2016 will be used to identify a small proportion of restaurant foods for inclusion under one or two categories within the master list of foods: 1) foods from fast food restaurants, and 2) foods from dine-in restaurants that are specifically marketed to children, as per Potvin Kent et al. (2017 [submitted])4. 12.2.3 Proposed NP Model Health Canada’s proposed NP model will allow marketing of foods that Health Canada encourages for healthy eating and child development, but will restrict marketing foods that contain sugars, sodium, or saturated fat above specific thresholds1. Two levels of nutrient threshold options are proposed: 5% of the DV of saturated fat, sugars, or sodium (option 1), or 15% of the DV of saturated fat, sugars, or sodium (option 2)1. Marketing would be restricted for foods exceeding one or more of these thresholds1. Ultimately, the proposed NP model will classify each food as “allowed” or “not allowed” to be marketed to children. Both these threshold options will be assessed. Each option will be applied to the master list and will be compared to the results from the survey of nutrition professionals (refer to Section 12.2.4). 12.2.4 Survey of Nutrition Professionals 12.2.4.1 Nutrition Professionals This study is modelled after a validation study conducted by Scarborough et al. (2007), who tested a NP model using data from a survey of nutrition professionals8,9. In the study by Scarborough et al. (2007), the survey was sent to a total of 3,517 nutrition professionals, of which 850 were from the British Dietetic Association and 2,667 were from the British Nutrition Society, with approximately 15% of individuals belonging to both organizations8,9. A total of 850 responses to the survey were received, resulting in a 24% initial response rate8,9. Of the 850 responses, 148 (17%) were considered non-suitable according to pre-defined criteria (described below) and were excluded; thus, 702 responses were eligible for analysis, which equates to a final response rate of 20%8,9. For this study, a non-nationally representative sample of nutrition professionals will be recruited through outreach from the three organizations that represent nutrition professionals in Canada to their constituents: DC with approximately 6,000 members; l’Ordre professionnel des diététistes du Québec (OPDQ) with approximately 3,175 members; and the Canadian Nutrition Society (CNS) with approximately 400 members. Based on an estimate whereby 15% (n=60) of the CNS members also belong to their respective dietetic organizations, the survey link will be sent 9,515 nutrition professionals across the three organizations (6,000 from DC; 3,175 from OPDQ; 340 from CNS).

109



Recruitment will be conducted via dissemination of the introductory email (refer to Appendix B) to all members of the three organizations (DC, OPDQ, and CNS). The recruitment email will provide a brief introduction to the study and a weblink to the online survey, which will be hosted by SurveyGizmo. As discussed in Section 12.2.4.5, the information letter and informed consent form (refer to Appendix C) will be provided as preamble to the survey questions (refer to Appendix D). Based on the response rates observed in the Scarborough et al. (2007) study8,9, initial and final response rates of 24% and 20% would correspond to 2,284 and 1,903 responses, respectively, in this study. If a more conservative estimate is used involving initial and final response rates of 10% and 8%, respectively, it is estimated that 952 responses will be received and 761 responses will be eligible for analysis in this study. The same response quality exclusion criteria used by Scarborough et al. (2007) 8,9 to ensure that the survey questions were not completed carelessly or maliciously will be applied in this study; specifically, responses will be excluded if: 1) The respondent categorized less than 30 of the 40 (<75%) foods presented; 2) The respondent placed more than 80% of foods within a 15% range of the continuous scale; or 3) The respondent categorized more than 50% of foods using discordant classifications and scores

(e.g., a food that was selected as “not allowed” for marketing was given a score of 100, which indicates that the food is “healthiest”).

With 2/3 of participants randomized into the “without PDP” group and the remaining randomized into the “with PDP” group, it is anticipated that of the estimated 761 eligible responses, 510 will be from the “without PDP” group and 251 will be from the “with PDP” group. With 40 foods viewed per participant each of the foods from the master list of 200 products (presented without PDP) and from the subset of 100 products (presented with PDP), each product will be viewed about 100 times within the randomized groups. 12.2.4.2 Provision of Nutrition Information The following information for each food will be provided to the nutrition professionals on a single screen: 1) either the generic product name (e.g., chocolate covered granola bar) for participants in the “without PDP” group or the PDP for participants in the “with PDP” group; 2) NFt; and 3) ingredient list (refer to Appendix D). Given the inclusion of OPDQ members, the languages in which the nutrition information is presented must be considered. For certain food products in FLIP 2017, the French text may not be available for the PDP, NFt, and/or ingredient list. As such, foods with PDPs containing both English and French text will be presented whenever possible; otherwise, foods with PDPs containing English only will be presented. NFts containing both English and French will be presented. For all foods, the ingredient lists will be presented in English only. For foods in Menu-FLIP 2016, French text may not be presented in the image of the restaurant food nor on the NFt. As such, the images of the restaurant foods containing English only will be presented; however, NFts containing both English and French will be presented. Ingredient lists are not available for restaurant foods. Due to the inclusion of both English and French nutrition professionals, a disclaimer that some of the information will be presented in English only will be included in the survey (refer to Appendix D). 12.2.4.3 Classification of Foods Nutrition professionals will be asked two questions related to the healthfulness of each food: 1) “Should the food be allowed or not allowed to be marketed to children?”, and 2) “How would you rate the healthfulness of the food, with two anchor descriptors ‘healthiest’ and ‘least healthy’?” The first question will permit analysis of the results as a categorical variable, whereas the second question will permit analysis of the results as a continuous variable. Nutrition professionals will be asked to score the foods compared to all foods, rather than foods from a similar category.

110



12.2.4.4 Informed Consent The information letter and informed consent form (refer to Appendix C) will be provided as preamble to the online survey questions. The information letter and informed consent will detail the study procedures, conditions for participation, risks and benefits, confidentiality, compensation, and contact information for the study. Nutrition professionals will be able to check one of two boxes corresponding to “I agree” or “I do not agree” to study participation. Those who selected “do not agree” will be prompted to exit the webpage. Those who selected “agree” will be directed to start the survey questions (refer to Appendix D). Participants will be able to decline participation in the study at any time by simply closing the webpage. 12.2.4.5 Survey Questions A single survey will be administered to all nutrition professionals. The survey platform, SurveyGizmo, will allow the participant to select the language (English or French) of the survey. The time for completion of the survey will be approximately 20 minutes. The survey questions are presented in Appendix D. The survey will consist of two parts for a total of 50 numbered questions. With respect to Part A, seven questions will be asked to ascertain demographics of the nutrition professionals (e.g., degrees acquired, years of nutrition experience, type of nutrition experience, experience working with children, etc.). Part B will be related to the classifications of the foods. Each participant will be randomized to classify either i) 40 foods for which the generic product name, NFt, and ingredient list will be presented or ii) 40 foods for which the PDP, NFt, and ingredient list will be presented. A disclaimer that some of the nutrition information is presented in English only will be included at the start of the food classifications (Part B). After classifying all 40 foods, two questions will be asked to ascertain the factors considered in the rating of foods (e.g., name of food, Nutrition Facts information, ingredient list, brand, claims, etc.). A final question will be asked to ascertain whether the respondent is familiar with Health Canada’s consultation document on marketing to children that was published in June 2017. The survey will be available for two weeks until a specified end-date, at which time the weblink to the survey will expire. 12.2.4.6 Pilot Study A pilot study with a convenience sample of approximately 20 English and French respondents (e.g., from the University of Toronto and Health Canada) will be conducted to ensure feasibility of the completion of the survey within the targeted time limit (i.e., 20 minutes) and clarity of the questions. Adjustments will be made to the survey if there are any concerns regarding clarity or time. 12.3 DATA ANALYSIS As discussed in Section 12.2.2.4.1, the same response quality exclusion criteria used by Scarborough et al. (2007) 8,9 to ensure that the survey questions were not completed carelessly or maliciously will be applied in this study; specifically, responses will be excluded if: 1) The respondent categorized less than 30 of the 40 (<75%) foods presented; 2) The respondent placed more than 80% of foods within a 15% range of the continuous scale; or 3) The respondent categorized more than 50% of foods using discordant classifications and scores

(e.g., a food that was selected as “not allowed” for marketing was given a score of 100, which indicates that the food is “healthiest”).

12.3.1 Primary Outcome The primary outcome will be the agreement in the proportions of “allowed” or “not allowed” foods determined using option 1 (5% DV) of Health Canada’s proposed NP model and by the nutrition professionals, assessed using the Kappa statistic and McNemar’s test for discordant pairs. Responses from participants randomized to the “without PDP” group will be used to determine this outcome. Each participant in the “without PDP” group will assess 40 foods using the generic name, NFt, and ingredient list. These 40 foods will be selected from the master list of 200 foods. It is anticipated that approximately 510 participants randomized to the “without PDP” group will have responses eligible for analyses; thus, each of the 200 foods will be assessed by approximately 100 participants.

111



12.3.2 Secondary Outcomes A secondary outcome will be the agreement in the proportions of “allowed” or “not allowed” foods determined using option 2 (15% DV) of Health Canada’s proposed NP model and by the nutrition professionals. Additional secondary outcomes that may be explored include: • An analysis of the factors (e.g., claims, FOP logos or symbols) that were used in making the determination of whether a food should be allowed to be marketed to children. • Determine whether or how the PDP influenced the nutrition professionals’ classifications. Responses from participants randomized to the “with PDP” group will be used to determine this outcome. Each participant in the “with PDP” group will assess 40 foods using the PDP, NFt, and ingredient list. These 40 foods will be selected from a subset of 100 foods from the master list. It is anticipated that approximately 251 participants randomized to the “with PDP” group will have responses eligible for analyses; thus, each of the 100 foods will be assessed by approximately 100 participants. The difference in the 100 ratings of each food from the “with PDP” and the “without PDP” groups will be determined. • Although the intent of the proposed NP model is not to generate a score, the derivation of a NP score may allow for comparisons to the score generated by the nutrition professionals. For example, the derivation of a score may be based on the three nutrients of public health concern (i.e., saturated fat, sugars, and sodium) considered in the proposed NP model, such that a score of 1 as “healthiest” when none of the limits are exceeded; score of 2 when the limit for one of the three nutrients is exceeded; score of 3 when the limits for two of the three nutrients are exceeded; or score of 4 when the limits for all three nutrients are exceeded. The secondary outcomes that may be examined include, for example:

- Difference in the average scores of a food determined using option 1 (5% DV) of the proposed NP model and by the nutrition professionals.

- Difference in the average scores of a food determined using option 2 (15% DV) of the proposed NP model and by the nutrition professionals.

- Based on the classification and scores generated by the nutrition professionals, a comparison of the mean score across all “allowed” foods versus mean score across all “not allowed” foods. 12.4 REFERENCES 1. Canada H. Toward Restricting Unhealthy Food and Beverage Marketing to Children: Discussion paper

for public consultation. June 2017. Internet: https://s3.ca-central-1.amazonaws.com/ehq-production-canada/documents/attachments/9bced5c3821050c708407be04b299ac6ad286e47/000/006/633/original/Restricting_Marketing_to_Children.pdf (Accessed: 17 June 2017).

2. Labonté ME, Poon T, Mulligan C, Bernstein JT, Franco-Arellano B, L’Abbé MR. Comparison of global nutrient profiling systems for restricting the commercial marketing of foods and beverages of low nutritional quality to children in Canada. Submitted to the American Journal of Clinical Nutrition, revisions requested, July 24, 2017.

3. Advertising Standards Canada. Canadian Children’s Food and Beverage Advertising Initiative. Uniform Nutrition Criteria White Paper. September 2014. Internet: http://www.adstandards.com/en/childrensinitiative/CAIUniformNutritionCriteriaWhitePaper.pdf (Accessed: 19 September 2016).

4. Potvin Kent M, Smith JR, Pauzé E, L’Abbé M. The food and beverage industry's self-established Uniform Nutrition Criteria are not improving the healthfulness of food and beverage advertisements viewed by Canadian children on television. Under revision; submitted to International Journal of Behavioural Nutrition and Physical Activity, May/June, 2017.

5. Australian Government. ComLaw. Legislative Instrument Compilation. Australia New Zealand Food Standards Code - Standard 1.2.7 - Nutrition, Health and Related Claims - F2015C00967. December 7, 2015. Internet: https://www.comlaw.gov.au/Details/F2015C00967 (accessed 17 January, 2016).

6. Schermel A, Emrich TE, Arcand J, Wong CL, L'Abbe M R. Nutrition marketing on processed food packages in Canada: 2010 Food Label Information Program. Appl Physiol Nutr Metab 2013;38:666-72. doi: 10.1139/apnm-2012-0386.

112

http://www.comlaw.gov.au/Details/F2015C00967



7. Government of Canada. Food and Drug Regulations (C.R.C., c. 870), Sections B.01.001, B.01.002A and D.01.001. Ottawa: Minister of Justice. 2016. Internet: http://laws-lois.justice.gc.ca/eng/regulations/c.r.c.,_c._870/page-165.html#h-353 (Accessed: 16 March 2016).

8. Scarborough P, Boxer A, Rayner M, Stockley L. Testing nutrient profile models using data from a survey of nutrition professionals. Public Health Nutr 2007;10:337-45. doi: 10.1017/s1368980007666671.

9. Scarborough P, Rayner M, Stockley L, Black A. Nutrition professionals' perception of the 'healthiness' of individual foods. Public Health Nutr 2007;10:346-53. doi: 10.1017/s1368980007666683.

(b) Attach a copy of all questionnaires, interview guides and/or any other instruments. (c) Include a list of appendices here for all additional materials submitted (e.g., Appendix A – Informed Consent; Appendix B – Interview Guide, etc.): Appendix A – List of Relevant Food Categories Appendix B – Recruitment Email Appendix C – Informed Consent Form Appendix D – Survey Questions 13. PARTICIPANTS, DATA AND/OR BIOLOGICAL MATERIALS (a) Describe the participants to be recruited list the eligibility criteria, and indicate the estimated sample size (i.e. min-max # of participants). Where applicable, please also provide a rationale for your choice in sample size and/or sample size calculation. Refer to Section 12.2.4.1. (b) Where the research involves extraction or collection of personally identifiable information, please describe the purpose, from whom the information will be obtained, what it will include, and how permission to access the data is being sought. (Strategies for recruitment are to be described in section #15.) No personally identifiable information will be collected or used from the participants in this study. (c) Is there any group or individual-level vulnerability related to the research that needs to be mitigated (for example, difficulties understanding informed consent, history of exploitation by researchers, power differential between the researcher and the potential participant)? If so, please provide further details below. As nutrition professionals will be recruited for this study, there is no group or individual-level vulnerability related to the research that needs to be mitigated. (d) If your research involves the collection and/or use of biological materials (e.g. blood, saliva, urine, teeth, etc.), please provide details below. Be sure to indicate how the samples will be collected and by whom. This study does not involve the collection and/or use of biological materials. 14. EXPERIENCE OF INVESTIGATORS WITH THIS TYPE OF RESEARCH (a) Please provide a brief description of previous experience by (i) the principal investigator/supervisor or sponsor, (ii) the research team and (iii) the people who will have direct contact with the participants. If there has not been previous experience with this type of research, please describe how the principal investigator/research team will be prepared.

113



Principal Investigator: Dr. Mary L’Abbé is the Earle W. McHenry Professor and Chair, Department of Nutritional Sciences, University of Toronto and former Director of the Bureau Nutritional Sciences, Health Canada. She is internationally recognized for her leadership of nutrition policy for public health. She has extensive experience in nutrition research, including assessing the nutritional quality of the Canadian food supply; nutrition labelling policies for population health; consumer nutrition knowledge, attitudes and behaviours and development of an online food composition database (FLIP); and nutrient profiling. Dr. L’Abbé and her graduate students have conducted and published six previous consumer research studies using online panelists. Her group has successfully developed other nutrition web tools and apps (e.g., BigLife Salt Calculator and One Sweet App). Co-investigators: Theresa Poon is currently a MSc graduate student researching the validity of several global NP models for the assessment of the nutritional quality of foods in a Canadian context. She also contributed to the other NP research projects with Dr. L’Abbé, namely on the systematic review of global NP models and the use of NP models for marketing restrictions to children. In this study, she will provide expertise in evaluating the validity of the proposed NP approach. Dr. Marie-Eve Labonté was a CIHR Postdoctoral fellow conducting research on NP methods for public health nutrition and nutrition regulation. She specifically developed expertise in the use and adaptation of various NP systems for the evaluation of the nutritional quality of the Canadian food supply, overall and in association with characteristics that drive consumer choices (e.g., food prices). Dr. Labonté recently accepted a position as an Assistant Professor at Université Laval, Québec City, effective June 1st, 2017, but will continue to participate in this study. Jodi Bernstein is currently a PhD graduate student researching sugars in the Canadian prepackaged food supply. She has expertise in utilizing data from the FLIP database. In this study, she will provide assistance with the data collection and analyses. Database Manager: Alyssa Schermel is the Database Manager of FLIP. She obtained her MSc. in Nutritional Sciences at the University of Toronto under Dr. L’Abbé. In this study, she will provide assistance with respect to the FLIP database as needed. People who will have direct contact with the participants: None of the investigators will be in contact with the participants as the survey completed by the nutrition professionals will be administered online. 15. RECRUITMENT OF PARTICIPANTS Where there is recruitment, please describe how, by whom, and from where the participants will be recruited. Where participant observation is to be used, please explain the form of insertion of the researcher into the research setting (e.g., living in a community, visiting on a bi-weekly basis, attending organized functions). If relevant, describe any translation of recruitment materials, how this will occur and whether or not those people responsible for recruitment will speak the language of the participants. Refer to Section 12.2.4.1 and Appendix B.

Attach a copy of all posters, advertisements, flyers, letters, e-mail text, or telephone scripts to be used for recruitment as appendices.

114



16. COMPENSATION Please see U of T’s Compensation and Reimbursement Guidelines. (a) Will participants receive compensation for participation? Financial Yes No In-kind Yes No Other Yes No (b) If Yes, please provide details and justification for the amount or the value of the compensation offered. Not applicable (c) If No, please explain why compensation is not possible or appropriate. This research will be conducted as part of the CIHR Project Grant, entitled “the IMPACT of food environment policies on the Canadian food supply, dietary intakes and health: evidence to inform policy action”. As no identifying information on participants is collected, we cannot compensate participants. Also, the professional associations consider the completion of this survey as public service, as we are not a commercial entity, and do not request payment for the distribution of this survey to their members. (d) Where there is a withdrawal clause in the research procedure, if participants choose to withdraw, how will compensation be affected? Not applicable SECTION C –DESCRIPTION OF THE RISKS AND BENEFITS OF THE PROPOSED RESEARCH 17. POSSIBLE RISKS (a) Please indicate all potential risks to participants as individuals or as members of a community that may arise from this research: (i) Physical risks (e.g., any bodily contact or administration of any substance): Yes No (ii) Psychological/emotional risks (e.g., feeling uncomfortable, embarrassed, or upset): Yes No (iii) Social risks (e.g., loss of status, privacy and/or reputation): Yes No (iv) Legal risks (e.g., apprehension or arrest, subpoena): Yes No (b) Please briefly describe each of the risks noted above and outline the steps that will be taken to manage and/or minimize them. There are no expected risks associated with this study. 18. POSSIBLE BENEFITS • Describe any potential direct benefits to participants from their involvement in the project • Describe any potential direct benefits to the community (e.g., capacity building)

115

http://www.research.utoronto.ca/wp-content/uploads/2010/01/Guidelines-for-Compensation-and-Reimbursement-of-Research-Participants-Approved-Feb-16-11.pdf



• Comment on the potential benefits to the scientific/scholarly community or society that would justify involvement of participants in this study

There will be no direct benefit to the participants for their involvement in this study. Information learned from this study may benefit nutrition professionals, consumers, and researchers, collectively, in the future. Participation in this study is important, as the results will provide understanding with regards to nutrition professionals’ perceptions on the healthfulness of foods and whether certain foods should be restricted from marketing to children. In addition, the results will be used to test the validity of the proposed NP model. The U of T will share the results of this study with Health Canada, and these results may be used to inform the use of the proposed nutrient profile model in the policy on restricting the marketing of unhealthy foods to children. SECTION D – INFORMED CONSENT 19. CONSENT PROCESS (a) Describe the process that will be used to obtain informed consent and explain how it will be recorded. Please note that it is the quality of the consent, not the form that is important. The goal is to ensure that potential participants understand to what they are consenting. Refer to Section 12.2.4.4 and Appendix C. (b) If the research involves extraction or collection of personally identifiable information from or about a research participant, please describe how consent from the individuals or authorization from the data custodian (e.g., medical records department, district school board) will be obtained. No personally identifiable information will be obtained from the participants in this study. 20. CONSENT DOCUMENTS (a) Attach an Information Letter/Consent Form For details about the required elements in the information letter and consent form, please refer to our informed consent guide (http://www.research.utoronto.ca/wp-content/uploads/documents/2014/10/GUIDE-FOR-INFORMED-CONSENT-V-Oct-2014.pdf) Additional documentation regarding consent should be provided such as:

- screening materials introductory letters, letters of administrative consent or authorization

(b) If any of the information collected in the screening process - prior to full informed consent to participate in the study - is to be retained from those who are later excluded or refuse to participate in the study, please state how potential participants will be informed of this course of action and whether they will have the right to refuse to allow this information to be kept. No information will be collected prior to obtaining full informed consent. 21. COMMUNITY AND/OR ORGANIZATIONAL CONSENT, OR CONSENT BY AN AUTHORIZED PARTY (a) If the research is taking place within a community or an organization which requires that formal consent be sought prior to the involvement of individual participants, describe how consent will be obtained and attach

116

http://www.research.utoronto.ca/wp-content/uploads/documents/2014/10/GUIDE-FOR-INFORMED-CONSENT-V-Oct-2014.pdf

http://www.research.utoronto.ca/wp-content/uploads/documents/2014/10/GUIDE-FOR-INFORMED-CONSENT-V-Oct-2014.pdf



any relevant documentation. If consent will not be sought, please provide a justification and describe any alternative forms of consultation that may take place. Dr. L’Abbé spoke directly with the leadership of the professional associations to obtain their permission to distribute the survey to their members. Permission was granted by DC on July 11, 2017. Permission was granted verbally by the president and executive director of CNS on July 27, with final sign-off received from the board of directors at their meeting on August 23, 2017. Permission from OPDQ was received by email Sept 11, 2017. (b) If any or all of the participants are children and/or individuals that may lack the capacity to consent, describe the process by which capacity/competency will be assessed and/or, the proposed alternate source of consent. As nutrition professionals will be recruited for this study, it is anticipated that all participants will have the capacity to consent. (c) If an authorized third party will be used to obtain consent:

i) Submit a copy of the permission/information letter to be provided to the person(s) providing the alternative consent

ii) Describe the assent process for participants and attach the assent letter.

Not applicable 22. DEBRIEFING and DISSEMINATION (a) If deception or intentional non-disclosure will be used in the study, provide justification. Please consult the Guidelines for the Use of Deception and Debriefing in Research Deception or intentional non-disclosure will not be used in the study, as the objective of the study will be disclosed to the participants in the recruitment email and informed consent form (refer to Appendices C and D, respectively). (b) Please provide a copy of the written debriefing form, if applicable. (c) If participants and/or communities will be given the option of withdrawing their data following the debriefing, please describe this process. Not applicable (d) Please describe what information/feedback will be provided to participants and/or communities after their participation in the project is complete (e.g., report, poster presentation, pamphlet, etc.) and note how participants will be able to access this information. No feedback will be provided directly to participants after study participation. Results of the study will be disseminated through research publications and at scientific conferences. 23. PARTICIPANT WITHDRAWAL (a) Where applicable, please describe how participants will be informed of their right to withdraw from the project and outline the procedures that will be followed to allow them to exercise this right.

117

http://www.research.utoronto.ca/wp-content/uploads/2009/09/Deception_and_Debriefing_Guidelines.pdf



Participants will be informed of their right to withdraw from the study in the Informed Consent Form (refer to Appendix X). They will be provided with the contact information of the primary investigator, research team, and Research Oversight and Compliance Office – Human Research Ethics Program. Participants will be able to decline participation in the study at any time by closing the webpage and not completing the survey. (b) Indicate what will be done with the participant’s data and any consequences which withdrawal may have on the participant. Any data collected from a participant that withdraws consent from the study will not be used in the analyses. There will be no consequences for participation withdrawal. (c) If participants will not have the right to withdraw from the project at all, or beyond a certain point, please explain. Ensure this information is included in the consent process and consent form. Participants will have the right to withdraw consent at any stage of the study. SECTION E – CONFIDENTIALITY AND PRIVACY 24. CONFIDENTIALITY Data security measures must be consistent with UT's Data Security Standards for Personally Identifiable and Other Confidential Data in Research. All identifiable electronic data that is being kept outside of a secure server environment must be encrypted. (a) Will the data be treated as confidential? Yes No (b) Describe the procedures to be used to protect the confidentiality of participants or informants, where applicable Data collected from each participant will be assigned a unique study identification number known only by a member of the research team tasked with assigning the numbers. No personally identifiable information will be obtained from the participants in this study. All data files will be encrypted and retained on a secure server. (c) Describe any limitations to protecting the confidentiality of participants whether due to the law, the methods used, or other reasons (e.g., a duty to report) Not applicable 25. DATA SECURITY, RETENTION AND ACCESS (a) Describe how data (including written records, video/audio recordings, artifacts and questionnaires) will be protected during the conduct of the research and dissemination of results. All data files, containing only the study subject identification numbers and no personally identifiable information, will be encrypted and retained on a secure server. Access to the data will be granted only to members of the research team. (b) Explain how long data or samples will be retained. (If applicable, referring to the standard data retention practice for your discipline) Provide details of their final disposal or storage. Provide a justification if you

118

http://www.research.utoronto.ca/wp-content/uploads/documents/2013/05/datasecurity1.pdf

http://www.research.utoronto.ca/wp-content/uploads/documents/2013/05/datasecurity1.pdf



intend to store your data for an indefinite length of time. If the data may have archival value, discuss how participants will be informed of this possibility during the consent process. The encrypted electronic files will be stored for 10 years and deleted thereafter. Ten years is the standard retention practice for data collected from non-randomized controlled trials conducted at the Department of Nutritional Sciences, Faculty of Medicine. (c) If participant anonymity or confidentiality is not appropriate to this research project, please explain. Not applicable (d) If data will be shared with other researchers or users, please describe how and where the data will be stored and any restrictions that will be made regarding access. Once data analysis is completed, the research team will share with Health Canada the results in aggregate form via scientific publications, technical reports, or presentations. The raw data will be accessible only by Dr. L’Abbé and the research team. SECTION F – LEVEL OF RISK AND REVIEW TYPE See the Instructions for Ethics Review Submission Form for detailed information about the Risk Matrix. 26. RISK MATRIX: REVIEW TYPE BY GROUP VULNERABILITY and RESEARCH RISK (a) Indicate the Risk Level for this project by checking the intersecting box

______________________Research Risk____________________________ Group Vulnerability Low Medium High Low 1 1 2 Medium 1 2 3 High 2 3 3 (b) Explain/justify the level of research risk and group vulnerability reported above: There are no expected risks associated with this study. The nutrition professionals recruited for this study are expected to have a low level of vulnerability. (Please note that the final determination of Review Type and level of monitoring will be made by the reviewing University of Toronto REB) Based on the level of risk, these are the types of ethics review that an application may receive: Risk level = 1: Delegated Review; Risk level = 2 or 3: Full Board Review For both delegated and full reviews (SSH&E, HS, or HIV), please submit one electronic copy of your application and all appendices (e.g., recruitment, information/consent and debriefing materials, and study instruments) as a single Word document or a pdf. Do not submit your entire research proposal. Please ensure that the electronic signatures are in place and e-mail to [email protected]

119





The deadline for delegated review (SSH&E or HS) is EVERY Monday, or first business day of the week, by 4 pm. Information about full REB meeting and submission due dates are posted on our website (SSH&E, HS or HIV). HIV REB reviews all applications at full board level but applies proportionate review based on the level of risk. All other submissions (e.g., amendments, adverse events, and continuing review submissions) should be sent to [email protected] SECTION G – SIGNATURES 27. PRIVACY REGULATIONS My signature as Principal Investigator, in Section G of this application form, confirms that I am aware of, understand, and will comply with all relevant laws governing the collection and use of personally identifiable information in research. I understand that for research involving extraction or collection of personally identifiable information, provincial, national and/or international laws may apply and that any apparent mishandling of personally identifiable information must be reported to the Office of Research Ethics. As the UofT Principal Investigator on this project, my signature confirms that I will ensure that all procedures performed will be conducted in accordance with all relevant University, provincial, national and international policies and regulations that govern research involving human participants. I understand that if there is any significant deviation from the project as originally approved I must submit an amendment to the Research Ethics Board for approval prior to implementing any change.

Signature of Principal Investigator:_____________________________________ Date: Dec 8, 2017

As the Departmental Chair/Dean, my signature confirms that I am aware of the requirements for scholarly review and that the ethics application for this research has received appropriate review prior to submission. In addition, my administrative unit will follow guidelines and procedures to ensure compliance with all relevant University, provincial, national or international policies and regulations that govern research involving human participants. My signature also reflects the willingness of the department, faculty or division to administer the research funds, if there are any, in accordance with University, regulatory agency and sponsor agency policies. Print Name of Departmental Chair/Dean (or designate): Signature of Departmental Chair/Dean: ___________________________ Date: (or authorized designate)

120





http://www.research.utoronto.ca/wp-content/uploads/2012/08/Chair-Reps-on-Protocol-REPAC-approved-April-2012.pdf

http://www.research.utoronto.ca/wp-content/uploads/2012/08/Chair-Reps-on-Protocol-REPAC-approved-April-2012.pdf



APPENDIX A – LIST OF RELEVANT FOOD CATEGORIES

Health Canada’s Indicator Food Categories (n=21)1

Food Categories with Packaging Specifically Marketed to Children, as per Labonté et al. (2017 [submitted]) (n=16)2

Canadian Children’s Food and Beverage Advertising Initiative Uniform Criteria (n=8)3

Foods Specifically Marketed to Children on Television, as per Potvin Kent et al. (2017 [submitted]) (n=19)4

1. Vegetables, with added fat, sugars or sodium

1. Bakery products 1. Milk and Alternatives

1. Cakes (including pudding)

2. Fruit, with added fat, sugars or sodium

2. Beverages 2. Grains 2. Candy (without chocolate)

3. Fruit, without added fat, sugars or sodium

3. Cereals, other grain products

3. Soups 3. Cold cereals (i.e. ready-to-eat products marketed as breakfast foods)

4. Juice (100% pure) 4. Dairy products, substitutes

4. Meat and Alternatives

4. Cheese (excluding cottage cheese)

5. Breads, plain rice and plain pasta

5. Desserts 5. Fruit and Vegetables

5. Chocolate bars (excluding boxed chocolate and candy with chocolate)

6. Breakfast cereals (no sugar added)


6. Occasional Snacks

6. Compartment snacks and lunch kits (i.e. prepackaged products comprised of two or more ingredients in separate compartments sold as portable snacks or meals)

7. Breakfast cereals (sugar added)

7. Marine, fresh water animals

7. Mixed Dishes 7. Cookies

8. Sweet snacks, desserts, cookies, granola bars

8. Fruit, fruit juices 8. Meals on the Go

8. Ice cream (including frozen yogurt, sherbet, sorbet or frozen treats made from these foods)

9. Savory snacks 9. Meat, poultry, their products, substitutes

9. Pizza (not sold in restaurants)

10. Fluid milk or soy based beverages (unsweetened)

10. Miscellaneous 10. Portable snacks (i.e. cereal/protein/fruit bars or squares and fruit snacks)

11. Fluid milk (sweetened) 11. Combination dishes 11. Fast food restaurants

12. Yogourts with sugar added

12. Nuts, seeds 12. Non-fast food restaurants (i.e. dine-in restaurants)

13. Yogourts without sugar added


13. Snack foods (e.g. chips, popcorn, pretzels, cheese puffs and meat-based snacks like jerky)

14. Cheeses 14. Snacks 14. Diet and regular soft drinks

15. Meat and alt 15. Soups 15. Energy drinks 16. Processed meats 16. Sugars, sweets 16. Sports drinks

121



Health Canada’s Indicator Food Categories (n=21)1

Food Categories with Packaging Specifically Marketed to Children, as per Labonté et al. (2017 [submitted]) (n=16)2

Canadian Children’s Food and Beverage Advertising Initiative Uniform Criteria (n=8)3

Foods Specifically Marketed to Children on Television, as per Potvin Kent et al. (2017 [submitted]) (n=19)4

17. Composite dish (ready made meals, mixed dishes)

17. Juices

18. Confectionary 18. Drinks and nectars (excluding water, milk and alternatives, tea and coffee drinks, cocktail mixers, and alcoholic beverages)

19. Sugar and artificially sweetened beverages (soda, energy drinks, juice drinks) and vegetables juices with added sodium

19. Yogurt (including yogurt beverages)

20. Meals from fast food outlets

21. Miscellaneous 1 Canada H. Toward Restricting Unhealthy Food and Beverage Marketing to Children: Discussion paper for public consultation. June 2017. Internet: https://s3.ca-central-1.amazonaws.com/ehq-production-canada/documents/attachments/9bced5c3821050c708407be04b299ac6ad286e47/000/006/633/original/Restricting_Marketing_to_Children.pdf (Accessed: 17 June 2017). 2 Labonté ME, Poon T, Mulligan C, Bernstein JT, Franco-Arellano B, L’Abbé MR. Comparison of global nutrient profiling systems for restricting the commercial marketing of foods and beverages of low nutritional quality to children in Canada. Submitted to the American Journal of Clinical Nutrition, revisions requested, July 24, 2017. 3 Advertising Standards Canada. Canadian Children’s Food and Beverage Advertising Initiative. Uniform Nutrition Criteria White Paper. September 2014. Internet: http://www.adstandards.com/en/childrensinitiative/CAIUniformNutritionCriteriaWhitePaper.pdf (Accessed: 19 September 2016). 4 Potvin Kent M, Smith JR, Pauzé E, L’Abbé M. The food and beverage industry's self-established Uniform Nutrition Criteria are not improving the healthfulness of food and beverage advertisements viewed by Canadian children on television. Under revision; submitted to International Journal of Behavioural Nutrition and Physical Activity, May/June, 2017.

122



APPENDIX B – RECRUITMENT EMAIL1

Dear nutrition professional, We are requesting your participation in a research study involving nutrition professionals conducted by the University of Toronto with funding from the Canadian Institutes of Health Research. You have been contacted because you are a member of Dietitians of Canada, l’Ordre professionnel des diététistes du Québec, and/or the Canadian Nutrition Society. The study aims to assess the healthfulness of pre-packaged and restaurant foods and whether or not these foods should be allowed to be marketed to children. We will use the results to test the validity of a nutrient profile model that has been proposed for use in Canada. Your involvement will be important, as the results of this study may be used to inform the policy on restricting the marketing of unhealthy foods to children in Canada. Study participants will be asked to complete an online survey, which will involve classifying the healthfulness of a random set of 40 foods. If you are interested in participating or learning more about this study, the information letter and informed consent form is provided as preamble to the online survey, which is hosted by SurveyGizmo. The survey is available in English or French. Please access the survey here [hyperlink], or at the following weblink: [weblink]. The survey will be available for two weeks until [date]. If you have any questions regarding the study, please email our research team at with the subject line "survey on marketing foods to children". Thank you for your consideration in participating in this study. Mary R. L'Abbé, PhD Earle W. McHenry Professor and Chair, Department of Nutritional Sciences Faculty of Medicine, University of Toronto

Tel ; Cell Email: Web: Assistant to the Chair: Don Newton Tel: ; Email:

1 The recruitment email will also be presented in French.

123



APPENDIX C – INFORMED CONSENT FORM2 Information Letter and Informed Consent Form Principal Investigator: Dr. Mary L’Abbé Earle W. McHenry Professor and Chair, Department of Nutritional Sciences Faculty of Medicine, University of Toronto

Phone: Email: Title: Assessing the healthfulness of foods for restricting the marketing of unhealthy foods to children You are being asked to participate in a study to assess the healthfulness of pre-packaged foods and whether or not these foods should be allowed to be marketed to children. This research is conducted by the University of Toronto (U of T) with funding from the Canadian Institutes of Health Research. The U of T will use the results to test the validity of a nutrient profile model that has been proposed for use in Canada. The U of T will share the results of this study with Health Canada, which may be used to inform the policy on restricting the marketing of unhealthy foods and beverages to children. Before agreeing to participate in this study, it is important that you read and understand the following explanation of the study procedures. You should understand enough about the study risks and benefits to be able to make an informed decision before deciding whether you wish to participate. This is known as the informed consent process. Procedure We are inviting approximately 9,000 participants to take part in this study. Participants include nutrition professionals from Dietitians of Canada, l’Ordre professionnel des diététistes du Québec, and/or the Canadian Nutrition Society. This study takes place through an online survey delivered using SurveyGizmo. Participation includes:

1. Providing consent for us to use the data collected from the survey, including demographics (e.g., degrees acquired, years of nutrition experience, type of nutrition

2 The informed consent form will also be presented in French.

124



experience, experience working with children) and your classifications of foods. Please note that some of the nutrition information of the foods may be presented in English only.

2. Completing the survey, which involves reviewing 40 foods and should take approximately 20 minutes to complete.

Conditions for Participating Your participation in this study is voluntary, and you may withdraw from or refuse to participate in the study at any time. You may decline to answer any question in the survey without any negative consequences. Risks and Discomforts There are no expected risks or discomforts associated with this study. Benefits There is no direct benefit to you for your involvement in this study. Information learned from this study may benefit you and other nutrition professionals, consumers, and researchers in the future. Your participation in this study is important, as the results will help us understand nutrition professionals’ perceptions on the healthfulness of foods and whether certain foods should be restricted from marketing to children. The U of T will use the results to test the validity of a nutrient profile model that has been proposed for use in Canada. The U of T will share the results of this study with Health Canada, which may be used to inform the policy on restricting the marketing of unhealthy foods to children. Confidentiality All information obtained during the study will be held in strict confidence. You will be identified only by a study-specific identification number. No names or personal identifying information will be collected or used in any publication or presentations. No information identifying you will be transferred outside this study. Compensation Compensation will not be provided for participation in this study. In no way does agreeing to participate in this study waive your legal rights, nor does it relieve the investigators, sponsors, or involved institutions from their legal and professional responsibilities. Questions and Contact Information This study has been reviewed and received ethics approval from the U of T (Human Study REB# ). If you have any questions or concerns about your rights as a participant in this study, you can contact the Research Oversight and Compliance Office – Human Research Ethics Program at or . This contact is not involved with the research project in any way, and calling them will not affect your participation in the study.

125



If you have any questions regarding the study in general, please email our research team at with the subject line "survey on marketing foods to children". Consent By selecting the “I agree” option below, I consent to take part in the study with the understanding that I may withdraw at any time.

I agree. I do not agree.

126



APPENDIX D – SURVEY QUESTIONS3

Part A: Demographics

1. Which province or territory do you reside in? Alberta British Columbia Manitoba New Brunswick Newfoundland and Labrador Northwest Territories Nova Scotia Nunavut Ontario Prince Edward Island Quebec Saskatchewan Yukon Prefer not to say

2. What degree(s) and/or professional designations have you acquired? (Select all that apply.)

BSc MSc PhD Post-doctoral fellowship RD or DtP Other (please specify): [Open field] Prefer not to say

3. How many years of working/professional nutrition experience do you have? <1 year 1 to 3 years 4 to 6 years 7 to 9 years 10 or more years Prefer not to say

4. What type of nutrition experience do you have? (Select all that apply.) Clinical practice Public health Research Academia

3 This information will be presented in French in the French version of the survey.

127



Industry Other (please specify): [Open field] Prefer not to say

5. Are you a parent or guardian of a child or children aged 17 years or younger? Yes a. If yes, please list age(s) of child(ren): [Open field]

No Prefer not to say

6. Do you have experience working with children? Yes a. If yes, please list age(s) of child(ren): [Open field] b. If yes, please specify the context in which you worked with children. (Select all

that apply.) Education Clinical practice Community or public health Research Other (please specify): [Open field] Prefer not to say

No Prefer not to say

128



Part B: Classification of Foods [for the subset of responders who will be shown the generic product name, NFt, and ingredient list of the food]

You will be asked to classify a total of 40 foods. Note: Some of the nutrition information (names of the foods, ingredient lists) is presented in English only.

[The specific food shown below is an example and will vary for each question. The following will be repeated for questions 1 to 40 in Part B.]

Granola bar, cereal bar, oatmeal

1a. Should the food be allowed to be marketed to children? Yes No

1b. How would you rate the healthfulness of the food on a scale from “least healthy” to “healthiest”? Note: Rate the food compared to all foods, rather than foods from a similar category.

least healthy [visual analogue scale from 1 to 100] healthiest

INGREDIENTS: WHOLE GRAIN ROLLED OATS, WHOLE WHEAT FLOUR, SUGAR, CANOLA OIL, TAPIOCA SYRUP, GLYCERIN, CHICORY ROOT EXTRACT (INULIN FIBRE), DRIED EGG YOLK (CONTAINS SODIUM SILICOALUMINATE), MOLASSES, RAISIN JUICE CONCENTRATE, PALM AND PALM KERNEL OIL, BAKING SODA, DRIED EGG WHITE, SALT, MODIFIED MILK INGREDIENTS, CINNAMON, NATURAL FLAVOUR, MALTODEXTRIN, SOY LECITHIN, ROSEMARY EXTRACT, CITRIC ACID.

129



Part B: Classification of Foods [for the subset of responders who will be shown the PDP, NFt, and ingredient list of the food]

You will be asked to classify a total of 40 foods. Note: Some of the nutrition information (food packages, ingredient lists) is presented in English only.

[The specific food shown below is an example and will vary for each question. The following will be repeated for questions 1 to 40 in Part B.]

1a. Should the food be allowed to be marketed to children?

Yes No

1b. How would you rate the healthfulness of the food on a scale from “least healthy” to “healthiest”? Note: Rate the food compared to all foods, rather than foods from a similar category.

least healthy [visual analogue scale from 1 to 100] healthiest

INGREDIENTS: WHOLE GRAIN ROLLED OATS, WHOLE WHEAT FLOUR, SUGAR, CANOLA OIL, TAPIOCA SYRUP, GLYCERIN, CHICORY ROOT EXTRACT (INULIN FIBRE), DRIED EGG YOLK (CONTAINS SODIUM SILICOALUMINATE), MOLASSES, RAISIN JUICE CONCENTRATE, PALM AND PALM KERNEL OIL, BAKING SODA, DRIED EGG WHITE, SALT, MODIFIED MILK INGREDIENTS, CINNAMON, NATURAL FLAVOUR, MALTODEXTRIN, SOY LECITHIN, ROSEMARY EXTRACT, CITRIC ACID.

130



Part C: Final Questions [for the subset of responders who will be shown the generic product name, NFt, and ingredient list of the food]

1. What factor(s), if any, did you consider in deciding whether the food should or should not be marketed to children? (Select all that apply.)

[The response fields illustrated in the figure below will be presented in a multi-stage manner. For example, participants will be presented with the five responses in green initially. Depending on the green responses selected, the responses in blue will be presented thereafter, if applicable; and so on.]

2. Are you familiar with Health Canada’s consultation document on restricting the marketing of unhealthy foods and beverages to children that was published in June 2017?

Yes No

Name of the food

Nutrition Facts table

Nutrients to limit

Energy

Total fat

Saturated fat

Trans fat

Sodium

Total sugars

Cholesterol

Nutrients to encourage

Mono- or polyunsaturated fats

Protein

Fibre

Vitamins or minerals

Serving size Other (please specify)

Ingredient list

Number of ingredients listed

Order of ingredients

Fruits

Vegetables

Whole grains

Refined grains

Hydrogenated oils

Partially-hydrogenated oils

Added/free sugars

Low/no-calorie sweeteners

Other flavouring agents

Monosodium glutamate (MSG)

Preservatives

Food colouring

Other food additives

Allergens

Ingredients not recognized

Other (please specify)


None of the above

131



Part C: Final Questions [for the subset of responders who will be shown the generic product name, NFt, and ingredient list of the food]

1. What factor(s), if any, did you consider in deciding whether the food should or should not be marketed to children? (Select all that apply.)

[The response fields illustrated in the figure below will be presented in a multi-stage manner. For example, participants will be presented with the five responses in green initially. Depending on the green responses selected, the responses in blue will be presented thereafter, if applicable; and so on.]

2. Are you familiar with Health Canada’s consultation document on restricting the

marketing of unhealthy foods and beverages to children that was published in June 2017?

Yes No

Food package

Brand

Nutrient content claims

Health claims

Symbols or logos


Nutrition Facts table

Nutrients to limit

Energy

Total fat

Saturated fat

Trans fat

Sodium

Total sugars

Cholesterol

Nutrients to encourage

Mono- or polyunsaturated fats

Protein

Fibre

Vitamins or minerals

Serving size Other (please specify)

Ingredient list

Number of ingredients listed

Order of ingredients

Fruits

Vegetables

Whole grains

Refined grains

Hydrogenated oils

Partially-hydrogenated oils

Added/free sugars

Low/no-calorie sweeteners

Other flavouring agents

Monosodium glutamate (MSG)

Preservatives

Food colouring

Other food additives

Allergens

Ingredients not recognized



None of the above

132

133

Appendix C

Systematic Review of NP Models This systematic review was registered on PROSPERO(41). The manuscript was

submitted in January 2018 to Advances in Nutrition for publication: Labonté MÈ, Poon

T, Gladanac B, Ahmed M, Franco-Arellano B, Rayner M, L’Abbé MR (unpublished).

Nutrient profile models with applications in government-led nutrition policies aimed at

health promotion and noncommunicable disease prevention: a systematic review.


this research, as indicated in Table C-1. The manuscript of the systematic review is

provided as a supplemental file to this thesis.

Table C-1 Authors’ contributions to systematic review of NP models

Task Author

Prior to January 2016 when TP started MSc

Designed research study Labonté, L’Abbé Identified models from Rayner’s catalogue Labonté, Ahmed Conducted literature searches Labonté Screened title/abstracts of publications Labonté, Ahmed After January 2016 when TP started MSc

Screened full-text publications Labonté, Poon Screened reference lists Labonté, Poon Assessed eligibility of NP models Labonté, Poon Extracted data Gladanac (majority), Labonté, Poon, Franco-Arellano Verified extracted data Labonté, Poon Prepared tables and figures Labonté, Poon

Wrote manuscript Labonté Provided comments/revisions to draft manuscript Poon Provided comments/revisions to final manuscript Poon, Gladanac, Ahmed, Franco-Arellano, Rayner,

L’Abbé

134

Copyright Acknowledgements Appendix A is reproduced with permission from “Labonté MÈ, Poon T, Mulligan C,

Bernstein JT, Franco-Arellano B, L’Abbé MR. Comparison of global nutrient profiling

systems for restricting the commercial marketing of foods and beverages of low

nutritional quality to children in Canada. Am J Clin Nutr 106, 1471-1481.”

examination of the validity of nutrient profiling models for …€¦ · to dr. marie-Ève...

Documents