lable at ScienceDirect

Regulatory Toxicology and Pharmacology 80 (2016) 183e194

Contents lists avai

Regulatory Toxicology and Pharmacology

journal homepage: www.elsevier .com/locate/yrtph

Performance standard-based validation study for local lymph nodeassay: 5-bromo-2-deoxyuridine-flow cytometry method

Ilyoung Ahn a, Tae-Sung Kim a, Eun-Sun Jung b, Jung-Sun Yi b, Won-Hee Jang c,Kyoung-Mi Jung c, Miyoung Park c, Mi-Sook Jung d, Eun-Young Jeon d, Kyeong-uk Yeo e,Ji-Hoon Jo e, Jung-Eun Park e, Chang-Yul Kim e, Yeong-Chul Park e, Won-Keun Seong a,Ai-Young Lee f, Young Jin Chun g, Tae Cheon Jeong h, Eui Bae Jeung i, Kyung-Min Lim j,SeungJin Bae j, Soojung Sohn a, Yong Heo e, *

a Korean Center for the Validation of Alternative Methods, National Institute of Food and Drug Safety Evaluation, MFDS, Chungbuk, Republic of Koreab National Institute of Food and Drug Safety Evaluation, Toxicological Evaluation and Research Dept., Toxicological Screening & Testing Division, MFDS,Chungbuk, Republic of Koreac Medical Beauty Research Division, AmorePacific Corp., R&D Center, Yongin, Republic of Koread Biotoxtech Co., Ltd., Pharmacology Efficacy Team, Chungbuk, Republic of Koreae Catholic University of Daegu, College of Medical and Public Health Sciences, Dept. Occupational Health and GLP Center, Gyeongbuk, Republic of Koreaf Dongguk University, Ilsan Hospital, Gyeonggi-do, Republic of Koreag Chung-Ang University, College of Pharmacy, Seoul, Republic of Koreah Yeungnam University, College of Pharmacy, Gyeongbuk, Republic of Koreai Chungbuk National University, College of Veterinary Medicine, Chungbuk, Republic of Koreaj Ewha Womans University, College of Pharmacy, Seoul, Republic of Korea

a r t i c l e i n f o

Article history:Received 24 February 2016Received in revised form10 June 2016Accepted 12 June 2016Available online 16 June 2016

Keywords:Performance standardsValidation studyLocal lymph node assayLLNA: BrdU-FCMSkin sensitizationFlow cytometry

* Corresponding author. Catholic University of DaPublic Health Sciences, Dept. Occupational Health, Gpublic of Korea.

E-mail address: yheo@cu.ac.kr (Y. Heo).

http://dx.doi.org/10.1016/j.yrtph.2016.06.0090273-2300/© 2016 Elsevier Inc. All rights reserved.

a b s t r a c t

Local lymph node assay: 5-bromo-2-deoxyuridine-flow cytometry method (LLNA: BrdU-FCM) is amodified non-radioisotopic technique with the additional advantages of accommodating multiple end-points with the introduction of FCM, and refinement and reduction of animal use by using a sophisti-cated prescreening scheme. Reliability and accuracy of the LLNA: BrdU-FCM was determined according toOECD Test Guideline (TG) No. 429 (Skin Sensitization: Local Lymph Node Assay) performance standards(PS), with the participation of four laboratories. Transferability was demonstrated through successfullyproducing stimulation index (SI) values for 25% hexyl cinnamic aldehyde (HCA) consistently greater than3, a predetermined threshold, by all participating laboratories. Within- and between-laboratory repro-ducibility was shown using HCA and 2,4-dinitrochlorobenzene, in which EC2.7 values (the estimatedconcentrations eliciting an SI of 2.7, the threshold for LLNA: BrdU-FCM) fell consistently within theacceptance ranges, 0.025e0.1% and 5e20%, respectively. Predictive capacity was tested using the finalprotocol version 1.3 for the 18 reference chemicals listed in OECD TG 429, of which results showed 84.6%sensitivity, 100% specificity, and 88.9% accuracy compared with the original LLNA. The data presented areconsidered to meet the performance criteria for the PS, and its predictive capacity was also sufficientlyvalidated.

© 2016 Elsevier Inc. All rights reserved.

1. Introduction

Concern over allergic contact dermatitis and immune skin

egu, College of Medical andyeongsan-si, Gyeongbuk, Re-

disorders induced by new chemicals or other hazardous substancesis growing. Consequently, the safety evaluation of new substancesis becoming more important. The local lymph node assay (LLNA),Organization for Economic Co-operation and Development (OECD)Test Guideline (TG) No. 429 (Skin Sensitization: Local Lymph NodeAssay), has been used to evaluate the skin sensitization potential ofchemicals since 2002 (OECD, 2010a). The LLNA is a skin sensitiza-tion test in mice that measures the proliferation of murine local

I. Ahn et al. / Regulatory Toxicology and Pharmacology 80 (2016) 183e194184

auricular lymph node cells after topical exposure to chemicals.Activation of T lymphocytes is the fourth key event of the adverseoutcome pathway for skin sensitization (OECD, 2012). In addition,this assay is used in the European Union’s Registration, Evaluation,Authorization, and Restriction of Chemicals (REACH) legislation,and notably, it is considered a priority among the in vivo tests inAnnex VII (REACH, 2006). The United States Environmental Pro-tection Agency (EPA) also adopted the LLNA in its Health EffectsTest Guideline and has used it as a method of evaluating skinsensitization since 2003 (EPA, 2003).

However, 3H-methyl thymidine (analogue of thymidine) or 125I-iododeoxyuridine is used in the LLNA; thus, the assay requires fa-cilities for handling the radioisotope and has the risk of radioactivecontamination. For that reason, the Japanese Center for the Vali-dation of Alternative Methods (JaCVAM) developed and validatedthe LLNA: BrdU-ELISA (OECD TG 442B) and the LLNA: DA (OECD TG442A), in which radioisotopes are not needed, with the aim ofupgrading the LLNA. Since 2010, those two test methods have beenused in evaluating chemicals (OECD, 2010b, 2010c).

The Local Lymph Node Assay: 5-bromo-2-deoxyuridine-flowcytometry method (LLNA: BrdU-FCM) was developed with thefinancial support from Korea’s Ministry of Food and Drug Safety(MFDS) in 2009. From 2010 to 2011, the usefulness of this testmethod was assessed, and its protocol was optimized (Jung et al.,2010; 2012). The LLNA: BrdU-FCM can be used to identify skinsensitizing chemicals and evaluate skin sensitization potency in thesame way as the LLNA, LLNA: BrdU-ELISA, and LLNA: DA. This testmethod does not require a radioisotope, which makes it easier totest and ensure the safety of experimenters and the environmentsimilarly to LLNA: BrdU-ELISA or LLNA: DA. However, in contrast toLLNA: BrdU-ELISA (in which the BrdU-content of lymph nodes ismeasured), the LLNA: BrdU-FCM can quantitatively and selectivelycount BrdU-incorporated auricular lymph node cells throughdouble-staining with 7-aminoactinomycin D (7-AAD). Further-more, other endpoints, such as cell surface markers or intracellularcytokines, can be measured through using a multi-color FACS ma-chine along with specific antibodies. Our pre-validation studydemonstrated the usefulness of the test method because it cananalyze B cells and T cells with flow cytometry and cytokines (Junget al., 2012). Flow cytometry is commonly used in immunologicaltests, and therefore, the LLNA: BrdU-FCM would be more useful tocharacterize skin sensitization of test substances. Moreover, in theLLNA: BrdU-FCM, BALB/c mice are used instead of CBA mice, ofwhich availability is limited in some countries.

The transferability, within- and between-laboratory reproduc-ibility, and predictive capacity of the test method were evaluatedfrom 2012 to 2015 to find out if it could be internationally acceptedas a method for skin sensitization tests. Previously, within-laboratory reproducibility (WLR) and between-laboratory repro-ducibility (BLR) with protocols 1.0 and 1.1 were shown to beacceptable, based on the OECD TG429 PS (Yang et al., 2015). Inaddition, predictive capacity using the previous protocol versions(1.1 and 1.2) was evaluated using combined data (Kim et al., 2016).However, the LLNA: BrdU-FCMmethod has been revised to version1.3 to accommodate a sophisticated prescreening scheme to reduceanimal sacrifice and to refine the solubilization procedure. Here, thepredictive capacity of LLNA: BrdU-FCMwas re-tested using the finalprotocol version 1.3 for the 18 reference chemicals listed in OECDTG 429. In addition, we assessed overall if the validation of LLNA:BrdU-FCM was well conducted in compliance with the OECDguidance document (GD) 34 and the OECD TG 429 Annex 1 per-formance standards (PS) (OECD, 2010a).

2. Material and methods

2.1. General conditions of the study

2.1.1. Validation management team (VMT)The validation study was coordinated by the VMT organized by

the Korean Center for the Alternative Methods (KoCVAM). The VMTcomprised the following: scientific advisory members with expe-rience in toxicology, pharmacology, veterinary medicine, ordermatology; KoCVAM members responsible for chemical selec-tion, coding/decoding, distribution, and overall management of thestudy; a statistician; and a representative from a lead laboratory.The VMT coordinated the overall validation process, includingreviewing and statistically analyzing test results, selecting anddistributing test substances, drawing final conclusions of the vali-dation study, and writing a final validation study report. Study di-rectors of the lead laboratories participated in VMT meetings toreport study progress and discuss but were not involved in testsubstance selection and coding. The chemical manager checkedand distributed the test chemicals used in proficiency tests, WLR,BLR, and predictive capacity to all testing sites. Among the testchemicals, the ones selected for the evaluation of predictive ca-pacity were coded. The bio-statistician was responsible for col-lecting Excel spreadsheets, including test results from thelaboratories, analyzing them, and writing a statistical report.

2.1.2. LaboratoriesThe AmorePacific Corporation R&D Center served as the lead

laboratory (lead laboratory 1) from the technology transfer phase tothe first predictive capacity test; however, the laboratory adopted apolicy of not using animals to test cosmetic products. Therefore, itwas replaced by the Toxicological Screening & Testing Division ofthe National Institute of Food and Drug Safety Evaluation (leadlaboratory 2). The two other participating laboratories were theLaboratory of Immunology for Public Health at Catholic Universityof Daegu (participating laboratory 1) and the Safety EvaluationTeam at Biotoxtech Co., Ltd. (participating laboratory 2).

All laboratories complied with the study plan and standardoperating procedures (SOPs) according to OECD Good LaboratoryPractice (GLP) principles. The quality assurance unit (QAU) of eachlaboratory checked if tests were conducted and data were managedproperly. In addition, the QAU of lead laboratory 1 inspected eachparticipating laboratory from 2012 to 2013.

2.1.3. Design of the validation studyThe validation study was conducted, and data were generated

based on themodule approach in OECD GDNo. 34 and suggested bythe European Union Reference Laboratory for Alternatives to Ani-mal Testing (EURL ECVAM) (Hartung et al., 2004; OECD, 2005).However, the overall study procedure and components wereestablished in accordance with the OECD TG 429 Annex 1 PS.

2.1.3.1. Transferability. Lead laboratory 1 transferred the testmethod to each laboratory, and then a test was performed to see ifthe test method was properly established. According to OECD TG429, 25% hexyl cinnamic aldehyde (HCA) was chosen as a positivecontrol with an acetone: olive oil (4:1, v/v, AOO) mixture as avehicle control. The main purpose of this test was to identifywhether the stimulation index (SI) for 25% HCA was �3.

Each participating laboratory’s proficiency in the conduct of theassay was assessed using eugenol. Eugenol was coded anddistributed to each testing site, but the information on concentra-tions and the vehicle (AOO) was given. Eugenol is known as a weaksensitizer, and the VMT selected it as an appropriate substance forthe assessment of each laboratory’s competence in the assay.

I. Ahn et al. / Regulatory Toxicology and Pharmacology 80 (2016) 183e194 185

2.1.3.2. Within-laboratory reproducibility. The evaluation of theWLR was repeated four times at an interval of more than 1 week atlead laboratory 1 and participating laboratories 1 and 2, using 5%,10%, and 25% HCA, based on OECD TG 429 Annex 1.

2.1.3.3. Between-laboratory reproducibility. The BLR was evaluatedat lead laboratory 1 and participating laboratories 1 and 2 using0.05%, 0.1%, and 0.25% 2,4-dinitrochlorobenzene (DNCB), based onOECD TG 429 Annex 1. As for HCA, the results obtained in the WLRevaluation test were used in the BLR evaluation.

2.1.3.4. Predictive capacity. After the evaluation of reproducibilitywas completed, the predictive capacity of the LLNA: BrdU-FCMwasassessed to identify whether the assay could reliably distinguishbetween skin sensitizers and non-sensitizers in accordance withOECD TG 429 Annex 1. The reference chemicals with high puritylisted in Table 1 were purchased and distributed to each testing site.Those chemicals were coded and tested under blind conditions.

The first test was performed at lead laboratory 1, and the secondtest was at lead laboratory 2 and participating laboratories 1 and 2,using 22 reference chemicals using protocol version 1.1 and 1.2,respectively. In the third test, 18 essential reference chemicals weretested at participating laboratory 2, using protocol 1.3.

2.2. Test chemicals

The VMT selected the reference chemicals suggested in theOECD TG 429 Annex 1 (Table 1). HCA used for the transferabilityevaluation and for each experiment as a positive control was pur-chased and tested by each laboratory. The reference chemicalsselected for the predictive capacity phase and eugenol for theproficiency assessment were purchased and distributed by thechemical manager under blind conditions with random tables(www.random.org). All vehicles used in the validation study werepurchased by each laboratory. Chemical information includingChemical Abstracts Service Registration Number, manufacturer,

Table 1List of the reference chemicals selected for the predictive capacity evaluation.

No. Chemical name CAS no. EC3 (%)a

1 CMI/MI 26172-55-4/2682-20-4 0.0092 DNCB 97-00-7 0.0493 4-Phenylenediamine 106-50-3 0.114 Cobalt chloride 7646-79-9 0.65 Isoeugenol 97-54-1 1.56 2-Mercaptobenzothiazole 149-30-4 1.77 Citral 5392-40-5 9.28 HCA 101-86-0 9.79 Eugenol 97-53-0 10.110 Phenyl benzoate 93-99-2 13.611 Cinnamic alcohol 104-54-1 2112 Imidazolidinyl urea 39236-46-9 2413 Methyl methacrylate 80-62-6 9014 Chlorobenzene 108-90-7 2515 Isopropanol 67-63-0 5016 Lactic acid 50-21-5 2517 Methyl salicylate 119-36-8 2018 Salicylic acid 69-72-7 2519 Sodium lauryl sulfate 151-21-3 8.120 Ethylene glycol dimethacrylate 97-90-5 2821 Xylene 1330-20-7 95.822 Nickel chloride 7718-54-9 5

AOO, acetone: olive oil (4:1, v/v); CAS No., Chemical Abstracts Service Number; CMI/MI,dimethylformamide; DMSO, dimethyl sulfoxide; DNCB, 2,4-dinitrochlorobenzene; EC3, etest result (i.e. TG 406); HCA, hexyl cinnamic aldehyde; Liq, liquid; LLNA, murine local lymstimulation index <3; NT, not tested; Sol, solid; Veh, test vehicle; *, presumed to be a non-included as a patch test kit allergen, and no case reports of human sensitization were lo

a OECD, 2010a.

and purity is described in Table 1. Chemical nameswere unveiled bythe chemical manager after each phase was finished, and raw datawere sent to the biostatistician. Material safety data sheets(MSDSs), which can be used when an emergency occurs, wereindividually sealed and sent to each laboratory. The unopenedMSDSs were returned and checked by the VMT.

2.3. Experimental methods

2.3.1. ProtocolThe LLNA: BrdU-FCM is a “me-too” test, and it complied with the

essential test method components described in OECD TG 429 An-nex 1. The LLNA: BrdU-FCM protocol has been revised three timessince 2012. The tests following each version of the protocol are asbelow: protocol version 1.0 for transferability, proficiency, andWLR; version 1.1 for BLR and first test for the predictive capacityevaluation; version 1.2 for proficiency (lead laboratory 2) and sec-ond test for the predictive capacity evaluation; version 1.3 foradditional test and third test for the predictive capacity evaluation.Experimental procedure for LLNA: BrdU-FCM is briefly described inSupplementary material 1. Protocol 1.0 was revised into 1.1 toinclude a pre-screen test for selecting doses and a vehicle withhighest solubility among AOO, N, N-dimethylformamide (DMF),and dimethyl sulfoxide (DMSO). The protocol was upgraded from1.1 to 1.2 by including methyl ethyl ketone (MEK) as a vehicle anddescribing the flow cytometric analysis procedure in detail. Also, aconcentration of 75% for all compounds was removed from the pre-screen test since it was not a concentration selected in OECD TG429. In protocol 1.3, the dissolution procedurewas refined to obtainthe maximum solubility of highly viscous chemicals (e.g. imida-zolidinyl urea) by extending themagnetic stirring time from 30minto 60 min.

2.3.2. AnimalsFemale BALB/c mice (7 weeks old) were purchased from Orient

Bio Co. (Seongnam, Korea) and acclimated for at least 5 days before

0.5�e2.0 � EC3a LLNAa Humana Manufacturer Purity

0.0045e0.018 þ þ Rohm and Hass e

0.025e0.099 þ þ Aldrich 99%0.055e0.22 þ þ Sigma 98%0.3e1.2 þ þ Sigma-Aldrich 97%, 98%0.77e3.1 þ þ Aldrich 98%0.85e3.4 þ þ Aldrich 97%4.6e18.3 þ þ Aldrich 95%4.8e19.5 þ þ Aldrich 85%5.05e20.2 þ þ Fluka 99%6.8e27.2 þ þ Aldrich 99%10.5e42 þ þ Aldrich 98%12e48 þ þ Aldrich 95%45e100 þ þ Aldrich 99%NA e * Sigma-Aldrich 99.8%NA e þ Sigma-Aldrich 99%, 99.5%NA e * Fluka 90%NA e e Sigma-Aldrich 99%NA e e Sigma 99%4.05e16.2 þ e Sigma 98.5%14e56 þ þ Aldrich 98%47.9e100 þ e Sigma-Aldrich 98.5%NA e þ Aldrich 98%

5-Chloro-2-methyl-4-isothiazolin-3-one/2-methyl-4-isothiazolin-3-one; DMF, N,N-stimated concentration needed to produce a Stimulation Index of 3; GP, guinea pigph node assay result (i.e. TG 429); MEK, methyl ethyl ketone; NA, not applicable sincesensitizer in humans, as based on no clinical patch test results were located; it is notcated.

I. Ahn et al. / Regulatory Toxicology and Pharmacology 80 (2016) 183e194186

experiments. The animals were housed in an animal room main-tained at a temperature of 22 ± 3 �C and a relative humidity of 30%e70%. The roomwas lit by artificial light for 12 h per day. The animalswere allowed to have free access to solid diets and sterilizeddrinking water. The mice were randomly allocated into test groups(4 or 5 mice per group) with an even distribution according to bodyweight. All the laboratories obtained ethical approval from theirrespective Institutional Animal Care and Use Committee, and ex-periments were performed in accordance with national and inter-national standards regarding the use of animals in scientificresearch.

2.3.2.1. Observation of general symptoms and erythema.General symptoms were observed and recorded on a daily basis.Erythema in the chemical-treated part was scored in accordancewith the Draize test method each day prior to the application of testsubstances. The mice were weighed on days 1 and 6, and theaverage thickness of both ears was calculated on day 1 (before testsubstance application), 3 (before test substance application), and 6(before sacrifice) (Supplementary material 1).

2.3.2.2. Application of test substances and injection of BrdU solution.Doses of 100%, 50%, 25%, 10%, 5%, 2.5%, 1%, or 0.5% were chosenaccording to OECD TG 429. In the pre-screen test, the solubility ofeach chemical was measured, and the maximum concentrationswere determined (See 2.3.2.5). Diluted test substances, vehiclealone, or the positive control chemical (25 mL) was applied to thedorsum of each ear on a daily basis for 3 consecutive days (Sup-plementary material 1). A vehicle selected for dissolving the testsubstances was used as a negative control and 25% HCA in AOOwasused as a positive control throughout the experiments. A singleintraperitoneal injection of 100 mL BrdU solution (20 mg/ml) wasgiven to each mouse at 24 ± 2 h before sacrifice. The auricularlymph nodes collected were placed in 6-well plates filled with coldphosphate-buffered saline (PBS) and maintained on ice.

2.3.2.3. Lymph node cell preparation and flow cytometry analysis.Lymph node cells (LNCs) were prepared by disintegrating thelymph nodes through 70-mm nylon mesh (BD Biosciences, FranklinLakes, NJ, USA) into 1 ml of PBS. The LNCs (1.5 � 106) were washedonce with PBS by centrifugation at 300 � g for 7 min and thenresuspended prior to the fixation and permeabilization steps, ac-cording to the instructions provided with the BrdU flow kit (BDPharmingen, Franklin Lakes, NJ, USA). Cells were permeabilizedwith Cytoperm permeabilization buffer plus and then treated withDNase at 37 �C in awater bath for 1 h to expose BrdU epitopes. Afterwashing, FITC-conjugated anti-BrdU antibodies were added for20 min at room temperature in the dark. After one more wash, theLNCs were resuspended in 7-AAD solution to label the DNA.

2.3.2.4. Evaluation of skin sensitization potency. The number ofBrdU-incorporating cells within the viable 7-AAD-expressing cellpopulation (104 cells) was counted in a FACSCalibur flow cytometer(BD Biosciences), as previously described (Jung et al., 2010; 2012;Yang et al., 2015). Briefly, a gating based on light scatter was appliedto the viable lymphocyte population, followed by the setup of atwo-parameter dot plot (FITC-anti BrdU for y-axis and 7-AAD for x-axis). An upper-right quadrant (Q2) was set up to exhibit no LNCsfrom a mouse in which neither BrdU nor test substance wasapplied. Then, the Q2 quadrant was adjusted to display 1% events,using LNCs from the mouse in which BrdU was injected in theabsence of test substance application. After completion of thesegating procedures, LNCs from the vehicle control or the test sub-stance application group were analyzed to obtain the percentage ofBrdU-incorporating cells within the viable 7-AAD-expressing cell

population. Then the total number of BrdU-incorporating LNCs wascalculated by multiplication of the percentage with total number ofLNCs. The SI value was obtained through dividing the total numberof BrdU-incorporating LNCs from each individual mouse appliedwith a test substance by the mean number of BrdU-incorporatingLNCs in the vehicle control group.

2.3.2.5. Selection of the vehicle and highest concentrations. In thepredictive capacity evaluation phase, each laboratory was requiredto perform the vehicle selection procedure in accordance withOECD TG 429 because there was no information on vehicles for thelaboratories about the coded test substances. A vehicle showing thehighest solubility was selected among the AOO, DMF, MEK andDMSO. A vehicle that dissolves test substance completely enough tohave no particles and layers has been selected. If more than onevehicle is selected, prioritize them in the order of AOO, DMF, MEKand DMSO. Magnetic stirring for 30e60 min was applied tocompletely dissolve test substances, and its complete solubility waschecked through visual inspection and/or microscopicexamination.

The pre-screen test, which was divided into two phases, wasperformed to determine test concentrations for the main test. Thepre-screen test could contribute to reduce the number of test ani-mals and inflict less pain or distress to animals. The three con-centrations that did not induce systemic toxicity or severe irritationwere chosen in the first and second phases of the pre-screentoxicity/irritation tests (Fig. 1). If a test animal is first exposed to aconcentration of 100%, excessive toxicity could be induced. For thisreason, the first pre-screen test was performed at 25%, and in thesecond test, the highest concentration was determined. If notoxicity or irritation were found at 25%, only 50% and 100% con-centrations were needed in the second test, not requiring lowerconcentrations (less than 25%). If toxicity or irritationwere found at25%, concentrations need to be lowered and severe toxicity wasinduced at 50% or 100%, thereby relieving more pain and distress inlaboratory animals and reducing animal testing.

2.4. Statistical analysis

Raw data obtained from each participating laboratory wererecorded in a template (Excel format) prepared by the VMT. Datawere sent directly sent to the VMT via e-mail. Descriptive statistics(WLR and BLR) were analyzed to evaluate whether the reportedestimated concentration of a test substance required to produce anSI of 2.7 (EC2.7) was within the 0.5 � e2.0 � ECt range specified inthe OECD TG 429 Annex 1 PS. ECt values were calculated by asimple linear interpolation method in which the x axis was set forconcentrations and the y axis for SI values. If an ECt value is below0% or above 100%, set y-intercept value as 1. Statistical analysis ofpredictive capacity focused on the concordance of predictions (yes/no), allowing the subsequent calculation of sensitivity, specificity,and overall accuracy. Receiver operating curve (ROC) analysis setthe optimal cutoff for the SI, which allowed test compounds to becategorized as sensitizers or non-sensitizers during the predictivecapacity phase (ROC analysis is commonly used to obtain optimalcutoff values for diagnostic tests) (Hanley and McNeil, 1982; Linet al., 2002). EC2.7 was also used to evaluate the WLR and BLR.

3. Results

3.1. Training and transfer of the test method

Lead laboratory 1 trained participating laboratories 1 and 2regarding protocol 1.0 and demonstrated flow cytometry operationand the way to measure BrdU content. Lead laboratory 1 then

Fig. 1. Procedure for the selection of the highest concentration. The serious irritation indicates 25% or more increase of ear thickness, or erythema (3 or more score), and thesystemic toxicity indicates death or weight loss (a decrease of more than 5% from Day 1 to Day 6).

Table 2Results of the transferability evaluation conducted by lead laboratory 2 and participating laboratories 1 and 2.

Experiment SI* (per animal) SI (mean ± SD) Result

Participating lab 1a 1 4.8, 8.9, 15.8, 18.1 11.9 ± 6.1 Pass2 1.3, 5.5, 14.4, 16.3, 17.7 11.0 ± 7.3 Pass

Participating lab 2a 1 10.6, 13.0, 14.3 12.6 ± 1.9 Pass2 13.5, 20.9, 25.8, 27.6, 29.8 23.5 ± 6.5 Pass

Lead lab 2b 1 3.8, 5.4, 7.0, 8.0 6.1 ± 1.8 Pass

* The SI for 25% HCA was calculated for each mouse.SD, standard deviation; SI, Stimulation Index.

a Performed with protocol 1.0.b Performed with protocol 1.2.

I. Ahn et al. / Regulatory Toxicology and Pharmacology 80 (2016) 183e194 187

confirmed each laboratory’s performance, such as if the assay wasclearly understood and properly implemented. A successful trans-fer of the test method was confirmed in accordance with OECD TG429, using 25% HCA as a positive control and AOO as a vehiclecontrol. Lead laboratory 2 also received theoretical and proceduraltraining from lead laboratory 1 later on. Lead laboratory 2 con-ducted a test using 25% HCA and AOO to check if the test methodwas properly transferred. All laboratories met the acceptancecriteria for transferability, namely, the SI for 25% HCA was �3(Table 2).

3.2. Demonstration of proficiency in the test method

Each laboratory conducted a proficiency test using coded testchemicals with protocol 1.0. The proficiency of each participatinglaboratory was evaluated using 25% HCA and 5%, 10%, and 25%

Table 3Proficiency test results submitted by lead laboratory 2 and participating laboratories 1 a

Eugenol concentration (%)

Participating lab 1 a 51025

Participating lab 2 a 51025

Lead lab 2 b 51025

* EC3: Estimated concentration of a test substance needed to produce a Stimulation IndeSD, standard deviation; SI, Stimulation Index.

a Performed with protocol 1.0 (5 animals per group were tested).b Performed with protocol 1.2 (4 animals per group were tested).

eugenol. The EC3 concentration of eugenol should be in the range of5.05e20.2% (0.5�e2� EC3), as described in OECD TG 429 Annex 1.Those chemicals were assigned to the laboratories, and their nameswere veiled until the test was completed. The information on avehicle (AOO) and test doses (5%, 10%, 25%) was given to eachlaboratory. Because 0.5 � e2 � ECt values for eugenol were in therange of 5.05e20.2% at each participating laboratory (participatinglaboratory 1: 8.2% and 2: 11.1%), the VMT concluded that theirproficiency was validated. Lead laboratory 2 also performed aproficiency test. Its proficiency was demonstrated because ECtvalues for eugenol (13.3%) were in the range of 5.05e20.2%(Table 3).

3.3. Within- and between-laboratory reproducibility

TheWLR and BLR were evaluated with the participation of three

nd 2.

SI (mean ± SD) EC3 * Result

1.3 ± 0.3 8.2 Pass4.4 ± 1.78.6 ± 2.42.1 ± 0.7 11.1 Pass2.5 ± 1.15.8 ± 2.21.3 ± 0.4 13.3 Pass2.8 ± 1.34.9 ± 0.5

x of 3.

Table

4Agree

men

tbe

twee

ntheclassification

predictedon

thethirdpredictivecapacitytest

*an

dthat

published

inOEC

DTG

429.

No.

Chem

ical

nam

eOEC

DTG

429

LLNA:BrdU-FCM

Agree

men

t

Categ

ory

EC3

Veh

icle

Categ

ory

Con

c.(%)

SIva

lues

(mea

n±SD

)MAX.S

IEC

2.7

Veh

icle

NC

LM

HPC

1CMI/MI

Cat1A

0.00

9DMF

Cat1A

2.5,5,10

1.00

±0.32

9.99

±3.45

13.42±1.85

13.55±1.77

6.24

±1.44

13.55

1.06

2*DMF

Yes

2DNCB

Cat1A

0.04

9AOO

Cat1A

0.12

5,0.25

,0.5

1.00

±0.39

15.57±4.25

38.97±6.12

47.29±9.45

9.01

±1.75

47.29

0.01

6*AOO

Yes

34-Ph

enylen

ediamine

Cat1A

0.11

AOO

Cat1A

0.5,1,2.5

1.00

±0.32

3.24

±1.34

6.48

±1.48

10.02±3.26

6.24

±1.44

10.02

0.10

1DMF

Yes

4Cob

altch

loride

Cat1A

0.6

DMSO

Cat1A

0.25

,0.5,1.0

1.00

±0.25

2.60

±1.00

7.80

±4.06

12.83±6.03

11.86±3.15

12.83

0.19

9DMF

Yes

5Isoe

uge

nol

Cat1A

1.5

AOO

Cat1A

5,10

,25

1.00

±0.32

7.66

±1.33

19.3

±4.37

34.91±11

.73

5.42

±1.48

34.91

1.19

8*AOO

Yes

62-Mercaptobe

nzo

thiazo

leCat1A

1.7

DMF

NI

5,10

,25

1.00

±0.25

1.44

±0.58

1.13

±0.27

1.30

±0.50

11.86±3.15

1.44

eDMF

No

7Citral

Cat1B

9.2

AOO

Cat1B

10,25,50

1.00

±0.32

1.98

±1.02

5.05

±1.85

8.88

±4.43

5.42

±1.48

8.88

13.08

AOO

Yes

8HCA

Cat1B

9.7

AOO

Cat1B

5,10

,25

1.00

±0.27

1.14

±0.40

1.75

±0.95

4.34

±0.74

6.50

±3.78

4.34

15.11

AOO

Yes

9Eu

genol

Cat1B

10.1

AOO

Cat1B

5,10

,25

1.00

±0.38

0.71

±0.19

2.01

±0.75

3.94

±0.74

3.89

±3.28

3.94

16.46

AOO

Yes

10Ph

enyl

benzo

ate

Cat1B

13.6

AOO

Cat1B

10,25,50

1.00

±0.39

6.85

±1.96

9.44

±3.00

15.41±5.82

9.01

±1.75

15.41

5.53

7*AOO

Yes

11Cinnam

icalco

hol

Cat1B

21AOO

Cat1B

10,25,50

1.00

±0.38

0.49

±0.15

2.29

±0.89

2.78

±0.65

3.89

±3.28

2.78

44.28

AOO

Yes

12Im

idazolidinyl

urea

Cat1B

24DMF

Cat1B

10,25,50

1.00

±0.35

1.05

±0.37

2.11

±1.04

4.10

±1.52

12.27±4.27

4.10

32.02

DMF

Yes

13Methyl

methacrylate

Cat1B

90AOO

NI

25,50,10

01.00

±0.27

0.57

±0.10

0.65

±0.19

0.87

±0.34

6.50

±3.78

0.87

eAOO

No

14Chlorobe

nze

ne

NI

NA

AOO

NI

10,25,50

1.00

±0.39

1.12

±0.49

1.25

±0.36

1.67

±0.23

9.01

±1.75

1.67

eAOO

Yes

15Isop

ropan

olNI

NA

AOO

NI

25,50,10

01.00

±0.32

0.86

±0.29

0.83

±0.24

0.89

±0.27

5.42

±1.48

0.89

eAOO

Yes

16Lactic

acid

NI

NA

DMSO

NI

10,25,50

1.00

±0.25

1.15

±0.32

1.18

±0.16

1.28

±0.50

11.86±3.15

1.28

eDMF

Yes

17Methyl

salic

ylate

NI

NA

AOO

NI

10,25,50

1.00

±0.38

1.77

±0.33

1.61

±0.52

1.14

±0.27

3.89

±3.28

1.77

eAOO

Yes

18Sa

licylic

acid

NI

NA

AOO

NI

1.0,2.5,5.0

1.00

±0.35

1.76

±0.49

2.26

±0.33

2.57

±0.22

12.27±4.27

2.57

eDMF

Yes

S,sensitize

r;N,non

-sen

sitize

r;AOO,aceton

e:olive

oil(4:1);

CMI/MI,

5-Chloro-2-m

ethyl-4-isothiazo

lin-3-one/2-methyl-4-isothiazo

lin-3-one;

DMF,

N,N-dim

ethylform

amide;

DMSO

,dim

ethyl

sulfox

ide;

DNCB,2,4-

dinitroch

lorobe

nze

ne;

HCA,h

exyl

cinnam

icaldeh

yde;

SD,standarddev

iation

;SI,S

timulation

Index

.*Pe

rformed

withprotoco

l1.3(5

anim

alsper

grou

pweretested

).**Se

ty-ax

is¼

1be

cause

anEC

2.7va

lueis

below

0%or

abov

e10

0%.

I. Ahn et al. / Regulatory Toxicology and Pharmacology 80 (2016) 183e194188

laboratories. In addition, the cut-off value for the test methodwas optimized to meet the acceptance criteria of OECD TG429 PS with the data using statistical approaches that we re-ported previously (Yang et al., 2015). Briefly, the statisticalanalysis of the WLR and BLR results indicated that EC2.5, 2.6, or2.7 was better than EC3.

The VMT concluded that the WLR and BLR of the LLNA:BrdU-FCM was successfully demonstrated (Supplementarymaterial 2 and 3). Lead laboratory 2, which joined the validationstudy in May 2014, did not perform a WLR test because theresults abovemet the acceptance criteria ofWLR and BLR statedin OECD 429 PS and unnecessary animal tests needed to beavoided in accordance with the 3R principle.

Besides the BLR results performed according to the OECD TG429 PS (Supplementary material 3), BLR could be furtherconfirmed since many test substances were concurrently usedat three predictive capacity tests by three different laboratories.Five test substances including isopropanol, methyl methacry-late, phenyl benzoate, cinnamic alcohol, and chlorobenzenewere consistent for prediction among the three laboratories(lead lab. 1 at the first predictive capacity test, lead lab. 2 orparticipating lab. 1 or participating lab. 2 at the second one,participating lab. 2 at the third one). Furthermore, 2-mercaptobenzothiazole was unanimously predicted as non-sensitizer at all three laboratories (lead lab. 1 at the first pre-dictive capacity test, lead lab. 2, participating lab. 1 & 2 at thesecond predictive capacity test, participating lab. 2 at the thirdone). (Supplementary material 4, 5, and Table 4).

In addition, as the result of the second and third tests per-formed at participating lab. 2, lactic acid, methyl salicylate and2-mercaptobenzothiazole were all classified into non-sensitizers, thereby further confirming the WLR of the testmethod.

3.4. Predictive capacity

The predictive capacity was evaluated from 2012 to 2014,based on OECD TG 429 Annex 1, and the 18 reference chemicalslisted in Table 1 were tested with protocol version 1.1, 1.2, and1.3. Concordance rates for the reference chemicals were calcu-lated. The four optional substances listed in Table 1 were alsotested with protocol 1.2, thereby demonstrating that the per-formance of the LLNA: BrdU-FCM was comparable to those ofthe existing LLNA test methods.

The first test for the predictive capacity evaluation wasconducted at lead laboratory 1 with protocol 1.1 from 2012 to2013. As a result of the first test for the predictive capacityevaluation, sensitivity was 76.9% and specificity was 60%.Therefore, the overall accuracy of the LLNA: BrdU-FCM reached72.2%. Among those 13 skin sensitizers, 2-mercaptobenzothiazole (moderate sensitizer), imidazolidinylurea, and methyl methacrylate (weak sensitizers) were mis-classified. Among the five non-sensitizing substances, chloro-benzene and methyl salicylate were misclassified(Supplementary material 4). After discussion, the VMT askedlead laboratory 2 and participating laboratories 1 and 2 toevaluate the predictive capacity after modifying the protocol.

The second test was conducted at the three laboratories,with protocol 1.2 in 2014. As a result, chlorobenzene andmethylsalicylate were correctly evaluated as negatives. 2-mercaptobenzothiazole, imidazolidinyl urea, and methylmethacrylate were evaluated as false negatives in the secondtest. The lowered test concentrations (50, 25,10%) with protocol1.2 for chlorobenzene could presumably contribute toward thecorrection since 75% test concentration was used for this test

Fig. 2. Stimulation index values of the 18 substances. The third test for the predictivecapacity evaluation was conducted at participating laboratory 2, using five mice foreach concentration. Symbols areB for vehicle control,C for low concentration, - formiddle concentration, : for high concentration, and ✕ for positive control. Abbre-viations: CMI/MI, 5-Chloro-2-methyl-4-isothiazolin-3-one/2-methyl-4-isothiazolin-3-one; DNCB, 2,4-dintrochlorobenzene; PD, 4-Phenylenediamine; CC, Cobalt chloride;IE, Isoeugenol; MBT, 2-Mercaptobenzothiazole; HCA, Hexyl cinnamic aldehyde; PB,Phenyl benzoate; CA, Cinnamic alcohol; IU, Imidazolidinyl urea; MM, Methyl meth-acrylate; CB, Chlorobenzene; IP, Isopropanol; LA, Lactic acid; MS, Methyl salicylate; SA,Salicylic acid.

I. Ahn et al. / Regulatory Toxicology and Pharmacology 80 (2016) 183e194 189

substance at the first predictive capacity evaluation. Furthermore,the flow cytometric analysis procedure was described in moredetail in protocol 1.2 for more accurate calculation of BrdU-incorporating cells, which might contribute toward on the correc-tion for methyl salicylate. As for the four optional chemicals,ethylene glycol dimethacrylate, xylene, and sodium lauryl sulfatewere classified as sensitizers, whereas nickel chloridewas classifiedas a non-sensitizer. Xylene and sodium lauryl sulfate indicated thesame false-positive results, and nickel chloride indicated the samefalse-negative results as those in the LLNA. (Supplementary mate-rial 5). The predictive capacity of the LLNA: BrdUeFCM was eval-uated previously using the combined data obtained from the firstand second tests (Kim et al., 2016). In addition, this paper showedthat the optimum cut-off value of the test method is 2.7 among 2.5,2.6, or 2.7, as suggested in WLR and BLR through establishing ROCcurves.

1. In response to the VMT’s suggestion, lead laboratory 2 per-formed an additional test using imidazolidinyl urea with modifiedprotocol (version 1.3) in 2015. The laboratory chose DMF as avehicle and 50% as the highest concentration for imidazolidinylurea according to the solubility test protocol. As a result of the test,its SI was 3.0 at 25% and 3.5 at 50%. EC2.7 was 26.8%, so imidazo-lidinyl urea was classified as a weak sensitizer.

The third test for the predictive capacity evaluation was con-ducted at participating laboratory 2 with protocol 1.3 in 2015. Theresults are described in Fig. 2 and Table 4. The 18 referencechemicals suggested in OECD TG 429 were chosen for the third test.Compared with the LLNA, sensitivity was 84.6%, specificity was100%, and overall accuracy was 88.9%. The results for a moderatesensitizer (2-mercaptobenzothiazole) and a weak sensitizer(methyl methacrylate) among the 13 skin sensitizers were incon-sistent with the classification reported in the LLNA and discordantfrom the human prediction as well, whereas the results for theother 11 chemicals were in concordance with the LLNA. All fivenon-sensitizers indicated the same results as those in the LLNA PS.Imidazolidinyl urea, which was falsely predicted in the second testwith protocol 1.2, was correctly classified as a sensitizer with an SIof 4.1 at 50%. Its EC2.7 was 32.0%, which is in the ECt range(12e48%) suggested in OECD TG 429 Annex 1. Considering thepredictive relevancy of LLNA: BrdU-FCM with other LLNA-basedtest methods or human classification, Cooper statistics werecalculated on the 18 reference test substances (Table 5). Whencompared with human classification, the LLNA: BrdU-FCMdemonstrated 78.6% sensitivity, 100% specificity, and 83.3% overallaccuracy. The three test substances, i.e., 2-mercaptobenzothiazole,methyl methacrylate and isopropanol, were misclassified as non-sensitizers in the LLNA: BrdU-FCM.

3.5. Evaluation of cut-off values

The optimal SI threshold was determined by statisticallyanalyzing the results of the WLR and BLR tests and the first andsecond predictive capacity evaluation (Kim et al., 2016; Yang et al.,2015). As a result, the optimal sensitivity and specificity weregenerated at 2.66 � cut-off SI < 4.66. In view of the results of theBLR, WLR, and first and second predictive capacity evaluation, 2.7was selected among the SI values (2.5, 2.6, and 2.7) because thehighest sensitivity was produced. That was reflected in the finalprotocol version 1.3.

The ROC analysis was re-conducted for the third test results. Thearea under curve (AUC) of ROC was 0.877, which was close to the0.885 calculated from the first and second tests’ combined data. Inaddition, the highest accuracy was produced at 2.6 � SI < 2.8(Fig. 3), which included 2.7, further supporting that the predictivecapacity was optimized at a cut-off value, SI 2.7.

4. Discussion

The WLR, BLR and predictive capacity of the LLNA: BrdU-FCMwere evaluated in accordance with the Performance Standards inOECD GD 34 and TG 429. It fully complied with the essential testmethod components described in the PS. All tests were undertakenby four laboratories over 4 years. After completing the secondpredictive capacity evaluation, the experts of the VMT and

Table 5Cooper’s statistics on LLNA: BrdU-FCM vs. LLNA, and LLNA: BrdU-FCM vs. Human data.

LLNA HumanSensitizer Non-sensitizer Total Sensitizer Non-sensitizer Total

LLNA: BrdU-FCM Sensitizer 11 0 11 11 0 11Non-sensitizer 2 5 7 3 4 7Total 13 5 18 14 4 18

Overall accuracy: 88.9%Sensitivity: 84.6%Specificity: 100%

Overall accuracy: 83.3%Sensitivity: 78.6%Specificity: 100%

LLNA: DA LLNA: BrdU-ELISASensitizer Non-sensitizer Total Sensitizer Non-sensitizer Total

LLNA: BrdU-FCM Sensitizer 11 0 11 11 0 11Non-sensitizer 4 3 7 2 3 5Total 15 3 18 13 3 16

Overall accuracy: 77.8%Sensitivity: 73.3%Specificity: 100%

Overall accuracy: 87.5%Sensitivity: 84.6%Specificity: 100%

I. Ahn et al. / Regulatory Toxicology and Pharmacology 80 (2016) 183e194190

International Cooperation of Alternative Test Methods suggestedevaluating the overall predictive capacity in accordance with thefinal version of the protocol, so the third test was conducted withprotocol 1.3. The sensitivity, specificity and predictive capacity ofour test method are 85%, 100% and 89%, respectively. The results forimidazolidinyl ureawere inconsistent with those in the LLNA in thesecond test, but in the third test the substance indicated the sameresults as those in the LLNA. Except for sensitizers 2-mercaptobenzothiazole and methyl methacrylate, which wereclassified as non-sensitizers, 16 among the 18 reference chemicalslisted in the OECD TG 429 Annex 1 PS indicated the same results.

Methylmethacrylate, aweak sensitizer referred in OECD TG 429,was once reported as a non-sensitizer with the LLNA and LNCC(lymph node cell count) method (Basketter et al., 2011). Indeed, thechemical is highly likely to be classified into a non-sensitizer sinceit was evaluated as a sensitizing substance at rather high 90%, 99%and 79% concentration in the LLNA, the LLNA: DA and the LLNA:BrdU-ELISA, respectively (ICCVAM, 2010a, 2010b). In addition, hu-man data for methyl methacrylate are insufficient (Basketter et al.,2014b; Teunis et al., 2014), and it was included in the referencechemicals in TG 429 on the basis of just one scientific paper (OECD,2010a). Kimber and Pemberton (2014) recently concluded methylmethacrylate as a contact allergen possessing no more than weakskin sensitizing potency. Taken together, very weak skin sensitizersmight be falsely predicted by the LLNA: BrdU-FCM.

OECD TG 429, in which 2-mercaptobenzothiazole was classifiedas a sensitizer, was also based on the results of only one scientificpaper (OECD, 2010a). In addition, the substance was considered tobe on the borderline as a sensitizer by the LLNA: DA and LLNA:BrdU-ELISA because its SI was 2.00 and 1.62, respectively (OECD,2010b, 2010c) (Table 6). This chemical was also defined as a veryweak sensitizing substance as its human No Observed Effect Level(NOEL) was 2269 mg/cm2, which is higher than 2000 mg/cm2 for aweak sensitizer, imidazolidinyl urea (Basketter et al., 2014b; Teuniset al., 2014). According to categorization of chemicals based onrelative human skin sensitizing potency (Basketter et al., 2014a), 2-mercaptobenzothiazole was categorized into the group 3. Chem-icals placed in the category 3 were defined to be contact allergenswith 500e2500 mg/cm2 NOEL value in which lower concentrationsof these chemicals may sensitize only small proportion or none ofthe exposed. Considering together, the LLNA: BrdU-FCM may havedifficulty to correctly classify weak or borderline chemicals assensitizers.

Isopropanol has been defined as a very weak human sensitizer,and also described as a human sensitizer in OECD TG 429. Thesubstance was classified into a non-sensitizer in the LLNA: BrdU-FCM as well as the LLNA and GPMT. Moreover, its human NOEL

value is not known due to insufficient data (Basketter et al., 2014;Teunis et al., 2014).

As for imidazolidinyl urea, which was suggested as a sensitizerin OECD TG 429, it was classified as a sensitizer at 50% because its SIwas 3.1 at 25% and 5.5 at 50% (Basketter and Scholes, 1992). Inaddition, its SI was on the borderline (1.6) in the LLNA: BrdU-ELISA.It was also classified as a weak sensitizer in the recently adoptedKeratinosens assay (EURL ECVAM, 2014) and direct peptide reac-tivity assay (DPRA) (EURL ECVAM, 2012; Natsch et al., 2013). Its SIwas 2.6 (lead laboratory 2) and 2.1 (participating laboratory 2) atthe highest concentration 50% in the second test of the predictivecapacity evaluation. The VMT suggested revising the protocol inorder to improve the solubility of imidazolidinyl urea throughextending the magnetic stirring time from 30 min to 60 min forsuch a highly viscous chemical. As a result of the test, SI values were3.0 at 25% and 3.5 at 50%, and EC2.7 was 26.8%. Thus, it wascorrectly classified as a weak sensitizer.

The OECD TG 429 PS classifies nickel chloride as a false-negativesubstance and sodium lauryl sulfate and xylene as false-positivesubstances. The potential for skin sensitization depends on theability of a test substance to penetrate the skin; therefore, hydro-philic metal compounds, such as nickel chloride, might not beeasily absorbed, resulting in a false-negative result (Basketter et al.,1995). In addition, nickel-induced skin sensitization was reportedthrough activation of human Toll-like receptor 4 (TLR4), whereasmouse TRL4 could not mediate this response (Schmidt et al., 2010).In this regard, the LLNA: BrdU-FCM assay was no better than thetraditional LLNA. Sodium lauryl sulfate, which is believed to be anon-sensitizing irritant, induced a weak positive response(EC2.7 ¼ 3.04) in the LLNA: BrdU-FCM. This misclassification alsooccurs when using the traditional LLNA (OECD, 2010b). Eventhough the LLNA: BrdU-FCM was no better than the traditionalLLNA with respect to the classification of SLS as a skin non-sensitizer, both of these alternative, in vivo-based, skin sensitiza-tion methods suggest that SLS-mediated positive responses occurthrough an as-yet-unknown, but probably non-immune-mediated,mechanism (Basketter et al., 1996).

Some substances (e.g., chlorobenzene, salicylic acid, and lacticacid), which were evaluated as false positives in the LLNA: DA andthe LLNA: BrdU-ELISA (ICCVAM, 2010a, 2010b), were correctlyclassified in the LLNA: BrdU-FCM. In addition, this test method canevaluate skin sensitization potential of chemicals in combinationwith the in vitro tests newly developed based on AOPs and theIntegrated Approaches to Testing and Assessment. This assay canalso correctly classify the test substances that are false-negative inthe in vitro tests; therefore, it is expected to complement thosetests.

Fig. 3. ROC curve in the third predictive capacity evaluation at participating lab 2 using18 reference chemicals according to protocol 1.3, and predictive capacity parameters ateach cut-off (AUC ¼ 0.877).

I. Ahn et al. / Regulatory Toxicology and Pharmacology 80 (2016) 183e194 191

Furthermore, the LLNA: BrdU-FCM uses a new mouse strain(BALB/c) and new analysis method (flow cytometry). This testmethod uses BALB/c mice, which are relatively cost-efficientcompared with CBA mice. In some countries, the price of CBAmice is high because they must be imported, making it harder to

obtain than widely used BALB/c. In addition, no significant differ-ence between those twomouse strains in terms of the prediction ofskin sensitization potency has been reported (Burns et al., 2010;Jung et al., 2010; Hou et al., 2015). DBA/2, B6C3F1, and BALB/cwere cited as appropriate mouse species, which could be addi-tionally used in the LLNA (Woolhiser et al., 2000). There was nostatistical difference in the test results of the LLNA: BrdU-ELISAusing BALB/c and CBA/JN, respectively (Hou et al., 2015).

In general, before the initiation of LLNA tests, a pre-screen test isrequired if a chemical’s toxicological information (e.g. acute toxicityand dermal irritation) is not available. The concentrations recom-mended in TG 429 and 442A/B (100%, 50%, 25%, 10%, 5%, 2.5%, 1%,0.5%) are commonly chosen. It is hard to identify at which con-centration an unknown chemical induces acute toxicity or dermalirritation without conducting a pre-screen test. However, if infor-mation on quantitative structure-activity relationship or irritationis available, the concentration range can be narrowed, or a pre-screen test will be no required. The LLNA: BrdU-FCM newlyestablished a new pre-screen test that is performed in two phases(Figs. 1 and 4 and Table 7). If the pre-screen test is conductedfollowing OECD TG 429, 442A/B, 9 mice (n ¼ 1/group) or 18 mice(n ¼ 2/group) will be needed with 9 concentrations. But if the pre-screen test is performed in two phases, the number of mice will bereduced by 1~8. In particular, with less-toxic chemicals of highconcentration, the number of mice will be reduced by up to 8(Fig. 4). But it should be noted that the number of animals dependson the number of selected concentrations. This pre-screen strategycould be also applied to the other LLNA-based skin sensitizationvalidation methods.

In addition, there is no significant difference in the entire testperiods for the LLNA: BrdU-ELISA and LLNA: BrdU-FCM. That isbecause the process of drying samples is not needed in the LLNA:BrdU-FCM, even though it contains the process of counting sam-ples. Moreover, substrates’ reactivity to reagents lasts long in theLLNA: BrdU-ELISA, so stop solutions are required sometimes. Incontrast, there is no such process in the LLNA: BrdU-FCM (Sup-plementary material 6 and 7).

The LLNA: BrdU-FCM was developed primarily to identify skinsensitizing substances. However, the assay could also be used toevaluate skin sensitizing potency, which is described in the UnitedNations (UN) Globally Harmonized System of Classification andLabeling of Chemicals (GHS) (UN, 2009). The results of potency sub-categorization in accordance with the GHS (1A or 1B) of the LLNAindicated that discordant results were produced for only twochemicals in the LLNA: BrdU-FCM (Table 4). Among four extreme orstrong skin sensitizers mentioned in the OECD TG 429 PS, DNCB, 4-phenylenediamine, and cobalt chloride were classified as strong orextreme skin sensitizers in that their EC2.7 values were 0.016, 0.101,and 0.199, respectively. 5-Chloro-2-methyl-4-isothiazolin-3-one/2-methyl-4-isothiazolin-3-one (CMI/MI) was classified as moderate(1.062). Test substances with an ECt <0.1 are generally consideredextreme skin sensitizers, and test substances with an ECt value(10e100) are thought to be weak skin sensitizers (Kern et al., 2010).Among the nineweak skin sensitizers in the OECD TG 429 PS (citral,HCA, eugenol, phenyl benzoate, cinnamic alcohol, imidazolidinylurea, methyl methacrylate, ethylene glycol dimethacrylate, andxylene), five (phenyl benzoate, cinnamic alcohol, imidazolidinylurea, ethylene glycol dimethacrylate, and xylene) were classified asweak skin sensitizers and three (citral, HCA, and eugenol) asmoderate skin sensitizers by the LLNA: BrdU-FCM (i.e., none ofthese chemicals were classified as extreme or strong skinsensitizers).

An additional test using the chemicals that are similar to thefalsely classified ones in terms of chemical or physical propertiesand skin sensitization potency could be performed to demonstrate

Table 6Comparison of the predictive capacities of the LLNA, the LLNA: DA, the LLNA: BrdU-ELISA, and the LLNA: BrdU-FCM for the 22 reference chemicals in the OECD TG 429 PS.

No. ChemicalHuman/

Guinea pig

LLNAa)

(EC3)

LLNA: BrdU-FCM

(EC2.7)

LLNA: DAb)

(highest SI)

LLNA: BrdU-ELISAc)

(highest SI)

1 CMI/MI +/++

(0.01)

+*

(1.062)

+

(7.50)

+

(4.83)

2 DNCB +/++

(<0.025)

+*

(0.016)

+

(9.96)

+

(6.84)

3 4-Phenylenediamine +/++

(0.1)

+*

(0.101)

+

(5.14)

+

(14.70)

4 Cobalt chloride +/++

(<0.25)

+*

(0.199)

+

(4.25)

+

(3.68)

5 Isoeugenol +/++

(2.2)

+*

(1.198)

+

(7.09)

+

(6.73)

6 2-Mercaptobenzothiazole +/++

(4.6)-* +***

(2.00)

+***

(1.62)

7 Citral +/+ +

(12.6)

+*

(13.08)

+

(4.40)

+

(16.35)

8 HCA +/+ +

(4.6)

+*

(15.11)

+

(5.50)

+

(3.40)

9 Eugenol +/+ +

(9.2)

+*

(16.46)

+

(7.07)

+

(3.30)

10 Phenyl benzoate +/+ +

(8.9)

+*

(5.537)

+

(4.24)

+

(3.37)

11 Cinnamic alcohol +/+ +

(25.2)

+*

(44.28)

+

(5.66)

+

(2.74)

12 Imidazolidinyl urea +/+ +

(15.9)

+*

(32.09)

+

(4.67)

+***

(1.61)

13 Methyl methacrylate +/+ - -* +***

(1.81)NA

14 Chlorobenzene -/-+

(45.6)-* +

(2.44)NA

15 Isopropanol +/- - -* -

(1.21)

-

(1.01)

16 Lactic acid -/- - -* -

(0.97)

+

(1.89)

17 Methyl salicylate -/-+

(32.8)-* -

(1.55)

-

(1.43)

18 Salicylic acid -/-+

(8.0)-* +***

(2.00)

-

(1.26)

19 Sodium lauryl sulfate -/-+

(2.9)

+**

(3.04)

+

(3.39)

+

(2.64)

20Ethylene glycol

dimethacrylate+/-

+

(45.1)

+**

(49.7)

+

(4.45)

+

(3.11)

21 Xylene -/ +

(39.1)

+**

(58.5)NA NA

22 Nickel chloride +/++

(3.5)-** -

(1.30)NA

NA, not available; CMI/MI, 5-Chloro-2-methyl-4-isothiazolin-3-one / 2-methyl-4-isothiazolin-3-one; DNCB, 2,4-dinitrochlorobenzene; HCA, hexyl

cinnamic aldehyde.

a) Basketter et al., 2011.

b) ICCVAM, 2010a.

c) ICCVAM, 2010b.

*3rd predictive capacity test with protocol 1.3.

**2nd predictive capacity test with protocol 1.2.

***Borderline value.

I. Ahn et al. / Regulatory Toxicology and Pharmacology 80 (2016) 183e194192

Fig. 4. Numbers of mice used for the pre-screen tests in the LLNA (OECD TG 429), the LLNA: DA (TG 442A), the LLNA: BrdU-ELISA (TG 442B), and the LLNA: BrdU-FCM.

Table 7Concentration series and number of animals used for the pre-screen test in the third predictive capacity evaluation with protocol 1.3.

No. Chemical name Concentration series (%) No. of animals (n ¼ 2for each concentration)**

1st step in pre-screen test 2nd step in pre-screen test

1 CMI/MI 25 10, 5, 2.5, 1, 0.5 16 (100% & 50% avoided)2 DNCB 25 10, 5, 2.5, 1, 0.5 16 (100% & 50% avoided)3 4-Phenylenediamine 25 10, 5, 2.5, 1, 0.5 16 (100% & 50% avoided)4 Cobalt chloride Insoluble 10, 5, 2.5, 1, 0.5 12 (100% & 50% avoided)5 Isoeugenol 25 100, 50 106 2-Mercaptobenzothiazole 25 50* 87 Citral 25 100, 50 108 HCA 25 100, 50 109 Eugenol 25 100, 50 1010 Phenyl benzoate 25 50* 811 Cinnamic alcohol 25 50* 812 Imidazolidinyl urea 25 50* 813 Methyl methacrylate 25 100, 50 1014 Chlorobenzene 25 100, 50 1015 Isopropanol 25 100, 50 1016 Lactic acid 25 100, 50 1017 Methyl salicylate 25 100, 50 1018 Salicylic acid 25 10, 5, 2.5, 1, 0.5 16 (100% & 50% avoided)

CMI/MI, 5-Chloro-2-methyl-4-isothiazolin-3-one/2-methyl-4-isothiazolin-3-one; DNCB, 2,4-dinitrochlorobenzene; HCA, hexyl cinnamic aldehyde; NT, not tested.*100%: insoluble.**Negative control group (n ¼ 2) was included in each step of the pre-screen test.

I. Ahn et al. / Regulatory Toxicology and Pharmacology 80 (2016) 183e194 193

that LLNA: BrdU-FCM can achieve the accuracy recommended inthe OECD TG 429 PS. Furthermore, other endpoints for evaluatingskin sensitization potential would need to be investigated toanalyze if this method could be used for the GHS categorization. Inaddition, further studies (e.g., analysis of specific lymphocyte acti-vation related to skin sensitization [T cell-specific surface markersor cytokines]) would be interesting to check the utility of the LLNA:BrdU-FCM for the characterization of the skin sensitizing potentialof test substances.

5. Conclusion

The purpose of this validation study was to propose the LLNA:BrdU-FCM as an alternative test method of the LLNA (OECD TG

429), LLNA: BrdU-ELISA (OECD TG 442B), and LLNA: DA (OECD TG442A) to international organizations. To this end, the trans-ferability, WLR, BLR, and predictive capacity of the test methodwere evaluated. As a result of the evaluation, the VMT concludedthat the LLNA: BrdU-FCM could be an appropriate test method foridentifying skin sensitization potential, compared with the otherLLNA methods (Supplementary material 6 and 7).

Acknowledgments

This research was supported by a grant (12182MFDS791,13172MFDS987, and 15181MFDS457) from theMinistry of Food andDrug Safety of Korea. We especially thank Eun-Kyung Ku for herkind grammatical check of the manuscript.

I. Ahn et al. / Regulatory Toxicology and Pharmacology 80 (2016) 183e194194

Transparency document

Transparency document related to this article can be foundonline at http://dx.doi.org/10.1016/j.yrtph.2016.06.009.

Appendix A. Supplementary data

Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.yrtph.2016.06.009.

References

Basketter, D.A., Scholes, E.W., 1992. Comparison of the local lymph node assay withthe guinea-pig maximization test for the detection of a range of contact aller-gens. Food. Chem. Toxicol. 30, 65e69.

Basketter, D.A., Scholes, E.W., Chamberlain, M., Barratt, M.D., 1995. An alternativestrategy to the use of guinea pig for the identification of skin sensitizationhazard. Food Chem. Toxicol. 33, 1051e1056.

Basketter, D.A., Gerberick, G.F., Kimber, I., Loveless, S.E., 1996. The local lymph nodeassay: a viable alternative to currently accepted skin sensitization tests. FoodChem. Toxicol. 34, 985e997.

Basketter, D., Kolle, S.N., Schrage, A., Honarvar, N., Gamer, A.O., van Ravenzwaay, B.,Landsiedel, R., 2011. Experience with local lymph node assay performancestandards using radioactivity and nonradioactive cell count measurements.J. Appl. Toxicol. 32, 590e596.

Basketter, D.A., Al�ep�ee, N., Ashikaga, T., Barroso, J., Gilmour, N., Goebi, C.,Hibatallah, J., Hoffmann, S., Kern, P., Martinozzi-Teissier, S., Maxwell, G.,Reisinger, K., Sakaguchi, H., Schepky, A., Taihardat, M., Templier, M., 2014a.Categorization of chemicals according to their relative human skin sensitizingpotency. Dermatitis 25, 11e21.

Basketter, D.A., Gerberick, G.F., Kimber, I., 2014b. The local lymph node assay in2014. Dermatitis 25, 49e50.

Burns, T., Strickland, J., Salicru, E., Allen, D., Stokes, W., 2010. Comparison of LLNAresponses between CBA and BALB/c mouse strains. Toxicologist 114, 65.

EPA, 2003. Health Effects Test Guidelines. OPPTS 870.2600 Skin sensitization.EURL ECVAM, 2012. Direct Peptide Reactivity Assay (DPRA) Validation Study Report

p.74.Hanley, J.A., McNeil, B.J., 1982. The meaning and use of the area under a receiver

operating characteristics (ROC) curve. Radiology 143, 29e36.Hartung, T., Bremer, S., Casati, S., Coecke, S., Corvi, R., Fortaner, S., Gribaldo, L.,

Halder, M., Hoffmann, S., Roi, A.J., Prieto, P., Sabbioni, E., Scott, L., Worth, A.,Zuang, A., 2004. A modular approach to the ECVAM principles on test validity.ATLA 32, 467e472.

Hou, F., Xing, C., Li, B., Cheng, J., Chen, W., Zhang, M., 2015. Application of BALB/cmouse in the local lymph node assay:BrdU-ELISA for the prediction of the skinsensitizing potential of chemicals. J. Pharmacol. Toxicol. Methods 72, 53e58.

ICCVAM, 2010a. ICCVAM Test Method Evaluation Report. Nonradioactive LocalLymph Node Assay: BrdU-ELISA Test Method Protocol (LLNA: BrdU-ELISA). NIHPublication No. 10e7552A/B. Research Triangle Park, NC: National Institute ofEnvironmental Health Sciences.

ICCVAM, 2010b. ICCVAM Test Method Evaluation Report. Nonradioactive LocalLymph Node Assay: Modified by Daicel Chemical Industries, Ltd., Based on ATPContent Test Method Protocol (LLNA: DA). NIH Publication No. 10e7551A/B.Research Triangle Park, NC: National Institute of Environmental HealthSciences.

Jung, K.M., Bae, I.H., Kim, B.H., Kim, W.K., Chung, J.H., Park, Y.H., Lim, K.M., 2010.

Comparison of flow cytometry and immunohistochemistry in non-radioisotopicmurine lymph node assay using bromodeoxyuridine. Toxicol. Lett. 192,229e237.

Jung, K.M., Jang, H.W., Lee, Y.K., Yum, Y.N., Sohn, S.J., Kim, B.H., Chung, J.H., Park, Y.H.,Lim, K.M., 2012. B cell increases and ex vivo IL-2 production as secondaryendpoints for the detection of sensitizers in non-radioisotopic local lymph nodeassay using flow cytometry. Toxicol. Lett. 209, 255e263.

Kern, P.S., Gerberick, G.F., Ryan, C.A., Kimber, I., Aptula, A., Basketter, D.A., 2010.Local lymph node data for the evaluation of skin sensitization alternatives: asecond compilation. Dermatitis 21, 8e32.

Kim, D.E., Yang, H., Jang, W.H., Jung, K.M., Park, M., Choi, J.K., Jung, M.S., Jeon, E.Y.,Heo, Y., Yeo, K.W., Jo, J.H., Park, J.E., Sohn, S.J., Kim, T.S., Ahn, I.Y., Jeong, T.C.,Lim, K.M., Bae, S.J., 2016. Predictive capacity of a non-radioisotopic local lymphnode assay using flow cytometry, LLNA: BrdU-FCM: comparison of a cutoffapproach and inferential statistics. J. Pharmacol. Toxicol. Methods 78, 76e84.

Kimber, I., Pemberton, M.A., 2014. Assessment of the skin sensitizing potency of thelower alkyl methacrylate esters. Reg. Toxicol. Pharmacol. 70, 24e36.

Lin, W.Y., Lee, L.T., Chen, C.Y., Lo, H., Hsia, H.H., Liu, I.L., Lin, R.S., Shau, W.Y.,Huang, K.C., 2002. Optimal cut-off values for obesity: using simple anthropo-metric indices to predict cardiovascular risk factors in Taiwan. Int. J. Obes. Relat.Metab. Disord. 26, 1232e1238.

Natsch, A., Ryan, C.A., Foertsch, L., Emter, R., Jaworska, J., Gerberick, F., Kern, P., 2013.A dataset on 145 chemicals tested in alternative assays for skin sensitizationundergoing prevalidation. J. Appl. Toxicol. 33, 1337e1352.

OECD, 2005. OECD Series on Testing and Assessment No. 34. Guidance Documenton the Validation and International Acceptance of New or Updated TestMethods for Hazard Assessment. OECD, Paris, France.

OECD, 2010a. Skin Sensitization: Local Lymph Node Assay. OECD Guideline for theTesting of Chemicals No. 429, OECD, Paris. Annex 1 Performance Standards forAssessment of Proposed Similar or Modified LLNA Test Methods for SkinSensitization. OECD, Paris, France.

OECD, 2010b. Skin Sensitization: Local Lymph Node Assay: BrdU-ELISA. OECDGuideline for the Testing of Chemicals No. 442B. OECD, Paris, France.

OECD, 2010c. Skin Sensitization: Local Lymph Node Assay: DA. OECD Guideline forthe Testing of Chemicals No. 442A. OECD, Paris, France.

OECD, 2012. OECD Guidance Document 168: the Adverse Outcome Pathway for SkinSensitisation Initiated by Covalent Binding to Proteins. OECD, Paris, France.

REACH, 2006. Regulation (EC) No. 1907/2006 of the European Parliament and of theCouncil of 18 December 2006 Concerning the Registration, Evaluation,Authorisation and Restriction of Chemicals (REACH), Establishing a EuropeanChemicals Agency.

Schmidt, M., Raghavan, B., Muller, V., Vogl, T., Fejer, G., Tchaptchet, S., Keck, S.,Kalis, C., Nielsen, P.J., Galanos, C., Roth, J., Skerra, A., Martin, S.F.,Freudenberg, M.A., Goebeler, M., 2010. Crucial role for human Toll-like receptor4 in the development of contact allergy to nickel. Nat. Immunol. 11, 814e819.

Teunis, M.A.T., Spiekstra, S.W., Smits, M., Adriaens, E., Eltze, T., Galbiati, V., Krul, C.,Landsiedel, R., Pieters, R., Reinders, J., Roggen, E., Corsini, E., Gibbs, S., 2014.International ring trial of the epidermal equivalent sensitizer potency assay:reproducibility and predictive capacity. ALTEX 31, 251e268.

UN, 2009. Globally Harmonized System for the Classification and Labelling ofSubstances (GHS). Part 3: Health Hazards. New York, NU, USA, and Geneva,Switzerland. United Nations Economic Commission for Europe.

Woolhiser, M.R., Munson, A.E., Meade, B.J., 2000. Comparison of mouse strainsusing the local lymph node assay. Toxicology 146, 221e227.

Yang, H., Na, J., Jang, W.H., Jung, M.S., Jeon, J.Y., Heo, Y., Yeo, K.W., Jo, J.H., Lim, K.M.,Bae, S.J., 2015. Appraisal of within- and between-laboratory reproducibility ofnon-radioisotopic local lymph node assay using flow cytometry, LLNA: BrdU-FCM: comparison of OECD TG429 performance standard and statistical evalu-ation. Toxicol. Lett. 5 (234), 172e179.