behaviour research and therapy - wordpress.com...maurice topper a, b, *, paul m.g. emmelkamp a, c,...

lable at ScienceDirect

Behaviour Research and Therapy 90 (2017) 123e136

Contents lists avai

Behaviour Research and Therapy

journal homepage: www.elsevier .com/locate/brat

Prevention of anxiety disorders and depression by targeting excessiveworry and rumination in adolescents and young adults: A randomizedcontrolled trial

Maurice Topper a, b, *, Paul M.G. Emmelkamp a, c, Ed Watkins d, e, Thomas Ehring f

a University of Amsterdam, Department of Clinical Psychology, Weesperplein 4, 1018 XA Amsterdam, The Netherlandsb University of Amsterdam, Cognitive Science Center Amsterdam, Weesperplein 4, 1018 XA Amsterdam, The Netherlandsc The Netherlands Institute for Advanced Study, Korte Spinhuissteeg 3, 1012 CG, The Netherlandsd University of Exeter, School of Psychology, Mood Disorders Centre, Queens drive Exeter Ex4 4QQ, United Kingdome University of Western Australia, 35 Stirling Highway, Crawley WA 6009, Perth, Australiaf LMU Munich, Department of Psychology, Leopoldstr. 13, D80802 Munich, Germany

a r t i c l e i n f o

Article history:Received 8 November 2015Received in revised form16 September 2016Accepted 20 December 2016Available online 23 December 2016

Keywords:PreventionAnxietyDepressionWorryRumination

* Corresponding author. Maelsonstraat 1, 1624 NP,E-mail address: [email protected] (M. Topper)

http://dx.doi.org/10.1016/j.brat.2016.12.0150005-7967/© 2016 Elsevier Ltd. All rights reserved.

a b s t r a c t

Background: This randomized controlled trial evaluated the efficacy of a preventive intervention foranxiety disorders and depression by targeting excessive levels of repetitive negative thinking (RNT;worry and rumination) in adolescents and young adults.Methods: Participants (N ¼ 251, 83.7% female) showing elevated levels of RNT were randomly allocatedto a 6-week cognitive-behavioral training delivered in a group, via the internet, or to a waitlist controlcondition. Self-report measures were collected at pre-intervention, post-intervention, 3 m and 12 mfollow-up.Results: Both versions of the preventive intervention significantly reduced RNT (d ¼ 0.53 to 0.89), andsymptom levels of anxiety and depression (d ¼ 0.36 to 0.72). Effects were maintained until 12 m follow-up. The interventions resulted in a significantly lower 12 m prevalence rate of depression (groupintervention: 15.3%, internet intervention: 14.7%) and generalized anxiety disorder (group intervention:18.0%, internet intervention: 16.0%), compared to the waitlist (32.4% and 42.2%, respectively). Mediationanalyses demonstrated that reductions in RNT mediated the effect of the interventions on the prevalenceof depression and generalized anxiety disorder.Conclusions: Results provide evidence for the efficacy of this preventive intervention targeting RNT and sup-port a selective prevention approach that specifically targets a known risk factor to preventmultiple disorders.

© 2016 Elsevier Ltd. All rights reserved.

1 Several effect size indices were reported in these meta-analyses, including

1. Introduction

Anxiety disorders and depression are highly prevalent (Kessler,Chiu, Demler,&Walters, 2005), are associated with reduced qualityof life for patients (Olatunji, Cisler,& Tolin, 2007; Strine et al., 2009),and enormous economic costs for society (Simon, VonKorff, &Barlow, 1995). Although evidence-based treatments for anxietydisorders and depression are available, large numbers of people donot seek treatment (Eisenberg, Hunt, Speer, & Zivin, 2011), do notrespond to treatment (Griffiths & Griffiths, 2015), or experience anew episode over time (Bruce et al., 2008; Curry et al., 2011).Therefore, it is increasingly being argued that treatment alone isnot sufficient to alleviate the individual and societal burden

Hoorn, The Netherlands..

associated with these disorders, but that this needs to be com-plemented by prevention (Topper, Emmelkamp, & Ehring, 2010).

A growing number of programs for the prevention of depressionand anxiety disorders have been developed and evaluated. Recentmeta-analytic studies have focused on the effects of universalprevention programs (i.e. interventions offered to all individualswithout pre-selection) on depressive symptom severities relativeto randomized control groups. Results demonstrate small effectsizes at post-intervention (d¼ 0.04-.121) (Horowitz&Garber, 2006;Merry et al., 2012; Stice, Shaw, Bohon, Marti, & Rohde, 2009) and

Cohen's d, the correlation coefficient r, and Hedges' g. To facilitate interpretation, alleffect size indices were converted to Cohen's d according to Cohen (Cohen, 1988),and Rosenthal (Rosenthal, Cooper, & Hedges, 1994). Criteria for Cohen's d are usedto classify effect sizes as small (0.20-0.30), medium (0.50) and large (0.80).

mailto:[email protected]

http://crossmark.crossref.org/dialog/?doi=10.1016/j.brat.2016.12.015&domain=pdf

www.sciencedirect.com/science/journal/00057967

http://www.elsevier.com/locate/brat

http://dx.doi.org/10.1016/j.brat.2016.12.015



2 One methodological issue to consider when comparing the different types ofpreventive interventions is that greater power is needed to demonstrate the effectsof universal interventions because a larger number of participants are not at risk ofdeveloping emotional problems, and there is a lower base-rate of incidence.

M. Topper et al. / Behaviour Research and Therapy 90 (2017) 123e136124

follow-up (d ¼ 0.02-0.12) (Horowitz & Garber, 2006; Merry et al.,2012; Stice et al., 2009). Similarly, meta-analyses evaluating uni-versal prevention programs for anxiety demonstrate small effectsizes at post-treatment (d ¼ 0.12-0.29) (Fisak, Richard, & Mann,2011; Teubert & Pinquart, 2011; Zalta, 2011) and follow-up(d ¼ 0.15) (Teubert & Pinquart, 2011). Evidence for efficacy of se-lective (i.e., provided for individuals at risk of psychopathology)and indicated prevention programs (i.e., offered to individualsshowing early symptoms of a disorder) is somewhat more favor-able. Programs focusing on the prevention of depression have beenfound to produce small to moderate effect sizes in the reduction ofdepressive symptoms levels at post-intervention (d ¼ 0.23-0.31)(Horowitz& Garber, 2006; Merry et al., 2012; Stice et al., 2009), andfollow-up (d ¼ 0.22-0.34) (Horowitz & Garber, 2006; Merry et al.,2012; Stice et al., 2009). For anxiety, meta-analytic studiesdemonstrate small to moderate effect sizes for these programs atpost-intervention (d ¼ 0.21-0.32) (Fisak et al., 2011; Teubert &Pinquart, 2011; Zalta, 2011), and follow-up (d ¼ 0.23) (Teubert &Pinquart, 2011).

Several researchers have argued that the comparison of symp-tom levels as described above does not reflect true prevention ef-fects (Cuijpers, van Straten, Smit, Mihalopoulos, & Beekman, 2008;Horowitz & Garber, 2006). Instead, it has been proposed that theincidence rates of disorders across intervention and control groupsshould be compared (Cuijpers et al., 2008). Following up on thisrecommendation, Garber et al. (2009) conducted an indicatedprevention study comparing a group cognitive behavioral preven-tion program to usual care in 316 adolescents of parents withdepression. At 9 months follow-up, the prevention programresulted in a 34% reduction of the risk of developing a depressivedisorder. In a recent meta-analysis, Merry et al. (2012) demon-strated that both universal and targeted (selective and indicated)prevention programs significantly reduced incidence rates ofdepression at post-treatment and 3e9 months follow-up. At 1 yearfollow-up, this effect disappeared for universal programs, butremained evident for targeted programs. However, a more recentmeta-analysis did not find significant differences in the reductionof incidence rates of depression between universal, selective andindicated prevention programs (Van Zoonen et al., 2014), with anaverage incidence reduction of 21%. Only a few studies haveinvestigated the effect of preventive interventions on the reductionof the incidence of anxiety disorders. For example, Seligman,Schulman, DeRubeis, and Hollon (1999) evaluated the effects of aselective prevention program targeting a maladaptive attributionalstyle in 231 college students. Participants assigned to a groupcognitive behavioral prevention program were less likely (14%) todevelop a diagnosis of generalized anxiety disorder than partici-pants assigned to a no-intervention control group (21%). Whenconducting a systematic literature search, Stockings et al. (2016)were able to identify only seven studies testing universal preven-tion, and one study testing selective and indicated prevention,respectively. Although the incidence of anxiety disorders wassignificantly reduced when compared to control groups at post-intervention, this effect was only maintained at longer-termfollow-up for indicated prevention. In sum, although the provi-sion of prevention for depression and anxiety disorders appearsgenerally promising, there is clearly room for improvement in thisarea. Moreover, despite the high levels of co-morbidity betweenanxiety and depression (Kessler et al., 2003), relatively few studieshave tested preventive interventions for both symptom clusterssimultaneously.

For example, in their systematic review and meta-analysis,Stockings et al. (2016) identified 12 studies testing universal pre-vention, 6 studies on selective prevention and 9 studies on indi-cated prevention targeting both depressive and anxiety symptoms.

Results of these studies mostly showed that the preventive in-terventions are efficacious in reducing both symptom clusters.Importantly, however, these effects were not maintained at the 12months assessment (universal prevention) or were not assessed(selective and indicated prevention).

A number of suggestions have been proposed in the literature toincrease the efficacy of preventive interventions. Many authorshave interpreted the findings described above as evidence that theeffects of universal prevention fall behind that of selective andindicated prevention and that prevention should therefore mainlyfocus on high risk individuals (Bienvenu & Ginsburg, 2007; Craske& Zucker, 2001; Horowitz & Garber, 2006). 2 In addition, it appearspromising to select participants based on risk factors that aremodifiable, and to then specifically target these risk factors in thepreventive intervention (Craske & Zucker, 2001; Zvolensky,Schmidt, Bernstein, & Keough, 2006). In contrast to preventionprograms consisting of broadband cognitive behavioral therapy(CBT) strategies, a focus on modifiable risk factors may ensure amore individualized approach that is tailored to the needs of anindividual and is thereby likely to boost motivation and engage-ment (Vitiello, 2011). Studies using this strategy in the past havetargeted factors such as maladaptive attributional style (Seligmanet al., 1999), anxiety sensitivity (Balle & Tortella-Feliu, 2010), bodydissatisfaction (Stice, Mazotti, Weibel, & Agras, 2000) or behavioralinhibition (Rapee, Kennedy, Ingram, Edwards, & Sweeney, 2005).Finally, the efficacy of prevention may be increased by focusing onpreventive interventions that target transdiagnostic risk factorsthat predispose for the development of a range of disorders (Dozois,Seeds, & Collins, 2009; Nehmy, 2010). There is some evidence thattargeting transdiagnostic risk factors, such as body dissatisfaction(Stice & Shaw, 2002), attributional style (Seligman et al., 1999), andperfectionism (Musiat et al., 2014) can reduce the risk for differenttypes of psychopathology.

In the current study, we applied the general strategy outlinedabove to develop a novel prevention program for depression andgeneralized anxiety disorder targeting repetitive negative thinking(RNT; e.g., worry, rumination). RNTappears to be a promising targetfor prevention for a number of reasons (see also Topper et al., 2010).First, there is substantial evidence showing that RNT is a trans-diagnostic risk factor. Longitudinal studies have shown that rumi-nation predicts future onset of major depressive episodes, diagnosisand symptom severities of PTSD, levels of anxiety, and bulimic aswell as substance abuse symptoms (Ehring & Watkins, 2008;Watkins, 2008). Similarly, worry has been found to predict futurelevels of anxiety and depressive symptoms. A relationship betweenRNT and emotional disorders is also found in childhood andadolescence (Abela, Brozina, & Haigh, 2002; McLaughlin &Hatzenbuehler, 2009; Nolen-Hoeksema, Stice, Wade, & Bohon,2007; Schwartz& Koenig, 1996), with increased RNT predicting theonset of depression (Wilkinson, Croudace,& Goodyer, 2013). Due tothe early onset of depression (Fergusson, Horwood, Ridder, &Beautrais, 2005) and anxiety disorders (McEvoy, Grove, & Slade,2011), preventive interventions are typically targeted at thisperiod of development. In experimental studies, the induction ofrumination has been shown to exacerbate already existingdysphoric mood and negatively impacts depressogenic processessuch as increased negative thinking and poorer problem solving(Hubbard, Faso, Krawczyk, & Rypma, 2015; Lyubomirsky & Nolen-Hoeksema, 1995). Experimental induction of worry has been

M. Topper et al. / Behaviour Research and Therapy 90 (2017) 123e136 125

shown to result in increased negative affect (Lyonfields, Borkovec,& Thayer, 1995; McLaughlin, Borkovec, & Sibrava, 2007), delayeddecision-making speed (Metzger, Miller, Cohen, Sofka, & Borkovec,1990), poor problem-solving confidence (Davey, 1994), and inter-ference with emotional processes associated with the extinction offear responses (Borkovec, Ray, & Stober, 1998).

Second, studies investigating the differences betweenworry andrumination, as the two main variants of RNT, have demonstratedthat worry and rumination are highly similar (Ehring & Watkins,2008). Recent studies using structural equation modellingshowed that factor solutions that include a common variancecomponent for worry and rumination provided a better fit thanfactor solutions in which worry and rumination were consideredseparate constructs (McEvoy & Brans, 2013). Moreover, this com-mon variance component of worry and ruminationwas sufficient topredict future levels of depression and anxiety (Topper, Molenaar,Emmelkamp, & Ehring, 2014a,b). This implies that worry andrumination can be reduced simultaneously using the same inter-vention strategies. Evidence suggests, however, that cognitivebehavioral strategies to reduce negative cognitions (e.g., thoughtchallenging) will not directly target worry and rumination, as RNTdoes not appear to be simply interchangeable with negativecognition in general. For example, RNT has been found to predictrisk for depression above and beyond other types of negativethinking and/or mediates the effects of other types of negativethinking, such as self-criticism (e.g., Spasojevic & Alloy, 2001;Verplanken, Friborg,Wang, Trafimow,&Woolf, 2007).With respectto treatment by CBT, symptoms for individuals reporting higherlevels of RNT improved at a slower rate from standard cognitivebehavioral approaches than those with lower levels of RNT (Ciesla& Roberts, 2002; Price & Anderson, 2011; Schmaling, Dimidjian,Katon, & Sullivan, 2002).

Third, a number of evidence-based treatments that are effectivein reducing RNT are available (Topper et al., 2010; Watkins, 2015;Watkins et al., 2011). The intervention principles within thesetreatments can be adapted to make them suitable for use in aprevention program. As a consequence, the development andevaluation of preventive interventions targeting RNT have beenrecommended (Topper et al., 2010; Watkins, 2015; Wilkinson et al.,2013). However, despite the robust evidence implicating RNT in thedevelopment of anxiety and depression, to date, no trial has spe-cifically evaluated a preventive intervention that explicitly targetsRNTas its main focus. Of course, it can be expected that participantsin existing CBT-based prevention programs also bring upworry andrumination as an intervention target. However, these programstypically use traditional cognitive restructuring to deal with thecontent of rumination or worry but do not include interventionsthat explicitly and specifically target the process of RNT.

The aim of the current study was to test the feasibility and ef-ficacy of a preventive intervention targeting RNT in adolescents andyoung adults. Following up on the above mentioned recommen-dations, the intervention was selectively offered to individualsshowing elevated levels of both worry and rumination. The contentof the preventive intervention was derived from a treatment pro-tocol (Rumination-focused CBT; RFCBT) that has been shown toeffectively reduce levels of RNT as well as symptom levels ofdepression, and to prevent relapse in individuals with residualdepression (Watkins & Moberly, 2009; Watkins et al., 2007, 2011).

Whilst still grounded within the core principles and techniquesof standard CBT for depression, RFCBT includes several novel ele-ments and features several key differences (seeWatkins, 2015, 2016for further details). First, the RFCBT intervention does not involvecognitive restructuring or challenging of individual thoughts, un-like many CBT programs, but rather is focused on helping in-dividuals to spot the sequence of RNT and trying to replace or

interrupt that pattern of thinking. Second, as part of that process, itexplicitly targets RNTas a habit (Watkins&Nolen-Hoeksema, 2014)by identifying antecedent cues to worry/rumination, controllingexposure to these cues, and by repeated practice of alternativehelpful responses to these cues, using behavioral activation (BA)techniques such as functional analysis and scheduling contingencyplans (Martell, Addis,& Jacobson, 2001). Third, building on researchindicating that an abstract, decontextualized, and global thinkingstyle, characteristic of RNT, causally contributes to its maladaptiveconsequences (Watkins, 2008), patients are taught to shift into amore adaptive concrete and specific thinking style, using imagery,behavioral experiments, and experiential approaches that includefocusing on recreating experiences of being absorbed (e.g., ‘flow’

experiences), and experiences of increased compassion to self orothers.

Two versions of the preventive intervention were tested. In thefirst condition, the intervention was delivered in a group format.Group formats are considered a cost-effective way to deliver pre-vention as one therapist can work with multiple individuals at thesame time. In the second condition, the intervention was deliveredvia the Internet. In the field of prevention, highly scalable in-terventions enabling widespread coverage and access are needed(Kazdin & Blase, 2011). While showing comparable efficacy(Christensen, Batterham, & Calear, 2014; Van der Zanden, Kramer,Gerrits, & Cuijpers, 2012), internet-based psychological in-terventions have a number of potential advantages relative to face-to-face formats, including greater accessibility, anonymity andconvenience.

We predicted that both versions of the preventive interventionwould reduce levels of worry and rumination. In addition, theinterventionwas hypothesized to reduce symptom levels of anxietyand depression as the primary target. Because symptoms of bulimiaand alcohol abuse (i.e., binge drinking) have also been associatedwith RNT (Nolen-Hoeksema et al., 2007), we predicted that thesesymptoms would also be reduced by the preventive interventions.Furthermore, we predicted that both versions of the interventionwould reduce the future prevalence of major depression andgeneralized anxiety disorder (GAD). As the group and internetversions followed the same basic principles, we did not have anypredictions regarding whether one or the other type of deliverywould be more efficacious.

A second aim of this study was to investigate the underlyingmechanisms of the preventive intervention. Specifically, we pre-dicted that the effect of the interventions on the prevalence ofdepression and GAD would be mediated by precedent changes inRNT.

2. Method

2.1. Participants

In linewith earlier research, an effect size of at least d¼ 0.66 wasexpected for the critical Time (pre-intervention, post-intervention,3 m follow-up assessment, 12 m follow-up assessment) x Condition(group intervention, internet intervention, wait list control group)interaction effect with rumination as the dependent variable(Watkins et al., 2011), whereas smaller effect sizes (d ¼ 0.20) wereexpected for symptom severity levels as the dependent variables(Merry et al., 2012). Using a conservative estimate of d¼ 0.20, alphawas set at 0.05 and power at 0.80. A two-tailed power calculation(using G*power) showed that a minimal sample size of 207 wasrequired, but more participants were recruited to hedge againstattrition, which was estimated at 20%.

All secondary schools (n ¼ 23) providing education to pupilspreparing for university within the greater Amsterdam area were


informed about the prevention trial and invited to participate.Pupils attending class at the final three grades (ages 15 to 18) of the13 schools that were willing to participate were informed andrecruited between September 2010 and May 2012. Their parentswere informed about the project via a letter. Following thisannouncement, research assistants visited the schools during reg-ular class to screen for excessive levels of worry and rumination byadministering the Penn State Worry Questionnaire (PSWQ) (Meyer,Miller, Metzger, & Borkovec, 1990) and the Ruminative ResponseScale (RRS) (Nolen-Hoeksema & Morrow, 1991).

In addition, two faculties of social sciences of two local univer-sities agreed to participate in this prevention trial. At these uni-versities, recruitment was organized through an e-mail, website,and poster campaign. 3 A link directed first-year students (ages18e22) interested in the project to a website where they could fillout the PSWQ and RRS.

To screen positive for elevated levels of worry and rumination,participants had to have a total score at or above the 75th percentileon one of the screening measures and a total score at or above the66th percentile on the other screening measure. For the PSWQ andRRS, the 75th percentiles corresponded to scores of 50 and 40,whereas the 66th percentiles corresponded to scores of 47 and 38respectively. 4

A CONSORT diagram (Schulz, Altman, Moher, & CONSORTGroup, 2010) illustrating participant flow throughout the study ispresented in Fig.1. Inclusion criteria included (1) being between theages of 15 and 22, (2) scoring above the cutoffs for excessive worryand rumination described above at both the initial screening andthe pre-intervention assessment, (3) absence of self-reported cur-rent diagnoses of major depression and/or generalized anxietydisorder as assessed by the Patient Health Questionnaire-9 (PHQ-9;Spitzer, Kroenke, Williams,& Patient Health Questionnaire PrimaryCare Study Group, 1999) and the Generalized Anxiety DisorderQuestionnaire-IV (GADQ-IV; Newman et al., 2002), and (4) absenceof concurrent treatment for mental health problems. Randomiza-tion was stratified by gender and student type (secondary schoolstudent, university), and was carried out by a person independentfrom the study via a true randomization process and delivered inclosed envelopes. The first author informed all participants abouttheir allocation to either the group intervention (n ¼ 82), theinternet intervention (n ¼ 84), or the wait list control condition(n ¼ 85). Participants’ demographic characteristics are shown inTable 1. The randomization method was successful in that therewere no significant differences between the three groups on anydemographic characteristic during the pre-intervention assess-ment. Participants were paid V7 for completing the post-intervention and 3 m follow-up (FU) assessments, and V20 forcompleting the 12 m FU assessment.

2.2. Measures

2.2.1. Self-report measures of repetitive negative thinking2.2.1.1. Penn State Worry Questionnaire (PSWQ). The PSWQ (Meyeret al., 1990) measures the tendency, intensity and uncontrollabilityof worry and consists of 16 items rated on a 5-point Likert scale,with values ranging from 1 (not at all typical of me) to 5 (verytypical of me) (sample items: “I am always worrying about

3 Although the recruitment procedure for secondary schools and universitiesdiffered, there were no pre-intervention differences in levels of RNT between pupils(PSWQ: M ¼ 58.36, SD ¼ 6.74; RRS: M ¼ 48.89, SD ¼ 8.51) and university students(PSWQ: M ¼ 59.21, SD ¼ 6.80; RRS: M ¼ 47.19, SD ¼ 7.58), t(249) ¼ �0.97, p ¼ 0.33,and t(249) ¼ 1.61, p ¼ 0.11, respectively.

4 These scores were based on the distribution of PSWQ and RRS scores from thefirst recruitment wave at the participating secondary schools (n ¼ 1258).

something”; “Once I start worrying, I cannot stop”). The PSWQ hasbeen shown to have high internal consistency in clinical and non-clinical samples (a ¼ 0.86-0.95), high test-retest reliability in avariety of samples (r ¼ 0.74-0.92), good convergent and discrimi-nant validity, and good predictive validity in the prediction ofdepression and anxiety (Hong, 2007; Meyer et al., 1990; VanRijsoort, Emmelkamp, & Vervaeke, 1999) (as in this study ¼ 0.76-0.89).

2.2.1.2. Ruminative Response Scale (RRS). The RRS (Nolen-Hoeksema & Morrow, 1991) consists of 22 items on a Likert-typescale, with values ranging from 1 (almost never) to 4 (almost al-ways). It assesses the tendency to respond to depressed mood witha focus on self, (sample item: “Why do I have problems that otherpeople don't have?”), symptoms (sample item: “Think about yourfeeling of fatigue and achiness.”), and possible consequences andcauses of this depressed mood (sample item: “I won't be able to domy job if I don't snap out of this.”). The RRS has been shown to havegood internal consistency (a ¼ 0.82-0.90), moderate to high test-retest reliability (r ¼ 0.47-0.80), good convergent validity as wellas good predictive validity in the prediction of depression andanxiety (Just & Alloy, 1997; Nolen-Hoeksema, 2000; Nolen-Hoeksema, Parker, & Larson, 1994) (as in this study ¼ 0.80-0.92).

2.2.1.3. Perseverative Thinking Questionnaire (PTQ). The PTQ(Ehring et al., 2011) is a 15-item questionnaire assessing the ten-dency to engage in RNT independent of a disorder-specific content.Items are rated on a scale ranging from 0 (never) to 4 (almost al-ways) (sample items: “The same thoughts keep going through mymind again and again.”; “I keep asking myself questions withoutfinding an answer.”). Assessment of the psychometric properties ofthe PTQ has demonstrated high internal consistency (a ¼ 0.93-0.95), acceptable test-retest reliability (r ¼ 0.69-0.75), goodconvergent validity as well as good predictive validity in the pre-diction of symptom levels of anxiety and depression (Ehring, Raes,Weidacker, & Emmelkamp, 2012) (as in the current study ¼ 0.86-0.93).

2.2.2. Self-report measures of symptomatology

2.2.2.1. Beck Depression Inventory-II (BDI-II). The BDI-II (Beck, Steer,& Brown, 1996) is a measure of depressive symptomatology. Re-spondents are asked to endorse 21 sets of statements varying inseverity from 0 (e.g., “I do not feel sad.”) to 3 (e.g., “I am so sad orunhappy that I can't stand it.”). The highest rating for each item issummed across all items to create a continuous measure ofdepressive symptoms. High internal consistency (a ¼ 0.88-0.92),and test-retest reliability (r ¼ 0.93-0.96) has been reported, as wellas high convergent validity with self-report measures and astructured interviewmeasure of depressive symptomatology (Becket al., 1996; Sprinkle et al., 2002) (as in this study ¼ 0.87-0.88).

2.2.2.2. Mood and Anxiety Symptom Questionnaire-D30 (MASQ-D30). TheMASQ-D30 (Wardenaar et al., 2010) is a 30-itemmeasuredesigned to measure nonspecific general distress and symptomsspecific to depression and anxiety. In this study, only the 10-itemAnxious Arousal (e.g., “My heart was racing or pounding.”) andGeneral Distress scales (e.g., “I felt inferior to others.”) were used, asdepressive symptomatology was measured with the BDI-II. Itemsare rated on a scale ranging from 0 (not at all) to 4 (extremely).Acceptable to high internal consistency has been demonstrated inclinical and non-clinical samples for the anxiety (a¼ 0.70-0.85) andnonspecific symptom (0.84-0.91) scale, as well as good convergentvalidity with other self-report measures of symptomatology(Wardenaar et al., 2010). (as in the current study ¼ 0.82-0.93).

Completed post-treatment assessment (n = 85)

Completed 3m follow-up assessment(n = 68, 82.9%)


Completed 3m follow-up assessment(n = 69, 82. 1%)


Completed screening (n = 5481)

Screened positive (n = 867)

Randomized(n = 251)

Unsuccesful Application (n = 398)• Declined (n = 292) • No contact (n = 106)

Exclusion based on pre-intervention assessment (n = 218)• No longer meets criterion excessive RNT (n = 91) • Self-report Diagnosis of depression and/or GAD (n =

104) • Receives treatment (n = 23)

Group intervention(n = 82)

Internet intervention(n = 84)

Waitlist control(n = 85)

Completed post-inter-vention assessment (n = 73, 89.0%)







Fig. 1. CONSORT Trial flowchart.

Table 1Sample characteristics.

Condition Group(n ¼ 82)

Internet(n ¼ 84)

Waitlist(n ¼ 85)

Difference Test

Mean age(SD)

17.32(1.97)

17.36(2.16)

17.67(2.15)

F (2,248) ¼ 0.72,p ¼ 0.49

Gender MaleFemale

14 (17.1%)68 (82.9%)

13 (15.5%)71 (84.5%)

14 (16.5%)71 (83.5%)

c2 (2) ¼ 0.08,p ¼ 0.96

School level Secondary schoolUniversity

48 (58.5%)34 (41.5%)

52 (61.9%)32 (38.1%)

52 (61.2%)33 (38.8%)

c2 (2) ¼ 0.22,p ¼ 0.90

Treatment history YesNo

2458

3153

3253

c2 (2) ¼ 1.57,p ¼ 0.46

Medication* YesNo

1270

777

1570

c2 (2) ¼ 3.25,p ¼ 0.20

Note* Exclusively for medical conditions (allergies, asthma) and ADHD medication (Ritalin, Concerta).


2.2.2.3. Eating Disorder Inventory-2, bulimia subscale (EDI-2-BU).The EDI-2 (Garner, 1991; Van Strien, 2002) is a 91-item question-naire assessing behavioral and attitudinal dimensions common ineating disorders. The bulimia scale consists of 7 items on a 1 (never)to 6 (always) scale designed to measure the tendency to binge andpurge (e.g., “I stuff myself with food.”; “I eat when I get upset.”).High internal consistency (a ¼ 0.78-0.93) and test-retest reliability(r¼ 0.75-0.94) of the bulimia subscale has been reported, as well asgood construct and discriminant validity (Thiel & Paul, 2006; VanStrien & Ouwens, 2003); as in this study ¼ 0.84-0.89).

2.2.2.4. Quick Drinking Screen (QDS). The QDS (Sobell et al., 2003)consists of 5 items assessing alcohol consumption. Respondents areasked to estimate the frequency and quantity of alcohol con-sumption. We adjusted the items of this measure so that they

applied to alcohol consumption over the last 2 weeks (e.g., “In thelast two weeks, how many times have you had 5 or more alcoholicbeverages?”). The QDS has shown to produce reliable brief sum-mary measures of alcohol consumption (Sobell et al., 2003). In thisstudy, we used the frequency of binge drinking as an outcomemeasure.

2.2.3. Self-report measures of clinical diagnoses

2.2.3.1. Patient Health Questionnaire-9 (PHQ-9). The PHQ-9(Spitzer, Kroenke, Williams, & He, 1999) consists of 9 items thatcan be used to make a tentative diagnosis of depression. The itemscorrespond to the DSM-IV criteria for major depression. For each ofthe nine symptoms of depression, respondents indicate whetherthe symptom has bothered them in the past 2 weeks: 0 ¼ not at all,1 ¼ several days, 2 ¼ more than half of the days, 3 ¼ nearly every


day (e.g., “Little interest or pleasure in doing things?”). Gilbody,Richards, Brealey, & Hewitt, (2007) in a meta-analysis of 17 vali-dation studies concluded that the PHQ9 has good diagnosticproperties for depression (sensitivity 92%; specificity 80%).

2.2.3.2. Generalized Anxiety Disorder Questionnaire-IV (GADQ-IV).The GADQ-IV (Newman et al., 2002) is a 9-itemmeasure that can beused to provide a self-report diagnostic assessment of generalizedanxiety disorder (GAD), as well as a severity score of generalizedanxiety symptoms. The items correspond to DSM-IV criteria forGAD. The validity of the GADQ-IV is supported by comparisonsbetween GADQ-IV diagnoses and clinician-administered AnxietyDisorder Interview Schedule (ADIS-IV) diagnoses in a treatment-seeking sample, and a non-anxious comparison group (Luterek,Turk, Heimberg, Fresco, & Mennin, 2002). The GADQ-IV showed asensitivity of 77% and a specificity of 96%.

2.2.4. Additional self-report measures2.2.4.1. Life Events Checklist (LEC). The LEC (Johnson &McCutcheon, 1980) is a measure of child and adolescent lifestress listing 46 life events (e.g., “Death of a parent.”). As recom-mended by Turner and Wheaton (Turner & Wheaton, 1995, pp.29e58), some items were adapted to increase the potential rele-vance to the current study sample. We used a simple unit ratingprocedure (sum of life events indicated as present within the pastyear) as an index of life stress. The LEC was administered at the pre-intervention and 12 m FU assessments.

2.2.4.2. Additional treatment/medication. At 12 m FU, participantsindicated whether they had received psychopharmacological and/or psychological treatment outside of the trial.

2.2.4.3. Treatment satisfaction evaluation form. This questionnairewas adapted from Gallego, Emmelkamp, Van Der Kooij, and Mees(2011). Perceived usefulness of the intervention (e.g., Did youexperience the program as relevant for your problems and diffi-culties?”), and participants’ satisfaction with the intervention (e.g.,“Have you been able to trust the facilitators?”) are assessed with anumber of yes/no responses; in addition, an overall satisfactionrating on a 1e10 scale is given.

2.2.5. Preventive interventionThe preventive intervention was developed by the authors and

consisted of a modified version of the Rumination-focused CBT(RFCBT) protocol developed byWatkins et al. (Watkins et al., 2007).The efficacy of RFCBT has recently been assessed in an RCT allo-cating 42 patients with medication-refractory residual depressionto treatment as usual (TAU) alone, or to TAU plus up to 12 sessionsof individual RFCBT (Watkins et al., 2011). TAU consisted of ongoingantidepressant medication and outpatient clinical management.TAU plus RFCBT significantly reduced rumination and depressionrelative to TAU alone (remission rates: TAU 21%; TAUþ RFCBT 62%),comparing favorably to remission rates (25%) found for TAU plusstandard CBT in another trial for residual depression (Paykel et al.,1999).

Our preventive intervention is based on (1) evidence showingthat worry and rumination are forms of avoidance (Giorgio et al.,2010; St€ober, Tepperwien, & Staak, 2000), and (2) experimentalresearch showing that dysfunctional forms of RNTare characterizedby an abstract and evaluative style of processing, which causallycontributes to a number of maladaptive consequences, includingpoor problem solving, increased emotional reactivity, relative to aconcrete and contextualized style of processing (Watkins &Baracaia, 2002; Watkins & Moulds, 2005; Watkins, 2008). Afunctional-analytic approach is used, in which RNT is

conceptualized as a learned habitual behavior that acts as a form ofavoidance and that develops through negative reinforcement(Watkins & Nolen-Hoeksema, 2014). Functional analysis examineshow, when and where RNT does occur, and its antecedents andconsequences, to formulate its possible functions and to makeplans that systematically reduce or replace it (Watkins, 2015, 2016).The intervention uses psycho-education, functional analysis, iden-tification of warning signs and the planning of alternative re-sponses in contingency or If-Then plans, reflective exercises/groupdiscussion, experiential exercises, behavioral activation, andbehavioral experiments designed to facilitate a shift fromdysfunctional worrisome/ruminative thinking into a more helpfulconcrete thinking style and to increase approach behavior. Theinternet and group version of the intervention only differed in theformat delivered, and were identical in content.

The group intervention was delivered in six weekly sessionslasting for 1.5 h each, and followed a session-by-session manual.The groups ranged in size from five to nine participants. Theinternet intervention consisted of six sessions that could becompleted on a designated website. The program was self-paced,yet participants were advised to complete at least half a sessionat a time. Upon completion of a session, therapists could securelyaccess the online platform to provide personalized written feed-back. In both versions of the intervention, participants receivedregular reminders for their sessions/online tasks.

Prior to administering the intervention, study therapists atten-ded a 2-day training workshop, led by the developers of theintervention. The group intervention was delivered by four thera-pists, graduated in clinical psychology, who had several years ofexperience in group treatment with the target groups. In addition, agraduate student served as a co-therapist during the sessions. Allsessions were audiotaped to guide supervisionwhich was providedby the second author after each session. Infractions in the form ofelements of the protocol left out and prohibited behaviors/in-terventions, were discussed during supervision to prevent futureprotocol violations. The internet intervention was delivered by sixdifferent therapists graduated in clinical psychology. All therapistshad at least six months of experience with internet treatment andfollowed a manualized response protocol. The content of theirfeedback on each of the completed sessions was checked by thefirst author. When necessary, adjustments were made before thefeedback was sent to the participant.

2.2.5.1. Content. The first sessions of the intervention were used todefine worry, rumination and avoidance, and to conduct a func-tional analysis of participants' use of these maladaptive strategies.Subsequent sessions included interventions that taught partici-pants to be more aware of when their attempts at coping withdistress (do not) work and to increase their use of strategies thatwork. Participants identified their triggers for worry and rumina-tion, and practiced alternative behaviors incompatible with RNT.Directed imagery was used to evaluate the cognitive, emotional andbehavioral effects of an abstract evaluative thinking style comparedto a more specific, concrete thinking style that is grounded in directexperience. Additional experiential exercises were used to recreateprevious mental states of being completely absorbed in an activity(e.g., ‘flow’ experiences), and experiences of increased compassionto self and others. Guidelines were provided that instructed par-ticipants on how to employ a more grounded, concrete andcompassionate thinking style. In the final session, participants werea) taught to increase action-oriented, assertive behavior whenengaging with others, and b) to create an individualized summaryof strategies that they considered to be useful.

In the group intervention, a workbook was used that guided theparticipants through the program content and exercises. Most of


the exercises were introduced using group discussion. In theinternet intervention, writing exercises were used to obtain inputfrom the participants and exercises were illustrated by film clips oftwo peer actors talking about their experiences. Both versions ofthe program contained homework exercises.

2.2.5.2. Adherence. Intervention adherence was only measured forthe group intervention as the written feedback in the internetintervention was standardized in a way that adherence was guar-anteed. Two audiotaped sessions for each group were randomlyselected (33%). Ratings of adherence in delivering the therapy wereprovided independently by the first author and a researcher whowas not involved in this study. For each session of the protocol,essential elements were listed such that the number of prescribedand prohibited infractions per session could be tallied. The inter-rater agreement between raters was high, k¼ 0.87. On average, 93%of the essential and required elements of the protocol werecompleted per session. Prohibited behaviors/interventionscomprised content that was not part of the protocol and originatedfrom other interventions (e.g., challenging thoughts). Ratersobserved an average of 1.13 infractions per session. There were nodifferences in the proportion of infractions to completed elementsbetween therapists, c2 (3) ¼ 0.73, p ¼ 0.87.

2.2.6. ProcedureEthical approval for the study was obtained from the local

institutional review board. In addition, the trial was registered atwww.clinicaltrials.gov (ID: NCT01223677). All assessments wereadministered online. After providing written informed consent,participants received an e-mail with a link directing to the pre-intervention assessment. Eligible participants were informedabout intervention allocation via telephone or e-mail. For all par-ticipants, the post-intervention assessment followed 1 week afterthe intervention had been completed, i.e., 8e10 weeks post-randomization. After the 12 m FU, participants in the waitlistconditionwere given the opportunity to follow the internet versionof the program.

2.2.7. Data analytic approachMultilevel regression analyses were conducted to evaluate the

effect of the intervention on RNT and symptom measures of anxi-ety, depression, bulimia, and alcohol abuse. Multilevel regression isan intent-to-treat procedure that does not impute missing data butdeals with incomplete data by assuming that the available data for agiven subject are representative of that subject's deviation from theaverage trends across time (Hedeker & Gibbons, 2006). The level-1model included the time variable, which captures within-personchange over time. In the level-2 model, between-person charac-teristics such as intervention condition were used to predict theslope estimates representing change in the dependent variables.We estimated a linear trend indicating the direction and rate ofchange, and a quadratic trend indicatingwhether the rate of changeincreased or decreased over time. The assumption that data weremissing at random was evaluated by using binary logistic regres-sion to predict measurement dropouts and by comparing the groupof participants with measurement dropout(s) (n ¼ 65) to the groupof participants without measurement dropout (n ¼ 186) on base-line measures. Baseline characteristics did not predict attrition anddid not differ significantly between conditions (all ps > 0.05)5.Allowing intercepts and slopes to covary did not significantlyimprove any of the models, and therefore unstructured covariance

5 Moreover, analyses performed over participants with complete data producedhighly similar results indicating no relationships with measurement attrition.

structures were used. Contrast coding was used to evaluate theeffect of the categorical variable intervention condition (Hox,Moerbeek, & Van de Schoot, 2010). The first contrast comparedboth interventions (coded 1 =3) to waitlist control (coded �2 =3), andthe second contrast compared the group RFCBT intervention(coded 1 =2) to the internet RFCBT intervention (coded�1 =2). Withingroup effect sizes (Cohen's d) for each outcome measure werecalculated by subtracting the mean score for each successivemeasurement from the pre-intervention mean and by dividing thisdifference score by the pooled standard deviation across timepoints. Between-group effect sizes (Cohen's d) from pre-to post-intervention were calculated using the linear trend estimate of thetime by contrast interactions divided by the pooled standard de-viation (Feingold, 2013).

Cox regression survival analyseswere performed to examine theeffect of the preventive intervention on self-reported episode on-sets of major depression and GAD. Participants were censored uponmeasurement dropout or end of study. Although condition was themain covariate included in the model, we also considered age,gender, school level, treatment history, medication, additionaltreatment, additional medication and life events during the studycourse. Similar to the multilevel regression described above,dummy variables were created for condition to compare both in-terventions together against the waitlist control, and against eachother.

Two separate mediation analyses were conducted to determinewhether reductions in RNT (PTQ6) mediated the prevalence of 1)major depression (PHQ-9) and 2) GAD (GADQ-IV) at the 12 m FU.Following the recommendations of Hayes (Hayes, 2009), a bias-corrected bootstrapping procedure with 5000 resamples was per-formed using structural equation modelling. This method offersmore power than more traditional approaches while maintainingreasonable control over the Type I error rate. As the outcomes of themediation analyses were dichotomous (0 ¼ does not fulfil criteriafor major depression or GAD, 1 ¼ fulfils criteria of major depressionor GAD), probit regression analyses were performed (MacKinnon,2008) using the WLSMV estimator in Mplus 5.0 (Muth�en &Muth�en, 2007). A prerequisite for conclusively establishing medi-ation is that changes in the mediator variable precede changes inthe outcome variables (Gu, Strauss, Bond, & Cavanagh, 2015). Tothis end, pre to post-intervention change scores were computed forthe PTQ. Confidence intervals (CI; 95%) were derived for the indi-rect effect of condition (group/online intervention vs. waitlistcontrol) via the hypothesizedmediator (change in RNT) on the 12mprevalence of depression and GAD. Mediation is said to occur if thederived confidence interval does not contain zero (Preacher &Hayes, 2004). To obtain an effect size of the mediation effect, wecalculated the proportion of the total effect that is mediated by pre-to post-intervention reductions in RNT (MacKinnon, 2008).

3. Results

3.1. Pre-intervention group differences

Table 1 demonstrates the demographic characteristics of par-ticipants per condition.

There was a significant difference in the number of life eventsexperienced in the year previous to the pre-intervention assess-ment, F (2, 248) ¼ 4.80, p ¼ 0.009. Participants in the waitlistcontrol condition (M ¼ 5.27, SD ¼ 2.66) had experienced a larger

6 The PTQ as a measure of RNT with items that do not refer to a disorder-specificcontent was chosen as a mediator, yet mediation analyses were also carried out forthe other two measures of RNT (PSWQ and RRS).

http://www.clinicaltrials.gov


number of life events compared to participants in the groupintervention (M ¼ 4.28, SD ¼ 2.34, p ¼ 0.01) and internet inter-vention conditions (M ¼ 4.21, SD ¼ 2.44, p ¼ 0.01). We thereforeentered this variable as a covariate in themain analyses. Differencesbetween conditions in the number of life events at the 12 m FUassessment just missed significance, F (2, 205) ¼ 3.024, p ¼ 0.05,but were nevertheless also entered as a covariate to account forpotential influences. Thirty-five participants had received addi-tional psychological treatment and 24 participants had takenadditional (psycho)pharmacological treatment during the FUperiod (allergies: n ¼ 6, pain medication: n ¼ 5, stomach problems:n ¼ 3, Crohn's disease: n ¼ 1, ADHD: n ¼ 6, sleep medication: n ¼ 2,anti-depressants: n ¼ 2), but there were no differences betweenconditions, c2 (2) ¼ 0.75, p ¼ 0.69, and c2 (2) ¼ 3.47, p ¼ 0.18.Descriptives for RNT and symptom severity measures per conditionare presented in Table 2. There were no significant differences be-tween conditions at the pre-intervention assessment (ps ¼ 0.09 to0.99), on any of these measures.

3.2. Intervention attrition

The percentage of non-starters was not significantly differentbetween the two active conditions (group: 6.1%; internet: 9.9%), c2

(2) ¼ 3.03, p ¼ 0.08. However, participants who had started thegroup intervention attended significantly more sessions (M ¼ 4.59;SD ¼ 1.43) than starters in the internet condition (M ¼ 3.96;SD ¼ 1.65), t (147) ¼ 2.28, p ¼ 0.02, d ¼ 0.38.

3.3. Effect of the intervention on measures of repetitive negativethinking

Both versions of the preventive intervention demonstratedmedium to large effects in the reduction of RNT, whereas thewaitlist control group demonstrated no to small effects in thereduction of RNT (see Table 2). The two intervention groups led tosignificantly greater reductions in RNT than the control group, withno significant differences between the two active conditions (seeTable 3).

3.4. Effect of the intervention on symptom measures

Both the group intervention and internet intervention led toreductions in symptoms of anxiety, depression, and general distressthat were maintained over time, whereas no significant reductionswere found in the control group (see Table 2). For all measures, theintervention groups demonstrated superiority to the control group(see Table 3). However, there were no significant differences be-tween the intervention groups and the waitlist control group in thereduction of bulimia symptoms and binge drinking. For all outcome

7 As explained in the method section, note that we used contrast coding tocompare the combined effect of the preventive interventions against waitlist controlfor the measures of RNT and symptom levels. The coding we used also allowed us tocompare the preventive interventions against each other, yet the contrasts did notallow us to compare each of the interventions separately to waitlist control.Separate group intervention vs. waitlist and internet intervention vs. waitlistcomparisons showed a similar pattern of results, except for the effect of reductionsin the anxious arousal scale of the MASQ which showed no significant difference inthe internet intervention group compared to waitlist control (p ¼ 0.09).

8 Although the number of sessions attended was higher for participants in thegroup intervention versus participants in the internet intervention, there were nosignificant differences between interventions on RNT and symptom levels overtime. Analyses exploring whether the mode of delivery (internet, group) and timeinteracted with demographic variables or initial RNT and symptoms levels showedno significant results (all p's < 0.05), and thereby provided no information on whobenefited most from the internet versus group mode of delivery.

measures, symptom reduction did not differ between the groupRFCBT intervention and the internet RFCBT intervention. 7,8

3.5. Effect of the intervention on prevalence of depression and GAD

Survival analysis indicated a significantly lower prevalence ofdepression at 12 m FU in the intervention conditions (group: 15.3%,internet: 14.7%) compared to the waitlist control group (32.4%) (seeFig. 2), Wald c2 (1) ¼ 4.89, p ¼ 0.03, hazard ratio ¼ 2.12 (95% CI,1.09e4.13). Similarly, a second survival analysis also indicated asignificantly lower prevalence of GAD in the intervention condi-tions (group: 18.0%, internet: 16.0%) compared to the waitlist con-trol group (42.2%) (see Fig. 3), Wald c2 (1) ¼ 9.07, p ¼ 0.003, hazardratio ¼ 2.52 (95% CI, 1.38e4.59). There were no differences in theprevalence of disorders between the two versions of the preventiveintervention for depression, Wald c2 (1) ¼ 0.02, p ¼ 0.88, and GAD,Wald c2 (1) ¼ 1.38, p ¼ 0.58. 9

3.6. Mediation

3.6.1. DepressionThe significant effect of condition on pre to post-intervention

changes in RNT was estimated at 4.83, p < 0.001. The significanteffect of pre to post-intervention changes in RNT on the prevalenceof major depression at the 12 m FU assessment was equal to 0.05,p < 0.001. The direct effect of condition on the prevalence of majordepression was not significant (0.364, p ¼ 0.49). The total indirecteffect of condition on the prevalence of major depression at 12 mFU was significant, Z ¼ 0.232, p ¼ 0.003, and the true indirect effectwas estimated to lie between 0.099 and 0.417 with 95% CI. Becausezero is not in the 95% CI, it can be concluded that the indirect effectis significantly different from zero at p < 0.05 and, thus, that changein RNT from pre-to post-intervention mediated the relationshipbetween condition and prevalence rates of major depression at12 m FU. The ratio of the indirect effect to the total effect suggeststhat reductions in RNT explained 38.9% of the effect on the preva-lence of major depression.

3.6.2. GADThe significant effect of pre to post-intervention changes in RNT

on the prevalence of GAD at the 12 m FU assessment was equal to0.05, p < 0.001. The direct effect of condition on the prevalence ofGADwas not significant (0.290, p¼ 0.22). The total indirect effect ofcondition on the prevalence of GAD at 12 m FU was significant,Z¼ 0.237, p¼ 0.002, and the true indirect effect was estimated to liebetween 0.109 and 0.408 with 95% CI. Because zero is not in the 95%CI, it can be concluded that the indirect effect is significantlydifferent from zero at p < 0.05 and, thus, that change in RNT frompre-to post-intervention mediated the relationship between con-dition and prevalence rates of GAD at 12 m FU. The ratio of theindirect effect to the total effect suggests that reductions in RNT

9 As we did for the measures of RNT and symptom levels, we also ran analyses inwhich we separately compared each of the preventive interventions to the waitlistcontrol condition. For the internet intervention condition, there was a significantlylower prevalence of depression, Wald c2 (1) ¼ 3.94, p ¼ 0.04, hazard ratio ¼ 2.36(95% CI, 1.01e5.52), and GAD, Wald c2 (1) ¼ 9.12, p < 0.01, hazard ratio ¼ 3.55 (95%CI, 1.56e8.09), compared to the waitlist control group. However, for the groupintervention condition compared to waitlist control, we did not find any differencesin the prevalence of depression, Wald c2 (1) ¼ 2.34, p ¼ 0.13, hazard ratio ¼ 1.91(95% CI, 0.83e4.40), and GAD, Wald c2 (1) ¼ 2.51, p ¼ 0.11, hazard ratio ¼ 1.77 (95%CI, 0.87e3.60).

Table 2Outcome measures at pre-intervention, post-intervention and follow-up.

Measure Condition PreM (SD)

PostM (SD)

d 3 m FUM (SD)

d 12 m FUM (SD)

d

PSWQ groupinternetwaitlist

58.20 (6.59)58.73 (6.96)59.15 (6.78)

51.29 (8.58)51.87 (8.85)57.80 (8.54)

0.98***0.79***0.24 ns

49.74 (9.33)49.51 (9.44)56.57 (8.58)

1.03***0.81***0.33*

50.46 (8.34)48.90 (9.60)56.59 (9.95)

1.04***0.94***0.28*

RRS groupinternetwaitlist

48.17 (8.65)48.29 (8.10)48.20 (7.89)

41.51 (10.29)41.81 (9.54)47.46 (9.66)

0.68***0.71***0.13 ns

39.40 (10.48)41.27 (10.66)45.25 (10.66)

1.12***0.64***0.27**

39.51 (10.24)39.88 (10.51)45.85 (12.34)

0.97***0.77***0.19 ns

PTQ groupinternetwaitlist

34.35 (6.84)34.64 (8.38)35.11 (6.75)

29.75 (8.70)30.54 (9.31)35.38 (8.15)

0.54***0.42**0.06 ns

27.60 (8.49)27.08 (10.00)33.64 (8.97)

0.84***0.62***0.16 ns

28.37 (9.24)27.43 (8.64)33.48 (9.47)

0.66***0.58***0.15 ns

BDI-II groupinternetwaitlist

10.57 (5.60)12.21 (6.73)12.55 (6.11)

6.38 (5.07)8.26 (6.03)13.00 (8.73)

0.80***0.51**0.05 ns

6.85 (6.11)7.72 (6.08)11.36 (9.24)

0.59***0.54**0.15 ns

7.84 (6.18)7.93 (6.52)11.55 (9.35)

0.43**0.44**0.10 ns

MASQ-AA groupinternetwaitlist

16.46 (5.09)16.73 (6.27)17.27 (7.22)

13.11 (3.98)14.61 (5.02)17.80 (7.96)

0.60***0.29 ns

0.08 ns

14.59 (6.47)14.34 (5.09)16.69 (7.04)

0.23 ns

0.26*0.09 ns

14.04 (5.28)14.33 (4.78)17.46 (8.51)

0.41**0.30*0.04 ns

MASQ-GD groupinternetwaitlist

22.72 (7.85)23.19 (7.45)22.69 (7.42)

17.62 (6.74)18.00 (7.30)24.18 (9.25)

0.60***0.64***0.19 ns

17.94 (8.77)17.41 (7.24)21.77 (8.97)

0.50**0.56***0.09 ns

18.16 (7.17)19.34 (7.88)22.68 (9.67)

0.45***0.36**0.01 ns

EDI-II bulimia groupinternetwaitlist

14.39 (5.25)15.44 (6.32)15.51 (6.03)

13.01 (5.12)13.99 (6.11)15.55 (7.11)

0.40***0.21 ns

0.04 ns

13.50 (5.56)14.02 (6.44)14.96 (6.00)

0.22*0.24 ns

0.12 ns

13.19 (5.66)13.81 (6.60)14.74 (6.21)

0.19 ns

0.21 ns

0.13 ns

QDS binge drinking groupinternetwaitlist

1.11 (1.26)0.69 (1.15)1.13 (1.16)

0.83 (1.32)0.94 (1.23)1.35 (2.05)

0.22 ns

0.22*0.13 ns

0.98 (1.64)0.78 (1.34)1.53 (1.77)

0.09 ns

0.07 ns

0.27**

0.98 (1.22)0.90 (1.70)1.32 (1.49)

0.10 ns

0.15 ns

0.11 ns

Note. PSWQ ¼ Penn State Worry Questionnaire; RRS ¼ Ruminative Response Scale; PTQ ¼ Perseverative Thinking Questionnaire; BDI-II.Beck Depression Inventory-II; MASQ-AA ¼ Mood and Anxiety Symptom Questionnaire e Anxious Arousal; MASQ-GD ¼ Mood and Anxiety.Symptom Questionnaire e General Distress; EDI-II ¼ Eating Disorder Inventory eII; QDS ¼ Quick Drinking Screen; FU ¼ follow-up; d ¼ effect size.Cohen's d (within-group) based on pre-intervention assessment, *p < 0.05, **p < 0.01,***p < 0.001., ns ¼ not significant.


explained 45% of the effect on the prevalence of GAD. 10

3.7. Acceptability of the intervention

The majority of participants allocated to the preventive inter-vention indicated that the training was adequate for their problems(group: 87%; internet: 93%), and that the trainers were sufficientlyproficient (group: 94%; internet: 92%), could be trusted (group:100%; internet: 98%), and were respectful (group: 100%; internet:97%). Participants in the group condition provided an overall ratingof 8.2 (SD ¼ 0.75; 1e10 scale), whereas the mean rating was 7.9(SD ¼ 0.52) in the internet condition.

4. Discussion

The present study is the first randomized controlled trialinvestigating the efficacy of a preventive intervention for depres-sion and anxiety that specifically and explicitly focuses on reducingRNT. Our results demonstrate that both versions of the new pre-ventive intervention reduced the tendency to engage in RNTas wellas symptom levels of anxiety and depression, whereas only mini-mal change took place in the waitlist control group. These inter-vention effects were maintained at 3 m and 12 m FU. Similarly, theprevalence of depression at 12 m FU was as high as 32.4% for par-ticipants on the waitlist, whereas depression prevalence rates forparticipants who had been offered the preventive interventionwere reduced to 14.7%e15.3%. This translates into a 45e47%reduction in depressive disorders achieved by our preventiveintervention. This compares favorably to the average decrease inincidence of 21% achieved in past depression prevention trials

10 Mediation of reductions in RNT as measured by the PSWQ and RRS producedsimilar results, except for the mediational effect of the PSWQ on the relationshipbetween condition and prevalence rates of GAD, total indirect effect, Z ¼ 0.072,p ¼ 0.377 (direct effect of condition was not significant, p ¼ 0.06).

according to the recent meta-analysis by Van Zoonen et al. (2014),and is consistent with the effects of a small number of earlierprevention trials (e.g., Brent et al., 2015; Clarke et al., 1995; Stice,Rohde, Gau, & Wade, 2010). Similarly, the prevalence of GAD at12 m FU was 42.2% in the waitlist condition compared to 16e18% inthe intervention conditions, which translated into a 38e43%reduction in GAD. To our knowledge, this is the first trial investi-gating prevention of GAD, and our findings show that targeting thetransdiagnostic risk factor for RNT can reduce the prevalence ofdepression and GAD to a similar degree. Importantly, our resultsalso support assumptions regarding mechanisms of change in thenew intervention as reductions in worry and rumination wereshown tomediate the effects of the interventions on the prevalencerate of major depression and GAD.

Both types of preventive interventions enabled participants toreduce their tendency to engage in worry and rumination towardsnormal levels, according to recently reported norms and de-scriptives for the PSWQ, RRS, and PTQ (Ehring et al., 2012; Topper,Emmelkamp, Watkins, & Ehring, 2014a,b; Van der Heiden, Muris,Bos, Van der Molen, & Oostra, 2009). The between-group effectsizes for the symptom measures of depression and anxiety rangefrom 0.36 to 0.72. These effect sizes are larger than those reportedin recent meta-analyses of universal, selective, and indicated pre-ventive interventions for depression and anxiety disorders (Topperet al., 2010). Participants who had been offered either variant of theprevention program were at least 2.5 times less likely to report adiagnosis of depression or GAD within the next year. Note that theresults on prevalence levels of depression and GADwere somewhatweaker for the group intervention compared to the internetintervention. Survival analyses comparing each of the interventionsseparately to the waitlist control intervention showed no signifi-cantly reduced prevalence levels for the group intervention,whereas prevalence levels remained significantly lower for theinternet intervention condition. However, from this result, it cannotbe concluded that the internet intervention is more effective than

40

50

60

70

80

90

100

post-treatment

3-monthfollow-up

12-monthfollow-up

% P

artic

ipan

ts r

emai

ning

with

out a

dia

gnos

is o

f de

pres

sion

group intervention

internet intervention

waitlist control

Fig. 2. Cumulative proportion of participants remaining without a self-reported diagnosis of depression (PHQ-9).

40

50

60

70

80

90

100

post-treatment

3-monthfollow-up

12-monthfollow-up

% P

artic

ipan

ts r

emai

ning

with

out a

dia

gnos

is o

f ge

nera

lized

anx

iety

dis

orde

r

group intervention

internet intervention

waitlist control

Fig. 3. Cumulative proportion of participants remaining without a self-reported diagnosis of generalized anxiety disorder (GADQ-IV).

Table 3Multilevel interaction effects for repetitive negative thinking and symptom levels.

Predictors B Confidenceinterval (95%)

p d

PSWQ contrast 1a � time �5.95 �8.61; �3.28 <0.001 0.89contrast 2b � time �1.75 �4.95; 1.46 0.28 0.26

RRS contrast 1 � time �4.56 �7.75; �1.38 0.005 0.53contrast 2 � time �1.40 �5.21; 2.42 0.47 0.16

PTQ contrast 1 � time �4.89 �7.62; �2.15 0.001 0.67contrast 2 � time �0.80 �4.08; 2.49 0.63 0.10

BDI-II contrast 1 � time �3.40 �5.46; �1.34 0.001 0.55contrast 2 � time �1.10 �3.56; 1.37 0.38 0.18

MASQ-AA contrast 1 � time �2.24 �4.18; �0.29 0.02 0.36contrast 2 � time �0.73 �3.06; 1.61 0.54 0.13

MASQ-GD contrast 1 � time �5.43 �8.09; �2.77 <0.001 0.72contrast 2 � time 1.71 �1.48; 4.90 0.29 0.22

EDI-II bulimia contrast 1 � time �0.91 �2.45; 0.62 0.24 0.15contrast 2 � time �0.89 �2.73; 0.96 0.35 0.15

QDS binge drinking contrast 1 � time �0.60 �1.28; 0.09 0.09 0.48contrast 2 � time �0.26 �1.09; 0.58 0.54 0.22

Note. Contrast 1 ¼ intervention conditions versus waitlist control condition; Contrast 2 ¼ group intervention condition versus internet intervention condition; time ¼ frompre-intervention to post-intervention; PSWQ¼ Penn State Worry Questionnaire; RRS¼ Ruminative Response Scale; PTQ¼ Perseverative Thinking Questionnaire; BDI-II BeckDepression Inventory-II; MASQ-AA ¼ Mood and Anxiety Symptom Questionnaire e Anxious Arousal; MASQ-GD ¼ Mood and Anxiety Symptom Questionnaire e GeneralDistress; EDI-II ¼ Eating Disorder Inventory -II; QDS ¼ Quick Drinking Screen; d ¼ effect size Cohen's d (between-group).



the group intervention. Both interventions resulted in similar re-ductions of RNT and symptom levels, and although the hazard ra-tios were still indicative of a preventive effect for participants in thegroup intervention condition, the study was not sufficiently pow-ered to detect this effect.

Together, the findings reported in this study point towards thepotential value of this prevention program for clinical practice. On amore general level, the findings underscore the potential value oftargeted prevention programs focused on modifiable risk factors asa means to improve the field of prevention.

One of the proposed advantages of the present targetedapproach was the possibility to affect multiple symptom clusters asa consequence of the transdiagnostic status of RNT. These trans-diagnostic effects were observed to the extent that the interventionchanged the course of both depression and anxiety symptoms.However, one finding in the current study undermines the trans-diagnostic status of RNT. The transdiagnostic effects did not extendto bulimic symptoms and alcohol abuse. An explanation for thisfinding could be that RNT is amore important risk factor for anxietyand depression than for bulimia and alcohol abuse. However, this isat odds with a recent study demonstrating that rumination in fe-male adolescents predicts increases in bulimic and substance abusesymptoms at least as well as it predicts increases in depressivesymptoms (Nolen-Hoeksema et al., 2007). A more plausibleexplanation for the symptom-dependent effects in this trial con-cerns lifetime prevalence data on mental disorders. The prevalencerates for bulimia and alcohol abuse are well below the rates foranxiety disorder and depression (Swanson, Crow, Le Grange,Swendsen, & Merikangas, 2011). Therefore, larger sample sizesare required to demonstrate effects for the former two symptoms.Furthermore, future studies should include a more thoroughassessment of bulimic symptoms and alcohol abuse, as currentmeasures were very brief and may lack content validity.

Although the number of sessions completed was lower in theinternet version of the program, the efficacy was nevertheless atleast as good as in the group version on all outcome measures,which replicates earlier findings on internet interventions(Richardson, Stallard, & Velleman, 2010). This suggests that theefficacy of the new intervention is independent of the mode ofdelivery, which may facilitate dissemination to different contexts.Although cost-effectiveness is often mentioned as one of the ad-vantages of internet interventions in the literature (Christensenet al., 2014), it should be noted that a guided online intervention(as opposed to online self-help interventions) is relatively timeconsuming and may not be more time efficient than a group ses-sion, as trainers reported to have spent at least 20 min to providefeedback on each completed internet session.

The current findings need to be considered in light of a numberof limitations. First, all participants were actively recruited viascreening procedures and advertisements and thus might not haveparticipated in the preventive intervention on their own. It is,however, likely that future implementation will adopt a similarapproach. Routine screening procedures at secondary schools car-ried out bymental health organizations are common practice in theNetherlands. Increased stigma as a result of participation in a tar-geted prevention program has arisen as a major point of concern(Offord, Kraemer, Kazdin, Jensen, & Harrington, 1998). Yet, levels ofperceived stigma related to participation in preventive in-terventions have been reported to be very low (Rapee et al., 2006),and can be circumvented by delivering intervention outside school-hours, either via internet or at a different location than the schoolsetting. The focus on targeting worry in this intervention may alsoreduce stigma and enhance engagement as worry is a commonexperience that young people can easily identify with, withoutnecessarily having connotations of mental illness.

Second, participants’ past history of psychopathological di-agnoses were not assessed upon entry into this study. Therefore, itcannot be established whether the effects observed concern theprevention of first episodes of depression and GAD vs. relapse/recurrence. Third, although we aimed to test the effect of thispreventive intervention acrossmultiple symptom clusters, wewereunable to include all potentially relevant groups of symptoms. Forexample, future studies should include a measure of social anxietyas social anxiety disorder is both prevalent in adolescents andyoung adults and related to RNT (Ehring & Watkins, 2008). Fourth,participants were mainly female and all prepared for or attendeduniversity, which may limit the generalizability of our findings.However, a predominance of female participants is a naturalconsequence of the risk factors screened for in this study. Alongsidehigher prevalence rates for depression and anxiety disorders(Bekker & van Mens-Verhulst, 2007), females consistently reporthigher levels of worry and rumination (Robichaud, Dugas, &Conway, 2003). Fifth, diagnostic interviews were not conductedduring the assessments and the observed effects solely relied onself-report measures. Moreover, these self-report measures onlyassessed the point prevalence of a diagnosis of depression and GAD.The period between assessments was ignored, which may reducesensitivity to a diagnosis of depression and GAD. The high preva-lence of depression and generalized anxiety disorder at 1-yearfollow-up (32.4% and 42.2% respectively in the waitlist controlgroup) could be due to the increased risks arising from selecting asample with elevated worry and rumination, although we need tobe cautious about false positives because of the reliance on self-report questionnaire data only. To obtain insight into how excep-tional the prevalence rates of depression and GAD are, it is temptingto compare them to what is observed in other prevention trials.However, it will be difficult to conclude anything from this com-parison, because of the preselection (elevated levels of RNT) thathas taken place before participants were enrolled in this trial,which was especially chosen because RNT is a risk factor for anxietyand depression, and, as such, we anticipated high prevalence ofdepression and GAD in this high-risk sample. Previous selectiveprevention trials have used different inclusion criteria and anydifference that may result from this comparison could signify thehigh risk to develop depression or GAD for individuals withelevated levels of RNT, as well as any false positive rates from self-reportedmeasures. Note, however, that high false-positives on bothself-report measures do not explain the differential prevalencerates reported between the waitlist control and the active in-terventions. Nevertheless, to overcome this limitation of potentialfalse positives, future trials on the efficacy of this interventionshould aim to include interview measures for clinical diagnoses.

Sixth, we used a waitlist control condition, as opposed to anactive or attention control condition. In future trials, we recom-mend the use of a placebo condition or an attention control, as itwill enable determination of the extent to which the benefits of theRFCBT interventions are due to non-specific effects such as positiveexpectancy versus specific effects of the interventions. Finally,preventive effects were assessed up to 1 year after intervention,whereas it would be interesting to see the potential of this pre-ventive intervention over the longer term. Results therefore need tobe replicated using structured clinical interviews and longer-termfollow-up intervals.

Despite these limitations, the results of this randomizedcontrolled trial provide the first indication of the efficacy for apreventive intervention for anxiety disorders and depression tar-geting excessive worry and rumination. On a broader level, thisstudy adds to emerging evidence for selective prevention programstargeting transdiagnostic risk factors as a promising approach toadvance the field of prevention.


Conflict of interest

The authors declare no conflict of interest.

Funding

This study was supported by a grant from the NetherlandsOrganisation for Health Research and Development (ZONMW)(Grant Nr. 200210001).

References

Abela, J. R. Z., Brozina, K., & Haigh, E. P. (2002). An examination of the responsestyles theory of depression in third- and seventh-grade children: A short-termlongitudinal study. Journal of Abnormal Child Psychology, 30(5), 515e527. http://dx.doi.org/10.1023/A:1019873015594.

Balle, M., & Tortella-Feliu, M. (2010). Efficacy of a brief school-based program forselective prevention of childhood anxiety. Anxiety, Stress & Coping, 23(1), 71e85.

Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Manual for the beck depression in-ventory-II. San Antonio, TX: Psychological Corporation.

Bekker, M. H., & van Mens-Verhulst, J. (2007). Anxiety disorders: Sex differences inprevalence, degree, and background, but gender-neutral treatment. GenderMedicine, 4, S178eS193.

Bienvenu, O. J., & Ginsburg, G. S. (2007). Prevention of anxiety disorders. Interna-tional Review of Psychiatry, 19(6), 647e654.

Borkovec, T. D., Ray, W. J., & Stober, J. (1998). Worry: A cognitive phenomenonintimately linked to affective, physiological, and interpersonal behavioral pro-cesses. Cognitive Therapy and Research, 22(6), 561e576. http://dx.doi.org/10.1023/A:1018790003416.

Brent, D. A., Brunwasser, S. M., Hollon, S. D., Weersing, V. R., Clarke, G. N.,Dickerson, J. F., & Iyengar, S. (2015). Effect of a cognitive-behavioral preventionprogram on depression 6 years after implementation among at-risk adoles-cents: A randomized clinical trial. JAMA Psychiatry, 72(11), 1110e1118.

Bruce, S. E., Yonkers, K. A., Otto, M. W., Eisen, J. L., Weisberg, R. B., Pagano, M., &Keller, M. B. (2008). Influence of psychiatric comorbidity on recovery andrecurrence in generalized anxiety disorder, social phobia, and panic disorder: A12-year prospective study. Focus, 6(4), 539e548.

Christensen, H., Batterham, P., & Calear, A. (2014). Online interventions for anxietydisorders. Current Opinion in Psychiatry, 27(1), 7e13. http://dx.doi.org/10.1097/YCO.0000000000000019.

Ciesla, J. A., & Roberts, J. E. (2002). Self-directed thought and response to treatmentfor depression: A preliminary investigation. Journal of Cognitive Psychotherapy,16(4), 435e453.

Clarke, G. N., Hawkins, W., Murphy, M., Sheeber, L. B., Lewinsohn, P. M., & Seeley, J. R.(1995). Targeted prevention of unipolar depressive disorder in an at-risk sampleof high school adolescents: A randomized trial of a group cognitive interven-tion. Journal of the American Academy of Child & Adolescent Psychiatry, 34(3),312e321.

Cohen, J. (1988). The effect size index: D. Statistical power analysis for the behavioralSciences.2nd Ed.New Jersey: Lawrence erl-baum associates (pp. 20e26).

Craske, M. G., & Zucker, B. G. (2001). Prevention of anxiety disorders: A model forintervention. Applied and Preventive Psychology, 10(3), 155e175.

Cuijpers, P., van Straten, A., Smit, F., Mihalopoulos, C., & Beekman, A. (2008). Pre-venting the onset of depressive disorders: A meta-analytic review of psycho-logical interventions. American Journal of Psychiatry, 165(10), 1272e1280.

Curry, J., Silva, S., Rohde, P., Ginsburg, G., Kratochvil, C., Simons, A., & Mayes, T.(2011). Recovery and recurrence following treatment for adolescent majordepression. Archives of General Psychiatry, 68(3), 263e269.

Davey, G. C. L. (1994). Worrying, social problem-solving abilities, and socialproblem-solving confidence. Behaviour Research and Therapy, 32(3), 327e330.http://dx.doi.org/10.1016/0005-7967(94)90130-9.

Dozois, D. J., Seeds, P. M., & Collins, K. A. (2009). Transdiagnostic approaches to theprevention of depression and anxiety. Journal of Cognitive Psychotherapy, 23(1),44e59.

Ehring, T., Raes, F., Weidacker, K., & Emmelkamp, P. M. (2012). Validation of thedutch version of the perseverative thinking questionnaire (PTQ-NL). EuropeanJournal of Psychological Assessment, 28(2), 102.

Ehring, T., & Watkins, E. R. (2008). Repetitive negative thinking as a transdiagnosticprocess. International Journal of Cognitive Therapy, 1(3), 192e205. http://dx.doi.org/10.1680/ijct.2008.1.3.192.

Ehring, T., Zetsche, U., Weidacker, K., Wahl, K., Sch€onfeld, S., & Ehlers, A. (2011). Theperseverative thinking questionnaire (PTQ): Validation of a content-independent measure of repetitive negative thinking. Journal of BehaviorTherapy and Experimental Psychiatry, 42(2), 225e232.

Eisenberg, D., Hunt, J., Speer, N., & Zivin, K. (2011). Mental health service utilizationamong college students in the United States. The Journal of Nervous and MentalDisease, 199(5), 301e308. http://dx.doi.org/10.1097/NMD.0b013e3182175123.

Feingold, A. (2013). A regression framework for effect size assessments in longi-tudinal modeling of group differences. Review of General Psychology, 17(1), 111.

Fergusson, D. M., Horwood, L. J., Ridder, E. M., & Beautrais, A. L. (2005). Subthresholddepression in adolescence and mental health outcomes in adulthood. Archives

of General Psychiatry, 62(1), 66e72.Fisak, B. J., Jr., Richard, D., & Mann, A. (2011). The prevention of child and adolescent

anxiety: A meta-analytic review. Prevention Science, 12(3), 255e268.Gallego, M., Gerardus Emmelkamp, P. M., van der Kooij, M., & Mees, H. (2011). The

effects of a dutch version of an internet-based treatment program for fear ofpublic speaking: A controlled study. International Journal of Clinical and HealthPsychology, 11(3), 459e472.

Garber, J., Clarke, G. N., Weersing, V. R., Beardslee, W. R., Brent, D. A., Gladstone, T. R.,& Shamseddeen, W. (2009). Prevention of depression in at-risk adolescents: Arandomized controlled trial. Jama, 301(21), 2215e2224.

Garner, D. (1991). EDI-2. Eating disorder Inventory-2.Professional manual.Gilbody, S., Richards, D., Brealey, S., & Hewitt, C. (2007). Screening for depression in

medical settings with the Patient Health Questionnaire (PHQ): A diagnosticmeta-analysis. Journal of General Internal Medicine, 22(11), 1596e1602.

Giorgio, J. M., Sanflippo, J., Kleiman, E., Reilly, D., Bender, R. E., Wagner, C. A., &Alloy, L. B. (2010). An experiential avoidance conceptualization of depressiverumination: Three tests of the model. Behaviour Research and Therapy, 48(10),1021e1031.

Griffiths, C. A., & Griffiths, L. J. (2015). Recovery and reliable change rates for pa-tients scoring severe on depression, anxiety or impaired functioning in a psy-chological therapies service: Iapt. Mental Health Review Journal, 20(1), 28e35.

Gu, J., Strauss, C., Bond, R., & Cavanagh, K. (2015). How do mindfulness-basedcognitive therapy and mindfulness-based stress reduction improve mentalhealth and wellbeing? A systematic review and meta-analysis of mediationstudies. Clinical Psychology Review, 37, 1e12.

Hayes, A. F. (2009). Beyond baron and kenny: Statistical mediation analysis in thenew millennium. Communication Monographs, 76(4), 408e420.

Hedeker, D., & Gibbons, R. D. (2006). Longitudinal data analysis. John Wiley & Sons.Hong, R. Y. (2007). Worry and rumination: Differential associations with anxious

and depressive symptoms and coping behavior. Behaviour Research and Therapy,45(2), 277e290.

Horowitz, J. L., & Garber, J. (2006). The prevention of depressive symptoms inchildren and adolescents: A meta-analytic review. Journal of Consulting andClinical Psychology, 74(3), 401.

Hox, J., Moerbeek, M., & Van de Schoot, R. (2010). Multilevel analysis: Techniques andapplications. Routledge.

Hubbard, N. A., Faso, D. J., Krawczyk, D. C., & Rypma, B. (2015). The dual roles of traitrumination in problem solving. Personality and Individual Differences, 86,321e325.

Johnson, J. H., & McCutcheon, S. M. (1980). Assessing life stress in older children andadolescents: Preliminary findings with the life events checklist. Stress andAnxiety, 7, 111e125.

Just, N., & Alloy, L. B. (1997). The response styles theory of depression: Tests and anextension of the theory. Journal of Abnormal Psychology, 106(2), 221e229. http://dx.doi.org/10.1037/0021-843X.106.2.221.

Kazdin, A. E., & Blase, S. L. (2011). Rebooting psychotherapy research and practice toreduce the burden of mental illness. Perspectives on Psychological Science, 6(1),21e37.

Kessler, R. C., Berglund, P., Demler, O., Jin, R., Koretz, D., Merikangas, K. R., &Wang, P. S. (2003). The epidemiology of major depressive disorder: Results fromthe national comorbidity survey replication (NCS-R). Journal of the AmericanMedical Association, 289(23), 3095e3105.

Kessler, R., Chiu, W., Demler, O., & Walters, E. (2005). Prevalence, severity, andcomorbidity of 12-month DSM-IV disorders in the national comorbidity surveyreplication. Archives of General Psychiatry, 62(6), 617e627. http://dx.doi.org/10.1001/archpsyc.62.6.617.

Luterek, J. A., Turk, C. L., Heimberg, R. G., Fresco, D. M., & Mennin, D. S. (2002,November). Psychometric properties of the GAD-Q-IV among individuals withclinician-assessed generalized anxiety disorder: An update. In Annual meeting ofthe association for advancement of behavior therapy, reno, NV.

Lyonfields, J. D., Borkovec, T. D., & Thayer, J. F. (1995). Vagal tone in generalizedanxiety disorder and the effects of aversive imagery and worrisome thinking.Behavior Therapy, 26(3), 457e466. http://dx.doi.org/10.1016/S0005-7894(05)80094-2.

Lyubomirsky, S., & Nolen-Hoeksema, S. (1995). Effects of self-focused rumination onnegative thinking and interpersonal problem solving. Journal of Personality andSocial Psychology, 69(1), 176.

MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. Routledge.Martell, C. R., Addis, M. E., & Jacobson, N. S. (2001). Depression in context: Strategies

for guided action. New York: Norton Press.McEvoy, P. M., & Brans, S. (2013). Common versus unique variance across measures

of worry and rumination: Predictive utility and mediational models for anxietyand depression. Cognitive Therapy and Research, 37(1), 183e196.

McEvoy, P. M., Grove, R., & Slade, T. (2011). Epidemiology of anxiety disorders in theaustralian general population: Findings of the 2007 australian national surveyof mental health and wellbeing. Australian and New Zealand Journal of Psychi-atry, 45(11), 957e967.

McLaughlin, K. A., Borkovec, T. D., & Sibrava, N. J. (2007). The effects of worry andrumination on affect states and cognitive activity. Behavior Therapy, 38(1),23e38. http://dx.doi.org/10.1016/j.beth.2006.03.003.

McLaughlin, K. A., & Hatzenbuehler, M. L. (2009). Mechanisms linking stressful lifeevents and mental health problems in a prospective, community-based sampleof adolescents. Journal of Adolescent Health, 44(2), 153e160.

Merry, S. N., Hetrick, S. E., Cox, G. R., Brudevold-Iversen, T., Bir, J. J., & McDowell, H.(2012). Cochrane review: Psychological and educational interventions for

http://dx.doi.org/10.1023/A:1019873015594

http://dx.doi.org/10.1023/A:1019873015594

http://refhub.elsevier.com/S0005-7967(16)30235-2/sref3













http://dx.doi.org/10.1023/A:1018790003416

http://dx.doi.org/10.1023/A:1018790003416











http://dx.doi.org/10.1097/YCO.0000000000000019

http://dx.doi.org/10.1097/YCO.0000000000000019


























http://dx.doi.org/10.1016/0005-7967(94)90130-9








http://dx.doi.org/10.1680/ijct.2008.1.3.192

http://dx.doi.org/10.1680/ijct.2008.1.3.192







http://dx.doi.org/10.1097/NMD.0b013e3182175123




























































http://dx.doi.org/10.1037/0021-843X.106.2.221

http://dx.doi.org/10.1037/0021-843X.106.2.221










http://dx.doi.org/10.1001/archpsyc.62.6.617

http://dx.doi.org/10.1001/archpsyc.62.6.617





http://dx.doi.org/10.1016/S0005-7894(05)80094-2

http://dx.doi.org/10.1016/S0005-7894(05)80094-2
















http://dx.doi.org/10.1016/j.beth.2006.03.003








preventing depression in children and adolescents. Evidence-Based Child Health:A Cochrane Review Journal, 7(5), 1409e1685.

Metzger, R. L., Miller, M. L., Cohen, M., Sofka, M., & Borkovec, T. D. (1990). Worrychanges decisionf making: The effect of negative thoughts on cognitive pro-cessing. Journal of Clinical Psychology, 46(1), 78e88.

Meyer, T. J., Miller, M. L., Metzger, R. L., & Borkovec, T. D. (1990). Development andvalidation of the penn state worry questionnaire. Behaviour Research andTherapy, 28(6), 487e495. http://dx.doi.org/10.1016/0005-7967(90)90135-6.

Musiat, P., Conrod, P., Treasure, J., Tylee, A., Williams, C., & Schmidt, U. (2014).Targeted prevention of common mental health disorders in university students:Randomised controlled trial of a transdiagnostic trait-focused web-basedintervention. PloS one, 9(4), e93621.

Muth�en, L. K., & Muth�en, B. O. (2007). Mplus user's guide (5th ed.). Los Angeles:Muth�en & Muth�en.

Nehmy, T. J. (2010). School-based prevention of depression and anxiety in Australia:Current state and future directions. Clinical Psychologist, 14(3), 74e83.

Newman, M., Zuellig, A., Kachin, K., Constantino, M., Przeworski, A., Erickson, T.,et al. (2002). Preliminary reliability and validity of the generalized anxietydisorder questionnaire - IV: A revised self-report diagnostic measure ofgeneralized anxiety disorder. Behavior Therapy, 33(2), 215e233. http://dx.doi.org/10.1016/S0005-7894(02)80026-0.

Nolen-Hoeksema, S. (2000). The role of rumination in depressive disorders andmixed anxiety/depressive symptoms. Journal of Abnormal Psychology, 109(3),504e511. http://dx.doi.org/10.1037/0021-843X.109.3.504.

Nolen-Hoeksema, S., & Morrow, J. (1991). A prospective study of depression anddistress following a natural disaster: The 1989 loma prieta earthquake. Journalof Personality and Social Psychology, 61, 105e121.

Nolen-Hoeksema, S., Parker, L. E., & Larson, J. (1994). Ruminative coping withdepressed mood following loss. Journal of Personality and Social Psychology,67(1), 92.

Nolen-Hoeksema, S., Stice, E., Wade, E., & Bohon, C. (2007). Reciprocal relationsbetween rumination and bulimic, substance abuse, and depressive symptomsin female adolescents. Journal of Abnormal Psychology, 116(1), 198e207. http://dx.doi.org/10.1037/0021-843X.116.1.198.

Offord, D. R., Kraemer, H. C., Kazdin, A. E., Jensen, P. S., & Harrington, R. (1998).Lowering the burden of suffering from child psychiatric disorder: Trade-offsamong clinical, targeted, and universal interventions. Journal of the AmericanAcademy of Child & Adolescent Psychiatry, 37(7), 686e694.

Olatunji, B. O., Cisler, J. M., & Tolin, D. F. (2007). Quality of life in the anxiety dis-orders: A meta-analytic review. Clinical Psychology Review, 27(5), 572e581.http://dx.doi.org/10.1016/j.cpr.2007.01.015.

Paykel, E. S., Scott, J., Teasdale, J. D., Johnson, A. L., Garland, A., Moore, R., & Pope, M.(1999). Prevention of relapse in residual depression by cognitive therapy: Acontrolled trial. Archives of General Psychiatry, 56(9), 829e835.

Preacher, K. J., & Hayes, A. F. (2004). SPSS and SAS procedures for estimating indirecteffects in simple mediation models. Behavior Research Methods, Instruments, &Computers, 36(4), 717e731.

Price, M., & Anderson, P. L. (2011). The impact of cognitive behavioral therapy onpost event processing among those with social anxiety disorder. BehaviourResearch and Therapy, 49(2), 132e137.

Rapee, R. M., Kennedy, S., Ingram, M., Edwards, S., & Sweeney, L. (2005). Preventionand early intervention of anxiety disorders in inhibited preschool children.Journal of Consulting and Clinical Psychology, 73(3), 488e497.

Rapee, R. M., Wignall, A., Sheffield, J., Kowalenko, N., Davis, A., McLoone, J., et al.(2006). Adolescents' reactions to universal and indicated prevention programsfor depression: Perceived stigma and consumer satisfaction. Prevention Science,7(2), 167e177.

Richardson, T., Stallard, P., & Velleman, S. (2010). Computerised cognitive behav-ioural therapy for the prevention and treatment of depression and anxiety inchildren and adolescents: A systematic review. Clinical Child and Family Psy-chology Review, 13(3), 275e290.

Robichaud, M., Dugas, M. J., & Conway, M. (2003). Gender differences in worry andassociated cognitive-behavioral variables. Journal of Anxiety Disorders, 17(5),501e516.

Rosenthal, R., Cooper, H., & Hedges, L. (1994). Parametric measures of effect size. TheHandbook of Research Synthesis, 231e244.

Schmaling, K. B., Dimidjian, S., Katon, W., & Sullivan, M. (2002). Response stylesamong patients with minor depression and dysthymia in primary care. Journalof Abnormal Psychology, 111(2), 350e356.

Schulz, K. F., Altman, D. G., Moher, D., & Group, C. O. N. S. O. R. T. (2010). CONSORT2010 statement: Updated guidelines for reporting parallel group randomisedtrials. BMC Medicine, 8. http://dx.doi.org/10.1186/1741-7015-8-18, 18e7015-8-18.

Schwartz, J. A., & Koenig, L. J. (1996). Response styles and negative affect amongadolescents. Cognitive Therapy and Research, 20(1), 13e36.

Seligman, M. E., Schulman, P., DeRubeis, R. J., & Hollon, S. D. (1999). The preventionof depression and anxiety. Prevention & Treatment, 2(1), 8a.

Simon, G. E., VonKorff, M., & Barlow, W. (1995). Health care costs of primary carepatients with recognized depression. Archives of General Psychiatry, 52(10),850e856.

Sobell, L. C., Agrawal, S., Sobell, M. B., Leo, G. I., Young, L. J., Cunningham, J. A., et al.(2003). Comparison of a quick drinking screen with the timeline followback forindividuals with alcohol problems. Journal of Studies on Alcohol and Drugs, 64(6),858.

Spasojevi�c, J., & Alloy, L. B. (2001). Rumination as a common mechanism relating

depressive risk factors to depression. Emotion, 1(1), 25e37.Spitzer, R. L., Kroenke, K., Williams, J. B., & Patient Health Questionnaire Primary

Care Study Group. (1999). Validation and utility of a self-report version ofPRIME-MD: The PHQ primary care study. Jama, 282(18), 1737e1744.

Sprinkle, S. D., Lurie, D., Insko, S. L., Atkinson, G., Jones, G. L., Logan, A. R., et al.(2002). Criterion validity, severity cut scores, and test-retest reliability of thebeck depression inventory-II in a university counseling center sample. Journal ofCounseling Psychology, 49(3), 381e385. http://dx.doi.org/10.1037//0022-0167.49.3.381.

Stice, E., Mazotti, L., Weibel, D., & Agras, W. S. (2000). Dissonance prevention pro-gram decreases thin-ideal internalization, body dissatisfaction, dieting, nega-tive affect, and bulimic symptoms: A preliminary experiment. InternationalJournal of Eating Disorders, 27(2), 206e217.

Stice, E., Rohde, P., Gau, J. M., & Wade, E. (2010). Efficacy trial of a brief cogniti-veebehavioral depression prevention program for high-risk adolescents: Ef-fects at 1-and 2-year follow-up. Journal of Consulting and Clinical Psychology,78(6), 856.

Stice, E., & Shaw, H. E. (2002). Role of body dissatisfaction in the onset and main-tenance of eating pathology: A synthesis of research findings. Journal of Psy-chosomatic Research, 53(5), 985e993.

Stice, E., Shaw, H., Bohon, C., Marti, C. N., & Rohde, P. (2009). A meta-analytic reviewof depression prevention programs for children and adolescents: Factors thatpredict magnitude of intervention effects. Journal of Consulting and ClinicalPsychology, 77(3), 486.

St€ober, J., Tepperwien, S., & Staak, M. (2000). Worrying leads to reduced concretenessof problem elaborations: Evidence for the avoidance theory of worry.

Stockings, E. A., Degenhardt, L., Dobbins, T., Lee, Y. Y., Erskine, H. E., Whiteford, H. A.,et al. (2016). Preventing depression and anxiety in young people: A review ofthe joint efficacy of universal, selective and indicated prevention. PsychologicalMedicine, 46(1), 11e26.

Strine, T. W., Kroenke, K., Dhingra, S., Balluz, L. S., Gonzalez, O., Berry, J. T., et al.(2009). The associations between depression, health-related quality of life,social support, life satisfaction, and disability in community-dwelling US adults.Journal of Nervous and Mental Disease, 197(1), 61e64. http://dx.doi.org/10.1097/NMD.0b013e3181924ad8.

Swanson, S. A., Crow, S. J., Le Grange, D., Swendsen, J., & Merikangas, K. R. (2011).Prevalence and correlates of eating disorders in adolescents: Results from thenational comorbidity survey replication adolescent supplement. Archives ofGeneral Psychiatry, 68(7), 714e723.

Teubert, D., & Pinquart, M. (2011). A meta-analytic review on the prevention ofsymptoms of anxiety in children and adolescents. Journal of Anxiety Disorders,25, 1046e1059.

Thiel, A., & Paul, T. (2006). Testeretest reliability of the eating disorder inventory 2.Journal of Psychosomatic Research, 61(4), 567e569.

Topper, M., Emmelkamp, P. M. G., & Ehring, T. (2010). Improving prevention ofdepression and anxiety disorders: Repetitive negative thinking as a promisingtarget. Applied & Preventive Psychology, 14(1e4), 57e71. http://dx.doi.org/10.1016/j.appsy.2012.03.001.

Topper, M., Emmelkamp, P. M., Watkins, E., & Ehring, T. (2014a). Development andassessment of brief versions of the penn state worry questionnaire and theruminative response scale. British Journal of Clinical Psychology, 53(4), 402e421.

Topper, M., Molenaar, D., Emmelkamp, P., & Ehring, T. (2014b). Are rumination andworry two sides of the same coin? A strcutural equation modeling approach.Journal of Experimental Psychopathology, 5(3), 363e381.

Turner, R. J., & Wheaton, B. (1995). Checklist measurement of stressful life events.Measuring stress: A guide for health and social scientists. Oxford University Press.

Van Rijsoort, S., Emmelkamp, P., & Vervaeke, G. (1999). The penn state worryquestionnaire and the worry domains questionnaire: Structure, reliability andvalidity. Clinical Psychology & Psychotherapy, 6(4), 297e307. http://dx.doi.org/10.1002/(SICI)1099-0879(199910)6, 4<297::AID-CPP206>3.0.CO;2-E.

Van Strien, T. (2002). EDI-II-NL manual.Van Strien, T., & Ouwens, M. (2003). Validation of the dutch EDI-2 in one clinical

and two nonclinical populations. European Journal of Psychological Assessment,19(1), 66.

Van Zoonen, K., Buntrock, C., Ebert, D. D., Smit, F., Reynolds, C. F., 3rd, Beekman, A. T.,et al. (2014). Preventing the onset of major depressive disorder: A meta-analyticreview of psychological interventions. International Journal of Epidemiology,43(2), 318e329. http://dx.doi.org/10.1093/ije/dyt175.

Van der Heiden, C., Muris, P. E. H. M., Bos, A. E. R., Van der Molen, H. T., & Oostra, M.(2009). Normative data for the dutch version of the penn state worry ques-tionnaire. Netherlands Journal of Psychology, 65(2), 69e75.

Van der Zanden, R., Kramer, J., Gerrits, R., & Cuijpers, P. (2012). Effectiveness of anonline group course for depression in adolescents and young adults: A ran-domized trial. Journal of Medical Internet Research, 14(3), e86. http://dx.doi.org/10.2196/jmir.2033.

Verplanken, B., Friborg, O., Wang, C. E., Trafimow, D., & Woolf, K. (2007). Mentalhabits: Metacognitive reflection on negative self-thinking. Journal of Personalityand Social Psychology, 92(3), 526e541.

Vitiello, B. (2011). Prevention and treatment of child and adolescent depression:Challenges and opportunities. Epidemiology and Psychiatric Sciences, 20(01),37e43.

Wardenaar, K. J., van Veen, T., Giltay, E. J., de Beurs, E., Penninx, B. W., & Zitman, F. G.(2010). Development and validation of a 30-item short adaptation of the moodand anxiety symptoms questionnaire (MASQ). Psychiatry Research, 179(1),101e106.








http://dx.doi.org/10.1016/0005-7967(90)90135-6















http://dx.doi.org/10.1016/S0005-7894(02)80026-0

http://dx.doi.org/10.1016/S0005-7894(02)80026-0

http://dx.doi.org/10.1037/0021-843X.109.3.504








http://dx.doi.org/10.1037/0021-843X.116.1.198

http://dx.doi.org/10.1037/0021-843X.116.1.198







http://dx.doi.org/10.1016/j.cpr.2007.01.015






































http://dx.doi.org/10.1186/1741-7015-8-18























http://dx.doi.org/10.1037//0022-0167.49.3.381

http://dx.doi.org/10.1037//0022-0167.49.3.381



























http://dx.doi.org/10.1097/NMD.0b013e3181924ad8

http://dx.doi.org/10.1097/NMD.0b013e3181924ad8














http://dx.doi.org/10.1016/j.appsy.2012.03.001

http://dx.doi.org/10.1016/j.appsy.2012.03.001











http://dx.doi.org/10.1002/(SICI)1099-0879(199910)6

http://dx.doi.org/10.1002/(SICI)1099-0879(199910)6





http://dx.doi.org/10.1093/ije/dyt175





http://dx.doi.org/10.2196/jmir.2033

http://dx.doi.org/10.2196/jmir.2033















Watkins, E. R. (2008). Constructive and unconstructive repetitive thought. Psycho-logical Bulletin, 134(2), 163.

Watkins, E. (2015). Psychological treatment of depressive rumination. CurrentOpinion in Psychology, 4, 32e36.

Watkins, E. R. (2016). Rumination-focused cognitive-behavioral therapy for depression.New York: Guildford Press.

Watkins, E., & Baracaia, S. (2002). Rumination and social problem solving indepression. Behavior Research and Therapy, 40, 1179e1189.

Watkins, E. R., & Moberly, N. J. (2009). Concreteness training reduces dysphoria: Apilot proof-of-principle study. Behaviour Research and Therapy, 47(1), 48e53.

Watkins, E. R., & Moulds, M. (2005). Distinct modes of ruminative self-focus: Impactof abstract versus concrete rumination on problem solving in depression.Emotion, 5, 319e328.

Watkins, E. R., Mullan, E., Wingrove, J., Rimes, K., Steiner, H., Bathurst, N., & Scott, J.(2011). Rumination-focused cognitive-behavioural therapy for residualdepression: Phase II randomised controlled trial. The British Journal of Psychiatry

: The Journal of Mental Science, 199(4), 317e322. http://dx.doi.org/10.1192/bjp.bp.110.090282.

Watkins, E. R., & Nolen-Hoeksema, S. (2014). A habit-goal framework of depressiverumination. Journal of Abnormal Psychology, 123, 24e34.

Watkins, E., Scott, J., Wingrove, J., Rimes, K., Bathurst, N., Steiner, H., & Malliaris, Y.(2007). Rumination-focused cognitive behaviour therapy for residual depres-sion: A case series. Behaviour Research and Therapy, 45(9), 2144e2154.

Wilkinson, P. O., Croudace, T. J., & Goodyer, I. M. (2013). Rumination, anxiety,depressive symptoms and subsequent depression in adolescents at risk forpsychopathology: A longitudinal cohort study. BMC Psychiatry, 13, 250. http://dx.doi.org/10.1186/1471-244X-13-250, 244, X-13-250.

Zalta, A. K. (2011). A meta-analysis of anxiety symptom prevention with cognitive-behavioral interventions. Journal of Anxiety Disorders, 25(5), 749e760.

Zvolensky, M. J., Schmidt, N. B., Bernstein, A., & Keough, M. E. (2006). Risk-factorresearch and prevention programs for anxiety disorders: A translationalresearch framework. Behaviour Research and Therapy, 44(9), 1219e1239.


















http://dx.doi.org/10.1192/bjp.bp.110.090282

http://dx.doi.org/10.1192/bjp.bp.110.090282








http://dx.doi.org/10.1186/1471-244X-13-250

http://dx.doi.org/10.1186/1471-244X-13-250








behaviour research and therapy - wordpress.com...maurice topper a, b, *, paul m.g. emmelkamp a, c,...

Documents