hsnet - got it! · got it! final evaluation report debbie plath consulting & family action...

Got It! Final Evaluation Report Debbie Plath Consulting & Family Action Centre, The University of Newcastle

1

Got It! Final evaluation

report

Evaluation of the Getting on Track in Time Got It! program

May 2014


2

This report has been prepared for MH – Children & Young People, in the NSW Ministry of Health, by

Debbie Plath Consulting and the University of Newcastle. It is the final report from a two-year

evaluation of the Got It! program. It completes contracted deliverables for the project HAC 12/27.

While every effort has been made to ensure the accuracy of information in this document, it relies

on economic data, economic forecasting and program outcome measures which have inherent

uncertainties. This means that Debbie Plath Consulting and the Family Action Centre, University of

Newcastle, and their employees are unable to make any warranties in relation to the information

contained herein and disclaim liability for any loss or damage which may arise as a consequence of

any person relying on the information contained in this document and attachments.

Key Evaluation Team Members:

Debbie Plath PhD, BA, BSW hons, MSW, MAASW, Research Consultant – Project Leader

Penny Crofts BSW, DipMan, MSW, MAASW, Acting Director, Family Action Centre

Deborah Hartman BA/Dip.Teach, Grad.Dip.TESOL, M.Ed, Lecturer, Family Action Centre

Graeme Stuart PhD, BMus, BSocSci, MLitt, Lecturer, Family Action Centre.

Technical assistance with statistical analysis has been provided by the University of Newcastle

Statistical Support Service.

Newcastle, NSW

May 2014


3

Got It!: Final evaluation report

Contents

Executive summary ............................................................................................................. 6

Key outcome findings................................................................................................................. 7

Key process findings: universal components ............................................................................. 8

Key process findings: targeted components ............................................................................ 10

Key process findings: organisational features ......................................................................... 12

Key economic findings ............................................................................................................. 13

Considerations for the future development of the Got It! program ....................................... 14

Overall conclusion .................................................................................................................... 18

1. Introduction ................................................................................................................... 19

2. Disruptive behaviour in children: research and literature ................................................ 23

2.1 Child-conduct concerns: nature, prevalence and implications.......................................... 23

2.2 Interventions for conduct concerns in childhood .............................................................. 25

3. Got It! pilot program ...................................................................................................... 28

3.1 Policy context and funding for Got It! pilot ....................................................................... 28

3.2 Got It! model of care .......................................................................................................... 29

3.3 Pilot sites ............................................................................................................................ 31

3.4 Systems, protocols and interventions ................................................................................ 37

4. Process Evaluation ......................................................................................................... 41

4.1 Design and methods .......................................................................................................... 41

4.2 Findings: model of care components ................................................................................. 49

4.3 Findings: organisational features of program implementation ......................................... 99

4.4 Findings: families’ experiences of Got It! ......................................................................... 113

5. Outcome Evaluation ..................................................................................................... 138

5.1 Design and methods ........................................................................................................ 138

5.2 Findings: baseline data..................................................................................................... 145

5.3 Findings: post-intervention outcomes ............................................................................. 160

5.4 Findings: qualitative and quantitative perspectives on outcomes .................................. 169


4

6. Economic Evaluation .................................................................................................... 174

6.1 Program benefits ............................................................................................................. 174

6.2 Program costs .................................................................................................................. 181

6.3 Benefit–cost analysis ........................................................................................................ 182

7. Conclusions .................................................................................................................. 184

References ....................................................................................................................... 186

Glossary of terms and acronyms ....................................................................................... 191

Appendices ...................................................................................................................... 192

Data collection instruments……………………………………………………………………………………………….192

Cost-Benefit Analysis spread-sheets…………………………………………………………………………………..211

List of figures and tables

Figure 1: Got It! Evaluation data collection timeline ................................................................... 21

Figure 2: Health and Education staff: online survey respondents by position ............................ 46

Figure 3: Health and Education staff: online survey respondents by site ................................... 47

Figure 4: Health and Education staff: ratings of success of teacher training .............................. 54

Figure 5: Health and Education staff: ratings of success of parent information campaign ......... 61

Figure 6: Health and Education staff: ratings of success of screening......................................... 66

Figure 7: Health and Education staff ratings of success of assessment process ......................... 73

Figure 8: Health and Education staff: ratings of success of small group program ...................... 83

Figure 9: Health and Education staff: ratings of success of child behaviour management ......... 87

Figure 10: Health and Education staff: ratings of success of referral process ............................. 96

Figure 11: Health and Education staff: ratings of quality of Got it! information ....................... 102

Figure 12: Age group of parent/carer screening respondent .................................................... 147

Figure 13: Primary care arrangement for children screened..................................................... 148

Figure 14: Number of children in the families of children screened ......................................... 148

Figure 15: Employment status of parent/carer screening respondents .................................... 149

Figure 16: Highest education level of parent/carer screening respondents ............................. 149

Figure 17: Age of children in targeted group program .............................................................. 157

Figure 18: Parents’ and teachers’ ratings of improvement in behaviour of individual children

following targeted intervention (%)........................................................................................... 164

Figure 19: Parents’ and teachers’ ratings of helpfulness of group intervention for

individual children (%) ............................................................................................................... 165

Figure 20: Health and Education staff ratings of change in key groups following Got It! ......... 170

Figure 21: Life trajectory probability map for children with emerging conduct problems ....... 176


5

Table 1: Got It! pilot sites: demographic data (ABS 2011 Census) ....................................................... 33

Table 2: Got It! pilot sites: overseas born, one-parent, median income data (2011 ABS) ................... 34

Table 3: Index of relative socio-economic disadvantage (IRSD) Australia, NSW & Got It! pilot sites,

based on 2011 ABS Census data ........................................................................................................... 35

Table 4: AEDI 2012 data: Proportion of children in Developmentally Vulnerable range ...................... 36

Table 5: Health and Education staff views on Health–Education partnership ................................... 105

Table 6: Involvement in Got It! program by parents not in targeted group (n=40) ........................... 113

Table 7: Standardised measures included in Got It! evaluation ......................................................... 141

Table 8: Baseline sample ..................................................................................................................... 146

Table 9: Relationships between demographic variables and SDQ scores .......................................... 151

Table 10: Proportions in SDQ score bands across sites ...................................................................... 153

Table 11: Demographic variables associated with elevated SDQ by site ........................................... 154

Table 12: Proportions of SDQp conduct bands for children in and not in targeted groups ............... 155

Table 13: Proportions of SDQt conduct bands for children in and not in targeted groups ................ 156

Table 14: Demographic differences between children in targeted groups and other K–2 ................ 158

Table 15: Pre- and post-datasets for child behaviour and parenting practice measures ................... 161

Table 16: Outcomes of Got It! intervention (targeted group program): Pre-Post differences .......... 162

Table 17: Proportions of targeted group participants in child behaviour bands pre- and post datasets

............................................................................................................................................................ 163

Table 18: Targeted groups: shifts in child behaviour bands from pre- to post-intervention ............. 163

Table 19: Parent and teacher ratings of improvement in child’s behaviour and helpfulness of

intervention ........................................................................................................................................ 164

Table 20: Site comparisons: differences in K–2 SDQ scores between two data collection points ..... 168

Table 21: Targeted group completions as proportions of K–2 ........................................................... 169

Table 22: Discounted lifetime costs per person of behaviours associated with childhood conduct

disorder (2013 dollars) ........................................................................................................................ 177

Table 23: Benefits from universal components of Got It! program using WTP values (2013 dollars) 178

Table 24: Outcomes of Got It! Intervention (targeted group program): significant improvements in

scores .................................................................................................................................................. 178

Table 25: Diversions from long-term behaviours associated with conduct disorder: base case and

best case scenarios ............................................................................................................................. 180

Table 26: Estimated benefits from six-month Got It! targeted program gained by diverting children

from behaviours associated with conduct disorder (2013 dollars) .................................................... 180

Table 27: Expenditure on Got It! program for six months in 2013 ..................................................... 181

Table 28: Summary of modelled costs and benefits of six-month delivery of Got It! program across

three pilot sites (2013 $ prices) .......................................................................................................... 182


6

Executive summary

He has changed a lot … He listens more and he cares about others’ feelings … It’s good if you

can get them before the problems get too bad. (Parent interview at end of targeted program.)

Getting on Track in Time – Got It! is an early intervention program, delivered by Child and

Adolescent Mental Health (CAMHS) clinicians in schools. The program is directed toward children

from Kindergarten to Year 2 (K–2) and their parents/carers. Got It! is designed to reduce the

frequency and severity of disruptive behaviours and ultimately to reduce the incidence of conduct

disorder amongst children. A Got It! teams is involved with each school for 6 months, engaging in

behaviour screening, universal interventions, a targeted group program and school capacity building.

Overall findings from the Got It! evaluation:

Children with elevated conduct problems who participated in the targeted group programs

showed improvements on standardised behaviour measures. Qualitative feedback from

teachers and parents further supported that positive behaviour changes had occurred.

Changes were largely sustained at the 6 month follow-up point.

Parents in the targeted group programs developed skills in more effectively managing child

behaviour and also reported improved relationships with their children, other parents and

the school community.

The universal aspects of the program enhanced engagement by mental health services with

families in schools and facilitated access to the targeted group program.

Got It! is well regarded in schools and is an example of an effective Health-Education

partnership program.

Got It! program components and principles are supported by published research evidence.

Findings from the evaluation have, however, indicated a number of ways in which the

implementation of the program model could be further enhanced.

A best case economic evaluation scenario produced a positive net present value ($446,000,

in 2013 dollars) at 4% p.a. discounting with a 30 year projection.

A mixed-method methodology was used to evaluate the Got It! program across the three pilot sites

(Dubbo, Mt Druitt and Newcastle). Information was gathered on the experiences, impact and

outcomes of the Got It! program for the range of stakeholder groups: parents, children, teachers,

clinicians, management, referral organisations and government departments. A variety of

standardised measures and purpose-designed data collection instruments were used to generate

qualitative and quantitative data. The data was further informed by a review of international

research literature. Financial data was collected and a benefit–cost analysis, modelling different

scenarios over the long term, was conducted. The research design and data-collection instruments

were approved by a Human Research Ethics Committee.

Using standardised child behaviour and parenting practice measures, baseline data was collected on

1061 children and 997 parents/carers in 12 schools where the Got It! program was offered. Across

these 12 schools, 63 families completed the targeted group-intervention program. Post-intervention

data was collected on 973 children in the universal and targeted programs and 48 parents who

completed the targeted group programs. Analysis produced findings regarding the level of behaviour


7

problems, associations between elevated behaviour problems and other variables, outcomes from

the intervention programs, and the degree to which the targeted group program was reaching the

children with elevated behaviour problems. The qualitative data provides insights into the

experiences of families and the impact that Got It! had for children and families.

It’s been good to learn different ways of handling situations … just simple strategies as well as

big strategies, but the little ones work just as good … You’re aware that you’re learning this so

you’ll practice. I’m practising at home … and coming each week, and another little bit helps,

and another little bit helps. So you’re climbing the ladder … Learning more about myself and

how to change myself. How to behave … We need to stop and just think ‘Why are we

fighting?’ Where before I’d just go, ‘Oh, look, just shut up’. Like, you know, ‘Stop it, Stop!’ …

It’s just changed my way of thinking. (Parent at end of targeted group program)

Key outcome findings

Pre- to post-intervention changes: targeted program

For children who completed the targeted group program, significant improvements were found on

disruptive behaviour measures (Strengths & Difficulties Questionnaire parent (SDQp) and teacher

(SDQt), Eyberg Child Behaviour Inventory (ECBI)). Depending upon the measure, it was found that

between 18% and 33% of children made a positive shift into the ‘normal’ or ‘borderline’ bands for

behaviour following the targeted group intervention. Improvements were maintained by around

85% of children at the six-to-eight month follow-up point.

Eighty-three per cent of parents in targeted groups indicated that their child had been helped ‘quite

a lot’ or ‘a great deal’ and teachers regarded 41% of children in targeted groups to have been helped

‘quite a lot’ or ‘a great deal’.

A significant improvement on the Arnold Parenting Scale was found for parents who had

participated in the targeted program. The majority of parents were continuing to improve at the six-

to-eight month follow-up point.

Level and characteristics of children with elevated behaviour scores

The following factors were found to be associated with higher behaviour difficulties scores:

Being male.

Not being cared for by two parents living together.

Having a parent/carer with lower education level.

Having a parent/carer who is not employed.

Being Aboriginal.

Having a parent/carer in their 20s.

Speaking a language other than English was not associated with higher behaviour difficulties scores;

in fact, the opposite was indicated.

The schools serviced by the Dubbo Got It! team showed higher behaviour difficulties scores than

occurred in either Mount Druitt or Newcastle. This reflects the higher representation of Aboriginal

children, parents with lower education levels, and parents in their 20s in those schools.


8

In contrast to prior research, only a weak correlation was found in the baseline data between parent

assessments of child behaviour and parenting practice measures. No significant correlation was

found between teachers’ assessments of child behaviour and the parenting measures.

Families in targeted group program

Elevated behaviour scores and family assessments were used appropriately to select children,

together with a parent/carer, to participate in the targeted groups. Children in targeted groups were

found to be representative of the wider group of children with elevated conduct scores for all

demographic variables measured, except for ‘language spoken at home’. Language is presenting a

barrier to participation in Got It! targeted groups for families that speak languages other than

English.

The requirements that a parent/carer also attend the 10-week program, that the child and parent

are suited to a group program, and that the number of families in the group remain small (maximum

of eight), limit the proportion of families participating. About 80% of children with elevated conduct

problems are not selected for the targeted groups.

The Newcastle site had the highest number of participants in the targeted program, but the three

sites were comparable in terms of targeted group participants as a proportion of total number

screened and as a proportion of children with elevated SDQt conduct scores.

Universal program outcomes

The varied nature of the universal program across schools and the inability to control other variables

potentially impacting on scores made it difficult to obtain strong findings relating to the universal

program. A delay in commencing the universal program in one site (four schools) meant that

changes in scores could be compared between K–2 children in schools that had and had not

participated in a universal classroom program. It was found that the two sites that had fully

implemented the universal program had significantly greater levels of improvement on child

behaviour scores at the second data collection point than was found in the site where teachers had

not yet implemented the classroom program. Within the limitations of the study design, this could

be taken as tentative support for the impact of the universal program in K–2 classes. Process findings

illustrate the impact of the universal program for families across the schools.

Key process findings: universal components

Professional development for teachers

[Fun Friends] training gives teachers confidence and the resources to deliver the program. Even if

it’s stuff teachers already know, it gives them the mastery and a package they can deliver … It

was enthusiastically received and appears to have gone over well. (Teacher)

Professional development assists to engage schools and establish understanding of the goals and

processes of the Got It! program.

Professional development has included:

training for teachers to deliver packaged programs in K–2 classrooms

sessions developed or tailored to address other training needs/requests


9

information on the Got It! program and service context.

Having Got It! teams certified to train teachers in particular universal programs is an efficient way

of rolling out the program. Training has had positive feedback from teachers when it:

allows teachers to reflect and build on their experiences and skills

has clear links to national curriculum requirements

involves a program with teaching resources for implementation

is responsive to school requests.

Social-emotional learning programs delivered by K–2 teachers

Fun Friends has been well received by teachers, parents and children as a universal program

delivered by teachers in K–2 classrooms. It has been adopted in two pilot sites.

Parent information campaign

I picked up some ideas from the newsletter and the meeting [parent information session] … I

try not to lose my cool with them. I try to slow down and think. (Parent)

Multi-pronged information strategies are needed. These work best when negotiated with and

tailored to the particular schools, using systems and events at schools whenever practicable.

Whilst families who engage with information dissemination events are often families who are

doing well, these universal information strategies can also make an impact on some families in

need.

Generating a realistic and positive image of Got It! and enthusiastically engaging with staff and

families are key principles for effective information dissemination.

Screening of K–2 children

Screening is a resource-intensive and complex process that entails engagement, information

provision, support, follow up, data entry and analysis.

Return rates for SDQ screening forms vary considerably between schools. During the evaluation

period the range was 37% to 91% (mean 67%).

The screening process was experienced as daunting by some parents, and the purpose was not

clear to them. An emphasis on positive engagement with school communities and dissemination of

information about Got It! prior to screening is likely to improve response rates for screening

questionnaires.

Strategies for increasing return rates of SDQ screening forms have included information provision

(verbal and written), incentives, follow up of individual families by teachers, provision of forms in

community languages, and offering assistance to complete forms.

High return rates rely on involvement and commitment by classroom teachers, both in completing

the teacher version (SDQt) and facilitating return of the parent version (SDQp). Funds for teacher

release allocated to the NSW Department of Education & Communities (DEC) from the NSW

Ministry of Health (MoH) have been crucial in enabling teachers to attend to these tasks.


10

The additional questionnaires distributed as part of the outcome evaluation were found to have

had no negative impact on the return of SDQ screening forms.

The Got It! database, designed by the evaluation team, has enabled data entry and the

identification of children with elevated scores to be more efficient and streamlined.

Screening data ownership and where it is stored (health or school) is not consistent across pilot

sites.

Key process findings: targeted components

Comprehensive assessment of children with elevated conduct problems

I pushed to be selected. In fact, I really went proactive about it … really showed my interest to

the teacher and said ‘I’d really like X to go for this. I think he would really benefit.’ (Parent)

SDQ (p & t) scores, teacher and school counsellor consultations, and observations are used by Got

It! teams to select families for comprehensive assessment. SDQ domains and cut-off scores vary

across schools and sites.

Seventy-five per cent of the families identified by Got It! teams as suitable for an assessment were

assessed. The number of families assessed in each school is generally between 10 and 25.

The assessment process focused predominantly on selection for the targeted group program and,

as a result, some early intervention opportunities have been missed. Coordination between the

Got It! teams and school counsellors is valuable in assessments, but is often difficult to achieve.

Teams struggled in some schools to fill places in the groups, whilst in other schools there were interested and suitable families who could not be offered a place.

40 parent/carers of children with elevated conduct scores who were not selected for targeted group were interviewed. Of these:

70% were exposed to other components of Got It!

15% said that Got It! had a positive impact for their child

30% said that Got It! had a positive impact for them as parents/carers

20% said they were disappointed that they didn’t get more from Got It!

several didn’t understand why they were not selected for the group program

most who had attended an assessment interview were satisfied that they were not selected for the group, trusting that Got It! staff selected those in greatest need

many commented that there was inadequate information on all the components of the Got It! program and access to group is difficult for working parents.

Child-focused group led by mental health clinicians

When I first started, I was a little bit silly … From going to the Got It! program, that’s making

me feel very good and I’m starting to get happier, and if I get happier I get more work done

and I get to play more with people and I get to be nicer. It makes me feel very nice. (Child in

focus group at end of targeted group program)


11

Exploring Together is the intervention program adopted by all three Got It! teams. It comprises a

child group, parent/carer group and interactive child–parent group. It is appropriate for the setting

and children enjoy attending, but it is resource intensive.

The school staff member group facilitator plays a key role but requires thorough preparation.

Children who attended could identify some learning and techniques that they got from the groups.

Parenting group for parents

I’ve just learnt to be more consistent with him and not give in so easily. Because it’s what I

used to do, was give in too easily and just cave in to him a lot when he used to cry … But

they’ve taught me now like you’ve got to stick with it and not cave in. But then if you come to

the point where you’re feeling like caving in, walk away for five minutes, 10 minutes and then

go back and try and do it again. (Parent at end of targeted group program)

Exploring Together has also been an appropriate program for parents/carers. Parents who

attended were able to identify what they had learnt, changes they had made in their parenting

practices, and changes in their child.

Parents were very positive about the groups. Only one parent expressed disappointment that the

group was not sufficiently tailored to individual circumstances.

At the six-to-eight month follow-up point, parents said that the positive impacts on child

behaviour and parenting practices were continuing.

… still occasionally has emotional outbursts, and we try to work the ‘Stop, Think, Do’ steps. But

these outbursts are much more rare than before Got It!. I thought the Got It! program was

excellent. It opened my eyes to a lot of things I could improve in my parenting and awareness

of myself and my child. My daughter often says she wants to go back to Got It!. She really

loved it. It helped us both. Thank you very much. (Written comment by parent six months

after targeted group)

The group program strengthened protective factors through positive engagement between:

parent and child

parents and school

parents and other parents

parents and mental health service providers.

The attendance rate for parents and children together was 88% across 10 groups.

Advanced group facilitation skills are required for program effectiveness.

Offering childcare for younger children during the Got It! group and transport to the group

improves interest in, and access to, the group program.

External factors and family dynamics beyond the group program can affect the impact of the

program. For example, the existence of family violence and other significant family crises and

stresses is likely to limit the potential for positive impacts from the Got It! targeted group.


12

Individual behaviour management for children with extreme conduct problems

The behaviour management component of the model of care is not well defined and practice has

varied between pilot sites. The extent of individual work possible is also limited by staff resources.

Teachers expressed a desire to have more input from Got It! staff in designing and implementing

individual plans for children. Many teachers voiced concerns that Got It! only seemed to be for the

very small number of children in the group program and that these were often not the children

with the most concerning behaviours.

Referral for further assessment and intervention

Referral practices have not been consistent across the pilot sites and may include:

direct referral of a family to a specialised or generalised service, via phone or written

referral report, with or without follow up

provision of information to families for self-referral (verbal or written, individually tailored

or general, with or without follow up)

provision of a letter of introduction or support for a particular service that families contact

themselves, with or without follow up

educating school staff on referral services and procedures, to build capacity within schools

to make and support referrals. This may entail general information and/or individualised

recommendations for particular families.

The referral networks used by Got It! teams have been quite limited, generally with reliance on

other health services. Barriers to effective referral include:

family resistance

limited availability of suitable local services

lack of staff time to make or follow up referrals

referrals given a lower priority to screening and group intervention.

Key process findings: organisational features

Health–Education partnership

Using a Partnership Analysis Tool (VicHealth 2011), it was found that with the Got It! program a

partnership based on genuine collaboration has been established. The challenge is to maintain its

impetus and build on current success.

The quality of the Health–Education partnership is pivotal to the success of the Got It! program.

Attention to engagement strategies, communication, and decision-making mechanisms is an ongoing

requirement for success.

Funding for release of teachers in schools to undertake tasks associated with Got It! has supported

implementation and facilitated a partnership approach.


13

Eighty per cent of Health and Education staff regard the information provided on the Got It! program

as ‘above average’ or ‘excellent’.

The continual process of engagement with new schools is demanding. Got It! staff require skills in

engaging school staff with the Got It! program and generating enthusiasm to participate.

School selection for Got It!

School selections have been made on the basis of DEC regional processes and the interest of school

principals.

A three-year cycle through schools appears to be achievable within the three pilot sites.

Sustaining the impact of Got It! in schools

In some schools, strategies involving staff, children, families and school policy have continued after

Got It!. Got It! teams have attended more to supporting schools with sustainability strategies over

the time of the pilot.

School ownership and responsibility for Got It! and a principal committed to the program are key

factors contributing to sustainable impacts.

Organisational and management issues

Got It! teams are appropriately located within CAMHS. Working across several CAMHS services is

difficult for the rural pilot site.

Got It! early intervention and prevention principles differ from the dominant clinical services model

within CAMHS.

Each of the local health district (LHD) pilot sites have faced difficulties recruiting suitably qualified

staff to Got It! teams. Got It! teams require a mix of clinical, group work, educational and

community engagement skills.

Key economic findings

On the basis of available financial and intervention outcome data, and in the context of findings

from international research on the long-term costs of conduct disorder, a best case scenario (10%

recidivism) produced a positive net present value ($446,000, in 2013 dollars over 30 years) at 4% p.a.

discounting. A break-even result at 7% was also very close (BCR = 0.92), but the base case scenario

(50% recidivism) did not produce a positive net present value (BCR = 0.74 at 4% discounting).

In light of the pilot status of the program and the inability to quantify some of the anticipated

benefits, these results support an expectation that the Got It! program would provide value for

money in the long term through diverting children from the costly behaviours associated with

conduct disorder.


14

Considerations for the future development of the Got It! program

State-level issues

Collaborative strategies between DEC, MoH and LHDs, guided by the findings of this evaluation,

would help to further develop and clarify the Got It! model of care.

Got It! would benefit from negotiation and agreements between stakeholders regarding the aspects

of the model of care to be standardised and the aspects that should remain flexible and responsive

to local requirements. Clarity will support ongoing evaluation and fidelity monitoring.

The future of the Got It! program would benefit from a plan for systematic ongoing evaluation of

outcomes.

The re-establishment of an active state-level DEC–MoH decision-making body for Got It! would

provide the structure for forming agreements on the respective roles and responsibilities of Health,

Education and non-government organisations in relation to universal and screening components of

the model of care.

Strong leadership, advocacy, supervision and service development roles are required in LHDs to

achieve an optimal level of success. State-level leadership and a state-wide approach to program

development and the implementation of recommendations emerging from the evaluation are also

indicated, should Got It! receive ongoing funding.

Australian Early Development Index (AEDI) and Index of Relative Socio-Economic Disadvantage

(IRSD) data provide useful indicators of geographic areas with elevated early child behaviour

problems and areas of socio-economic disadvantage. In line with international research findings, the

current evaluation findings indicate that the level of conduct problems, and hence the need for early

intervention programs, is associated with indicators of social disadvantage. Targeting schools and

geographic areas with high levels of socio-economic disadvantage is recommended.

The association of disruptive behaviour problems with social factors suggest the importance of wider

social and community programs to address disadvantage in communities. Supportive programs and

resources to assist Indigenous communities, address unemployment, offer pathways and options for

education, and support young parents are likely to have flow-on effects in child behaviours.

Got It! resources for schools and resources for universal interventions (e.g. web-based parenting

information, podcasts, brochures, posters, multi-media presentations, seminar programs),

including materials in different community languages, could be more efficiently developed on a

state-wide basis. A state-wide position to develop and distribute resources could also work across

LHDs to support implementation of the model of care.

Close attention to budget planning and communication is needed to ensure the appropriate

allocation of teacher release funds provided by MoH and administered through DEC. Funds should

be expended to support optimal engagement by schools with the Got It! program.

The model of care

Attention to all components of the Got It! model of care, and communication about these to the

whole-school community, are likely to broaden the impact of Got It!


15

Whilst the group program component is well developed, assessment, individual behaviour

management and referral aspects of the secondary prevention could be strengthened.

The Got It! model of care could be re-defined as a four-stage model of care comprising:

• engagement

• screening and assessment

• interventions

• follow-through and referral.

Focusing more directly on the ‘engagement’ and ‘Follow-through’ phases in the Got It! model could

also support sustainability strategies. Sustained impacts from Got It! in schools begins with schools

taking on ownership and responsibility at the outset of involvement with Got It!.

The engagement phase with schools, in the term leading up to screening, entails:

marketing the Got It! program to get school staff ‘on board’

generating enthusiasm and ownership for the program amongst school staff

making collaborative decisions on how existing school systems (meetings, procedures,

timeframes) could be used for Got It! implementation

providing engaging information on Got It! to distribute to parents and carers

engaging with local community / cultural groups.

The follow-through phase is when Got It! teams would provide advice and assistance to schools

implementing strategies that flow on from the Got It! program.


Regular review of the universal program and associated training is recommended to ensure that it

continues to be appropriate and well received in schools. Investment in training for one particular

universal intervention program should not be at the expense of considering alternative programs

in the future and addressing other education needs in schools.

Web-based Got It! program information and resources for teachers could further support

professional development and understanding of the program.

Social-emotional learning programs for K–2

Decisions about which program to adopt should be based on collaborative assessments of potential

programs and associated evidence, as well as school capacities, systems and motivation to

implement. Got It! teams may take on the role of program trainers and/or consultants in the

program selection and implementation. They could also have a role in overseeing program fidelity.

Parent information campaign

Web-based resources on the Got It! program and associated parenting information, with links to

other well-credentialed websites, could be incorporated into parent information campaigns.


16

With some standardisation of the universal program across sites, promotional and web-based

resources could be developed on a state-wide basis. This would allow local Got It! teams to focus

more on the clinical components of program delivery.

In collaboration with community groups and non-government organisations (NGOs), explore the

development of parenting information resources and intervention options for engagement of

families from culturally and linguistically diverse backgrounds. This could also incorporate alternative

group, individual, assessment and information/referral intervention options.

Screening

Given the partnership nature of Got It!, agreements on the storage and ownership of screening data

are warranted.

The potential for schools to take on responsibility for the collection, entry and storage of screening

data (e.g. at Kindergarten entry) and to provide access to data by Got It! teams could be explored.

Ongoing use of the Got It! database, initiated for evaluation purposes, is recommended.

Regardless of the engagement strategies that are implemented, the reality is that families have the

right to decline involvement in screening. The delivery of quality universal programs becomes more

important for these families.

Comprehensive assessment of children with elevated conduct problems

Comprehensive procedures are needed to respond to those children and families who are identified

as potentially suited to the targeted group program but subsequently not selected. This should

incorporate information provision, intervention and referral procedures. Both individually tailored

responses and standardised information may be needed.

Child-focused group

An induction session on program content, group work principles, and professional and ethical

practice should be offered as standard practice to prepare the school staff members to be involved

in Got It! groups.

Parent group

Got It! teams should be attuned to potential violence, mental illness, substance abuse, and grief and

loss in families attending group programs and refer to appropriate specialised services if required.

Individual behaviour management

The individual behaviour management component of the Got It! model of care requires

clarification, including how the role intersects with DEC student welfare policies and the roles of

school counsellors and district guidance officers. The role of Got It! could be strengthened to sit

alongside the targeted group program and supported referrals as the suite of Got It! early

interventions for children with elevated conduct problems.

Individual behaviour management could be approached by Got It! teams in different ways:


17

Assessment, collaborative planning and referral.

Consultation and support to classroom teachers to develop and implement plans for

individual children (whether in targeted group or not).

Providing individual family intervention sessions for children not selected for the group,

likely to benefit from individual work and unlikely to access an alternative service.

Provision of individual family sessions as an adjunct to the group program.

Classroom teachers could also be better engaged in behaviour management strategies emerging

from the targeted group program through regular updates to K–2 staff on individual children and

the group program.

Referral to other services

Closer attention to a follow-through phase could better address the referral component in the Got

It! model of care with the review of individual plans, follow up on referrals, provision of additional

information and consultation with school staff.

A referral policy and procedures for each stage of Got It! is needed. Too narrow a focus on the group

intervention could detract from a key role of Got It! in facilitating and supporting longer-term

engagement of families with general community and specialist services. Referral procedures cover:

referral points in Got It! model of care

critical timeframes for referral or information provision

standard documentation and resources

processes and resources for families to build familiarity with services for self-referral

processes and resources to support referral by school staff

relationship-building strategies between Got It! and other referral services

processes to enhance collaborations with school counsellors throughout the time of

involvement in a schools

processes to avoid multiple assessments

procedures to manage wait periods.

Attention should be given to making referrals as early as possible in the intervention phase (group or

individual work).

Developing an integrated, collaborative approach to service provision and referral with Early

Intervention and Placement Prevention Program (EIPPP) services and Family Referral Services (FRS)

should be a priority for the future of the Got It! program.

Educating school staff on key referral organisations and referral procedures is a worthwhile core

component of professional development sessions delivered by Got It! teams in schools.

Further collaborative developmental work is needed to enhance and coordinate the role that school

counsellors have with the Got It! program, specifically in the areas of assessment, group program

delivery and follow-up referrals. Workshops between Got It! staff and school counsellors could assist

with the development of protocols and expectations.

Familiarity and trust of other support services could be built up amongst families by inviting

representatives from services to targeted group sessions.


18

Overall conclusion

The results of the Outcome Evaluation, when considered alongside the findings from the Process

Evaluation, show that the Got It! pilot program has demonstrated the capacity to produce positive

outcomes through a combination of universal and targeted components. The targeted component is

reaching families with children with elevated behaviour problems, and post-intervention

improvements in both parenting practices and child behaviour are evident. The positive changes in

child behaviours were found to be maintained at the six-to-eight month follow-up point for the

majority of children. Considering these findings in light of the pilot status of the program, and with

attention to the strategies and principles for future development that have emerged from the

evaluation, it is proposed that Got It! could play an effective role in reducing disruptive behaviours

and conduct disorder in the future.

The results of the Economic Evaluation indicate that on the basis of available financial and

intervention outcome data, and in the context of findings from other published research, there can

be optimism that the Got It! program would provide value for money in the long term through

diverting children from the costly behaviours associated with conduct disorder. With attention to

ongoing evaluation and quality improvements, this Education–Health partnership program has the

potential to offer both value for money and a stimulus for changing the trajectory of young people’s

lives. As an early intervention strategy, Got It! focuses on the prevention of a specific and costly

mental health disorder. Findings from this evaluation indicate that the program has the potential to

be of both social and economic value.

It’s changed our whole life … He can control his anger … He has strategies and we have

strategies … The harmony in our house is amazing. We feel like a normal family now … The

Got It! program was the single one thing that made the change. (Parent six months after

targeted group program)


19

1. Introduction

Getting on Track in Time (Got It!) is an early intervention program, delivered by Child and Adolescent

Mental Health (CAMHS) clinicians in schools. The program is directed toward children from

Kindergarten to Year 2 (K–2) and their parents/carers. Through screening, universal interventions, a

targeted group program and capacity building in schools, Got It! is designed to reduce the frequency

and severity of disruptive behaviour problems and ultimately to reduce the incidence of conduct

disorder amongst children. This is done through building skills, knowledge and capacities amongst

children, family carers and teachers over a six-month period of involvement in each school. Got It!

team members in CAMHS include mental health practitioners with occupational therapy,

psychology, psychiatry, social work, nursing and teaching qualifications. Funded by the NSW

Government’s Keep Them Safe strategy through the NSW Ministry of Health (MoH), Got It! is

delivered in partnership with NSW Department of Education and Communities (DEC). Program

funding is administered by Mental Health – Children and Young People (MH-CYP) in MoH.

The Got It! pilot program commenced in 2011 in three pilot sites: Dubbo, servicing the Western NSW

Local Health District (LHD); Mount Druitt, servicing the Blacktown Local Government Area (LGA) in

the Western Sydney LHD; and Newcastle, servicing the Newcastle LGA in the Hunter New England

LHD. By mid 2012, Got It! teams were operational in each of the three LHDs, with local structures

and processes in place to facilitate the DEC and MoH partnership approach.

In June 2012, Debbie Plath Consulting – in partnership with the Family Action Centre at the

University of Newcastle – was engaged by MoH, through a competitive tendering process, to

undertake a two-year evaluation of the pilot Got It! program. In line with the Request for Tender –

Project Specifications: HAC 12/27, three components of the evaluation have been completed:

Process Evaluation, Outcome Evaluation, Economic Evaluation.

The evaluation team had a period of consultation and engagement with stakeholders in preparation

for the formal data-collection process. This served to gain critical background and contextual

information on the Got It! program and to establish working relationships with the DEC and LHD

staff who would be play a role in the implementation of the evaluation design. The evaluation

approach involved working with the three Got It! teams in CAMHS in order to understand the

program implementation processes and to facilitate collection of good quality data. The evaluation

team also engaged with DEC and local schools to provide information on the evaluation project,

consult directly with school staff, and encourage participation in data collection processes. This

personal engagement with the stakeholders throughout the evaluation facilitated a good

understanding by the evaluators of the Got It! program delivery model, a standardised approach to

data collection across the sites and a high data return rate. The evaluation team also designed a

database for demographic data and scores from standardised measures. This worked to streamline

the school screening process for Got It! teams and the collection of pre- and post-intervention data

for the evaluation. The database can be used for ongoing clinical assessment and evaluation

purposes. The database templates were provided to the three pilot sites in January 2013, with

ongoing data entry support.


20

A mixed-method approach to the evaluation was adopted in order to understand the experiences,

impact and outcomes of the Got It! program for the range of stakeholder groups: parents, children,

teachers, clinicians, management, referral organisations and government departments. A variety of

standardised measures and purpose-designed data collection instruments were used to generate

qualitative and quantitative data. Data collection methods were shaped to suit contexts and

participant groups. The goal was to generate good quality and widely encompassing evaluation data

through ethical and respectful means. Data collection instruments are described in the ‘Design and

Methods’ sections in the Process Evaluation and Outcomes Evaluation reports in chapters 5 and 6

respectively.

Data collected for the evaluation relates to potentially vulnerable children and families. As such, it is

important that the safeguards associated with an external committee review, monitoring and

complaints process were established at the outset of the project. In addition, the research findings

from this evaluation have the potential to make an important contribution to the evidence base for

early intervention programs in Australia. In order to publish findings in the public arena, formal

Human Research Ethics Committee (HREC) approval is required. A National Ethics Application Form

(NEAF) was submitted for approval to the Hunter New England Health HREC, which has authority to

approve applications for multiple sites across NSW. The committee reviewed and approved the

research design and data collection protocols, including information sheets, signed consent forms

and data collection instruments for all participant groups. The participant groups comprise:

K–2 children

children selected for targeted groups

parents/carers of K–2 children

parents/carers participating in targeted groups

teachers and other DEC staff

Got It! team members and managers

staff in referral organisations.

All research conducted in DEC schools must also be approved through the State Education Research

Approval Process (SERAP). Additional requirements beyond HREC approval included Working with

Children criminal record checks for each of the evaluation team members and additional safeguards

whilst gathering data on school grounds (e.g. have a school staff member present whilst conducting

focus groups with children). Ethics approval reference details are as follows:

HREC approval – HREC/12/HNE/318

HNE Ref. 12/09/19/4.07.

Site-specific authorisations in LHDs:

SSA/12/WMEAD/477

SSA/13/GWAHS/4

SSA/12/HNE/452.


21

The University of Newcastle HREC endorsement:

Ref. H-2013-0008.

State Education Research Approval:

SERAP No. 2012267.

A review of research literature on the prevalence and consequences of early disruptive behaviours in

children and the outcomes of preventative intervention programs was prepared by the evaluation

team for MoH early in 2013. This literature review has been drawn upon as a resource document

across the three components of the evaluation.

A six-month sample period of program delivery across the three sites was selected for data

collection for the evaluation. The Got It! program was delivered in 12 schools during this period (four

in each of the three sites). On the basis of consultation in the early phase of the evaluation, this level

of program delivery was determined to represent full capacity. Pre- and post-intervention data was

collected. This included standardised measures of child behaviour and parenting practices collected

before the intervention, on completion of the intervention and at a six-to-eight month follow-up

point, together with a range of qualitative and process data. As the Got It! delivery schedule was not

the same at each of the 12 schools, the data collection at each point occurred over a number of

months. The timeline for data collection is illustrated in Figure 1.

Figure 1: Got It! Evaluation data collection timeline

In this report, Chapter 2 provides a brief review of research and literature on child conduct concerns

and early intervention programs. Background on the policy context that led to the establishment of

the Got It! pilot program, together with a descriptive overview of the three pilot sites, the Got It!

service model and the Got It! implementation process to date is given in Chapter 3. The Process

Qualitative data


22

Evaluation component of the project is reported in Chapter 4, including a description of methods

and findings. Chapter 5 reports on the Outcome Evaluation, with a description of the methods, an

overview of the baseline data and findings relating to post-intervention outcomes. The Economic

Evaluation, which utilises a benefit–cost analysis methodology, is reported in Chapter 6.


23

2. Disruptive behaviour in children: research and

literature

The Got It! program was developed in response to concerns about the long-term and generational

impact of early disruptive behaviour problems in children. Childhood conduct problems can escalate

to a level that is diagnosable as the mental health disorder ‘conduct disorder’ which generally

continues on to destructive adolescent and adult behaviours. Accompanying such concerns, there is

growing recognition of the potential for early interventions to divert children from this path, with

evidence for the effectiveness of parenting skills education programs. An overview of data on the

prevalence of conduct disorders is provided in this chapter, along with some findings from

international research on the outcomes of early intervention programs.

2.1 Child-conduct concerns: nature, prevalence and implications

There is a spectrum of conduct problems and antisocial behaviours in children that range in their

severity, frequency and impact. Early disruptive behaviours may or may not progress to a clinical

diagnosis of conduct disorder. The American Psychiatric Association (APA) in the DSM-V (2013)

provides a long list of behaviours indicative of conduct disorder (e.g. bullying, fighting, fire setting,

using a weapon, stealing, running away from home), three or more of which must have been present

in the previous 12 months to warrant a clinical diagnosis. Hence, the presenting behaviours

associated with conduct disorder are diverse. The essential criterion is that there is a ‘repetitive and

persistent pattern of behaviour in which the basic rights of others or major age-appropriate societal

norms or rules are violated’ (APA 2013, p. 469).

There is wide acceptance of standardised child-conduct screening tools as measures of risk in

developing what is clinically defined as ‘conduct disorder’. Given the wide range of behaviours

associated with emerging conduct problems and conduct disorder, concern has, however, been

expressed about the potential to label and pathologise children exhibiting disruptive behaviours

which they are likely to outgrow (Keenan et al. 2011). From a social perspective, the diagnosis of

children with conduct disorder can also be viewed as pathologising individuals who are reacting to

social and economic circumstances such as poverty, marginalisation and alienation from mainstream

society. For each individual child, the picture associated with conduct problems is generally a

complex one involving personal, family, social and cultural characteristics that influence how a child

behaves in certain circumstances. From the perspectives of different service sectors, the child may

be viewed as in need of support, special education or mental health intervention (Bywater 2012).

There is broad agreement in the international literature that conduct disorder is the most common

mental health concern amongst children, with disruptive behaviours accounting for up to half of

children seen by mental health services (Thomas 2010). Whilst statistics vary to some degree

between countries and research studies, international research indicates that conduct problems

affect about 5–10% of the childhood population worldwide at a level of severity that warrants a

diagnosis (Baker-Henningham et al. 2012, Bonin et al. 2011, Bywater et al. 2009, Bywater et al. 2011,

Bywater 2012, Edwards et al. 2007, Foster et al. 2007, Friedli & Parsonage 2007, Hughes 2010,


24

Sainsbury Centre for Mental Health 2009, Webster-Stratton & Reid 2010). An Australian survey,

however, found that the rate of conduct disorder in childhood (assessed by parents according to the

clinical definition) was 3%, which is lower than the rates reported internationally (Sawyer et al.

2000). Using a behaviour checklist, the same survey also reported that 13% of 4–12 year olds had

other conduct problems including delinquent behaviour (7%) and aggression (6%) (Sawyer et al.

2000). Many more children are believed to have conduct problems than have been clinically

diagnosed (Bonin et al. 2011). Some research also indicates that the rate of conduct problems

amongst children is increasing (Hughes 2010, Sainsbury Centre for Mental Health 2009, Zubrick et al.

2000).

Much higher rates of conduct disorder, at 20–35%, have been reported in some disadvantaged

communities (Bywater et al. 2009, Bywater 2012, Webster-Stratton & Reid 2010) and 37% amongst

fostered children (Bywater et al. 2011). Sawyer et al. (2000) found rates of conduct disorder to be

three times higher in the lowest income band than the highest band. An Australian population study

found, however, that whilst neighbourhood socio-economic status is associated with child conduct

disorders, this appears to be mediated by perceptions of neighbourhood safety and of

neighbourhood belonging (Edwards & Bromfield 2009).

In the UK there are about twice as many boys as girls diagnosed with conduct disorder (Bywater

2012). Foster et al. (2007) report a lifetime prevalence of 12% for boys and 7% for girls. Hughes

(2010) found that boys aged 5–10 were 2.5 times more likely than girls to be diagnosed with conduct

disorders. In Australia the reported rates are 4.4% for males and 1.6% for females (Sawyer et al.

2000). This gender difference is likely to reflect both biological and social factors influencing

behaviour patterns.

The median age of onset for conduct disorder is 11 years, but it can be diagnosed as early as three

years (Foster et al. 2007). There is wide acceptance that if early disruptive behaviour problems are

not addressed in childhood, they have a strong chance of progressing into significant and costly

longer-term problems such as school failure, criminal behaviour, teenage pregnancy,

unemployment, substance abuse, domestic violence and a range of mental health disorders (Baker-

Henningham et al. 2012, Beauchaine et al. 2010, Bonin et al. 2011, Charles et al. 2011, Dretzke et al.

2009, Foster et al. 2007, Freidli & Parsonage 2007, Hughes 2010, Scott et al. 2010, Thomas 2010).

Children with early onset of conduct disorder tend to develop more severe behaviours that progress

into more concerning behaviours and consequences in adolescence and adulthood (Foster et al.

2007, Hutchings et al. 2007).

It has been estimated that up to 40% of children with early behavioural problems such as aggression

and non-compliance go on to develop conduct disorder (Hutchings et al. 2007). Those who develop

conduct problems in adolescence tend to have less severe behaviours than those who develop

conduct problems early, and adolescent-onset conduct problems tend not continue into adulthood

(Jones et al. 2002). Estimations of the proportion of childhood conduct disorders that develop into

problem anti-social behaviours and criminality in adulthood range from 40–50% (Bonin et al. 2011,

Bywater et al. 2009, Charles et al. 2011, Freidli & Parsonage 2007, Mihalopoulos et al. 2007,

Sainsbury Centre for Mental Health 2009, Scott et al. 2001, Thomas 2010). Adults with anti-social

personality disorder generally have a history of childhood conduct disorder (Thomas 2010). Ninety

per cent of repeat juvenile offenders were found to have conduct disorder in childhood (Scott et al.


25

2001). An estimated 30% of crimes committed in the UK have been attributed to those with conduct

disorders in their childhood, with a further 50% of crimes attributed to offenders exhibiting other

conduct problems in childhood (Sainsbury Centre for Mental Health 2009). Whilst it is not the case

that all children with conduct disorders go on to engage in criminal activity, it appears that

somewhere between 14% and 60% are later imprisoned (Sainsbury Centre for Mental Health 2009).

Whilst individual life trajectories are shaped by a wide range of influences and opportunities, there is

a high probability that children with early conduct problems will go on to develop more severe, anti-

social behaviours and that these behaviours will lead to poorer outcomes later in life and costly

implications for communities in both the short term and long term.

2.2 Interventions for conduct concerns in childhood

Interventions to reduce conduct problems in children of different age groups have been developed

internationally and in Australia. These include school-, home- and clinic-based programs using a

variety of therapeutic and educational approaches. There is strong support in the research literature

for early intervention (preschool and early school years) in light of the persistence of untreated

early-onset conduct problems into adulthood and difficulties with engaging older children in

treatment (Bywater 2012, Foster et al. 2007, Hughes 2010, Murrihy et al. 2010, Raphael 2000,

Sainsbury Centre for Mental Health 2009, Webster-Stratton & Reid 2010). The case for prevention

programs has been widely presented as a less costly and more effective way forward than tertiary

mental health treatment (Raphael 2000, Zubrick et al. 2000). Primary prevention aims to stop early

disruptive behaviour problems developing into conduct disorder. Secondary prevention aims to treat

those with conduct disorder to reduce the long-term negative effects.

Preventative group programs cannot directly replace clinical treatment, as programs may not be

suited to the nature or extent of the presenting problems, or there may be issues in accessing a

group program. A well-established range of clinic-based treatment models for children and families

have also been developed from a strong evidence base and can be seen as alternatives or adjuncts

to group programs (Eyberg & Bussing 2010, Hilarski 2007, Kazdin 2010, Kazdin & Wassell 2000,

Murrihy, Kidman, & Ollendick 2010, Zisser & Eyberg 2010). These interventions generally attend to

both cognitive and problem-solving skill development for children, together with parenting skills

training for parents. Bywater (2012) points out that clinic-based programs are, however, rarely

accessed before a crisis point is reached, by which time treatment becomes more intense and

expensive. Only around 25% of children with conduct disorder access specialised treatment, and this

is generally in adolescence when problems are more severe (Scott et al. 2010, Waddell et al. 2007).

Whilst still not fully understood, the body of research on the risk and protective factors in relation to

conduct problems has been drawn upon in developing a range of evidenced, standardised

intervention programs (Hughes 2010, Morrison et al 2000, Murrihy et al. 2010). These risk and

protective factors can be grouped into social, relational, cognitive and biological factors which

interact to influence the development of conduct problems and become the focus for intervention

and change (Hughes 2010, Thomas 2010). Research indicates that protective environments at the

family, school and community levels can work to regulate the impulsivity trait that can develop into

anti-social behaviours (Beauchaine et al. 2010, Morrison et al. 2000).


26

Parenting style has been identified as the most important factor that can mediate other

environmental factors that impact on child behaviour (Bonin 2011, Hutchings et al. 2007, Sainsbury

Centre for Mental Health 2009, Scott 2010). In particular, this relates to the nature of parental

supervision, consistency or harshness in discipline, and the clarity of expectations that are set

(Bywater 2012). Australian research supports international findings regarding the risk pathways in

parenting style that contribute to disruptive behaviour disorders in children, including inconsistent

discipline and negative emotional expressiveness (Duncombe et al. 2012).

The past 10–15 years has seen an international expansion in the availability and adoption of

standardised group-based preventative programs for families with children exhibiting disruptive

behaviour problems (Dretzke et al. 2009). The Incredible Years program, developed in the US, has

been adopted in several countries and has a large and growing body of research evidence to support

its effectiveness (Bywater 2012, Foster 2010, Hughes 2010, Webster-Stratton & Reid 2010). Like Got

It!, The Incredible Years takes a multi-component intervention approach that targets school, family,

individual and/or peer systems in an interactional way. Multi-system approaches are increasingly

recognised as the most effective way forward to reduce the incidence of conduct problems (Bywater

2012, Foster et al. 2007, Woolgar & Scott 2005). Such programs draw on well-established practice

theories in group work, social learning and family processes and are based on research evidence on

risk and protective factors. They rely on well-trained staff to deliver programs in a group setting and

can require considerable multi-agency collaboration to establish (Bywater 2012, Trentcosta & Shaw

2012).

There is a growing body of research literature examining the outcomes and efficacy of such

programs and a number of systematic reviews of studies have been published. In 2012 a Cochrane

review of group-based parenting programs for early-onset conduct problems in children aged three

to 12 was published (Furlong et al. 2012). The review of 13 trials involving 1078 participants

concluded that, based on parent and independent assessments, parenting programs produced

significant reductions in child conduct problems and in negative or harsh parenting practices.

Significant improvements in parental mental health, based on parent self-assessment, were also

found. No conclusions could, however, be drawn on long-term outcomes of programs (Furlong et al.

2012).

Similarly positive conclusions were drawn by Dretzke and colleagues (2009), who conducted a

systematic review of parenting programs comprising a meta-analysis of findings and qualitative

synthesis of 157 randomised controlled trials. Whilst sample sizes in these studies tended to be

small, the review found consistent results of positive outcomes for intervention groups in

comparison to the controls. No conclusions could, however, be drawn about the relative benefits of

any one type of parenting program over others (Dretzke et al. 2009).

Waddell and colleagues (2007) carried out a systematic review of preventative programs for mental

health disorders in children, including nine randomised controlled trials for programs to prevent

conduct disorder. The programs researched included preschool, school home-visiting and group-

based programs targeting children aged nought to eight. All trials demonstrated significant

reductions in at least one conduct-related symptom or measure (Waddell et al. 2007).

Bonin et al. (2011) conducted a meta-analysis of randomised controlled trials of prevention

programs for children with a behaviour problem warranting professional intervention (i.e. in ‘clinical


27

problem’ range as measured by the Eyberg Child Behaviour Inventory (ECBI), Eyberg & Pincus 1999).

They found an average of 34% reduction in conduct problems (range: 20–68%) from pre-

intervention to post-intervention if families completed programs. This analysis included research on

home, clinic and community-based programs.

A review of evidence for parenting programs provided in Australia (Wade et al. 2012) demonstrates

that there is a vast array of programs on offer, but that the research to determine the efficacy of

programs has not always kept pace. A well-established parenting program in Australia is the Triple P

Positive Parenting Program, which is designed for adoption in a variety of settings and targets

parents of children in different age groups and with different levels of severity of behaviour

problems (Mihalopoulos et al. 2007). The evidence base for Triple P is quite strong, as evaluative

research has responded to the extent of implementation and adaptation of the program in Australia

(Wade et al. 2012). Triple P does not, however, simultaneously work with parent, child and school

systems to strengthen these relationships.

The goal of facilitating positive interactions between school, family and child has, similar to the

Incredible Years program, shaped the Got It! model of care. This distinguishes it from the parent

education approach of Triple P. The Got It! model of care adopts a partnership approach between

health and education systems in the delivery of universal screening and capacity building in the

school community and a targeted group program focused on parent and child development in

interactive group programs run in schools (NSW Ministry of Health 2011). The Got It! model of care

is based on the CAMHS and Schools Early Action (CASEA) program developed in Victoria, Australia,

which showed promising outcomes in an early evaluation (Brann et al. 2007, Department of Health,

n.d.).

In summary, there is a large and growing body of research into the outcomes of preventative

programs for children with early disruptive behaviour problems and conduct disorders that are

yielding positive results. The sustainability of outcomes into adulthood is, however, an important

issue for ongoing research. The Got It! model of care builds on the positive outcomes that are

associated with early intervention, universal access and the promotion of positive relationships

between child, family and school systems. As an early intervention program for K–2 children, the

targeted component of Got It! seeks to address emerging behaviour problems at a time when

intervention is likely to be most effective and in a context where children and families can be

actively engaged within the school setting.


28

3. Got It! pilot program

In this chapter, background information on the establishment of the Got It! pilot is provided. This

includes the policy context and funding for the pilot program, the development of the model of care,

profiles of the three pilot sites and the systems and protocols adopted in the pilot sites. An

introduction to the intervention is provided as background to the Process Evaluation report in the

next chapter, where intervention processes are examined in detail.

3.1 Policy context and funding for Got It! pilot

In March 2009 the NSW Government released Keep Them Safe – A Shared Approach to Child

Wellbeing, a whole-of-government response to the Report of the Special Commission of Inquiry into

Child Protection Services in NSW. The Keep Them Safe strategy aims to enhance the wellbeing of all

children and young people through the provision of early support to families and, through so doing,

preventing children and young people from requiring statutory child protection intervention. Keep

Them Safe programs are expected to support families, communities, government agencies and

NGOs to work together to enhance the wellbeing of children, young people and families.

The NSW Government’s Child Wellbeing strategy is consistent with a national social policy agenda

directed towards optimising child development, strengthening social inclusion, and addressing social

disadvantage. This policy agenda includes strategies targeting:

• the quality, standards, and availability of early education and care services for the 0–5 year-

old age group

• child protection and wellbeing through a national framework

• closing the gap in health, education, economic and social outcomes for Indigenous

Australians

• domestic and family violence

• economic and social participation

• disability care.

Policy and program development is increasingly informed by a public health model which combines

universal prevention initiatives with target early interventions for vulnerable groups, and reduction

in the use of statutory intervention (Council of Australian Governments 2009).

The Got It! pilot is funded by the NSW Government Keep Them Safe strategy. Got It! responds to

recommendations in the Keep Them Safe Action Plan regarding service availability – that services

should be ‘integrated, multidisciplinary and co-located, wherever practicable’ (recommendation

10.4) – and child and family services should be established in locations of greatest need, by outreach

if necessary. Got It! also aligns with the Keep Them Safe Action Plan regarding strengthening early

intervention and community-based services as well as strengthening partnerships across community

services sector. The NSW pilot site locations for Got It! were selected to be in close proximity to the

NSW Family Referral Services in order to provide opportunities for partnership. The Family Referral

Services (FRS), also funded under the Keep Them Safe initiative, link vulnerable children, young


29

people and families with appropriate support services in their local area. It is intended that Got It!

clinicians refer families to FRS to assist children, young people and families who do not meet the

statutory threshold for intervention but would benefit from accessing specific services to address

current problems and prevent escalation. Got It! specifically addresses one of the Keep Them Safe

actions, that the NSW Department of Education and Communities and NSW Health examine

strategies for expanding counselling services for parents.

The Got It! pilot program was funded under NSW Keep Them Safe for a four-year period from 2010–

11 to 2013–14. At the state level the Got It! program is administered by MoH, but as a partnership

program with DEC it was decided that the structures established for the School-Link program were

to be utilised for decision making and strategic oversight of Got it!. The School-Link committee has

been in operation for about 14 years and was formalised by a Memorandum of Understanding

between MoH and DEC in 2009. A key focus of the School-Link initiative is to facilitate the interaction

between MoH and DEC on issues relevant to the management of children and young people with

mental health problems and in the provision of shared care and collaborative support to students.

Through the School-Link committee, DEC and MoH worked together to develop the Got it! program

and model of care

Following selection of the three participating pilot sites (Blacktown LGA, Newcastle LGA and Western

NSW), funding was provided to LHDs (Western Sydney, Hunter New England and Western NSW) for

program implementation. Got It! teams are located within CAMHS services and, whilst Got It! is an

innovative outreach model of service provision and the partnership with DEC on the project is new,

community mental health services in LHDs have a long history of outreach programs and

collaboration with other organisations. Location of Got It! teams within CAMHS and LHDs was

appropriate given their role and expertise and could be incorporated into the existing management

and service provision structures.

In addition to funding LHDs, MoH has provided Keep Them Safe funding to each of the three regional

offices of DEC to provide teacher relief for staff to participate in the Got It! program in their schools.

This funding releases appropriate teachers to engage in planning, training, screening and program

delivery for the universal and targeted components of Got It!.

3.2 Got It! model of care

Based on the CASEA model in Victoria, and drawing on principles underpinning the Incredible Years

program in the US and UK, Got It! is a multi-level model of care. It incorporates universal primary

prevention involving children, teachers and parents in the early years at selected schools. This

includes providing training for teachers to better manage conduct problems in the classroom and to

deliver universal social-emotional skills development programs in class. Primary prevention

strategies also include providing information to parents through newsletters and presentations. The

secondary prevention strategy entails assessment of children identified through the screening as

having elevated disruptive behaviours and the provision of a child-focused group program integrated

with a parenting education program for parents/carers. The 8–10 week group intervention program

is led by mental health clinicians and supported by school staff. The group program is supplemented

with individual behaviour management and referral as appropriate. The intervention is depicted in

the diagram below from the model of care document (NSW Ministry of Health 2011). Detail on how


30

the model of care has been implemented is provided in the Process Evaluation findings in the next

chapter.

Got It! multi-level model of early intervention (NSW Ministry of Health, 2011)

The Got It! model of care is not a standardised or manualised program. Rather, it provides a

framework and a set of principles that allows for the incorporation of a range of strategies and

evidence-informed intervention programs. Fundamental to Got It! is the partnership between health

and education systems, which is intended to promote positive family and school interaction and

capacity building that supports social-emotional skill development in children. Essential to the Got It!

model is the combination of primary prevention, screening and secondary prevention. The model

can, however, encompass different specific intervention programs. A list of programs is suggested by

MoH in the Program Description and Model of Care document (NSW Ministry of Health 2011).

The Got It! model of care allows some flexibility in how the program is implemented so that it can be

adapted to different capacities and skills within a Got It! team and to different school contexts. This

adaptability has benefits in that the model can be adjusted to suit cultural, resource or logistical

factors. It does, however, present limitations for an outcome evaluation. Evaluating the impact of a

program that is not standardised means that there are more uncontrolled variables that limit the

ability to draw strong conclusions about program impact. During the pilot phase there has been

some consistency in how the model is implemented, including the particular targeted intervention

program used (Exploring Together), as well as aspects of screening and the universal intervention.

There have, however, also been differences between pilot sites and differences between the

participant schools, each of which has a particular culture and characteristics. Past, present and

future strategies incorporated into the Got It! intervention program may be different, whilst still

being true to the Got It! model of care. This is important to take into account in any extrapolation of

evaluation findings into the future. Qualitative and process data from program participants, schools,

Got It! teams and management at each pilot site was gathered in order to understand the contextual


31

factors and differences in program delivery in particular settings. Findings are presented in the

Process Evaluation report in the next chapter.

3.3 Pilot sites

The pilot site locations for Got It! (Blacktown LGA, Newcastle LGA and Western NSW) were selected

to align with the NSW Family Referral Services in these areas. This was designed to support and

develop partnerships across a range of organisations, in line with the Keep Them Safe initiative. Both

programs were intentionally located to service populations regarded as having high levels of

vulnerability and service need. The pilot sites are very different from each other, which provided the

opportunity to evaluate the implementation of the Got It! model in different demographic contexts.

It was late 2011 before the recruitment process started for Got It! teams in the three pilot sites, and

the three LHDs varied in the timing of recruitment and team establishment. By mid 2012 the three

teams had established protocols with DEC, selected participant schools and had begun running

programs in schools. Whilst each of the Got It! teams have dealt with recruitment delays and staff

leaving on maternity leave, the associated staff shortages are within the normal experience of

service provision in CAMHS, which is faced with the nationwide challenge of recruiting experienced

mental health clinicians in short supply. This is particularly so in rural, regional and outer suburban

areas where the Got It! teams are located. Clinicians recruited to Got It! teams have qualifications in

education, nursing, occupational therapy, psychiatry, psychology, social work or social sciences. For

the purpose of the evaluation, the Got It! program was regarded as fully operational in the three

pilot sites by the start of data collection in 2013.

Pilot site 1: Mount Druitt, servicing the Blacktown LGA in the Western Sydney LHD. Blacktown LGA is

a heavily populated outer area of metropolitan Sydney, with 301,099 people as at the 2011 Census

(ABS 2013a). Thirty-two per cent of the people who live in this LGA have a non–English-speaking

background, and nearly 55% of the population had both parents born overseas. The region has

transitioned from one of the earliest sites of European settlement to a major outer urban residential

and employment centre.

Pilot site 2: Newcastle, servicing the Newcastle LGA in the Hunter New England LHD. Newcastle LGA

is a large regional port city with a population of 148,535 at the 2011 census (ABS 2013a). Newcastle

is the capital of the Hunter region and is a major provider of health and education services, as well as

the gateway to significant mining, light manufacturing and agricultural sectors in its hinterland.

Pilot site 3: Dubbo, servicing the Western NSW LHD. The Western NSW LHD comprises 18 LGAs in

inland NSW with a total population of 227,125, as at the 2011 census (ABS 2013a). These LGAs range

in population from under 2,000 people to main service towns with populations of close to 40,000

people. Agriculture, mining and tourism are the major industries in this region. Health services are

delivered across a very wide area and access to appropriate specialist services is an ongoing issue for

communities in this very large inland region.

Demographic data from the 2011 Census (ABS 2013a) relevant to the Got It! program is presented in

tables 1 and 2. As can be seen, each of the pilot sites had a higher proportion of Aboriginal and

Torres Strait Islander (ATSI) people than the state and national averages. While representing just

over 3% of the NSW total population in 2011, Western NSW had an overall ATSI population of


32

22,186 people, which was nearly 13% of the total state ATSI population. Six LGAs in the Western

NSW LHD had 20% or more of the population who identified as Aboriginal and Torres Strait Islander.

Blacktown and Western NSW regions had higher proportions of children aged 5–9 than the state and

national levels, at 7.7% and 7% respectively. Only one LGA in the Western NSW region had a lower

proportion of 5–9 year olds than the state or national levels. Newcastle LGA had a lower than state

or national proportion of children aged 5–9.

As already noted, Blacktown LGA had a much higher proportion of people born overseas than

Newcastle and Western NSW regions. Only 57.7% of the Blacktown population was born in Australia,

compared to 82.3% for Newcastle and 87.3% for Western NSW. This compares to state and national

proportions of 69.8% and 68.6% respectively of people born in Australia (see Table 2).

As shown in Table 2, all of the Got It! pilot regions had higher than the state and national

proportions of one-parent families as at the 2011 Census. Compared to a state proportion of 16.3%

of families, and a national proportion of 15.9%, 19.5% of families in Blacktown were one-parent

families. The rates for Newcastle and Western NSW were 18.5% and 17.7% respectively. Thirteen of

the 18 LGAs in Western NSW had higher than state and national proportions of one-parent families,

ranging from 29.3% in Brewarrina to 16.6% in Warren.

Median weekly family incomes in Western NSW LHD ranged from $1,579 in Parkes LGA to $2,521 in

Mid-Western LGA, which was the only LGA in the Western NSW LHD to record a median weekly

family income higher than state or national medians. A mining operation in this LGA probably

accounts for this higher income level. The median weekly family income of $2,274 in Blacktown LGA

was slightly less than the state and national median family income of $2,370 and $2,310

respectively. The median family income in Newcastle LGA of $2,388 was slightly higher than the

state and national medians.

Socio-Economic Index For Areas (SEIFA) combines a range of variables from census data to rank

geographic areas across Australia in terms of their relative advantage and disadvantage. The Index of

Relative Socio-economic Disadvantage (IRSD) is one of the four SEIFA indices. The IRSD percentile

rankings for the LGAs in the Got It! pilot sites are presented in Table 3. A low percentile indicates a

high level of disadvantage in comparison to other LGAs in the state and nationally. Nine of the 18

LGAs in the Western NSW LHD ranked in the 25th percentile of disadvantage in the state and

nationally, with five of these in the 10th percentile of disadvantaged LGAs in the state, and four in

the 10th percentile of disadvantage Australia wide. Blacktown LGA had a national ranking of 45,

indicating that 45% of LGAs across Australia are more disadvantaged, and a state ranking of 51.

Newcastle LGA is ranked at the 64th percentile nationally and the 70th percentile in NSW. Based on

the IRSD, the Dubbo site is dealing with the most disadvantaged communities, whilst Newcastle and

Mount Druitt communities are in the mid-range of disadvantage across Australia and NSW.


33

Table 1: Got It! pilot sites: demographic data (ABS 2011 Census)

Region Population

Pop'n 5–9 ATSI

No. % No. %

Australia 21,507,717 1,351,921 6.3 548,369 2.5

New South Wales 6,917,658 434,608 6.3 172,621 2.5

Blacktown LGA 301,099 23,216 7.7 8,200 2.7

Newcastle LGA 148,535 8,332 5.6 3,927 2.6

Western NSW LHD 227,125 15,844 7.0 22,186 9.8

Western NSW LGAs:

Bathurst LGA 38,519 2,677 7.0 1,634 4.2

Blayney LGA 6,985 521 7.5 206 2.9

Bourke LGA 2,868 258 9.0 867 30.2

Brewarrina LGA 1,766 155 8.8 1,043 59.1

Cobar LGA 4,170 347 7.4 606 12.9

Coonamble LGA 4,030 302 7.5 1,178 29.2

Cowra LGA 12,147 790 6.5 793 6.5

Dubbo LGA 38,805 2,713 7.0 4,985 12.8

Forbes LGA 9,170 588 6.9 882 9.6

Gilgandra LGA 4,368 280 6.4 533 12.2

Mid-Western LGA (Mudgee/Gulgong) 22,318 1,458 6.5 870 3.9

Narromine LGA 6,585 538 8.2 1,290 19.6

Oberon LGA 5,040 303 6.0 160 3.2

Orange LGA 38,057 2,598 6.8 2,049 5.4

Parkes LGA (incl. Peak Hill & Trundle) 14,592 1,055 7.2 1,211 8.3

Walgett LGA 6,454 495 7.7 1,812 28.1

Warren LGA 2,758 187 6.8 367 13.3

Wellington LGA 8,493 579 6.8 1,700 20.0

Total: 227,125 15,844 7.0 20,552 9.8


34

Table 2: Got It! pilot sites: overseas born, one-parent, median income data (2011 ABS)

Region

Pop'n Australian

Born

Pop'n Both

Parents Overseas

Born One-parent Family

Median

Income

Families

with Chn

# % # % #

% All

families $

Australia 15,017,847 69.8 6,876,586 34.3 901,637 15.9 2,310

New South Wales 4,747,372 68.6 2,363,793 36.7 297,904 16.3 2,370

Blacktown LGA 173,765 57.7 154,113 54.7 15,661 19.5 2,274

Newcastle LGA 122,228 82.3 23,385 16.9 7,022 18.5 2,388

Western NSW LHD 198,282 87.3 18,870 8.3 10,444 17.7 NA

Western NSW LGAs:

Bathurst LGA 33,178 86.1 3,836 10.9 1,628 16.8 1,659

Blayney LGA 6,169 88.3 510 7.8 281 15.2 1,779

Bourke LGA 2,344 81.8 128 5.4 124 19.3 1,839

Brewarrina LGA 1,601 90.7 56 3.4 122 29.3 1,902

Cobar LGA 3,976 84.4 389 9.1 175 14.8 1,947

Coonamble LGA 3,642 90.3 133 3.6 234 22.1 1,600

Cowra LGA 10,557 86.9 937 8.4 454 14.2 2,136

Dubbo LGA 34,747 89.5 2,600 7.2 1,970 19.3 1,958

Forbes LGA 8,342 91.0 435 5.1 414 17.0 1,769

Gilgandra LGA 3,923 89.8 203 5.0 200 16.9 2,109

Mid-Western LGA

(Mudgee/Gulgong) 19,088 85.5 2,299 11.1 905 15.2 2,521

Narromine LGA 5,963 90.6 313 5.1 349 20.0 2,240

Oberon LGA 4,318 85.6 589 12.7 182 13.4 2,204

Orange LGA 32,454 85.3 4,137 11.7 1,798 18.5 2,090

Parkes LGA 12,847 88.0 883 6.6 699 18.4 1,579

Walgett LGA 5,118 79.3 805 14.1 345 22.8 1,714

Warren LGA 2,500 90.6 136 5.2 124 16.6 1,849

Wellington LGA 7,515 88.5 481 6.5 440 20.5 1,963

Total: 198,282 87.3 18,870 8.3 10,444 17.7 NA


35

Table 3: Index of relative socio-economic disadvantage (IRSD) Australia, NSW & Got It! pilot sites,

based on 2011 ABS Census data

Local Government Area

IRSD Percentile

Australia*

IRSD Percentile

NSW*

Blacktown LGA 45 51

Newcastle LGA 64 70

Western NSW LHD LGAs:

Brewarrina LGA 7 1

Walgett LGA 8 3

Coonamble LGA 9 4

Wellington LGA 10 5

Gilgandra LGA 14 8

Narromine LGA 19 17

Cowra LGA 20 18

Bourke LGA 22 21

Warren LGA 25 25

Parkes LGA (incl. Peak Hill and Trundle) 27 28

Forbes LGA 30 32

Cobar LGA 37 43

Mid-Western LGA (Mudgee/Gulgong) 41 46

Oberon LGA 49 57

Dubbo LGA 50 59

Orange LGA 50 58

Blayney LGA 54 62

Bathurst LGA 62 68

*Lower rank = more disadvantaged (1-100)

Population data from the 2012 Australian Early Development Index (AEDI) is of direct relevance to

the concerns of the Got It! program. The AEDI draws on very large samples and presents child

behaviour and development data on a geographic basis. This triennial national data collection

initiative collates teacher assessments of child development at the time of school commencement.

Teachers assess each child’s development across five domains: physical health and wellbeing; social

competence; emotional maturity; language and cognitive skills; and communication skills and

general knowledge (Australian Government 2013). The most recent survey, involving 289,973

children in their first year of school in over 7,415 schools in 2012, captured data on 96% of Australian

children enrolled in their first year of formal schooling in 2012. Results were released in April 2013.

Of most relevance to the Got It! program are results relating to ‘social competence’ and ‘emotional

maturity’ measures as these two domains incorporate the sub-domains of ‘respectful and

responsible behaviour towards others’, ‘aggressive behaviour’ and ‘hyperactive and inattentive

behaviour’, which are associated with conduct problems in children. Table 4 reports AEDI data

provided for the Got It! evaluation project on the proportion of children in the pilot site regions that

fall in the developmentally vulnerable category for the relevant domains and sub-domains. This data

was available for LGAs. Aggregate data was not available for the Western NSW LHD pilot site, but


36

data is reported for the 18 communities that comprise this region. Data on the sub-domains were

not obtained for these smaller communities.

Table 4: AEDI 2012 data: Proportion of children in Developmentally Vulnerable range

No. children surveyed

Social Competence

domain (% developmentally

vulnerable)

Emotional Maturity

domain (% developmentally

vulnerable)

Aggressive Behaviour sub-domain

(% develop.

vulnerable)

Hyperactive

Inattentive sub-domain

(% develop.

vulnerable)

Australia 289,973 9.3 7.6 9.1 10.3

New South

Wales

94,572 8.5 6.2 7.4 9.1

Pilot site 1:

Blacktown LGA

5,149 9.7 7.6 8.9 10.8

Pilot site 2:

Newcastle LGA

1,897 7.1 5.6 6.9 9.3

Pilot site 3:

Western NSW:

Provided below by AEDI community NA NA

Bathurst 552 7.4 4.7

Blayney 99 6.4 5.3

Bourke 69 18.2 24.6

Brewarrina 30 31.0 10.3

Cobar 78 10.3 3.8

Coonamble 66 17.7 13.1

Cowra 165 11.6 3.9

Dubbo 569 12.6 7.0

Forbes 118 4.7 5.6

Gilgandra 59 15.5 10.3

Mid-Western

(Mudgee/Gulgong) 342 10.4 11.9

Narromine 124 6.7 7.6

Oberon 75 5.8 11.6

Orange 609 13.6 9.6

Parkes 251 11.2 6.3

Walgett 112 15.6 16.7

Warren 43 7.3 2.4

Wellington 129 22.6 15.7

Note: Note all children that were part of the AEDI 2012 data collection were eligible to be assessed on all or any of the domains

Of the three pilot sites, Newcastle had lower than national and state proportions of children in the

developmentally vulnerable range for both ‘social competence’ and ‘emotional maturity’ domains

and for the ‘aggressive behaviour’ and ‘hyperactive/inattentive’ sub-domains. There is, however,

considerable variation within the area, with a number of localities within Newcastle that have

vulnerability levels above the state and national percentages. Blacktown had a higher proportion of

children in the developmentally vulnerability range than both national and NSW state levels for the


37

‘social competence’ domain and for the ‘hyperactive/inattentive’ sub-domain. Blacktown had close

to the national proportions, but higher than NSW proportions, for ‘emotional maturity’ and

‘aggression’. In Western NSW, 12 of the 18 AEDI communities in the LHD had higher proportions in

the developmentally vulnerable category for the ‘social competence’ domain than national and NSW

results. Ten of these communities had higher proportions of vulnerable children in the ‘emotional

maturity’ domain than nationally, whilst 12 communities had higher proportions compared to NSW

results. Based on this AEDI data, the Got It! pilot sites offer varied contexts for evaluation of the pilot

program. Whilst AEDI results for children in the Mount Druitt and Newcastle sites are close to results

at state and national levels, higher levels of vulnerability are evident within the Dubbo pilot site.

3.4 Systems, protocols and interventions

The partnership arrangement between DEC and MoH, central to the delivery of the Got It! program,

requires clear systems and protocols that are compatible across two different government

bureaucracies, each with their own established systems and cultures. In order to implement the Got

It! model of care, structures have been put in place at state, regional and school levels to facilitate

the partnership and establish Got It! protocols and practices. It is apparent that each of the Got It!

teams, with the assistance of their CAMHS managers and DEC regional office staff, have worked to

establish collaborative practices and appropriate protocols to implement the Got It! model of care.

Implementing an innovative new program is a challenging undertaking. This appears to have been

approached by each of the Got It! teams and steering committees in professional, creative and

committed ways.

Alongside the School-Link committee, which is the formal decision-making body for the Got It!

program, a State Implementation Group is also facilitated by MH-CYP in MoH. This group comprises

DEC representatives from student welfare and counselling services in state and regional offices,

together with LHD managers and Got It! team members. This group provides a collaborative

information exchange and problem-solving forum to support implementation across the three sites.

MH-Kids assists with resources and support; for example, in preparing a Got It! brochure for use

across the three sites, distributing information relevant to program delivery, and providing an annual

state forum that brings together Got It! clinicians and DEC representatives for professional

development and exchange of ideas.

At the regional DEC / LHD level, the Got It! implementation group manages dissemination of

information on the Got It! program to schools in the catchment areas and selection of schools for

participation. School principal representatives and DEC regional managers of student welfare

services play key roles on these groups in advising on potential participant schools and linking Got It!

teams with interested principals. Got It! team members also give presentations on the Got It!

program to relevant DEC forums. Schools that are already participating in the DEC Positive Behaviour

for Learning (PBL) program have been approached as a priority in the regions, given the expectation

that in these schools there would be an existing framework for addressing behaviour concerns. The

selection of schools is made each semester by representatives of DEC and CAMHS on the local

implementation group.


38

Once schools have been engaged for the upcoming semester, a key contact person and a school

action group are appointed in each school. During the first term of involvement with the school, Got

It! teams complete the following processes:

• Awareness raising for staff and parents/carers on the focus of the Got it! program, through

activities such as presentations at staff meetings, information stalls in the playground, items

for school newsletter.

• Screening of all K–2 children to identify potential children for the targeted group

intervention program.

• Professional development for teachers.

• Parenting information to parents/carers in written or verbal formats, using forums

appropriate to the school context.

• Assessment, selection and preparation of child and parent/carer participants for the

targeted group intervention program.

Each of the Got It! teams has developed protocols, information flyers, introduction letters, consent

forms and a variety of other documents to implement these processes, and has used standardised

assessment and screening tools. Whilst there has been some opportunity to share and collaborate

between the three pilot sites, this has been quite limited with protocols and documents in each of

the three pilot sites developing largely independently. The State Implementation Group offers a

consultative role but has not been directly involved in coordinating and guiding these processes.

Likewise, MH-CYP monitors but does not direct implementation. The development of protocols and

operational guidelines has been the responsibility of LHDs at the frontline of Got It! implementation.

Following the school engagement and screening phase, universal and targeted intervention

programs are run in the second term of involvement with a school. Four schools per semester has

become the standard across the three Got It! teams. One of the Got It! teams completes screening

and teacher development for all schools in one term and then runs the intervention programs for

the four schools in the following term. Another team has adopted a staggered approach, with two

schools being screened and two intervention programs run each term. The third Got It! team has

tried both approaches, as well as commencing screening part way through a term and running

programs across two part-terms.

The school action groups tend to comprise the K–2 teachers who facilitate the engagement of the

Got It! team with the school community. Got It! funding provided to DEC is distributed to

participating schools to release teachers to take part in the screening, assessment and group

intervention activities associated with the implementation of Got It! in the school. Teachers play a

vital role in the Got It! screening process, both in completing the child-behaviour measure, the

Strengths and Difficulties Questionnaire (the SDQ, described in detail in the outcome evaluation

chapter of the report) and coordinating the return of the SDQp (parent version) completed by

parents/carers. The SDQp is distributed to parents/carers in a variety of ways in line with school

practices, including by post, with the school newsletter or to children in class. Children return forms

completed by parents/carers to school, again in a way that fits with practices adopted in the school;

for example, giving forms to the teacher, placement in a box in the school office or returning in an

envelope. At times schools, with the support of the Got It! teams, use rewards to promote return of


39

the forms, such as chocolates or entry in a raffle draw. Items in the school newsletter, information

stands and presentations by Got It! teams to parent forums are also used to promote awareness of

Got It! and participation. The SDQt (teacher version) completed by teachers and the SDQp are

returned to the Got It! teams along with the class rolls. Data from the SDQ forms are used to

calculate which children fall into the Abnormal or Borderline bands from both parents’ and teachers’

perspectives.

SDQ screening is the starting point for more thorough assessment of children with elevated scores

and the selection of families for the targeted group programs. The screening is followed by

interviews with teachers to discuss all of the potential group participants in their class, then

interviews with a parent/carer and the child. A key component of the targeted group is that a

parent/carer must be committed and available to participate with the child in the group program.

Through this process of standardised screening, teacher reviews, clinical assessment interviews and

discussion of group participation logistics, decisions about suitable participants are made by the Got

It! team, who are then invited to the group, which commences in the following term.

The intervention programs are examined in detail in the Process Evaluation chapter of the report,

but for the purpose of providing background, a general introduction is provided here. The universal

intervention program, or primary prevention component of the Got It! model of care, has been

interpreted and implemented in different ways by the Got It! teams in the three sites. It has included

one or more activities designed to build knowledge and capacities in managing child behaviour,

targeted at either parents/carers or teachers, or both. The universal program activities may be

delivered using different mediums and forums and can be adapted to the requirements of the

particular school context. In the early implementation of the Got It! programs, attention was paid

more closely to screening, assessment and targeted intervention programs, with the universal

program taking a bit longer to establish its place in the program. As resource people for the school

communities, the Got It! teams respond in a variety of ways to the particular presenting needs and

opportunities in schools, such as contributing at parent forums, setting up information stands,

providing newsletter items and presenting at staff forums. Alongside this, attention to training

teachers to deliver established social-emotional skills development programs in their K–2 classes has

emerged as an important component of the universal / primary prevention component of Got It!

During the period of the evaluation, the primary prevention program adopted by two of the pilot

sites is a program called Fun Friends, which teachers are trained by Got It! staff to deliver in their

classrooms. Whilst not specifically targeting conduct problems, the Fun Friends program is designed

to build cognitive-behavioural strategies in children to increase social and emotional strength. The

program is based on the premise that most children will experience pressure and their resilience at

times of pressure will be enhanced if taught strategies relating to relaxation skills, emotional self-

regulation, empathy and self-awareness, building relationships, promoting self-esteem, dealing with

conflict, attempting new experiences, and positive thinking and coping (Pathways to Health and

Research Centre 2007). Initial research has indicated that the program achieves positive outcomes

for children (Pahl & Barrett 2010). Program delivery is assisted by resources for use in the classroom.

Fun Friends is one of a number of such programs that could be incorporated into the Got It!

program. Got It! clinicians have trialled different packaged programs and strategies for the universal

program, as well as developing their own.


40

The targeted group-intervention program, or secondary prevention component of the Got It! model

of care, currently being used by all Got It! teams is called Exploring Together. This is a multi-pronged

program that includes a parent and child interactive group, child group, parent group, partner

sessions and teacher meetings (Hemphill & Littlefield 2001). The children’s group focuses on

developing interpersonal skills, including anger management, conversation, problem solving,

assertiveness, decision making, perceptiveness and perspective. Parents participate in group

discussion to enhance their understanding of the underlying causes of children’s behaviour, to

challenge their unhelpful beliefs and attitudes, to promote awareness and the use of strengths,

capacities and resources, to address parenting, relationship and family issues, and to learn behaviour

management techniques. The combined group is an important feature of the program and provides

an opportunity to respond to specific parent–child interaction issues, and promote positive

interaction, problem solving, conflict resolution, play and fun. Partner and teacher meetings are also

included in the program (Hemphill & Littlefield 2001). Exploring Together has not been researched as

extensively with well-designed studies as is the case with some other programs adopted

internationally, but initial research comparing the outcomes for 106 children in the intervention

group with 39 children in a waitlist control group found movement from the clinical to the non-

clinical range to be significantly higher for the intervention group than the control group on the basis

of parent assessments. This was largely maintained at six-month and 12-month follow ups (Hemphill

& Littlefield 2001).

Each of the Got It! teams has adopted different methods for collecting and recording data on the

nature and impact of the interventions. Apart from all using the SDQp and SDQt for screening, each

of the teams has used different measures and tools in their assessment and evaluation processes. In

the past, Got It! teams compiled both qualitative and quantitative data on program participation and

outcomes for their internal evaluation and some of this data has been passed on to the evaluation

team. This includes a DVD of parents talking about their experiences in the Got It! group and a

number of written reports using a variety of pre- and post-intervention measures. The teams

commenced their own internal evaluations as the external evaluation project was not contracted

until over 12 months after the Got It! pilot commenced. The three LHDs piloting the Got It! Program

are also required to report on mandatory key performance milestones to the NSW Mental Health

and Drug & Alcohol Office and financial expenditure to NSW Kids and Families. The evaluation team

found a willingness amongst Got It! team members to examine program effectiveness and an

enthusiastic commitment to engage with the formal evaluation process.


41

4. Process Evaluation

The Process Evaluation is in four parts. The first part describes the evaluation design, participant

groups and data collection methods. The second part presents findings relating to the Got It!

intervention. The process of implementation of each of the nine components of the Got It! model of

care is described in turn in this part and associated issues are identified. The third part focuses on

findings relating to organisational features of program implementation, including the Health–

Education partnership. In the fourth part, the views and experiences of parents and children are

presented.

4.1 Design and methods

Qualitative and process data was gathered from all relevant program stakeholder groups. Data

collection instruments and methods for engaging with the participant groups were designed to suit

the particular respondent context. In total, over 150 people contributed information on their

experiences of the Got It! program. Data collection methods are detailed below. In summary,

process data collection comprised:

Individual interviews with parents on completion of the Got It! targeted group programs

during a six-month evaluation period in 2013: 11 face-to-face interviews and two phone

interviews, with at least one parent from each school.

Follow-up phone interviews with 12 parents from the targeted groups, six to eight months

after completion of the program.

Focus groups with children who participated in the targeted group programs in three schools

in 2013: three focus groups, with a total of 16 child participants.

Individual interviews by phone with parents/carers of children with elevated SDQ conduct

scores but who did not participate in the targeted group programs in 12 schools where Got

It! was running in 2013: 40 participants (1–5 participants from each school).

Online survey of DEC and Child & Adolescent Mental Health Service (CAMHS) staff involved

with Got It! implementation during the pilot phase: 79 respondents.

Focus group with Got It! staff in each of the pilot sites: three focus groups with a total of 15

participants.

Individual interviews by phone with Got It! managers and staff: four participants.

Individual phone interviews with NSW Ministry of Health staff: four participants.

Process journals compiled by Got It! teams during the implementation of Got It! in 12

schools in 2013: 22 entries across three pilot sites.

Individual interviews by phone with DEC staff involved with Got It!, representing a range of

roles in the organisation: eight participants.

Contextual and background information gleaned from meetings with K–2 teaching staff at

each of 12 schools that participated in the Got It! program during 2013.

School profiles and participation data compiled for the evaluation team by Got It! staff for

the 12 schools included in the evaluation period. Data collected includes attendance,

attrition, referral rates and a profile of the nature of involvement with each school.


42

Individual interviews by phone with external and referral organisations associated with the

Got It! program: six participants.

Qualitative data from all sources were analysed thematically and then integrated with findings from

surveys in order to draw together process findings in relation to each topic area. Quotes from

qualitative data are used throughout the Process Evaluation findings to illustrate and offer further

insights into the issues identified. The second and third parts of the Process Evaluation draw upon

data gathered from service providers: Got It! teams, CAMHS managers, DEC staff, MoH staff and

external/referral organisations. It is not intended to privilege these views over those of families by

placing them first. For the purpose of a logical report structure, however, the procedural information

in these sections assists with background and context to the experiences of families, which are

presented in the fourth part of the Process Evaluation.

All potential participants were provided with information sheets on the evaluation project and

signed consent was required prior to participation. Versions of the information sheets were tailored

to the respective participant groups. Further detail on these participant groups and the data

collection processes are provided below. All interview schedules and survey forms developed by the

evaluation team are provided in the appendix.

4.1.1 Parents in targeted groups: interviews by evaluation team

Information on the evaluation was distributed to all K–2 parents/carers in the 12 schools where the

Got It! program was run during the six-month evaluation period in 2013. Families who participated

in the targeted group program (n=68) were asked for written consent if they wished to participate in

an interview at the conclusion of the group program and/or a follow-up phone interview six to eight

months after the conclusion of the program. From those who provided signed consent and

completed the group program, a participant was selected for interview from each school group on

the basis of availability. Attempts were made to contact potential interview participants by phone,

but if there was no response or they were unavailable on the day the interview was scheduled to be

conducted, then the next parent on the list was contacted.

A total of 13 parents were interviewed from the 12 schools within a few days of the final session of

the group program. Two fathers and 11 mothers took part in the interviews. English was not the

language spoken at home for one of the participants. Respondents were contacted in advance by

phone to organise a convenient time and place to meet. Interviews with nine parents were held at

their school, one parent requested to meet at her place of work, one parent requested to be

interviewed at home and two interviews were conducted by phone. The interviews were conducted

in an informal, conversational manner. Participants were asked about their experiences with various

aspects of the program and the perceived impact for themselves and their child. Each of the

interviews was of around 30 minutes duration. Face-to-face interviews were audio-recorded and

transcribed. Detailed notes were taken during the two phone interviews. The qualitative data was

thematically analysed and illustrative quotations were extracted.

Six to eight months after the completion of the group program, phone interviews were conducted

with 12 parents from the targeted groups. Ten of those parents who had been previously

interviewed in person by the evaluation were included in these follow-up phone interviews. As not

all from the original interview participants could be contacted for the follow-up phone interview,


43

additional parents were interviewed in an attempt to include a participant from each school. In the

end 12 parents from targeted groups in 11 of the 12 schools were contacted for follow-up phone

interviews, including 10 of the original interviewees. The phone interviews were brief, lasting around

10 minutes. Parents were asked to comment on the impact of the Got It! program for them and their

child now that it was several months down the track, whether any positive changes had been

maintained, whether improvements were continuing and what was found to help or hinder

behaviour. In addition, all parents who completed the targeted groups, who provided consent to

participate in the evaluation and for whom postal address details were still current at this time

(n=57) were sent a letter requesting any written feedback on the impact of the program together

with a reply-paid envelope. Four parents took up this offer, three of whom were also in the group

interviewed by phone.

4.1.2 Children in targeted groups: focus groups by evaluation team

Three focus groups were run with a total of 16 children. Participants were drawn from the 65

children who participated in the targeted group program across 12 schools in the first half of 2013.

All were in Kindergarten to Year 2, aged five to eight years. Parents/carers of these 65 children were

provided with a consent form to sign if they agreed to their child taking part in a focus group

discussion about their experiences in the Got It! group. A focus group was organised at one school in

each pilot area, being the school that had the highest number of children with consent to participate

in the focus group. Three groups were run with seven, five and four participants respectively, making

a total of 16 participants. As these children were part of the targeted group component of the Got It!

program, all had SDQ ‘conduct’ scores in the Abnormal or Borderline bands.

The focus group format was designed with age-appropriate activities, including a sheet of ‘emotions

faces’, ‘Bears’ picture cards (used as a trigger to discuss behaviours and what the children had learnt

about these behaviours) and a mock radio interview, in which children shared their experiences of

Got it! ‘on the radio‘ to children in other schools. The groups were run for about 30 minutes in a

room at the school. One or two members of the evaluation team facilitated each focus group session

and a teacher from the school was also present. The focus groups were audio-recorded and

transcribed for analysis.

4.1.3 Parents/carers not in targeted groups: phone survey by evaluation team

Information on the evaluation was distributed to all K–2 parents/carers in 12 schools together with

the screening questionnaires for the Got It! program. A consent form regarding a phone survey to

provide feedback on the Got It! program was also included. From those parents/carers who provided

signed consent to take part in the phone survey, a purposive sample of 40 respondents was taken.

The goal was to select a group that had characteristics similar to participants in the targeted group

program. Using SDQ score results, only parents/carers of children with Abnormal or Borderline

scores in the ‘conduct’ sub-scale of the SDQ (parent or teacher version) were included. The

proportion of participants from each pilot site was close to the proportion across the targeted

groups. There were 12 participants from the Dubbo area (30%), 12 participants from the Mount

Druitt area (30%) and 16 participants from the Newcastle area (40%). Between one and five

participants were included from each of the 12 schools (determined by availability during the phone

survey period).


44

The 40 parents/carers in the sample were phoned after the Got it! program had finished in the

school. They were asked about the nature of their involvement in the Got It! program, perceived

impacts on their child, themselves and the wider school community, views on the selection process

for the targeted groups, and any general comments or feedback. The SDQp was then completed as a

post-intervention measure for the outcome evaluation of the universal program.

4.1.4 Families in targeted groups: group participation data collected by Got It! teams

During the six-month evaluation period in 2013, the Got It! teams collected a range of data for the

evaluation. This largely comprised measures of child behaviour and parenting practises collected for

the Outcome Evaluation component. For the Process Evaluation, data on the level of participation in

the targeted groups was collected on a weekly basis across the 12 schools and entered into a

spreadsheet provided by the evaluation team. This provided information on group attendance

patterns for each child in the targeted group and their parent/carer, along with any additional

comment on factors impacting on attendance. If partner sessions were run as part of the group

program, attendance was also recorded. Attendance data from 11 of the 12 schools was provided by

the Got It! teams to the evaluation team at the conclusion of the group programs.

4.1.5 School populations: school profiles completed by Got It! teams

The Got It! teams were also provided with a ‘school profile’ template to record the nature of their

involvement in each of the 12 schools. This was set out in line with each of the components of the

Got It! model of care. Got It! teams were asked to describe the activities they engaged in for each

school and to record the associated number of participants that took part. The school profiles were

sent through to the evaluation team at the end of the period of involvement with each school.

4.1.6 Got It! teams: process journals completed in Got It! team meetings

The day-to-day implementation of the Got It! program has largely been the responsibility of a Got It!

team in each of the three pilot sites (Dubbo, Mount Druitt and Newcastle). During the period of data

collection in 2013 there were several staff changes, absences and departures within the Got It!

teams. During the evaluation period, the teams comprised between two and four individual Got It!

staff members. The teams are multi-disciplinary, with education, occupational therapy, psychiatry,

psychology and social work professions represented. Got It! teams report to managers in the Child

and Adolescent Mental Health Service (CAMHS) within the Local Health District (LHD) where the

pilot site is located.

Got It! teams were provided with a process journal template to guide discussions during team

meetings on process issues of interest to the evaluation. This included information on what was

regarded to have worked well, what hadn’t, the factors contributing and what was learnt from this.

The template also included a section for challenges and key decisions. It was left to the teams to

decide how frequently they completed these. One team completed 14 entries over a six-month

period and the other two teams completed four entries each. The journal templates were forwarded

to the evaluation team as they were completed. These were reviewed by the evaluation team as

they were provided by the Got It! teams so that any key process issues could be identified for

ongoing attention as part of the process evaluation. All process journals were reviewed again during


45

the data analysis phase of the evaluation. The process journals proved to be effective in capturing ‘in

the moment’ reflections on key process and implementation issues for the Got It! program.

4.1.7 Got It! staff and managers: focus groups and interviews by evaluation team

A focus group was conducted for the Got It! team and manager in each of the three pilot sites. A

total of 15 staff participated in the three focus groups, comprising current team members (11), past

team members (two) and CAMHS managers/team leaders (two). The focus groups were structured

to work through each of the nine components of the Got It! model of care in turn, with participants

asked to draw on their own experiences of implementation and comment on what works, what

doesn’t, what has changed over time, and suggested improvements for that component in the

model of care. Participants were also asked, toward the end of the focus group, to comment on

organisational issues and sustainability issues. Each focus group lasted for two hours and was audio-

recorded and fully transcribed for thematic analysis and extraction of quotes.

The option of an individual interview was also offered to Got It! staff and managers either if they

could not attend the focus group or if they wished to provide additional comment. Four staff took up

this option, two of whom had also participated in a focus group. These interviews were conducted

by phone.

4.1.8 Health and Education staff: online survey by evaluation team

An online survey was designed for completion by Health and Education staff involved with the Got

It! program during the pilot phase. Respondents were asked to identify their role with the Got It!

program, to rate the impact of the program for the various target groups, rate the success of the

different components of the Got It! model of care, and evaluate the effectiveness of the partnership.

There were also several open questions providing opportunities for comment and suggestions for

improvement or future development. The online survey did not include a question regarding the

implementation of universal social-emotional programs in K–2 classes. Due to the varying degrees to

which this has been implemented by Got It! across the pilot sites (in some schools there was no

defined social-emotional learning program), the evaluation team was concerned that such a

question could be misinterpreted to refer to any of a range of programs that could be defined as

‘social-emotional learning’ that are being run in schools. To avoid confusion about whether or not

responses related to the Got It! program, a question regarding this component of the Got It! model

of care was not included.

The survey included 18 questions adapted from the VicHealth Partnership Analysis Tool (VicHealth

2011). The original survey has 35 questions rated from Strongly Disagree to Strongly Agree, with five

points on the scale. Three percentage bands are published for the tool (Bottom 35%, Mid 30% and

Top 35%) with associated statements summing up the status of the partnership relationship in each

band. The 18 questions included in the online survey were drawn from each of the seven sections of

the partnership tool and were selected on the basis of their relevance to the Got It! program. Some

minor wording changes (including adding the names of organisations) were made in order to make

the questions more easily understood by respondents and directly relevant to Got It!. Such

modifications were deemed appropriate given that this is a tool designed to assist reflection and

development of partnerships, rather than being a standardised scale. At the end of the survey,


46

respondents were asked to provide contact details if they wished to do a follow-up interview with

the evaluation team.

Email addresses of potential respondents were gathered from Got It! teams and schools involved

with the Got It! program. The link for the online survey was emailed to 133 staff in Health and

Education who have been involved with Got It! during the pilot period. There was a request in the

email to forward the link on to others who may have been missed, so the final number of people to

whom the survey link was emailed is not known. There were 79 respondents to the online survey

and 68 of these completed the survey to the end. A profile of the positions held by respondents and

the site with which respondents were associated is provided in figures 2 and 3. The largest group of

respondents is K–2 teachers, comprising 38 (48%) of the respondents. Newcastle was the site with

the largest group of respondents, comprising 30 (39%) of the respondents.

Figure 2: Health and Education staff: online survey respondents by position

05

10152025303540

Online survey respondents by position (n=79)


47

Figure 3: Health and Education staff: online survey respondents by site

4.1.9 DEC staff: interviews and meetings with staff groups by evaluation team

A member of the evaluation team met with K–2 teaching staff groups at the 12 schools where Got It!

programs were run in the first half of 2013. The purpose of these meetings was partly to facilitate

the evaluation and associated data collection processes in the schools and to provide school staff

with some feedback on initial findings, but it also provided the opportunity to gain a more nuanced

understanding of the school-based context for Got It! implementation and local reactions and

experiences. Whilst this did not comprise a formal part of the data collection process it offered

useful background understanding. At these meetings, attendees were alerted to the upcoming

online survey and email addresses were collected to forward the survey link. They were also

provided with information sheets on the evaluation and a consent form if they wished to take part in

an individual interview to provide further comment on the Got It! program.

DEC staff who indicated in the online survey, or through return of a signed consent form, that they

wished to participate in an interview were contacted by email and/or phone to set up an interview

time. Several people who had indicated an interest did not respond to these contacts and were not

pursued. Eight DEC staff were interviewed, comprising, two principals, one K–2 assistant principal,

one K–2 teacher, two district guidance officers, a school counsellor and a regionally based Positive

Behaviour for Learning coordinator. This offered good coverage of the range of DEC roles involved

with the Got It! program and provided useful qualitative data and local examples to complement the

data gathered from the online survey. There were four DEC staff from one pilot site, three from

another site and one from the third site. Individual interviews were conducted by phone and

detailed notes were taken. DEC staff were asked about their involvement with Got It!, perceived

impacts, Health–Education partnership issues, school selection, engagement with families, universal

and targeted components of the intervention and sustainability of impacts in the school. Interviews

were around 30 minutes in duration.

0

5

10

15

20

25

30

35

Dubbo Mt Druitt Newcastle

Online survey respondents by site (n=77)


48

4.1.10 NSW Ministry of Health staff: interviews by evaluation team

Toward the end of the process data collection period, phone interviews were held with four key staff

involved with the establishment and oversight of the Got It! program within the Ministry of Health.

These interviews were designed to facilitate information on issues associated with the state-wide

administration of the program and perspectives on how the Got It! program fits within the broader

picture of state health priorities and trends.

4.1.11 External and referral services: interviews by evaluation team

As a time-limited early-intervention program it is important to evaluate how effectively Got It! can

integrate with the wider network of specialist services and family/parenting support services. The

evaluation team sought to interview external and referral organisations in the service networks in

each of the pilot sites where Got It! programs are operating. Each of the Got It! teams were asked to

provide a list of referral organisations and contact names for service providers who were in a

position to comment on the role of the Got It! program in relation to their service and to illustrate

this with some case examples. This request proved to be challenging for the Got It! teams. After

several requests from the evaluation team, one team was unable to provide contact details for any

referral organisation (within or outside Health) that would be able to provide comment. One team

provided contact details for one service within the LHD. Two staff within this service volunteered to

be interviewed. The third team provided the names of seven organisations. Two of these

organisations were not contacted as there were no specific contact people identified, another two

did not respond to the phone and email requests to set up an interview time and one other was

contacted but said she had not heard of the Got It! program. Thus, two interviews eventuated for

this site. In addition to these four interviews (two from each of two sites), the evaluation team was

contacted by the coordinator of a potential referral organisation who had heard about the

evaluation of the Got It! program and wished to have some input. An interview was subsequently set

up. Got It! teams also make referrals to school counsellors. Input into the evaluation was made by

school counsellors through the online survey and DEC interviews. The evaluation team also made

contact with a representative from the KidsMatter program, which is a national initiative supporting

mental health strategies in schools. Whilst KidsMatter is not a referral organisation for families, the

organisation’s charter has clear parallels with the goals of the Got It! and it was regarded by the

evaluation team as a useful organisation to consult for the purposes of the evaluation.

In total, six phone interviews were conducted with relevant organisations external to Got It! and

DEC. Three respondents were from two health services within LHDs, one respondent was from a

Family Referral Service, one respondent was from an Early Intervention and Placement Prevention

Program service and, finally, KidsMatter was consulted. The interviews were tailored somewhat to

the organisation and nature of contact with Got It! in order to gather views on the positive aspects

and potential for Got It!, referral, coordination and strategic implementation processes, experiences

with referral of particular families, and suggestions for future development. Interviews were each of

about 30 minutes duration and detailed notes were taken for analysis.


49

4.2 Findings: model of care components

The Got It! model of care is described in the NSW Ministry of Health (2011) Getting on Track in Time

– Got It! Program Description and Model of Care. This document was provided to the three LHDs

that participated in the pilot to guide the implementation of the pilot program. The document

provides a framework and program principles along with a choice of targeted and universal

intervention programs. Within the Got It! model, there is room to choose intervention packages and

to adopt practices and procedures that are appropriate to local contexts.

This section describes how the Got It! model of care has taken shape within the LHDs, including

similarities and differences across the three pilot sites. Information gathered from focus groups with

Got It! staff was the starting point for this section of the report. The focus group findings are

enhanced with findings and examples from the process journals, interviews with Got It! staff, MoH

staff, DEC staff and referral organisations, and results from the online survey. The aim has been to

draw together findings from different stakeholder perspectives in relation to each component in the

model of care. The perspectives from participant families, presented later in this chapter, add

further insights into the model.

The Got It! model of care is innovative in its attention to both universal and targeted interventions

to reduce conduct problems and in that it is a partnership program between MoH and DEC.

Got It! teams based in the CAMHS work closely with schools to provide accessible interventions at

the school. Schools are provided with funds to release teachers from normal duties in order to

participate in the Got It! program. Across both Health and Education sectors, the Got It! program is

valued for the focused attention it gives to the goal of reducing disruptive behaviours in children and

for the potential it offers health services to connect with the wider community through the delivery

of programs to families within schools.

Each component of the Got It! model of care is discussed in turn in the following sections. Based on

data gathered from the stakeholder groups, the implementation of each component is described

and associated issues identified. In summary, the components in the Got It! model of care are:

Primary prevention – universal components

1. Professional development for teachers to assist in managing conduct problems and to deliver

universal interventions.

2. Delivery of Social-Emotional Learning programs by K–2 teachers.

3. Parenting information campaign for K–2 (seminars, newsletters etc.).

4. Screening of all K–2, by both teachers and parents, to identify students for intervention

program.

Secondary prevention – targeted components

5. Comprehensive assessment of children with elevated conduct problems using a range of

tools.

6. Child-focused group led by mental health clinicians and supported by school staff (to reduce

conduct problems, manage emotions, improve social skills, promote self-esteem and problem

solving).


50

7. Parenting group for parents (offered after hours if necessary, with the provision of transport

and child care as required).

8. Individual behaviour management for children with extreme conduct problems.

9. Referral to services (CAMHS and others) for further assessment and intervention.

Once teams were established in the three LHDs and the procedures for implementing the model of

care underway, a capacity of eight schools per annum became the standard for each team. With the

current staffing levels (including vacant positions and maternity leave absences), all teams felt

stretched in their capacity to fully implement the model of care. As to whether four schools

commence in the program at the same time or there is a staggered commencement has been

decided locally. There is no apparent advantage of one approach over the other and it seems

appropriate for such timing issues to continue to be decided at the local level at Education–Health

Steering Committee level.

4.2.1 Professional development for teachers

The Got It! model of care (NSW MoH 2011, p. 15) states that interventions will involve:

Intensive professional development for teachers addressing the management of conduct

problems and delivery of universal interventions to all children in the first three years of

primary school.

It has taken longer for the three pilot sites to define the nature of universal interventions and to

implement associated teacher training than has been required to establish the screening and

secondary intervention practices. This has partly been due to the higher level of need amongst the

targeted group children, which was prioritised by the Got It! teams, particularly in the face of Got It!

position vacancies. Alongside this, however, more time has also been required to determine the

most appropriate content for and approaches to teacher training. The pilot sites trialled and adopted

different approaches to teacher training and have learnt from their respective experiences.

Two sites have now adopted the Fun Friends program (Pathways Health and Research Centre, n.d.)

as the universal program. Most Got It! staff have been trained to provide training to K–2 teachers to

deliver the program in their classrooms. Teacher training for Fun Friends delivery entails six to eight

hours of training. Got It! teams deliver a packaged training program or teachers can also access

training through Pathways. One of the benefits of this program for the rural pilot site is that

Pathways provides an online training platform using webinar technology. This has enabled teachers

in different towns to join the training at a regular weekly timeslot over several weeks and to benefit

from interactions with teachers in other schools, thus overcoming the distance barrier. Whilst the

online training was found by Got It! staff to be less engaging and less interactive than face-to-face

training, it is an efficient way to provide teachers with the training credentials required by Pathways

to present the Fun Friends program. A Got It! team member comments:

It was impossible to get teachers to a whole day of training … I think we trained 37 teachers.

To get them all released on one day that was just impossible, so the online training was

something I guess that could be done that way.


51

Having Got It! staff trained as program trainers is an efficient way to roll out the Fun Friends

program. Training schedules can then be negotiated directly with schools.

One of the drawbacks that Got It! team members experienced as Fun Friends trainers was their own

lack of experience in running the program with children. Whilst running the program is not a

requirement for being a trainer, personal experience adds depth and richness to the examples and

suggestions used in training. Some Got It! team members felt uncomfortable in the role of a trainer

of teachers who had far more experience in running programs with children than they themselves

had. On the whole, teachers find delivering Fun Friends ‘a piece of cake’ (Got It! team member). As

well as being approved trainers in the Fun Friends program, Got It! team members have also been

required to use skills and approaches to engage teachers in training sessions; for example, drawing

out their existing knowledge and capacities. In adult education, trainers do not need to be experts

on a topic in order to structure and facilitate a group learning process. There was some feedback in

interviews with DEC staff that the early training sessions did not do this as well as it could have, but

that this improved over time. Got It! team members spoke about the usefulness of valuing and using

the expertise of teachers in the training sessions by getting them to share their own experiences and

providing feedback to each other. Got It! staff have also reported that they have become more

confident and effective over time in their delivery of the Fun Friends training:

I think the only thing that’s changed over time is the way we deliver it. We might be able to do

it more succinctly, have better examples, make it a little bit more exciting for them.

Having a Got It! team member who is teacher-trained appears to have assisted the understanding of

teachers’ experiences and the school context, which is useful in delivering training. Whilst still

maintaining the format and requirements for Fun Friends training, it can be tailored to match

teacher requirements, with particular aspects emphasised or enhanced. For example, in one of the

Got It! teams, a team member who delivered the training has mapped the Fun Friends content

against the K–2 syllabus for Personal Development Health and Physical Education (PDHPE).

Being a teacher has been a massive asset when it comes to the delivery … relating the program

to the new Australian Curriculum syllabus for K to 2 and all the components that the Fun

Friends program would meet within the syllabus … The feedback from teachers has just been

huge with how much time that would save them and how much more user friendly the

program will be from that. So that’s been a huge positive.

One of the Got It! teams chose to develop its own training program for teachers, rather than adopt a

standardised program like Fun Friends. This occurred partly because of feedback from teachers that

their training needs related to managing strong emotions in the classroom, rather than behaviour

management, in which they are generally well trained and competent. The Got It! team responded

by developing a training program on Emotion Coaching. Being unable to locate a packaged program

designed for teachers in the classroom, the team built up a program that draws on the Tuning in to

Kids program (Mindful Centre). They offer this in up to four sessions, as requested by the schools. In

the following focus group extract, members of the team talk about their experience of the program

they’ve developed:

We’ve been trying to develop a model which incorporates that, to give teachers some skill

around working with the strong emotions in conjunction with the behaviour management skills


52

they already have … There wasn’t a nice program that already existed to do that in schools.

We’ve kind of taken evidence-based stuff and turned it into something that could be used in

the classroom … The emotion coaching … fits really well in the classroom. I think we’ve found

though that teachers really like something very concrete that they can do. They really like it all

nicely set out … That’s what we’re now really working on is trying to put it in a form that’s

concrete, that they can take to their classroom and go ‘Here’s what I have to do. Here’s my five

steps‘.

Having the five steps … has made them actually more mindful of it, use it more often and do it

in perhaps a more structured way and perhaps pay attention to bits that maybe they left out.

Like actually sitting with the problem rather than jumping to problem solving.

Some teachers have noticed their classroom is calmer when they actually stop and pay

attention to those little things. So it is universal.

Developing a training program that leads to changes in practice in the classroom is a complex

process. Research into the implementation of evidence-based practice indicates that there is often a

gulf between new knowledge and changes in practice (Gray et al. 2013). Attention to practical ’how

to‘ steps within the specific practice context is vital in training, but a big challenge when developing

a new program, as this Got It! team has set out to do. Organisational support for changes in practice

also contributes to the uptake of evidence-based practice (Plath 2012). In one of the schools where

the Got It! team worked, aspects of the Emotion Coaching training have been incorporated into

school policy, which is likely to impact on wider adoption in the school. One of the benefits of the

Fun Friends program is the very clear application to the classroom setting with the inclusion of

quality resources such as books, CDs and props that can easily be incorporated into a classroom

program. The Fun Friends program attends well to the translation from theory and principles to

practice and actions. As a Got It! team member comments:

They love that it’s a package that’s condensed, that you can hand over to them. Usually

teachers go to training and they get given all this theory and all this information, but they are

not actually given the tools to do the program. They are just handed it so it’s so simple for

them to actually run in the classroom. Usually they have to run around getting bits and pieces

together and also try and interpret the program themselves, whereas we’ve shown them step

by step how this is to be run in the classroom. They can put their own modification on it and

they seem to really appreciate that and they love the concept.

From the perspective of a DEC staff member interviewed:

[Fun Friends] training gives teachers confidence and the resources to deliver the program.

Even if it’s stuff teachers already know, it gives them the mastery and a package they can

deliver … enthusiastically received and appears to have gone over well.

The Got It! teams are also providing other teacher training sessions that tend to be short, generally

one hour, sessions focused on a topic that is of interest to and requested by the particular schools.

Some of the topics covered include attachment, trauma, challenging behaviours and emotion

coaching. The Got It! teams are continuing to work on honing the focus of such sessions to make the

most of the limited time generally available. The process journals indicate that the Got It! teams are


53

continually reviewing and considering alternative approaches to teacher professional development

sessions. As well as consulting about the content and timing of these sessions, it has been important

that Got It! teams are mindful of the educational process, using adult education principles to build

on past experiences and existing skills of teacher participants. As a principal interviewed for the

evaluation emphasises:

They need to be mindful in training sessions that Got It! teams are not telling teachers how to

suck eggs … Teachers know a lot about managing kids’ behaviours.

Got It! teams have become aware of this dynamic over time, as one team member reports:

It’s gone from more delivering theory to being much more practical for the teachers and

getting them to give what they already know and share with the other teachers and reflect on

their experience as teachers as well. So, it went from us delivering a lot of education, to them

contributing a lot more to the sessions as well.

Similarly in a process journal, a team notes:

Teachers appear to appreciate this time to debrief and were happy to talk about their

struggles.

The Got It! model also allows some flexibility to be responsive to the needs of particular schools. As

well as formal sessions, Got It! teams can be available for informal consultation and can act as a

resource and conduit to other programs during the time they are involved with a school, as these

team members in a focus group describe:

We’re always available for them if they do have any questions about any of their kids. We

don’t say specifically the Got It! kids or specifically K to 2, it’s school wide …

They came in for issues that were probably beyond what the program can meet, mostly

trauma related. So we did offer to do an in-service to the teachers around trauma but instead I

just flicked them off a ‘Calmer Classrooms’ document, which is a Victorian Government

initiative … and the feedback was good.

A training issue that all three Got It! teams dealt with was negotiating training time and teacher

attendance to fit in with tight and busy school schedules. Which school staff attend and when

training can be held varies from school to school and there is a need for flexibility in the delivery of

training to fit in with the local requirements. Often this means reducing the length of sessions so

that they can be incorporated into scheduled staff meetings. Bringing schools together for longer

training sessions can also be a useful strategy and opportunity for interchange. The Got It! funding

administered through DEC at regional level for teacher release has been critical in supporting

teachers to attend the Fun Friends training. This has also been used to purchase kits of resources

and workbooks for the schools.

Comments made by DEC staff in the interviews indicated strong support for teacher training as it

provides opportunities to step outside and reflect upon classroom experiences. DEC interviews and

online survey comments point to the knowledge and support of the classroom teacher as critical for

the success of the Got It! program. The classroom teacher is at the frontline with children. Therefore


54

Got It! teams have a responsibility to support teachers to understand the program, and how they

can connect it back to the classroom and reinforce it. The teacher’s input is pivotal, but they are also

the ones in the position to refer on to more specialised support if needed (such as the school

counsellor or outside services). At times this preparatory training and information through the Got

It! program was not seen to be adequate. As a classroom teacher commented:

Absolutely it would have been better to have had training for teachers in the lead up to the Got

It! group, and there’s also a need for follow-up training afterwards … The targeted group is

intense, but it can be equally intense in the classroom, where you have 30 children and 15 of

them with needs … Teaching, once you are in it, no matter how good your intentions, it is very

easy to get caught up in requirements and schedules and ticking boxes. It’s hard to slow down

and acknowledge kids.

The responses of those who completed the online survey (see Figure 4) indicate quite positive

experiences of the teacher training, with 33% (24) regarding it as excellent, 43% (31) as above

average, 18% (13) as average and 5.5% (4) as below average. The results for the sub-group of

respondents who identified as teachers (n=37), and thus the group that is likely to have received the

training, the ratings were still good, but not quite as high as for the whole sample. For the teacher

sub-group, 27% (10) regarded the training as excellent, 40% (15) above average, 27% (10) average

and 5% (2) below average.

Figure 4: Health and Education staff: ratings of success of teacher training

0

5

10

15

20

25

30

35

40

45

50

% for All respondents (n=72)

% for Teachers (n=37)


55

Summary of findings: Professional development for teachers.

Professional development has included: - Training for teachers to deliver packaged programs in K–2 classrooms, with Fun

Friends (Pathways) being adopted as the program of choice in two of the three sites. - Training sessions developed by the Got It! teams, either as a series focused on

classroom strategies or other specific topics. - Information on the Got It! program and associated service context.

Training receives positive feedback from teachers when it: allows teachers to reflect and build on their experiences and skills has clear links to national curriculum requirements involves a program with teaching resources for implementation is responsive to school requests for education on a topic.

Having Got It! teams certified to train teachers in particular universal programs is an efficient way of rolling out the program.

Professional development is needed during the early engagement phase with schools in order to establish understanding of the goals and processes of the Got It! program.

Considerations for development of the Got It! model

A review of the universal programs and associated training is recommended on a regular basis to ensure that training continues to be appropriate, is well received in schools and meets teacher training needs.

Investment in training for one particular universal intervention program (e.g. Fun Friends) should not be at the expense of considering alternative programs in the future.

An ongoing financial commitment to training Got It! staff as trainers in suitable universal programs is needed at LHD level to ensure that there is an adequate number of trainers in the Got It! teams as new staff come on board and alternative programs are considered.

4.2.2 Delivery of social-emotional learning programs by K–2 teachers

The Got It! model of care states (p. 15) that interventions will involve:

Delivery of social and emotional learning classroom programs by teachers to promote social-

emotional development of all children in the first three years of school.

Two of the pilot sites have adopted the Fun Friends program as an integral part of the universal

component of Got It!. Following the training of K–2 teachers described above, the Fun Friends Social-

Emotional Learning program is implemented by teachers in their classrooms. Depending upon the

timing of the training and the other demands on classroom schedules this may run during the same

term as the Got It! targeted intervention or may commence later. Due to delayed release of funds

for training by one LHD, the Fun Friends program had only recently commenced in classrooms at the

time of data gathering for the evaluation. The benefit of running the classroom program at the same

time as the targeted group program is that Got It! team members are available to consult about the

program and to support its implementation. The Fun Friends program is expected to run weekly over

one term, but a number of schools chose to run it on a fortnightly basis over two terms. Feedback to


56

the evaluation team when meeting with teachers at these schools was that the fortnightly

implementation worked well. Teachers felt less pressured to get through the material and there was

more time over two terms to reinforce and practice the skills being learnt.

Feedback from teachers and Got It! team members regarding the Fun Friends program is very

positive. Fun Friends is a set program of activities with a range of associated resources to support

children’s learning. Teachers have the opportunity to incorporate these in different ways in their

classes and to enhance the program as they require. It is regarded as an engaging program that

children enjoy and that teaches children fundamental skills around emotions and relationships.

These skills are revisited, practised and reinforced in the classroom throughout the program.

Teachers have reported that the Fun Friends language and skills are apparent amongst children in

classes that have done the program. There were many positive comments, in DEC interviews and in

the online survey, about the quality and usefulness of the resources that come with the Fun Friends

program. One teacher comments:

Fun Friends has great resources. Good classroom resources are always a benefit and teachers

are more likely to implement programs if they come with good resources that they can pick up

and use.

The Fun Friends program has some scope for adaptability. As a Got It! team member observes:

Each individual teacher gives their flavour to it and it’s not like that have to do absolutely every

activity in each session. The only major requirement is that they’re doing a sequential

program.

Whilst it was beyond the scope of this evaluation to assess the fidelity of the individual intervention

programs being implemented, there were some concerns expressed by Got It! staff about how the

Fun Friends program may be modified, with resources and activities used out of context and by

untrained staff, as time goes by. There is a licence agreement with Pathways regarding training of

those delivering Fun Friends and purchase of materials, including a workbook for each child, a

manual for each teacher delivering the program and resources kits. Whilst Got It! teams have

ensured that such requirements are met whilst Got It! is running in the schools, the use of materials

after they leave the schools is out of their control. There were also some concerns expressed about

delays with orders, licence documents and other information from Pathways that has impacted on

program implementation.

The Fun Friends program includes a workbook for parents/carers to complete with their child. It is

expected that teachers send these home with the children and ask parents to complete activities on

a regular basis together with their child. Feedback from teachers indicates that they vary in their

approach to this component designed to impact carer–child relationships. Some teachers expected

that parents may not do these activities and as an alternative the teacher works with children to

complete these workbooks in class. Other teachers send the workbook home but do not follow up in

any way as to whether it is completed. Other teachers have emphasised the importance of

completing the workbook activities and have incorporated workbook sheets in the weekly

homework that is sent home with the children. The workbook does provide some potential to have

an impact on interactions at home and, whilst this is likely to be limited, strategies to optimise its

uptake at home are worthy of closer attention in Fun Friends training. Some teachers had suggested


57

modifications to the homework sheets to make them more engaging for children and parents. Some

insights into the unseen impact that the Fun Friends program can have for families is provided in

parents’ comments later in this chapter.


Whilst the Fun Friends program has now been adopted as a central feature of the universal program in two of the pilot sites and has been positively received, the Got It! model of care could include other appropriate social-emotional learning programs that teachers can deliver in their classrooms. Several potential programs are recommended in the Got It! model of care. It is important that Got It! teams continue to gather feedback on the impact of the programs delivered by teachers and remain informed on alternative programs, their suitability to schools and the evidence base, so that universal social-emotional learning programs can be responsive to changing contexts.

For the Got It! pilot site that has not taken up Fun Friends, but has developed a teacher training

program on Emotion Coaching, there is no social-emotional learning program as such delivered by

teachers as part of the Got It! program. Got It! staff have had feedback from teachers that they have

taken on concepts from Emotion Coaching’s five steps and some have incorporated emotional

literacy lessons in their classes but this is not delivered as a classroom program. As a Got It! staff

member described in the focus group:

It’s not really a direct link. I mean, we make suggestions around resources that they can use …

but we haven’t actually given them a program that they can teach.

Both the Positive Behaviour for Learning (PBL) framework and KidsMatter provide a whole-system

approach to dealing with social-emotional learning in schools. The staff working on these initiatives,

who were consulted as part of the evaluation, emphasised the importance of working with existing

resources and capacities in schools in order to maximise uptake and sustainability. From such

systemic perspectives, the social-emotional learning programs delivered by teachers as part of the

Got It! model would be considered in the context of other programs, strategies and frameworks

adopted at the school level. Got It! can be readily incorporated into such frameworks, but it does

mean that components of Got It! would sit alongside or be integrated with other social-emotional

learning programs within the schools. These may be offered at different stages or as whole-school

programs or strategies. Some Got It! team members expressed concern about the ‘mixing and

matching’ that can occur when teachers draw on different programs and the potential for this to

impact on the quality and fidelity of programs. MoH staff also expressed concerns about too much

flexibility and adaptability with the universal interventions at the local school level as the potential

to implement well-evidenced interventions can consequently be compromised.


58

Summary of findings: social-emotional learning programs delivered by K–2 teachers

Fun Friends has been well received as a universal program delivered by teachers in K–2 classrooms. Teachers find it easy to implement the program in classrooms, children enjoy it and engage with learning, and the accompanying teaching resources are appealing.

School systems and existing social-emotional learning programs and practices within schools impact on how the Got It! program is received and the preparedness to implement associated universal programs such as Fun Friends.


Assessments of individual schools, carried out in a collaborative way between Got It! teams and school staff, can determine the best social-emotional learning program/s for implementation in K–2 classrooms in that school. Assessments should examine school capacities, school systems, program evidence and existing/alternative programs to the one/s Got It! staff provide training in. In the process of decision making about social-emotional learning programs to implement in classrooms, the focus of Got It! should remain on the prevention of conduct problems. Got It! teams may take on the roles of program trainers and/or consultants in the program selection and implementation. They could also have a role in advising on the evidence base for programs and in overseeing program fidelity.

4.2.3 Parenting information campaign for K–2


Parenting information campaign for parents of all children in the first three years of school to

promote effective parenting strategies, to normalise and de-stigmatise participation in

parenting programs and to introduce the Got It! program. Strategies such as newsletters and

introductory parenting seminars could also be considered as part of this intervention.

Each of the three Got It! teams have utilised a number of different approaches to address the

challenge of engaging with parent/carer communities in order to disseminate parenting information.

Initially this entails distributing information about the Got It! program and what it will involve in the

school, but importantly also involves providing practical information that will assist in the tasks of

parenting and where to seek out help if needed. All Got It! sites have used the school newsletter to

provide information on the Got It! program along with regular ‘parenting tips’. Resources associated

with the Fun Friends program have been useful to draw on in preparing these newsletter items. This

is an easy way to disseminate information that is worthy of continuing as standard practice, but

concerns about how many parents read these items and the likely impact indicate the need for

additional strategies as part of an information campaign that continues throughout the two terms of

involvement by Got It! in a school. Parent comments presented later in this chapter indicate that

there are some unforeseen positive impacts from such strategies.

The parenting information campaigns adopted by the Got It! teams have been directed at the whole-

school community (K–6) and have included:


59

items in school newsletter

flyers sent home with children

information stands with flyers, brochures, family and health service brochures, parenting

resource information as well as information on the Got It! program

information sessions or ’seminars‘, of generally around an hour in length, on topics such as

Raising Resilient Families, Emotion Coaching or other general parenting topics

posters around the school.

One Got It! team is also working on developing a website. In the focus groups, each of the Got It!

teams spoke about a variety of different responses to, and levels of success with, the information

dissemination strategies in different schools. Their comments pointed to the need to be flexible,

creative and assertive, but not confronting, in order to engage parents.

It’s a bit like you’re a street peddler. You sort of have to engage them and let them know who

you are and what you’re on about, and you need to be very enthusiastic.

A duplication of efforts with the development of parenting information resources and promotional

information related to the Got It! program was apparent across the pilot sites. A MoH staff member

emphasised the potential to develop a professional package of promotional and parenting

information resources that could be drawn upon across the state. This could include web-based

material accessible to parents and teachers for use in schools as well as hard copies that could be

distributed by Got It! teams. A range of accessible multimedia resources could support the efforts of

Got It! teams in this component of the model. MoH staff pointed out the importance of Got It! team

members focusing on clinical treatment and the potential for non-government organisations being

engaged more in such community education aspects of the model.


Development of a multimedia range of promotional and parenting information resources to support the work of schools and Got It! teams.

Directing parents/carers to web-based parenting resources specifically developed by the Got It! program, and/or other well-credentialed websites, is a strategy worthy of incorporating into the parent information campaigns.

In summary the Got It! teams have found the following helpful:

Have the principal and teachers on side and willing to support efforts to provide information

to parents. The first step is to engage with the principal and teachers so they are clear on

what Got It! is trying to do and why.

Utilise local knowledge on what works in their school, regarding times when sessions are

held, how to advertise sessions, content, language that is engaging. For example, the drop-

off or pick-up timeslot won’t work as well in schools where most children travel to school by

bus.


60

Combine Got It! parenting information strategies with other events in the school. Some

examples were: Hold the parenting seminar on the night when the disco is held so that

childcare is dealt with and parents are already coming to the school; Have information stalls

at school assembly, special events such as multicultural day or mothers’ day breakfast,

school drop-off or pick-up time.

Use props and signs to attract attention and have freebies and resources for people to take

away from information sessions, seminars and information stands.

Promote word of mouth as a powerful way to engage participation. Having a parent who has

participated in a Got It! program at another school speak to parents at an information stall

or session can be effective. Teachers talking with particular parents and encouraging them

to attend a seminar was also found to be useful. Involving key community members, such as

Indigenous elders, to talk with people in their communities can also be effective.

Provide food.

Engage parents/carers in conversations and be prepared to ’hang around‘ to talk with

parents after events.

Continue the campaign throughout the two terms of involvement in a school as engagement

with the school can improve over time. Subsequent strategies can build on information

presented in early parts of the campaign.

Be prepared for strategies to be poorly attended and to try something else. What works well

in one school setting can be unsuccessful in another. Even if parents don’t come up to a stall,

just being noticed adds to the profile and acceptance of Got It! in the school.

A number of respondents reported that suspicion of the Got It! program has developed in a few

schools due to false perceptions that the program is associated with Community Services or Housing

and that some parents fear scrutiny of their parenting will lead to unwelcome interventions. Whilst

it is clear in Got It! written information that this is not the case, the need to reinforce this through

each of the information dissemination strategies was identified. As a Got It! team member

commented:

The general feeling is maybe that their parenting is being criticised, that there must be

something wrong with you … You’re standing there at a stall … but no-one comes up to you. I

think they’re just quite scared … I sort of go out and say, ‘How are you going? Do you have a

form?’ and whatever, and that works.

For the rural pilot site, distance presents a particular challenge in the process of engaging with a

school community. In considering activities at schools in towns outside where the team is located,

there is a constant weighing up of whether the trip and the time involved will be worth it.

A common experience of the Got It! staff is that it seems to be the parents that don’t really need

assistance with parenting who are more likely to attend information sessions. Alongside this,

however, there were examples of individuals for whom attendance at an information session was

key to their subsequent involvement in a Got It! group or in linking in with another service. As Got It!

team members point out:

It has captured families we might not otherwise have captured; it’s just not a large number.


61

We always make an effort to hang around … just listening to people and their concerns and

quite a few will often come up and ask for advice. Sometimes it’s about older children. In some

ways it’s a really valuable kind of early intervention where we can give information on who you

contact about this.

Some parents who are never going to be involved in the program really engage in that brief,

‘Oh, here’s a different way to interact with my kids‘. Some have even approached teachers

afterwards and said, ‘I’m going to do everything completely differently now!’

Feedback from parents on the information campaign is provided later in this chapter.

The online survey of Education and Health staff involved with Got It! revealed a reasonably positive

response to the provision of parenting information to the school community. As can be seen in

Figure 5, 48 (67%) of respondents regarded the parent information campaign as above average or

excellent, 11 (15%) as average and 5 (7%) as below average or poor. There were 11% of respondents

who indicated that they did not know about the parenting information provided.

Figure 5: Health and Education staff: ratings of success of parent information campaign

0

5

10

15

20

25

30

0 = poor 1 2=average 3 4=excellent Don't know

Respondents' ratings of success of parent info campaign (n=72)


62

Summary of findings: parent information campaign

A variety of strategies have been used by Got It! teams to disseminate parenting information to school communities. Indications are that multi-pronged information strategies, negotiated with and tailored to the particular schools, are required and that existing systems and events at schools should be used to disseminate information whenever practicable.

Generating a realistic and positive image of Got It! and enthusiastically engaging with staff and families are key principles for effective information dissemination.

Whilst families who engage with information dissemination events are often families who are doing well, these universal information strategies also engaged some families in need.

4.2.4 Screening of K–2 by teachers and parents to identify children with elevated conduct

scores


Screening of all children in the first three years of primary school regarding their behaviour

using Strengths and Difficulties Questionnaire (SDQ; Goodman 1997) as rated by both teachers

and parents. Children who show elevated conduct problems according to either (or both)

teachers or parents will be targeted for inclusion in the indicated interventions within the

model.

Effective screening relies upon getting information out to parents about the Got It! program,

preferably in the term prior to screening. A focus on Got It! information dissemination and

engagement with the school community prior to screening assists families to understand the context

of the whole Got It! program, rather than seeing the program purely in terms of the SDQ forms.

Screening in isolation from this program context can be experienced as an alienating process. Whilst

it has not always been possible, Positive promotion of Got It! and engagement with the parent

community best occurs prior to screening. This assists families to put the screening in context and is

likely to maximise the return rate. Familiarity with and trust in the program is expected to build up

over time. As one DEC staff member expects:

If Got It! was run again in the school there would be more interest and greater uptake

amongst families. There was some resentment and resistance initially.

A description of the SDQ screening tool can be found in the earlier Got It! Evaluation Interim Report.

SDQp questionnaires are distributed to K–2 parents through the school and the completed forms are

returned to the school for collection by the Got It! teams, along with SDQt forms completed by K–2

teachers. It is not uncommon for children to exhibit conduct problems in either, but not both, home

and school. Effective screening therefore relies on a high return rate from both parents and

teachers. Given that it takes a while for Kindergarten children to settle in to the school environment,

and the transition can have a negative impact on behaviour, some schools have decided not to

include Kindergarten children in the screening process in the first half of the year. This seems to be

appropriate if there are sufficient families to form a group amongst years 1 and 2. Delivery of the

universal program to all K–2 has, however, been supported.


63

A lot of attention and energy is put into maximising the SDQ returns in schools. Attention is paid by

Got It! teams to including clear information to parents/carers on how SDQ data is to be used and the

procedures for return of forms. An information stand in the school is another strategy used by Got

It! teams to answer questions and encourage the return of forms. Commitment by teachers to

follow up with parents who have not returned forms is also generally required to get a good

response rate. The evaluation team was informed at a number of schools that teachers made phone

calls to individual families and completed some SDQ forms over the phone or assisted parents in

person to complete forms. This has been valuable for parents/carers with literacy difficulties who

would not otherwise have been able to complete the forms. However, there were concerns

expressed in the focus group that some families may experience this as being harassed or pressured

by the school and that this therefore needs to be handled with delicacy. Got It! teams have also

been making use of SDQ forms translated into other languages for families from different language

backgrounds to facilitate higher response rates. One of the Got It! teams runs a raffle with an iPod

prize in each school, with a ticket given to each child returning an SDQp form. In other schools

teachers provide small incentive rewards to children returning forms. In this way children motivate

their parents/carers to return the forms. One school principal commented that there needs to be a

whole-school approach to screening that involves teachers, welfare officer, school counsellor and

administrative staff, in order to maximise the return rate. A Got It! team member comments:

A huge contributing factor with the screening is the school. So what works would definitely be

having very enthusiastic and committed staff pushing the program within the school. So

schools where we have huge return rates has been because the staff there have been really

excited about the program, chasing their kids up in the classroom.

During terms 1 to 3 in 2013 the return rate for the SDQ screening forms in 18 schools ranged from

37% to 91%. This is a wide variation, indicating a range of factors that are likely to be influencing the

return of SDQ screening forms. Across these 18 schools, 1,830 children were screened, representing

an overall return rate of 67%. This is the combined return rate for those 12 schools included in the

outcomes evaluation sample and another six school that were screened after the outcomes data

collection period. In response to comments that the additional questionnaires distributed for

evaluation purposes may have impacted negatively on the SDQ return rate, a comparison between

return rates for these two groups of schools was made by the evaluation team. It was found that

those schools not included in the outcomes evaluation group sample had a return rate of 57%

(range: 37–73%) and the return rate for the outcomes evaluation sample was 74% (range: 56–91%).

It was therefore concluded that the additional evaluation questionnaires did not impact negatively

on the return rates for schools included in the outcomes evaluation sample.

In their process journals, Got It! teams identified a number of barriers to achieving high SDQ return

rates. One was the addition of the evaluation questionnaires (which was not supported by

subsequent data collection), resistance by parents to revealing personal details about their child’s

behaviours due to suspicions about potential consequences, language or cultural barriers, poor

literacy skills and a lack of support for the screening by school staff. Minimising such barriers

through engaging teachers more to encourage families to complete forms, school staff completing

forms with parents (either face to face or by phone) and utilising interpreters and SDQ forms in

other languages have been used in efforts to maximise returns from families. This places additional

demands on staff time and requires communication between Got It! and school staff regarding


64

potential issues well in advance so that, for example, interpreters can be booked. According to data

in process journals, teachers are concerned about the number of children with concerning

behaviours for whom no parent/carer SDQ was returned, and hence are not eligible for the group

program.

Given that engaging families is an important precursor to effective screening, suggestions on

strategies to maximise engagement between the Got It! program and the school community were

sought in the online survey and DEC interviews. Following is a summary of suggestions:

Feed them – Then they tend to come out of the woodwork.

Develop a website so that information, resources and screening tool can be accessed online.

Flyer/information on all aspects of Got It! to go out to families in the term leading up to the

program to put screening in context

Market program positively with logo / branding.

Reinforce information on program through regular information in school newsletter.

More information prior to screening forms going home – including that child care and

transport will be provided.

Parent/carer information sessions before screening questionnaires go home – Got It! staff

come along to existing parent events, rather than organising a separate time (e.g. school

assemblies, Kinder orientation, parent-teacher interviews or other events that parents

attend).

Combine information on Got It! with other school events.

Incentives for return of screening questionnaires.

Go back to the same school whilst families still remember it. Involve families who have been

through the Got It! program to promote it in the school.

Personal contact by teachers to explain program and encourage families to participate and

complete screening form.

Involving Aboriginal Education Officer as a group facilitator and to engage with Aboriginal

families.

Develop partnerships with multicultural service providers / community organisations, that

may assist to engage with culturally and linguistically diverse communities, promote the Got

It! program in those communities and assist with referrals to culturally appropriate services.

Offer extra support for families with little English or literacy.

Data from the SDQ forms are entered by Got It! teams in order to identify those children with

elevated conduct scores. Data entry is a time-consuming and tedious process. Whilst one Got It!

team has an allocated administrative assistant to support the team with tasks such as data entry, the

other two teams have struggled with inadequate administrative support. Consequently, valuable

clinician time has been taken up with data entry for screening purposes. As a Got It! team member

observes:

I found that a lot of hours go into this screening process. So many hours go into interviewing

teachers, interviewing parents, just getting all those forms back and scoring them. All the work

and admin for this six to eight families. I don’t know how to do that better, but it’s a huge

amount of time to work out who these families are going to be.


65

Got It! teams reported that the Got it! Database developed by the evaluation team has increased the

efficiency of data entry and screening. In the Got It! database, domain and total scores are

calculated automatically and data for all children can be included in one spreadsheet. Whilst

screening is a labour-intensive process, there was general agreement across DEC and Got It! teams

that the process is worthwhile and that the SDQ is a suitable tool for screening for potential conduct

problems. It is a relatively brief and accessible tool that has worked to flag children for earlier

intervention that would not otherwise have been identified. The SDQ scores do provide a useful

indication of problems, but cannot be relied upon without follow-up assessment. Got It! team

members referred to examples where parents had obviously filled in the wrong boxes accidentally

and the comments made in the interview clearly didn’t match with the scores. Got It! staff reported

that generally when SDQp scores were discussed with teachers, there were ‘no surprises’ and that

the SDQ works appropriately as a screening tool. The SDQ does not, however, stand alone. It is the

starting point for an assessment process.

During the pilot phase, Got It! teams have developed a strict timetable in order for screening to be

completed in time for interviews and targeted group selection before the end of a term. The teams

reported that they are now clearer in their communication of the requirements and expectations of

schools and how to facilitate a streamlined process. In particular, explaining the value of screening

to teachers and having them committed to supporting the process is critical for success. Having the

DEC funding available to schools to release teachers from class to complete the SDQ forms has also

been important in facilitating the screening process by K–2 teachers in schools. What is missing in

the timetable, however, is the time it takes to engage in a positive way with the school community

as a lead in to the screening process.

Whilst some Got It! team members expressed concern about the amount of SDQ screening data that

is collected and not subsequently used, they also were able to identify a number of benefits from

the screening process beyond selection for the targeted Got It! group. For example, the completion

of SDQt forms by teachers prompts them to stop and consider the behaviours of each child in their

class and can alert them to issues that they may not have otherwise have noticed. The process can

also prompt communication between teachers and parents. As one Got It! team member comments:

Sometimes parents don’t necessarily realise that the teacher is a little bit worried about this or

wondering about that. So it’s useful in that way.

Practises relating to the storage and ownership of SDQ data varied between Got It! sites. In one site

all SDQ screening data is provided to the school counsellor to be retained at the school and used in

the future for other purposes if deemed appropriate. In the other two sites screening data is kept on

file at the CAMHS service.

In some DEC interviews the need for more of a partnership approach to screening was proposed.

One interviewee pointed out that schools already collect a lot of screening data and that schools

may have the capacity to coordinate SDQ screening with other screening, stating:

There is a lot of data collection that goes on and very little analysis. There is an ethical

responsibility to do something about what you pick up on.


66


Given the partnership nature of the program, the storage and ownership of screening data needs to be negotiated. Should NSW Ministry of Health require standardisation of data management across the sites then negotiation with LHDs and the local Got It! steering committees would be required as practices vary across pilot sites.

The potential for schools to take on responsibility for the collection, data entry and storage of screening data and to provide access to data by Got It! teams is also worthy of exploration. SDQp data could, for example, be collected by schools with personal data at Kindergarten entry or the beginning of each school year. Again, this would need to be negotiated at the regional level with DEC and the steering committees.

The online survey of Education and Health staff involved with Got It! reveals a reasonably positive

regard for the screening processes conducted as part of the program. As depicted in Figure 6, 46

(64%) of respondents regarded the screening as being above average or excellent, 17 (24%)

regarded it as average and 6 (8.3%) as below average or poor. There were many comments in the

online survey expressing concern about the lack of interest by parents in completing the screening

forms or participating in the program. For those schools not delivering a universal program in K–2

classes, this was seen to effectively exclude children from taking part or gaining any benefit from the

Got It! program.

Figure 6: Health and Education staff: ratings of success of screening

0

5

10

15

20

25

30


Respondents' ratings of success of screening (n=72)


67

Summary of findings: screening of K–2 children

Screening is a resource intensive and complex process that entails engagement, information provision, support, follow up, data entry and analysis.

Return rates for SDQ screening forms vary considerably between schools. During the evaluation period the range was 37–91% (mean 67%).

An emphasis on positive engagement with school communities and dissemination of information about Got It! prior to screening is likely to improve response rates for screening questionnaires.

Strategies for increasing return rates of SDQ screening forms have included information provision (verbal and written), incentives, follow up of individual families by teachers, provision of forms in community languages and offering assistance to complete forms.

High return rates rely on involvement and commitment by classroom teachers, both in completing the teacher SDQt and facilitating return of parent SDQp. Funding for teacher release has been crucial for school-based screening.

The additional questionnaires distributed as part of the outcome evaluation were found to have had no negative impact on the return of SDQ screening forms.

The Got It! database designed by the evaluation team has enabled more efficient data entry and identification of children with elevated scores.

Screening data storage and ownership varies between pilot sites. In two sites data and forms are stored at CAMHS and at one site data and forms are provided to schools at the end of the intervention period.

Consideration for development of the Got It! model

The universal program is an important means by which some impact can be made for those children whose parents do not return the SDQ, and are therefore excluded from consideration for the targeted group program. Once all strategies for engagement in screening are implemented, the right of families to decline involvement needs to be respected.

4.2.5 Comprehensive assessment of children with elevated conduct problems

The Got It! model of care states (p, 15) that interventions will involve:

A comprehensive assessment by mental health clinicians for children exhibiting elevated

conduct problems and their families using standardised parent-rated questionnaires and

clinician-rated tools.

The assessment of children identified from the SDQ as having elevated conduct problems is the first

component of the targeted intervention phase in the Got It! Model of Care. The assessment process

has included interviews with teachers and other school staff, such as school counsellors, interviews

with family members at the school, classroom observations and consultation with external service

providers involved with the family. Whilst practices have varied to some degree between the sites,

the purpose of these assessments during the pilot phase has predominantly been to make decisions

about which families to include in the Got It! targeted group program. Whilst the families assessed

may present with a variety of needs, assessment for the targeted group naturally focuses on the


68

central concern of the Got It! program (reduction in conduct problems) and suitability to a group

intervention program. As two members of a Got It! team discuss:

They still need to meet the threshold of having the difficulties. It needs to be predominantly

behaviour, it can’t be predominantly something else.

Parents need to be willing to commit and able to be part of the group. There’s been some that

we’ve excluded because we met the parent and just went, ‘They’re not going to cope in a

group’.

The Got It! teams agree that a viable targeted group size is from four to eight children and the

assessment process includes determining which families are the best suited to the group. Inclusion

in the targeted group requires:

high conduct problem score on SDQ

a parent/carer willing to be engaged and commit to the group program

child and parent/carer able to manage in a group.

The attention given to those children with elevated conduct problems who are not selected for the

targeted group has varied between the Got It! sites. Through the focus group discussions and the

review of school profiles, it was evident that a number of context-dependent factors are assessed

and taken into account by Got It! clinicians in their decisions regarding which families to include in

the group program. Children have been assessed by Got It! teams as unsuitable for the group for one

or more of the following reasons:

Their behaviours or skill levels are likely to present difficulties for the group process.

They face complex or severe individual or family issues that warrant more specialised

intervention (which they may or may not currently be receiving).

They, or their families, are already receiving other services.

A family member is not able to commit to attend the Got It! group program (e.g. due to child

care issues, pending child birth, work commitments, lack of interest).

The available family member is assessed as likely to face difficulties in a group process (e.g.

due to poor English capacity, mental health issues, drug use).

Their behavioural issues or family circumstances are not as high a priority as other children

in the school.

For these children and families not included in the targeted group, Got It! can still facilitate early

intervention through the provision of information, referral, liaison with services that the family is

involved with, intervention with the child or family, or other follow up within the school. There

appears to be considerable energy and attention given by Got It! teams to determining which

families are included in the group, but there is a lack of clarity and consistency across the Got It!

teams with regard to responsibilities toward the other assessed families. Two of the Got It! teams

develop case plans for all assessed families, with the assistance of case review meetings within

CAMHS. Utilising such clinical structures within CAMHS plays a role in supporting the quality of

responses to families.


69

The potential for families to fall through the gaps is highlighted in the following comments from Got

It! team members:

We make a lots of recommendations to people, some of which we try to sort of follow up,

others we say to the school, ‘You should follow up’. But I guess we have a very wide target

group at this point. We make a lot of recommendations, but maybe only a few of them get

followed up … Some of them just sit there in a school counsellor file.

The screening is good in a sense, but we don’t actually have the staff resources to deal with the

information that we get in an adequate way. There’s a lot that come back that would need the

conversation and possibly a referral and because of, well whatever, we don’t have the time to

adequately follow those through to a level of professional responsibility.

The potential to miss some early intervention opportunities is further illustrated with comments

from parents provided later in this chapter. Despite these concerns, the Got It! teams nonetheless

provided examples of children and families being identified through the assessment process as

unsuitable for the group, but who have been still been provided with appropriate services through

other channels.

We’ve got to present all of them [those interviewed] for clinical review with the psychiatrist.

We’ve got a pretty good idea as to what the specific needs of the families are and we do pretty

comprehensive referral letters for them after the interview. We also have a discussion at

school with the counsellor about each of the children that presented at interview so she can

act as the one who follow up on my recommendation … That’s worked well, having the school

counsellor a bit more involved.

One of the concerns of Got It! teams in relation to follow up and referral following assessment

interviews was that they do not have the time and resources to respond to the needs of families. In

order to manage demands, teams try to focus on early intervention for emerging conduct problems

as the central concern and regard families where other services are involved as a lower priority. As

Got It! team members explained during one of the focus groups:

It’s difficult because a lot of children are already accessing quite a lot of services but the school

is still really struggling with them. So there’s that impulse for us to step in but it’s not really

appropriate because they already are getting a lot of other services. Just because they’re not

working doesn’t mean that they need to come into Got It!, which can be frustrating for

teachers.

They just don’t grasp the concept of early intervention. They want their immediate needs met.

This kid is a problem in my class. He’s already been to the GP and the paediatrician, he’s on

medication, its not working, he should be in the program. But again, it’s not early intervention.

Generally, Got It! teams have not had the time to comprehensively assess all of the children with

elevated scores and so teams have selected a manageable number to assess following the screening.

(Note: SDQ screening data for the 12 schools included in the evaluation period are provided in the

Outcomes Evaluation chapter of this report.) Teams vary in terms of the scores that are taken into

account in this initial selection. Whilst all consider the ’conduct‘ domain score of the SDQ (parent

and teacher), the ‘total’, ‘pro-social’ and ‘peer conduct’ are taken into account by some teams at


70

some times. Given the focus of the Got It! program, the ‘conduct’ score is clearly the most critical

concern. The ‘impact’ score can also give an indication of the degree to which the parent/carer

regards the behaviours with concern. Decisions about which other scores to consider can depend on

the particular schools. If there are many children in the Abnormal band for conduct then there may

be no opportunity to consider the ‘borderline’ band or other sub-scale scores. Screening cut-off

points have been determined on a school-by-school basis, depending upon the number of elevated

scores that emerge and the capacity of the team to manage the more comprehensive assessments.

The assessment process begins with Got It! teams interviewing classroom teachers. According to the

Got It! team process journals, teachers value the opportunity to talk about children in their classes

and the behaviours they observe. This provides useful information for the Got It! assessment, but

can also be very time consuming. Whilst teachers have important insights into a child’s behaviour

and may have some knowledge of the family circumstances, they do not always fully appreciate the

requirements for selection in the targeted groups and sometimes tensions need to be diplomatically

managed. As staff in two Got It! teams comment:

Sometimes there can be a little bit of tension with the school because they kind of say, ‘Well,

we wanted you to work with this family, this family and this family’. But at the end of the day

they may not be appropriate for the group, or because of their family situation they may not

return the forms … we found we had to be really clear up front and even being clear up front

doesn’t always calm that particular tension.

I think we’ve gotten better from the outset in working with the schools and constantly

reminding them that in the data we may find families that are struggling, but that the kids

aren’t showing any signs of struggling at school.

On the basis of comments from teachers, the Got It! teams then cull the number of potential

families for interview. Sometimes teachers’ suggestions also prompt them to consider other families

that were not initially in their list. The importance of not relying entirely on teacher assessments

was, however, highlighted by Got It! teams. A disparity between teacher and parent SDQ scores can

be an indicator of environmental factors at home for which the Got It! parent group could have a

valuable impact. Got It! teams have also incorporated classroom and playground observations of

children in their assessments. This is another useful source of information on the nature of

behaviours and also provides insights into how the small group process is likely to work for particular

children.

Got It! teams reported that it can be valuable to include information from school counsellors in the

assessment process, such as results from testing or information from prior involvement with the

families, but that this is often difficult to obtain from school counsellors. It was regarded as

worthwhile to engage school counsellors at this point in the Got It! program in preparation for

future referrals. All of the Got It! teams experienced some difficulties in connecting with school

counsellors who move between several schools and often have priorities apart from the Got It!

program. Following consultation with school staff, the list of families for interview is finalised.

Data collected in the 12 school profiles showed that between nine and 45 families were identified as

suitable to interview at each school, with an average of 18 at each school. The one school with 45

was particularly high (the next highest was 24) and it seems that decisions not to proceed to


71

interview were subsequently made in pre-interview phone conversations with some families, as only

27 were eventually interviewed. Between seven and 27 families were finally interviewed at each

school, with an average of 13 families at each school. Across the 12 schools, 74% of families

identified as suitable to interview (on the basis of SDQ scores and consultation between Got It!

teams and school staff) were eventually interviewed.

Parents/carers are phoned to organise to meet with Got It! team members at the school for an

assessment interview. Got It! teams may make this phone call directly to the parent/carers or ask

the school to do this. Sometimes it is a school administrative staff member who makes this contact.

In schools where this occurred, poor attendance at assessment interviews and parent confusion as

to why they were there when they did turn up was noted in process journals. Given the importance

of this initial contact for engagement with the parent/carer, its potential sensitivity and the value in

being able to provide detail on the Got It! program, it seems that contact by a Got It! team member

or a school staff member with a good knowledge of the Got It! program and/or family is appropriate.

The process journal of another team indicates that text message reminders of interviews, following

the phone call by a Got It! team member to set up the interview time, can be effective. They

recorded a 100% attendance rate for one school. First contact by the classroom teacher was also

recorded as an effective strategy.

Contacting parents/carers and organising to meet them at the school can present difficulties if they

work during the day, feel uncomfortable or threatened in the school environment, do not return

calls, or are unavailable at the time that interviews are held. This was found to be particularly

difficult in the rural pilot site where interviews in a school often needed to be confined to one day

due to the distance and time involved in travelling to that town.


Who invites families to assessment interviews is best decided in terms of who is most likely to engage the family. Some flexibility of procedures may thus be required. It is important that the person who contacts the family is sensitive to the potentially delicate nature of the approach, is familiar with the Got It! program and goals, and has an understanding of the purpose and processes of the assessment interview. Knowing the family may be useful and thus it may be appropriate for a teacher to make contact if the relationship with the classroom teacher is a positive one and families are not likely to view contact by the school as threatening.

The Got It! teams recognise the need to approach parents sensitively and diplomatically as, whilst

SDQ scores and teacher feedback provide some indication of potential behaviour difficulties,

parents/carers can vary in how a contact from the Got It! team, or the school, is perceived.

Sometimes the initial phone conversation is sufficient to determine that a family is not suitable for

the group (for example due to work commitments) and a decision is made not to offer a face-to-face

interview. In such circumstances, forwarding information about other services or other resources

may still be warranted. One Got It! team has considered running assessment interviews after hours

in order to increase access and participation by parents/carers.


72

Whilst the process is time consuming, Got It! teams have generally experienced the screening

process and assessment interviews as an appropriate way to determine participants for the early

intervention targeted groups. As one team member comments:

The good thing about that is often we do reach families who aren’t in crisis but are at a point

where we recognise they need help and they wouldn’t seek help at that point in time because

they haven’t recognised where they’re at because they don’t see the bigger picture around

families. So it’s a really valuable early intervention.

The group inclusion and exclusion criteria listed above provide some guidance for selection for the

targeted groups, but Got It! teams have found that they need to consider each group independently

in deciding the best combination of participants for a productive group. The Got It! teams reported

that there is considerable team discussion prior to decisions about who to invite to the targeted

group. As one Got It! team member notes:

We might say that too low an IQ might be excluded, but in some schools there might be lots of

kids with low IQs so we’ll think that that could form an appropriate group.

Based on experience as a group co-facilitator, the following comment was made in a DEC interview:

If there are too many challenging kids at the tip of the iceberg, they have a significant impact

on the whole group. The other children start to model their behaviours and it can be very

frustrating. It limits the potential positive impacts for the other children.

At times Got It! teams have struggled to identify a viable number of appropriate families to

participate in the targeted group in a school and at other times there have been so many suitable

participants that a second group has been run. Whilst the size of the school has some impact on

these numbers, this is not a good predictor as processes and culture in the particular school also play

a part. Whilst children in the targeted group generally have conduct scores in the Abnormal band,

sometimes the group composition and size has resulted in the inclusion of children with conduct

scores in the Borderline band.

Following interviews and team discussions, invitations are made to potential group members and a

back-up list of participants is kept in case some families decide not to commit to the group.

Generally there is some uncertainty right up until the start of the group regarding who will actually

commit to attend. This is complicated by the fact that the final organisation is done during the

school holidays in time for the group to start in week one of term. Got It! staff indicated that

generally parents/carers were relieved to be invited to the group as they felt they would be assisted

to work through problems that they have been struggling with. Parents’ experiences of being

selected, or not, for the group program are discussed later in this chapter.

In the online survey, 45 (62.5%) respondents regarded the family assessment process as above

average or excellent, 16 (22%) as average and 7 (10%) as below average or poor (see Figure 7).


73

Figure 7: Health and Education staff ratings of success of assessment process

Summary of findings: comprehensive assessment of children with elevated conduct problems

SDQ (p&t) scores, teacher and school counsellor consultations and observations are used by Got It! teams to select families for comprehensive assessment.

The number of families assessed in each school varies with Got It! team capacity, school size and distribution of scores, but is generally between 10 and 25.

SDQ domains and cut-off scores used to select for assessment vary across schools and sites.

The assessment process has focused predominantly on selection for the targeted group program (criteria: high SDQ conduct score, parent/carer commitment to attend group program, and child and parent/carer able to constructively participate in a group program).

Early intervention opportunities have been missed when attention is focused on group selection at the expense of follow up for those children not suited to the group program.

Coordination between the Got It! assessment process and the work of the school counsellors is valuable at this stage, but has often proved difficult to achieve.

During the evaluation period, around 75% of the families identified by Got It! teams as suitable for an assessment interview were actually assessed.

In some schools teams struggled to fill places in the groups, whilst in others there were interested and suitable families who could not be offered a place.

0

5

10

15

20

25

30


Respondents' ratings of success of assessment phase (n=72)


74


Comprehensive procedures are needed to respond to those children and families who are identified as potentially suited to the targeted group program but subsequently assessed to be unsuitable or of a lower priority to other families. This should incorporate information provision, intervention and referral procedures. These should be negotiated and endorsed at regional steering committee level and agreed to by individual schools and school counsellors to facilitate a coordinated approach.

In order to maximise the early intervention potential for Got It!, both individually tailored intervention responses and standard referral information to those families with children with elevated SDQ conduct scores are required. Good practice would entail informing parents/carers of children with elevated screening results in any of the SDQ domains, explanation of the limitations of screening and contact details for further assessment or follow-up services. For many families a standard, current flyer with information on where to access parenting information, support and education programs may be sufficient, but for other families a tailored response and referral plan is likely to be warranted.

4.2.6 Child-focused group led by mental health clinicians and supported by school staff

The Got It! model of care states (pp. 15–16) that interventions will involve:

An 8-week child-focused group intervention delivered in the school for children with elevated

conduct problems. These programs, led by mental health clinicians and supported by school

staff, should aim to reduce conduct problems by improving awareness and management of

difficult emotions such as anxiety and anger, improve social skills, promote self-esteem and

enhance problem solving skills by identifying and building on existing strengths.

These stated goals of the child-focused group align with the outcomes of the Exploring Together

program, which has been adopted by each of the three Got It! teams following a period of review

and trialling of potential programs. Whilst the Got It! model of care indicates an eight-week

program, Exploring Together is a 10-week program and hence takes the full term to run, which Got

It! teams regarded as appropriate to consolidate learning in the group. There is a small body of

evidence for the effectiveness of the Exploring Together program (Hemphill & Littlefield 2001).

Exploring Together has several components: a child-focused group; a parent-focused group; an

interactive child–parent group; partner evenings for other parent/carers not involved with the

regular group; and teacher meetings. This structure is supported by evidence on the value of

simultaneously working with different systems in a child’s life. Multi-component intervention

programs that target school, family, individual and/or peer systems in an interactional way, are

increasingly recognised as an effective way forward in reducing the incidence of conduct problems

(Bywater 2012, Foster et al. 2007, Woolgar & Scott 2005). The group sessions are also combined

with a more social time for parents/carers, children and facilitators. In one of the sites the

participants have breakfast together with facilitators before the group. The group sessions are

followed by debriefing sessions for all of the facilitators to review content and process and plan for

the next session.

When the children and parents/carers separate into their groups, the expectation is that two Got It!

staff and a school staff member (e.g. teacher, school counsellor, learning support officer, Aboriginal


75

education officer) work with the children and two Got It! staff work with the parents. With Got It!

staff absences and vacancies, this staffing level has not always been achieved. The groups have run

with less facilitators, or other school or CAMHS staff have been brought in to assist the Got It! team.

On the surface, the staffing level of five for a group program seems high, but in order to deal with

the group dynamics and behaviour of the children, two facilitators are required in both the child and

parent group. When group participant numbers are small, however, five facilitators may be

experienced as overwhelming by participants. A Got It! team member expressed concern that the

number of facilitators can also lead to confusion about who is taking the lead at any time in the

group.

The child-focused group is designed for children aged between six and 14. The program attends to

anger management, pro-social skills, conversation skills, problem solving and decision making. As the

Got It! target group is K–2, some of the content is advanced for the children, relying too much on

discussion of issues. Whilst keeping the core components, principles and structure, Got It! teams

have been able to simplify the content so that it is appropriately targeted for the age group and have

added in more activities and techniques built up from their own clinical practice. For example, one of

the teams has incorporated sensory stimulation activities. Got It! staff have built up confidence with

a range of activities and resources that they have found to work well.

Skills in group work with children have been important for the success of the children’s group.

Comments in Got It! team process journals indicate that a mix of facilitator skills is worthwhile and

that there are benefits in having a male facilitator. A successful group with the children involves

being enthusiastic and creating a fun and engaging environment. It was also found that having a

teacher-trained member of the Got It! team can help with age-appropriate group activities that

engage children.

Some of the school staff members involved with the child groups also played a role during the pilot

phase in shaping the child-focused group program. A school staff member interviewed for the

evaluation spoke about how she had assisted to revise and redesign some of the activities for the

child group, explaining how her teaching background helped to shape the type of activities that are

age appropriate and more likely to work with the particular children. The other important roles of

the school staff member are to ensure that the planning and implementation of the group fit in

appropriately with school systems and structures, to respond to queries from parents/carers and to

liaise with the principal and teachers about the content and progress of the group. Having a

committed school staff member to carry out this role is important for the coordination and success

of the Got It! program in schools. As pointed out in a DEC interview:

Teacher involvement is critical. They are the point of contact during the week when the Got It!

team isn’t there to provide information and then after the program.

Concern was, however, expressed in a number of DEC interviews that one person in a school gets

the role of Got It! coordinator, rather than all staff becoming informed and involved. A wider

commitment amongst school staff is required if the impact of the program is to be sustained.

The school staff member takes on an important role as the liaison between Got It! and the rest of

the school but may not have completed the training to run Exploring Together. The level of


76

involvement of the school staff member in group facilitation has varied from school to school. As a

Got It! staff member explains:

In the kids group you could bounce off each other and you really take it in turns and you can

keep that enthusiasm and motivation going but if there’s a behavioural issue that one person

can still facilitate the group while the other person takes the kid aside and addresses that

particular issue with the child. I know you’ve got the teacher in there with you as a co-

facilitator but really a lot of the time their contribution is quite minimal. It’s essential that you

have them there … but they might not know how to address that problem particularly.

In another Got It! team, a member comments:

It’s been really hard to clarify what we want from the support teacher … It’s a very key role and

when they do it well it’s terrific, and when they don’t … It’s not an educational group, it’s a

therapeutic group. So for them to understand that is huge and even things about

confidentiality … what’s appropriate behaviour by adults in the school … I think we make an

assumption that they will just get it, but they don’t, until we end up with a problem.

One of the team process journals examines this in more detail, with examples of the teacher in the

Got It! group disclosing parent–child interactions from the group in school staff meetings and also

discussing the mother’s involvement in the group with the child’s father from whom the mother had

separated. This reflects differences between the school and mental health contexts in terms of the

principles governing information sharing of information and points to the need for better

preparation of teachers for the role of group facilitator.


An induction session, addressing practice guidelines for family work, be provided to school staff involved with Got It! as standard practice in the preparation for Got It! groups.

Comments from all participant groups indicated that the children enjoy the Got It! group and look

forward to coming. Some of the comments from parents and children regarding the group

experience are provided later in this chapter. In particular, children look forward to having a parent

with them at school. One Got It! team member observes:

They always come in and are happy to see her. They’re happy to come to school on that day.

They want to come to school on that day to come in for the program.

None of the participant groups indicated that the children experienced any sort of stigma in

attending the Got It! group. Rather, it was viewed as a special activity that they enjoyed. There were

examples, however, of parents who decided that their child was not suitable for the group because

of concerns that they would be negatively influenced by other children in the group or labelled in

some way.

DEC staff interviewed for the evaluation commented on the positive impacts they had observed

amongst children who attended the group. This included improved self-esteem that led on to


77

improved behaviour. Some who were involved in early offerings of the Got It! program made

comments about families for whom they observed a lasting impact from the Got It! group. This

principal commented on the impact for children who took part in the group:

It’s an inclusive program. People underestimate kids. They know when there are problems and

that there are things that they need to learn … You wouldn’t believe how it’s changed them …

Because of the behaviour changes, they’ve become more engaged in academic learning and

this will be great for the future … If they’re better educated they are better able to make

better, rational choices.

Many DEC respondents to the online survey used the open question opportunities to note the

impacts that the small group had on some of the children who attended. These included being

calmer in class, being able to express feelings, providing them with practical strategies,

strengthening the parent–child relationship and building resilience. Whilst some noted no change in

the children who attended the small group, it was still recognised as a fun program that children

enjoyed attending.

Summary of findings: child-focused group led by mental health clinicians

Exploring Together is the intervention program adopted by all three Got It! teams and determined by them to be the most appropriate for the setting. The program comprises a child group, parent/carer group and an interactive child-parent component.

Exploring Together is a resource intensive program. For a group of four to eight families, five group facilitators, including a school staff member, are required.

The school staff member plays a key role in coordination of information with the school community. Thorough preparation for the role is important.

4.2.7 Parenting group for parents

The Got It! model of care states (p. 16) that the intervention will involve:

An 8-week parenting intervention, based on social learning theory, for parents of children with

elevated conduct problems, led by mental health clinicians and supported by schools staff.

Where parents are unable to attend a group-based parenting program during school hours, an

after-hours parenting group could be offered as an alternative. Assistance with practical

considerations of attending the group such as transport and childcare may also be offered.

The location of the group in the school is widely regarded as a constructive way to engage parents.

The school is familiar to parents and easy to access. Families often have good existing relationships

at the school. When parents/carers are not engaged in the school, the Got It! program can help to

promote more positive relationships, which is likely to have positive impacts for children in the long

term. Experiencing a mental health service on safe and familiar territory can build trust and

confidence in families to seek out such services in the future if required. Got It! team members

spoke about the benefits of running the parent group in the school:


78

There’s been huge benefits around running it in the schools because you’re building bridges

with Health and schools, but also with parents who have disengaged from schools. And

parents see their child and realise … ‘We thought the school was against our child, but actually

the teacher’s actually got some grounds to what they’re saying’.

For some parents, engaging with us in a positive way has meant that it’s probably facilitated

some time in the future for them to engage in other mental health services. It’s probably

broken down some barriers around what mental health service clinicians are like. That’s been

quite important.

A lot of the families who access this group I believe would not access individual therapy …

There was one particular family … had six children with huge, huge difficulties … probably been

referred to CAMHS over the years but had never really committed to something like this

before. The school said, ‘You would never have got them’ … So we just do capture, I think, quite

a unique group in that way. I think a lot of people feel comfortable coming into the school in a

kind of set program.

The following comments from interviews with DEC staff highlight the engagement of families in the

group program who would otherwise have been hard to reach, but also point to the challenges

faced:

Got It! plays an important role with families who are experiencing big challenges. It brings

parents into schools in a supportive environment. The program really coaches parents.

It has made links with lost families … Even if only five kids are helped, these may be the hardest

cases ever. These can be really difficult families … I really dislike the numbers game … Not all

families will be helped, but helping only two or three families in a school is a success.

It gives parents an early intervention into managing kids and behaviours … It targets

dysfunctional families well. The needs are huge, particularly families where there is mental

illness … The kids are up against a great deal of hardship … They realise that they’re not alone

and that their problems are shared problems … It offers a support system. The families who

were in the program are still pretty dysfunctional. It’s harder to change the parents.

A barrier to involvement in the school was experienced in one of the small schools where parents all

knew each other and thus were reluctant to participate in a group where they would be expected to

share personal experiences. It was determined that a parent group would not be viable and so the

Got It! team decided to run only a child group and to offer individual and parent–child interactive

sessions with a clinician as an alternative to the normal group program. Some parents took up this

option on a weekly basis. Apart from this creative solution to the particular dynamics at that school,

as a general rule the value of a group program held in the school was upheld by the Got It! teams.

Setting up a welcoming and non-threatening atmosphere in the groups is an important basis for

constructive work with families.

Everyone on the Got It! team is approachable and easy to talk to. They made parents feel

valued and were very professional. (Interview with teacher)


79

Got It! team members highlighted the importance of the interactive component of Exploring

Together where parents/carers can practice skills with their children in the group. Group facilitators

have the opportunity to observe children and parents together and provide feedback. They also

recognise the value in having a separate time when parents have the chance to discuss issues and

learn about different approaches. As a Got It! staff member comments:

When you have a child there we have more opportunity to coach in vivo. They’re having to

discipline the child right there in front of us. So they really get to practice the skills there in the

group. I don’t know of any other program that has this really heavy interactive component

which I think is really great and the children just love it. Their little faces when their mums are

there and touching them and playing with them and laughing is the best part of the group by

far.

The parent group in the Exploring Together program focuses on awareness raising regarding feelings

and relationships, developing understanding about factors underlying behaviours, teaching

behaviour management techniques and recognising and building on strengths. The parent program

also has space to cover additional areas. For example, one Got It! staff member spoke about adding

content on brain development and attachment issues in the parent group. Got It! team members

have found the program appropriate and effective for the target group of parents/carers, as a focus

group participant explains:

I think it’s very powerful in the repetition and I don’t know of any other program that does this

continuous repetition of the principles with the parents … for example saying, ‘Look at your

child’ and waiting for the parent to turn to their child, ‘Hold their hand’ and then waiting for

the parent to hold their hand, ‘Now give them the information’. That very basic step by step

that most of us would take for granted … But for parents that either have their own illness or

their own trauma or those sorts of things aren’t capable of taking on board, you know, ‘You

need to interact more with your child’. But the strength in this is in the repetition and the

modelling that happens by the facilitators and the immediate feedback that is given … because

the child responds and smiles to them.

The parent group is also an opportunity to reduce isolation and strengthen social networks amongst

parents. Facilitating parent interactions is therefore an important part of the group program. Some

schools have supported an ongoing ‘coffee and chat’ group after the Got It! group has finished.

Generally this requires a school staff member to organise this process. As one Got It! team member

comments:

That’s the first thing that parents always say is that they love meeting with other parents. They

love knowing they weren’t the only ones who have these issues and they often form quite a

tight group.

Got It! staff also pointed out the importance of focusing on using group facilitation skills to work

with presenting issues in the group and to balance this with delivering content. Attention to group

process, in order to engage participants and achieve outcomes, is emphasised in the Exploring

Together program and associated training. Group facilitators need to be able to create a comfortable

environment and to encourage participation. This can be done by working with participants in a


80

supportive way, rather than as an expert with the answers, as these staff from two Got It! teams

explain:

We’re working together with them. We’re not sitting there telling them how to do it. It’s

actually a kind of team effort. We’re there to support them rather than to give them advice.

We worked really hard on making it more reflective to the parents and to share their

experiences and getting them to kind of come up with the solutions and with just a little bit of

input from us.

Facilitators also need to be skilled in managing group conflict and dealing with dominating or

distracting participants. Group processes can be difficult to manage alone, but due to staff vacancies

in Got It! teams, at times parent groups have been facilitated by one worker. As one Got It! team

member explains:

They [parents] get quite passionate because we’re dealing with their children. So I think it

would be difficult for someone who hasn’t worked much to jump into the parent group. I think

it would be quite difficult without support.

Responsiveness to the dynamics in particular groups and the needs, issues and culture of

participants is required. Here, Got It! staff members explain how they have to adapt their approach

to the particular groups:

We’ve had a lot of Indigenous families in our program … they really enjoy just the space of

talking and talking and sometimes it’s been hard for me to manage … Taking the time to get to

know these families outside the group and just sitting with them and having lunch and those

type of little things I think has made a huge difference.

It really depends on the group of parents that you have. Because we ran, at the same time,

these two parenting groups that were so different. The needs of one group was really just like

a support group and the other parents were just really interested in ‘What’s the evidence

base?’ and ’What can I do better as a parent?’

Got It! teams have agreed that four to eight participants is a viable size for the parent group. Their

experiences indicate, however, that with a group of eight time can be short and not everyone gets

the opportunity to talk about their own experiences. In these situations it is important to attend to

processes and not to feel pressured to get through a program. At times it means individual follow up

with participants between group sessions.

Just paring it back to the essential stuff. Feeling like this process and a chance to speak is more

important than kind of giving information.

One of the perceived drawbacks of the Exploring Together parents group is the requirement that

only one parent attend the group. As a clinician from another service, who was referred a family in

the Got It! program, points out:

It flies in the face of evidence to exclude one parent.


81

Got It! staff have also recognised the limitation of working with only one parent/carer. In one focus

group they spoke about how they have ignored this Exploring Together principle in their parent

program:

I’ve been very flexible in who comes to the group, so in small groups I actually encourage both

parents to come if they can.

Yeah, that’s one downside of the Exploring Together program in putting emphasis on one

parent attending … the benefit is obviously greater if it’s a couple there.

So we’ve tried to engage a little bit more with the whole family … so it doesn’t kind of isolate

mum and the child from the rest of the family.

And we’ve had a lot of dads take us up on that and they have come to the group to see what

it’s all about. It’s happened a few times.

The Got It! staff went on to talk about how this arrangement appeared to work better than offering

partner evenings/ sessions, which tend to be poorly attended. There have also been a number of

families where different parents or carers have attended different sessions, but not both together.

Whilst this affects continuity, it has the positive impact of more than one carer engaging in the

program and learning strategies to relate more effectively with their child. In families where parents

have separated or where there are tensions or conflicts between parents and/or other carers in the

family, decisions about who comes to the group and who knows about the group can be delicate to

manage. Such instances were discussed in team process journals along with a recognition that an

agreement about these matters needs to be reached between relevant parties before a final

decision about inclusion in the group is made. The value in engaging fathers in the program and

strategies to do this has been given consideration by Got It! teams. Whilst there have been many

fathers who have attended the Got It! groups, by far the majority of participants have been female.

Two of the 13 parent group participants interviewed for the evaluation are fathers.

The Got It! model of care indicates that the parent group program could be offered after hours if

necessary and that transport and childcare could be provided if required, so as to increase the

accessibility of the program. One Got It! team provides a weekly $20 supermarket voucher as an

incentive for attendance at the group program. The Got It! teams have trialled different times for

the group. All three have found that running the group during school time, preferably just after

morning drop-off or just before afternoon pick-up, is most convenient for parent/carers to attend.

This is generally supplemented with one or two after-hours partner evenings that tend not to be well

attended. Two Got It! teams have been able to organise childcare providers to mind younger

children at schools where this is needed and reported that this has helped to improve attendance.

The third Got It! team has faced difficulties with organising childcare (space, licensing, access to LHD

funds) and this subsequently has been a barrier to attendance for some parents. Whilst transport

has been offered by Got It! teams, parents rarely take this up. Providing the offer of childcare and

transport up front was identified in a process journal as important in supporting commitment by

families to the group.

There were 12 Got It! groups run in 12 schools during the six-month evaluation data-gathering

period in 2013. A total of 68 families took part in the 12 groups (63 completed the program and five


82

families withdrew). There were between three and eight families in each group. The five

withdrawals were from five different groups. Weekly attendance data was provided to the

evaluation team for 11 of the 12 groups. The families who withdrew were excluded in calculating

attendance rates, as was data from one school where an alternative program of individual sessions

was offered in place of the standard parent group. The attendance rate across the remaining 10

groups was generally good. The proportion of sessions attended by both parent/carer and child was

88% across all groups, ranging from 67% to 100% for individual groups. An additional 5% of sessions

were attended by a child without their parent/carer present. A one-off partner session was offered

at four schools. A partner/other family member attended for 68% of families from the four targeted

groups.

It was those school staff who were more closely involved with the Got It! program who volunteered

to be interviewed for the evaluation. Comments from these DEC staff on the impact of the small

group program for families were generally very positive, as the following comments indicate:

The six families had really positive experiences of getting help and support and they didn’t feel

threatened … For the three families still in the school, the impact has been ongoing [gave

examples of taking up counselling, other groups and education subsequently] … I’m very

passionate about it … It’s very important.

One mother has completely turned around and now realises that her child has needs too. She’s

more engaged with the school and with the child too since Got It!.

A range of comments on the impact of the small group program for the families who participated

was provided by respondents to the online survey. These included the following categories of

responses:

Strengthening the connection between families and schools.

Taking the first step in what could be a long but positive process.

Developing skills in how to interact positively with children.

Positive changes in parent-child relationships.

Reports of improvements in home life.

Learning new strategies for dealing with parenting challenges.

Developing problem solving skills.

Normalising experiences and providing social support between parents who were previously

isolated.

Parents feeling empowered in the role of parent.

Seeing the value in and knowing how to seek mental health support in the future.

The online survey did not distinguish between the child and parent/carer components of the group,

but asked for ratings of the success of the small group program overall. As depicted in Figure 8, the

small group program is well regarded by Education and Health staff. The success of the group

program was rated as excellent by 34 (47%), with 52 (72%) rating it as either above average or

excellent. The small group program was rated by 12 (17%) as average and by 4 (5.5%) as below

average or poor.


83

Figure 8: Health and Education staff: ratings of success of small group program

Further insights into the impacts of the groups, from the perspectives of parents, are provided below

and outcome data from standardised scales are presented in the Outcome Evaluation.

The major concern expressed about the targeted group program in the online survey was the low

level of interest in the group program within the school community, with only a small number of

families participating. Whilst more families could benefit from this early intervention program, many

respondents said that family interest, engagement and participation were poor. This challenge,

recognised by the Got It! teams, is also faced by other services working with children and families. In

some schools, however, there have been very high levels of interest in the group program and not all

interested families could be accommodated. Growing familiarity with the Got It! program within

school communities is likely to make an impact on how it is viewed. The principal at a school that is

taking up the Got It! program a second time made the comment:

This time there is a much greater pick-up of the program. People were more hesitant the first

time. Parents are lobbying to get into the program this time.

Another concern expressed in the online survey was a lack of information being provided back to

teachers on how children are progressing through the group and how classroom strategies and

playground interactions by teachers could support the work that is being undertaken in the group

program. A number of respondents expressed a desire to be more informed and more involved in

the program in the school.

In schools where there was no defined universal program or well-developed universal parent

information campaign, the Got It! program was generally regarded simply as the small group

program. Even in schools where there was a universal program, the small group tended to dominate

the perceptions of what Got It! is about. Therefore, a commonly held view by school staff was that ‘It

is a considerable investment of staffing for a very small group‘ (online survey).

0

5

10

15

20

25

30

35

40


Respondents' ratings of success of small group program (n=72)


84

Summary of findings: parenting group for parents

Exploring Together offers a suitable program for parents. The interactive component with their children has been found to be particularly valuable for skill development.

Positive impacts of parent group have been observed, including strengthening positive engagement between: - parents and school - parents and other parents - parents and mental health service providers. All of which are protective factors for the future.

The attendance rate of child and parent/carer together was 88% across 10 groups.

Advanced group facilitation skills are required to maximise impact and outcomes of the group program for parents.

Involving only one parent/carer in the group program appears to have presented difficulties in some families as the other partner can feel excluded.

Negotiating group attendance when parents/carers are estranged can also present barriers to attendance.

Offering childcare for younger children during the Got It! group and transport to the group improves interest in and access to the group program.

4.2.8 Individual behaviour management for children with extreme conduct problems

The Got It! model of care states (p, 16 and p. 19) that the intervention will involve:

Individual behaviour management support for children exhibiting extreme levels of conduct

problems.

Further assessment and intervention for children identified as having severe conduct problems

through support from the wider CAMHS service and other relevant health and child and family

support agencies.

Got It! team members have provided individual treatment for children who have not been suitable

for the group program but who have significant conduct problems. This has been done during the

time of involvement with the school in particular circumstances where other referral services are not

available at the time. This type of individual / family work has not, however, been provided as

standard practice and is not offered in all schools or by all of the Got It! teams. Got It! team

members indicated that cases of extreme behaviour problems were not common in the schools they

have worked with, but that in some cases children and families have been referred on to other

services.

Having clinicians in the school does, however, provide the opportunity and flexibility to work with a

child and their parents/carers when particular needs present. This may be to establish or

supplement a longer-term behaviour management plan in conjunction with the school or other

referral services. The following examples of this were provided by Got It! team members:

I did that with a boy who actually wasn’t in the group. He was too high needs for the group …

and we were wanting to get him into Paediatric Health but their referral criteria meant that he

needed to have some primary intervention first. We didn’t think that they would turn up to a


85

private psychologist or whatever, so we gave him six sessions. Ultimately we managed to get

him in there … which was exactly the right place. (Focus Group)

Successful follow up with a high-needs family taken on late last year. Got It! provided an

interim service while waiting for Child Protection Counselling Service to be able to take patient

on. Family engaged surprisingly well and attended eight sessions … Child is much more settled

in school and they report improvements. (Process Journal)

The individual work for children who have not been selected for the Got it! group is valued by school

staff, as the interview with a district guidance officer indicated:

Got It! have worked with really pointy-end difficult families and linked them in with services.

Some have been the previously unreachable families who have been engaged and are now at

least willing to consider other services in the future.

With staff vacancies on Got It! teams and the focus on the group program, individual work has taken

a lower priority during the pilot phase. The limited resources of the Got It! teams require that

individual interventions are provided only when suitable services would otherwise not be available

or accessed. A time limit is required for such interventions (e.g. one team has set a limit of six

sessions) with the goal of transitioning to other services. There may also be situations where

children are identified to the Got It! team by schools, but families do not consent to intervention or

the expectations are too high, as this Got It! team member explains:

There’ve been other kids who are really high end that I’ve tried to engage the parents and they

just haven’t come. Like I said, they’ll come to one and that’s it. Obviously our resources are

limited and we can’t do too much. But I do think sometimes we go to schools and the kids are

quite high needs and we need to offer some opinion and advice. So, supporting where they

need to go.

There are also children identified in schools who have significant behaviour problems and high

needs, but are already linked in with other services. Pressure from school staff is sometimes

experienced by Got It! team members to ’fix it‘. When there are other services involved with a family

and given the time-limited and focused nature of the Got It! program, Got it! teams have declined

involvement with these families. Clarity about what is included in the Got It! model of care and the

limitations of the program is important for Got It! teams, as a staff member emphasises:

I think just being clear with schools about ’This is one intervention. We’ll do this. It’ll work for

some people. It won’t work for others. For some people it will help a little bit.’ I guess just

being clear about limitations.

Without actually providing any direct intervention with the child or family, Got It! team members

can also play an important role in offering teachers or families information, an opinion or details on

other services. Providing this type of consultation within the school was discussed by Got It! staff as

the most common role taken on in relation to children with specific behavioural problems.

Sometimes we offer an extra assessment or a bit of extra support, even though we can’t take

on that kid in the group. I think that goes a long way to engaging schools and helping them to


86

feel like they’ve got something valuable out of the process as well … There’s always extra work

that you could do.

A number of teachers commented that the children who needed the Got It! program most were the

ones that did not have the parent commitment to attend a group or even the parent SDQ screening

form completed. It is for these children that teachers were seeking guidance from Got It! teams on

how best to manage issues in the classroom. It appears from interview and survey comments that

teachers would appreciate more consultation with mental health clinicians to gain information and

guidance on behaviour management plans. Whether such work should be the role of Got It! teams

or whether other specialist services within DEC should be providing this consultation to teachers and

intervention for families is not clear. The need to clarify the role of Got It! in relation to individual

behaviour management work within schools and how this interfaces with other services in DEC was

evident.

In addition to this individual work for children outside the Got It! group program, during the pilot

phase one Got It! team provided some individual/family work as an adjunct to the group program.

They noted in the team process journal that making the time for these extra individual meetings

increases the effectiveness of the group program, if parents are keen. The behaviour of these

children is unlikely to be regarded as ’extreme‘ and therefore does not strictly fit under this

component of the Got It! model of care. It does raise the question, however, of whether individual

consultation with families in the group program should be incorporated into the Got It! model of

care. Some parents requested this type of intervention from the Got It! team, and one parent raised

this intervention gap in the evaluation interview, reported later in this chapter. Following is a

comment from the process journal of one of the Got It! teams:

Many parents seemed to appreciate one-on-one time and fed back that they would have like

more of this during the term.

Consultation with classroom teachers has tended to be haphazard during the pilot phase, with many

teachers commenting that they felt ’left in the dark‘ about the group program. Whilst a respect for

privacy of families may mean that some information is kept confidential to Got It! staff, securing

agreement by group participants to regular updates to teachers on individual progress and plans

could enhance the program. Based on comments from both teachers and parents, this would be

welcomed. In most of the interviews with DEC staff, and in many of the comments added to the

online survey, the need was identified for classroom teachers to be informed about the strategies

covered in the Got It! small-group program. The desire to be more involved was expressed in order

to support strategies more effectively in class. In some ways this links back to the teacher training

and the universal program, which if delivered effectively can work to complement the small group

program. This does not, however, replace more personalised information about individual children.

All teachers use some behaviour management techniques or programs in the classroom and also

have contact and communication with families.

The individual behaviour management component of the Got It! model of care was the only

component in the online survey that had over 10% of respondents rating it as below average or poor

in its success. As shown in Figure 9, 38 (53%) rated individual behaviour management programs as

above average or excellent, indicating that there is some good work being done, but with 5 (7%)

rating this component as poor, another 3 (4%) as below average and 10 (14%) as average, closer


87

attention to this component of the model is warranted. It is also worth noting that 16 (22%) of the

respondents did not know about this component of the Got It! model of care, including some Got It!

team members.

Figure 9: Health and Education staff: ratings of success of child behaviour management

There were a number of comments in the online survey indicating that teachers were seeking advice

and support in developing plans for managing children in their classes, but that the Got it! teams

were not available to assist due to attention being focused on the children selected for a very small

group. Acting as a consultant to teachers and providing information on referral services in such

circumstances can be part of developing a behaviour management plan that fits within the Got It!

model of care. This work could be done with teachers even if the Got It! team does not work directly

with the child or family. As one teacher put it:

I’m not sure what the Health people did except for run the small group program. They were

not helping others who have children with behaviour/social/emotional problems because

parents didn’t bring forms back. Perhaps ideas for teachers who have raised concerns through

SDQs as a whole-school approach rather than only six children.

Summary of findings: individual behaviour management for extreme conduct problems

The behaviour management component of the model of care is not well defined. The definition of ’extreme conduct problems‘ and the extent and nature of the individual behaviour management intervention to be provided has been unclear to the Got It! teams; hence, practice has been inconsistent.

The extent of individual work is limited by staff resources.

Teachers expressed a desire to have input from the Got It! teams in designing and implementing individual plans for children not in the group. Many voiced concerns that Got It! only worked with a very small number of children in the group program and that these were often not the children in greatest need.

0

5

10

15

20

25

30


Respondents' ratings of success of child behaviour management programs (n=72)


88


The role of Got It! teams with individual behaviour management in schools requires clarification in consultation with school counsellors, district guidance officers and other specialised services in DEC to ensure that roles are complementary and integrated. If appropriate in the context of other services, individual behaviour management could be strengthened to sit alongside the targeted group program and supported referrals as the suite of Got It! early interventions for children with elevated conduct problems.

Individual behaviour management could be approached by Got It! in different ways:

Assessment, collaborative planning and referral.

Consultation and support to classroom teachers to develop and implement a plan.

Providing individual family intervention sessions at the school for children not selected for the group and likely to benefit from individual work, but who are unlikely to access an alternative service.

Provision of one or more individual family sessions as an adjunct to the group program to develop an individualised behaviour management and referral plan and to provide the opportunity to address concerns that may not be appropriate to discuss in the group setting.

Engagement of classroom teachers in interventions could be strengthened through:

Consultation sessions with teachers to develop classroom strategies and plans for particular children who are not be attending the small group program.

Weekly updates to classroom teachers on group program content and progress. This could be a group presentation to K–2 teachers followed by brief discussion.

Consultation sessions with classroom teachers regarding individual children in the group program, providing an opportunity for teachers to have input into development of behaviour management plans and to ensure that classroom interventions are in line with the work in the group program. Such sessions could also involve the parent/carer.

4.2.9 Referral to services (CAMHS and others) for further assessment and intervention

The Got It! model of care states (p. 16) that the intervention will involve:

Streamlined referral of identified children and their families to CAMHS or other appropriate

services for further assessment and intervention.

The Keep Them Safe Interim Report (NSW Government 2013, p. 4) describes the early intervention

service system as increasingly ’complex and organisationally fragmented‘, with a growing number of

service providers. This in turn creates ’costs of collaboration: the time and effort required to identify

potential partners for collaboration and to engage with them when managing individual cases‘

(2013, p. 4). These comments reflect well the challenges faced by the Got It! program.

Got It! has a potential role to play in referring children and families at various points in the Got it!

process. General information on services for self-referral can be provided as part of the parent

information campaign and at parent seminars for the general school community. Referral for specific


89

follow up after an assessment interview may be provided for those accepted into the Got It! group,

those offered individual work, and for those where there is no further involvement by the Got It!

team. Referral may also be provided to follow through from the Got It! group program. Families

gaining knowledge on available services, and confidence to access these, are important aspects of an

early intervention process. Sometimes families will require more active support to take the step of

contacting a service and, as a Got It! team member points out, in these cases providing information

for self-referral is of limited value:

It can be a disservice to leave it with families to make contact with services … If we don’t

ensure referral and follow up then Got It! is just another fly-in and fly-out service.

Finding the time to manage referrals effectively within the resource and logistical confines of the Got

It! program has presented difficulties. Referral at a time when families are ready is also an issue as

the Got It! team may have left the school at the point when the family is ready to take the step.


Educating school staff on key referral organisations and referral procedures is a worthwhile core component of professional development sessions delivered by Got It! teams in schools.

In the school profiles that Got It! teams completed, there was a section on referrals. Responses

revealed that there are many types of interactions with and for families that could be included under

the term ‘referral’. It was not possible to simply identify the number of referrals made for families at

each school. Some of the activities that were recorded included discussions with families about

where they might seek assistance in the future, provision of a standard list of organisations for self-

referral, ‘recommendations’ to families (written or verbal), ‘suggestions’ to families, letters of

introduction/support for families to take to a GP or other service, referral to school counsellor, and

direct referrals to particular NGOs or other specialist services, including completion of referral

reports. At times it was families who were interviewed but not invited to join the group who were

being referred and at other times it was group participants who were being referred as a follow up

to the group program. No standard referral protocols were evident.

The Got It! teams provided examples of the referral process working well. This was particularly the

case with referrals within the Health service where co-location of services, a formal internal case

review mechanism or internal referral procedure streamlined the process, as discussed by this Got

It! team:

Being quite proactive … not waiting until the end of the program … So we try and get onto that

as quickly as we can and we write a supporting letter and get them to go and see their GP and

get the ball rolling as soon as we can … Because they are on our caseload we can access a

psychiatrist just like the CAMHS staff can so we can obviously bring it up at clinical review

meetings … Most of the time he’s going to say, ‘Look, you know I’d need a paediatrician’s

opinion on this’, but there has been one instance where he’s medicated a particular child and

you get a massive, massive outcome for that family.

The representatives from referral organisations interviewed as part of the evaluation were from

other health services as well as non-government organisations. It was, however, only those from


90

health services that could identify specific families that had been referred from Got It!. A dominant

theme in each of these interviews was the detailed referral process and strict criteria that had to be

met to get families in to be seen by these referral services. These are specialist clinical services for

children with mental health or complex developmental issues that required a clear diagnosis and

detailed referral information. Referral by Got It! was facilitated by pre-existing relationships (co-

location, clinical review meetings) and clinical mental health backgrounds. Feedback from the

referral services was very positive on the preparatory work done by the Got It! program with the

families and the value of a school-based program in providing access for families in need. One

clinician spoke about a child diagnosed with ADHD whose family had struggled for years without

knowing how to access services, assistance or information. It was through the school and the Got It!

program that this family was linked in with services. A clinician in another service spoke about the

benefit of Got It! ’getting in early‘ with a family with significant issues of mental health, drug use and

violence. The Got It! program was seen to provide practical assistance with parenting and to carry

out the initial work required to engage with a health service. A third clinician spoke about a referral

of a child with queried autism spectrum disorder that was subsequently confirmed. Again, the initial

work with the family provided by Got It! was described as a positive experience for the family, the

quality of information provided in the referral forms was high and the value in forging stronger links

with schools valued. This clinician identified:

The biggest plus is that it is early intervention that hopefully will change the trajectory for

some of these children.

These examples demonstrate the potential for Got it! to be linking families in to specialist health

services. Such services would not, however, be appropriate and could not be accessed by the many

families who require more generalised parenting support. There is a range of services provided by

different auspicing bodies and funded under the NSW Government Family & Community Services

Early Intervention and Placement Prevention Program (EIPPP) that include low level parenting

support and intensive family preservation programs. Whilst the Got It! teams mentioned some such

services that they had involvement with, they struggled to identify referral organisations for the

evaluation team to interview, despite several requests. This suggested to the evaluation team that

the Got It! teams did not have strong referral networks and that the challenge of establishing the

Got It! program within the early intervention service system was not fully met during the pilot

period. Staff in the Got It! teams are, however, aware of the issues relating to effective referral

systems and have been taking steps to strengthen this aspect of the implementation of the Got It!

model of care.

Two services funded under EIPPP were contacted as part of the evaluation. One service suggested by

a Got It! team was contacted but the coordinator had no recollection of the Got It! program. The

other service made contact with the evaluation team on hearing that an evaluation of Got It! was

taking place. This service provides case management for families incorporating information, referral,

planning and parenting support for up to one year through fortnightly home visits and regular phone

contact. Depending upon what is happening for the family, they can work with them on developing

pathways to employment, education, accommodation etc. They also offer parenting group programs

and children’s groups. It is this type of low level support in the home that many families who are

assessed for, and/or undertake, the Got It! group would also be suited to. The service coordinator

said that they were keen to receive referrals from Got It! and that a staff member could attend Got

It! group sessions and/or three-way meetings with families before the completion of the Got It!


91

program to assist a smooth and supported referral/transition process. The experience for this

organisation was, however, that Got It! was unresponsive to contacts by the organisation.


Early Intervention and Placement Prevention Program (EIPPP) funded services are community-based, flexible, low level, early intervention family support services that the Keep Them Safe strategy requires to work with families to provide support and prevent escalating problems. Referral to specialist health services is important for some children, but an integrated approach to service provision and referral to EIPPP services is likely to assist families with a wider range of needs.

Attention to developing an integrated, collaborative approach with EIPPP services should be a priority for the future of the Got It! program. Relationships with these services may be built up through attendance at interagency groups and forums. The need for agreements with specific services is also warranted.

The involvement of service providers at some Got It! group sessions, as a core component of the targeted group program, could assist to build familiarity and trust of other support services amongst families.

The other key organisation identified in the design of the Got It! pilot is the pilot Family Referral

Service (FRS). Got It! pilot sites were specifically chosen to be in the same areas as FRS pilots. Apart

from some consultation, however, none of the three Got It! teams identified FRS as playing a

significant role in service provision or referral in relation to the Got It! program. A service provider

from the FRS in one of the pilot sites was interviewed as part of the evaluation process in order to

examine the potential relationship between Got It! and FRS. She identified a range of ways that FRS

could support and follow through on the work of the Got It! program, if the pilot programs were

continued:

Comprehensive assessment to establish the most appropriate referral services, and/or

gather assessment information from other services with the consent of the family.

Case management for two to six weeks as a referral plan is being developed with family and

referrals made.

Assist families whilst they are on wait-list for other services within ’checking in‘ phone calls

to families and feedback on progress with referral / appointment to keep family engaged.

Provide consultation to Got It! staff in sorting through the best services or options for

referral for particular families / needs.

Some limited funds for service brokerage.

Follow up with family after the referral to see how the service is progressing.

Follow up with referral service to give and receive feedback on the referral process.

The FRS representative spoke about how providing families with a list of possible services or making

multiple referrals with the hope that one service will be picked up is unlikely to be successful.

Alternatively, a ‘consolidated referral’ approach promoted by FRS entails:

targeted, appropriate referral based on comprehensive assessment

full communication on which services are involved and nature of involvement


92

consent of family members

follow up to ensure that referral is implemented and further assistance is required.

Got It! team members were mindful of and spoke about the barriers they faced with referrals and

hence the missed opportunities for referral. These included families not being interested in referrals,

families not meeting the criteria for referral organisations, no suitable local services to refer to, time

pressures limiting opportunities to provide a supported referral and a loss of momentum as groups

finish in the last week of term. Got It! teams discussed some of these challenges in the focus groups:

We work out whether they’re willing to actually engage rather than send them to a service …

Sometimes they won’t even engage with us.

Behaviour problems aren’t necessarily seen as a mental health issue. Of course the floodgates

could open. You can understand why that distinction’s been made, but it does make the

referral a bit difficult. It does mean that the continuity of service doesn’t flow as well as we

would have liked it to.

We just send them a letter with recommendations on it basically …. Or phone with

recommendations whilst the letter’s being put together …. Most of them self-refer … It’s at this

point that we get squeezed for time … and the sense of having to move on with the small

group, with all the logistical stuff.

There’s actually no exit phase in our design, so there’s the set-up phase and the

implementation phase, but there’s no exit phase. Actually there’s a lot of work that happens in

the exit phase, which should happen in the exit phase. We should be making our referrals.

What we’ve ended up doing … is talking with families throughout the group … and trying to

get families to think about making their own referrals … But Ideally we would be able to have a

report written two weeks into the next term, sit down and talk about the referrals, talk about

which ones they need to make and which ones we can make. That’s our ideal but it hasn’t

happened … We do make a couple of referrals here and there on the phone and some of these

families need it in writing.

Got It! teams experience the school term following the group program as ’trying to fit in‘ follow-up

activities whilst focusing on starting up new groups or screening.


A ’follow-through‘ phase is required in the Got It! model of care. Closer attention to this phase would, however, have workload and resource implications for Got It! teams. A clearly identified follow-through phase ideally would entail scheduled appointments with each of the families who participated in the group, review of individual plans, follow up on referrals made, provision of additional information and consultation with classroom teachers, school counsellors and other school staff relevant to families for whom a group or individual intervention has been provided.

Establishing relationships with referral organisations and understanding their referral criteria is key

to effective referrals. These relationships may already exist because of prior work associations or co-


93

location, but often they require proactive engagement. The Got It! teams have tried strategies to

establish connections with referral services, including visiting and consulting with other services,

attending interagency meetings and holding information forums for local service providers. One Got

It! team in particular has put energy into engaging local referral services in a community exchange

forum and has noted the benefits in their process journal, both in terms of the increased knowledge

on services amongst service providers and in terms of subsequent appropriate referrals and follow

up with Got It! families. One of the Got It! teams has found that regular case discussion meetings

with a key referral service within CAMHS has improved the referral process as well as enhancing the

quality of their own clinical service provision through the peer review process. It has taken time to

build up a profile and relationships with referral organisations. As a new program, Got It! is still in

the early stage of establishing such a profile, but teams are making efforts to increase this profile, as

was explained in the focus groups:

We organised a networking day where we went around to different services in the area and

invited them for afternoon tea. So, trying to build our profile and letting other services know

and finding out about what they do. I think, as we go along, doing more of that would

probably be really useful just to get more linked in with services in the area.

Regardless of referral procedures, there remain some service gaps that mean that referrals are not

possible. This is particularly pronounced in rural towns. As a Got It! team member explains:

That’s my dilemma. When we don’t know the town and the clinicians within the town and the

skill base … I’d love to be able to refer to someone and say, ‘I have so much confidence that

you are going to get exactly the right service that you need’, but it’s just not the case … There

was one family … what do you do? There was nothing there that they could access.

Beyond the rural setting there were other service needs identified by the Got It! teams; for example,

for regular parenting coaching, including in-home, one-on-one family support. This type of

community-based primary prevention and family support is an important early intervention adjunct

to mental health services as many of the families identified by Got It! teams do not yet meet the

criteria for mental health services. These types of services have expanded in recent years under the

EIPPP and, as discussed above, there is a need for Got It! teams to focus attention on building

relationships with these referral organisations.

For specialist, and many secondary mental health services, a referral must be made by a primary

health care provider (e.g. GP) and cannot be made directly by the Got It! team. This is experienced

by Got It! teams as an unfortunate extra hurdle for families in need of services. One team has noted

that writing letters of support or introduction to primary care providers in such situations can assist

referral of families to appropriate services. When referrals are made directly by the Got It! teams,

detailed referral reports are often required, which are time consuming to complete. Once a referral

has been made and accepted there are commonly waiting periods of weeks or months before a

family can be seen. This has been the case even for services within the LHD, suggesting the need for

more streamlined internal referral processes.


94


In order to maintain continuity in service provision and capitalise on the momentum generated by the Got It! intervention, attention should be given to referrals as early as possible. For those in the targeted groups, it would be appropriate to pay attention to the development of a referral plan throughout the group program. Due to waiting times, there may still need to be a ’stopgap‘ measure. Family Referral Services may be available to assist families during waiting periods and could be brought in by Got It! teams.

The school counsellor is a key person in the school who can follow up children and families. It

became apparent through the evaluation that a closer involvement by school counsellors is likely to

contribute to lasting impacts of Got It! in the schools. Engagement by the school counsellor during

the assessment phase and/or through the group program is likely to support a more effective

referral and follow-up process. Establishing a relationship with the school counsellor and keeping

them informed about particular children and families could be made a higher priority. Whilst

attention to consultation and early referral by Got It! teams is likely to improve relationships, Got It!

teams found that the situation varied between schools, as the following comments indicate:

Some schools seem to have a really good connection with the school counsellor and they’ll

bring them in and they’ll make them part of the program and they’ll include them. Other

schools, you actually have to watch yourself not to get caught up in the split between schools

and school counsellor … There’s been a few where there’s been some conflict with the school

counsellor or they’re just not quite on the same page or they’re only in the school one day a

week … so it’s really dependent on the school and their relationship with the school counsellor.

I did some follow up with one child and I worked quite closely with the school counsellor for

probably about six months into the program. She was fantastic … The little girl ended up being

diagnosed with Asperger’s disorder and she was putting great stuff in place at school.

Interviews with school counsellors and district guidance officers (effectively senior school

counsellors) also provided some examples of positive involvement with Got It!, but there was also

the sentiment that a more integral role needs to be taken by school counsellors in the Got It!

program. Involvement by a school counsellor as a group facilitator can help to establish relationships

for ongoing involvement with families in schools as well as coordination of information in the school

setting. The following excerpts from interviews with a school counsellor and district guidance officer

present a view that the school counsellor is the most appropriate person to take on the school group

facilitator role:

If possible, it would be ideal to have the school counsellor involved with the small group

program … but it’s up to the principal.

The best thing is for the school counsellor to be involved in the group program … it is really

valuable … provided an avenue for involvement with families that is ongoing.

Depending upon the family’s concerns and expectations, however, it may not always be appropriate

to refer to the school counsellor. Got It! team members spoke further about their experiences with

referrals to school counsellors:


95

Sometimes if the school counsellor is involved we give it to them and say, ‘This is where they

need to go’ and leave it to them to follow up because they’re going to be there. We’re not.

Other times we do the process of maybe having a few individual sessions and doing a

handover as we kind of figure out the best place for them to go and work on that engagement

a little bit if they need it.

School counsellors generally work between several schools and may not be at the school on the days

that the Got It! team are there, which makes communication more difficult. It was explained by DEC

staff that the principal in each school decides the areas of priority for the school counsellor, which

may not include the Got It! program. Got It! teams have generally found school counsellors difficult

to contact and engage in the program, as the following comments indicate:

There’s a massive emphasis put on cognitive testing. The school counsellors don’t really get to

address much else.

It’s a mystery as to how school counsellors are factored into Got It!.

Involvement by school counsellors and referrals to school counsellors were described as ad hoc and

inconsistent. A teacher involved as a Got It! group facilitator, who was interviewed for the

evaluation, spoke about how the school counsellor had tried to get a parent social support group

going to follow on from the Got It! group, but that this was not successful as the parents did not

know or trust the school counsellor who had not been involved with the Got It! group.


Further collaborative developmental work is needed to enhance and coordinate the role that school counsellors have with the Got It! program, in the areas of assessment, group program delivery and follow-up referrals. Workshops between Got It! staff, school counsellors and other specialised services in DEC could be run at regional and/or state level as a strategy to assist with the development of protocols and expectations.

Comments made in DEC interviews indicated that the following factors contribute to constructive

relationships between school counsellors and Got It!, and could be enhanced in the future:

School counsellor contribution to initial assessment interviews.

Got It! clinician and school counsellor consult and exchange information regularly

throughout the small group program on progress of group and individual families.

School counsellor to visit the small group on at least two occasions to meet families during

social segment of the group and talk about ongoing role of school counsellor and supports

available.

School counsellor to complete Exploring Together training and co-facilitate groups in schools

when possible.

Run Got It! group on a day when the school counsellor is at that school if possible.

School counsellor engaged as early as possible with families who are likely to require follow

up and a plan developed in three-way meeting between family, Got It! staff and school

counsellor.


96

School counsellor act as a consultant to teachers in relation to behaviour plans for children

in their classes.

School counsellors and referral service providers interviewed for the evaluation expressed an

interest in and capacity for some involvement in the Got It! groups. Such involvement was regarded

as a strategy to break down the barrier of unfamiliarity and to facilitate a better referral and follow-

up process.


Involving school counsellors and other referral services in the targeted group program could support the uptake and coordination of subsequent family referrals. The level of involvement should best be negotiated and appropriate to the interests and needs of group participants. This could involve key referral personnel attending the social time attached to the group program to meet families or speaking in a group session for a few minutes about the service they offer or even being a co-facilitator for the group program. Building familiarity with referral services in this way can influence families to take up services in the future, either as a direct referral or as a self-referral at some later point when issues emerge in the family.

Results from the online survey presented in Figure 10 show that referral of families was the

component of the Got It! model of care that had the lowest number of above average or excellent

ratings with 34 (48%) combined. It was also the component for which the highest proportion of

respondents had a ’don’t know‘ answer at 21 (29%). Clearly there has also been some effective work

done, with 21 (29%) ranking referral processes as excellent, but in line with the comments made

above, this is a component of the Got It! model of care that is worthy of closer attention and

development for the future.

Figure 10: Health and Education staff: ratings of success of referral process

0

5

10

15

20

25


Respondents' ratings of success with referrals (n=72)


97

There is a body of literature on single-session interventions that could also be drawn upon in

developing an effective framework for dealing with referrals in the context of assessment interviews

that need to be approached as potentially a single contact (e.g. Gibbons & Plath 2005, 2009, 2012).

Summary of findings: referral for further assessment and intervention

Referral in the Got It! program can include a variety of practices, with varying levels of demand on staff time: - Direct referral of a family to a specialised or generalised service, via phone or

written referral report, with or without follow up. - Provision of information to families for self-referral: verbal or written, individually

tailored or general, with or without follow up. - Provision of a letter of introduction or support for a particular service that families

contact themselves, with or without follow up. - Educating school staff on referral services and procedures, to build capacity within

schools to make and support referrals. This may entail general information and/or individualised recommendations for particular families.

Each of these referral practices have occurred for some families during the pilot phase, but referral practices tend to be inconsistent and follow up is limited.

The referral networks used by the Got It! teams were found to be limited, with a reliance on referral to other health services.

Referred families were those who attended assessment interviews, including those who were selected for the targeted groups and those who were not. Given the variety of referral practices and minimal attention to follow up, it was not possible for the evaluation team to identify the number of referrals made or taken up.

Barriers to effective referral identified by the Got It! teams included: - family resistance - lack of staff time to make or follow up referrals - referrals given a lower priority to screening and group intervention.


There is a need to develop a referral policy and comprehensive referral procedures for each stage of the Got It! program in schools. Too narrow a focus on the group intervention can detract from the key role of Got It! in facilitating and supporting longer-term engagement of families with general community services and specialist services. Referral procedures could cover:

referral points in Got It! model of care

critical timeframes for referral or information provision

standard documentation and resources

processes and resources for families to build familiarity with local services and support self-referral

processes and resources to support referral by school staff

relationship-building strategies with other referral services (e.g. hold and attend forums, establish agreements)

processes to enhance collaborations with school counsellors throughout the time of involvement in a schools

processes to avoid multiple assessments

procedures to manage wait periods.


98

4.2.10 Conclusions on the model of care

The Process Evaluation revealed that, on the whole, the Got It! model of care has been successfully

implemented across the three pilot sites. There have been some differences between the sites in

relation to the universal aspects of the model and in the attention given to individual assessment,

individual behaviour plans and referral processes. The online survey has provided a picture of how

the impact of the Got It! program is perceived by Education and Health staff who have been involved

with the program. As discussed above, the vast majority of those staff indicated that the

components of Got It! have achieved a reasonable or above average level of success. Whilst still

rated reasonably well, the three components of the model relating to individual family interventions

have rated lower than the others, these being:

assessment of families

individual behaviour management

referral.

Individual behaviour management and referral are also the two components about which a high

proportion of respondents indicated that they did not have knowledge (22% and 29% respectively),

suggesting that these components have been given less attention in the pilot implementation.

A tension exists between the potential for the Got It! model to be flexible and responsive to

expectations in local school contexts and the benefits of implementing a standardised program that

can be systematically evaluated in the longer term and for which outcomes are demonstrated.

Whilst DEC respondents largely valued local adaptability, MoH staff emphasised the importance of

systematic interventions with demonstrable outcomes if recognition as an early intervention

program that prevents conduct disorder and receives ongoing funding is to be achieved. Further

clarification of the Got It! model of care, together with a plan for ongoing systematic evaluation of

outcomes, is indicated. Greater standardisation of the universal program across sites could be

supported by the state-wide development of promotional and web-based resources, which would

allow local Got It! teams to focus more on the clinical components of program delivery.

The procedures for effective referral (following assessment and group program phases) and

individual behaviour plans for children with extreme conduct problems have been identified as the

targeted components of the model requiring development. The professional approach that the Got

It! teams have taken to trialling and reporting on different approaches and procedures, along with

other information gathered for the evaluation, has yielded much useful information to assist in

developing the components of the Got It! model for the future.

Whilst the Got It! model of care has been generally understood as a two-term involvement in a

school, the experience has been that there is lead-in time required for school engagement and there

is also time involved with referral of families and follow up with the school and families in the term

following the group program. The school profiles in which Got It! teams were asked to document the

nature, timing and participation in Got It! activities in each school demonstrate that for many

schools involvement spans four terms.


99

Considerations for development of the Got It! model A state-wide position to facilitate the development of state-wide resources for universal interventions and communication about Got It! and to work with LHD staff to develop, clarify and monitor the Got It! model of care, in line with evaluation recommendations. The Got It! model of care could be re-defined as a four-stage model of care comprising: • engagement • screening and assessment • interventions • follow-through and referral ’Engagement‘ is a crucial process upon which the effectiveness of the screening and intervention phases rely. The term ‘follow-through’ has been adopted for the fourth phase, rather than ‘ending’, as the Got It! program is regarded as the beginning of an early intervention process that will continue for a period of time for many of the families. Whilst the ‘engagement’ and ‘follow-through’ phases are less time-intensive than the ‘assessment’ and ‘intervention’ phases, the evaluation has revealed that they are vital stages in an effective Got It! program.

4.3 Findings: organisational features of program implementation

The implementation of the Got It! model of care cannot be reviewed in isolation from the

implementation of the Got It! program at the organisational level. In this section, the organisational

context surrounding the establishment of Got It! teams and the nature of the partnership

arrangements between Health and DEC are addressed.

4.3.1 Health–Education partnership

Bringing together two organisations with different cultures, professional groupings, use of language,

procedures and systems is not an easy task. Recognition of the value of the Got It! model and the

potential to effect change in the lives of children and families has resulted in widespread

commitment to making the partnership between Health and Education work. Expected frustrations

and occasional misunderstandings aside, the partnership seems to have progressed in a healthy way

through the life of the Got It! pilot. Partnership steering committees and action-group meetings at

state, regional and school levels have provided the structures through which procedures are

discussed, decisions are made and plans are set. Significant changes to organisational structure

within DEC at state level during the pilot has, however, had an impact on state-wide decision

making. According to MoH staff, the state decision-making body has not met in two years. This is

being addressed with the re-establishment of the state-level Health–Education decision-making

group, in conjunction with the School Link initiative, which had been operating for a number of years

on strengthening relationships between health services and schools. There have also been other

forums for information exchange and collaboration, such as PBL conferences. At the frontline of

program implementation in schools, the Got It! program enables interaction and collaborative

practice between teachers and mental health clinicians, which otherwise is not common practice.

The quality of the Health–Education partnership is pivotal to the success of Got It!. As one school

staff member puts it:


100

It’s a very valuable program and, once schools and Health have worked out how to work

together, it can be a really useful program.

At the state departmental level, the partnership is valued but the need for clarity on respective roles

and responsibilities is also recognised. As a Ministry of Health staff points out, the concern of health

services is with the provision of expert advice, early intervention and clinical services. LHD staff in

Got It! have a role to play in promoting partnerships with DEC and with NGOs to provide non-clinical

components of the model of care, such as screening and community education. According to the

MoH staff, however, such universal aspects are not core business of LHDs, which should remain

focused on a systematic approach to early intervention to prevent the escalation of disruptive

behaviours into more severe conduct problems in the face of an increasing emphasis on the need for

early intervention and clinical services in health to have evidence-based links to outcomes. Current

policy changes within DEC include the development of integrated and comprehensive local services

through specialist centres and collaborative programs, and local schools having increased authority

to make local decisions (DEC 2013). This suggests there is potential for DEC to take on more of the

implementation responsibilities within the Got It! model of care. A revitalisation of a state-level

Education–Health decision-making body that has the power to clarify and implement roles and

responsibilities for Got It! within the respective state systems will be important for the ongoing

viability of the program. Shaping the Got It! model of care within the context of current policy trends

and initiatives within Health and Education offers considerable opportunity for the future

development of the Got It! model of care. The model hinges on the integration between universal

and targeted components. Partnerships between Health, Education and NGOs could enhance

implementation.

Considerations for development of the Got It! program

The re-establishment of an active state-level DEC – Ministry of Health decision-making body for Got It! would provide the structure for forming agreements on the respective roles and responsibilities of Health, Education and non-government organisations in relation to universal and screening components of the model of care.

The dedicated staffing of the Got It! program is with Got It! teams in LHDs. A true partnership,

however, relies on a staffing commitment by both parties. An important feature of the pilot program

has been the funding allocated to DEC (from NSW Ministry of Health under its Keep Them Safe

strategy) to provide release of teachers in schools to participate in a variety of tasks associated with

the Got It! program. In interviews with DEC and Got It! staff, this funding allocation has been

identified as critical for the implementation of the program. It should be noted, however, that in two

of the pilot sites there was a considerable underspend of this budget allocation. Sometimes schools

indicate that the teacher release time is not required, and sometimes suitable casual teaching staff

are not available. It was noted in one of the Got It! team process journals that some teachers were

resistant to completing the SDQ screening questionnaires because it was additional work for them. It

is not known whether this school was accessing the teacher release pool of funds and if not where

the barrier to accessing this lay.


101


Close attention to budget planning and communication is needed to ensure the appropriate allocation of teacher release funds provided by Ministry of Health and administered through DEC. Funds should be expended to support optimal engagement by schools with the Got It! program.

Restructuring in DEC and NSW Health during 2013 has also impacted on the relationships at regional

level. This has had an effect on communication, information dissemination, planning, school

selection and other aspects of program delivery. Got It! teams reported that processes have not

been as clear or streamlined over the past 12 months. Key personalities driving Got It! at the outset

were critical for building up momentum and the success of the partnership. There is a continuing

need to work on relationships and partnership forums as new people come on board and to

continue to disseminate clear information about Got It! and associated procedures.

At the school level, attention to strategies for engagement is important in the early stage before the

training and screening can commence. Having the principal on board as an advocate for Got It!

amongst the school staff was identified by respondents as the most important factor in establishing

effective working relationships in schools. Following on from this is the importance of having

committed staff who are enthusiastic about the potential for the Got It! program to make a

difference in the lives of children and families. Engagement begins at the start of the program, but

continued efforts are needed throughout the life of involvement by Got It! in a school. Whilst there

is one key staff member in each school identified as the liaison person and Got It! coordinator, there

were many comments in the interviews with DEC staff and in the online survey that indicated a need

for more information for and involvement by teachers across the school. Strategies for achieving

these goals need to be attended to in a collaborative way at school action committees. One school

staff member described the process of the school action group:

It was an interesting process … very different institutions … frustrations on both sides in terms

of organisation … By the end of it we’d worked it out! … Principals have got a lot of power.

They drive everything in schools.

The Got It! model of care does not adequately recognise the amount of time required to build

relationships between Got It! and school staff, to understand respective systems and procedures, to

communicate expectations and to develop plans. There were many examples in the Got It! team

process journals where ’poor communication‘ or ’inaccurate information‘ or ’poor timing‘ were

identified as reasons for activities not being as successful as they could have been. Before the Got It!

program can engage with school families, time is needed to work on engagement between schools

and Got It! teams. Suggestions from Got It! staff on ways to facilitate engagement in the term

leading up to screening included:

visits to schools to provide information on Got It! to school staff

taking along a principal from another school where Got It! has run to talk about the

experience

having a Got It! website available for school staff and families to refer to

providing engaging information on Got It! for distribution to families.


102


The engagement phase with schools, in the term leading up to screening, should entail:

marketing the Got It! program to get school staff ’on board‘, using online and printed resources, together with face to face meetings at schools

generating enthusiasm and ownership for the program amongst school staff

making collaborative decisions on how existing school systems (meetings, procedures, timeframes) could be used for Got It! implementation

providing engaging information on the Got It! program for schools to distribute to parents and carers as a start to building awareness of the program.

When teams have been stretched, with other logistical and client demands in the Got It! program, it

is these relationship-building and engagement strategies that have sometimes been given less

attention. Got It! teams have, however, become more proficient and efficient in providing

information and outlining expectations of the Got it! program in the set-up phase in schools, as the

following comment from a DEC member of a regional steering committee indicates:

Got It! have got their spiels worked out now. They are clear on their timeline and the

requirements in schools. It’s tried and true now and it’s easy for schools to work out and plan.

The online survey revealed very positive perceptions of the quality of Got It! information provided,

with 58 (80%) respondents regarding the provision of information to be above average or excellent.

As can be seen in Figure 11, no one considered the information provision to be below average. This

indicates that Got It! teams have appropriately handled concerns and have provided the information

that school staff were requiring.

Figure 11: Health and Education staff: ratings of quality of Got it! information

0

5

10

15

20

25

30

35


Respondents' ratings of quality of Got It! information (n=72)


103

As a partnership program, it should not be the case that Got It! teams are the sole drivers and

organisers but that a degree of ownership and responsibility for the Got It! program develops in

each school. There was a sense in some of the comments from Got It! staff that in order to establish

manageable procedures for themselves, they needed to allocate ’responsibility‘ for other tasks to

the school. This was talked about in terms of the school’s responsibility in the partnership

arrangement. This acceptance of responsibility, however, is only likely to be successful if school staff

already have a sense of ownership of and commitment to the Got It! program and if the activities

that they are given responsibility for are negotiated by them as part of the partnership relationship.

Whilst joint ownership is recognised at regional level, a sense of ownership was clearly not present

in some schools or amongst some school staff. There are school staff who regarded the Got It!

program as something they were required to do or had imposed on them. The efforts by Got It! staff

to be very clear on their roles and to hand over responsibility to schools can lead to a stand-off and it

is the families that consequently miss out. An example of this was in decisions about who phones

families to invite them to assessment interviews. If this is done by an uninterested school

administrative staff member who resents the imposition on their time, the response from families

can only be expected to be very small. The recommended approach by programs such as Positive

Behaviour for Learning (PBL) and KidsMatter, both of which were consulted as part of the

evaluation, is to initially focus on generating enthusiasm for a program and establishing a sense of

school ownership as a prerequisite for engagement. A PBL coordinator in DEC commented:

The Got It! team tends to set the agenda and puts it to the school. It may be better for the

school to set the agenda. Do it with the school, not to the school. Got It! should be considering

how best to support schools.

School-Link Coordinators in LHDs work on relationships between CAMHS and schools, but did not

feature as a part of Got It! processes in the discussions with teams. There appears to be some

potential to better integrate processes between Got It! and the NSW School-Link initiative.

Some further skill development in engagement strategies amongst Got It! staff, together with the

regional DEC staff who work on Got It! school selection and engagement, is warranted. The goal of

initial engagement should be to generate enthusiasm and ownership, with roles and responsibilities

being subsequently negotiated in the context of a partnership relationship. With dedicated,

experienced teams of Got It! staff in LHDs, however, recognition that Health will carry the bulk of the

work is also reasonable. The following comment from a regional steering committee member in DEC

reflects the partnership balance:

Even though the initiative comes from Health, it runs in schools and schools are appreciative of

having a packaged program come along and be delivered. Schools don’t otherwise have the

potential to offer a program with this level of intensity … Schools take part and benefit from

the outcomes.

Regardless of how skilled Got It! staff are in engagement strategies, a partnership program that

requires new partnership relationships to be forged with every new school will be demanding.

Developing constructive relationships with school staff in order to provide information on Got It! and

set procedures in place has often been challenging for Got It! staff. It was apparent from reading the

process journals that Got It! staff also require skills in working with resistance. There is one example


104

of the teacher allocated to be a co-facilitator for the Got It! group explaining to the Got It! team at

their first meeting that she had ’drawn the short straw‘ and that she isn’t ’into all of this fluffy-arsed

stuff‘. Further in the focus groups, the following comments were made:

Some relationships have been tricky. Like we’ve met some quite fierce resistance from a few

teachers, quite negative.

We do try and have pre- and post-program meetings with the teachers to explain to them

what’s going to happen and then what has happened and then addressing any of those needs

of kids that they do have in their classroom and services that they may be able to access …

Often the schools don’t know what’s available and because you do sit within a CAMHS team

you have those connections with the services that are available. You have that information.

In an interview with a PBL coordinator in DEC the importance of working with existing systems in

schools was emphasised as a strategy for success:

Schools have a lot heaped on them … Strategies for communication and intervention will be

different school by school. Building on the relationship between Education and Health is critical

… Ask which team is best placed to help, rather than set up yet another meeting. Connect with

the existing structures drawing on staff who have had relevant training or are already working

on similar programs. In some schools this may be the learning and support team. In other

schools this will be different.

There were examples in the Got It! team process journals of successfully utilising staff development

days and K–2 stage meetings for training and Got It! meetings rather than setting up extra times in

teachers’ busy schedules. Another excellent example of the Got It! program integrating with existing

DEC structures is where one Got It! team mapped the content of the Fun Friends universal program

against the new Australian Curriculum outcomes. For teachers this means that involvement in the

program not only provides a well-resourced social-emotional program to deliver in class, but it also

serves the purpose of clearly meeting required curriculum. Existing DEC structures can also be drawn

upon when partnerships are not working well. The following example from a Got It! focus group

illustrates this:

We had the parent information evening and they [teachers] kind of all just took off. And it was

just us, so we had to run it alone …. I’m feeling like we’re doing the work but we’re having

trouble getting them to participate and take up their end of it. So we invited the PBL officer

along and that kind of liaison actually really helped. I did sort of say, ‘Well, it’s not really going

to work unless you guys are also really involved‘. After that they actually stepped up a lot,

didn’t they? It did make a big difference.

The online survey of Education and Health staff has provided a useful overview of how the

partnership between the two sectors has been experienced by those involved. The survey comprised

18 questions adapted from the VicHealth Partnership Analysis Tool (VicHealth 2011). The results are

presented below in Table 5. The mean score was calculated for the ratings by the respondent group

and compared to the percentage bands published for the tool. The mean score of 51.63 fell into the

top band for the tool (top 35% in range of responses overall), which according to the tool indicates

‘A partnership based on genuine collaboration has been established. The challenge is to maintain its

impetus and build on current success.’ Whilst this is not a validated tool and it needs to be


105

acknowledged that the tool was also reduced in length and the language was tailored to suit the

context of the Got It! evaluation, this outcome is nonetheless encouraging. It provides an Australian

comparative standard that indicates the work put into developing the partnership for the Got It!

program has been worthwhile. Of all the questions relating to various aspects of the partnership, the

only one that had 50% or less of responses in the ‘agree’ or ‘strongly agree’ categories was ’There is

a history of good relations between DEC and Health‘ (50%). Given this information, the success of

the partnership is even more noteworthy.

Table 5: Health and Education staff views on Health–Education partnership

0 Strongly disagree

1 Disagree

2 Not sure

3 Agree

4 Strongly

agree

n

There is shared understanding & commitment to Got It! by DEC & Health

0% 0

1.43% 1

24.29% 17

47.14% 33

27.14% 19

70

The need for a partnership is understood by both DEC & Health

0% 0

0% 0

21.74% 15

43.48% 30

34.78% 24

69

DEC & Health are willing to share ideas & resources to fulfil Got It! goals

1.43% 1

1.43% 1

24.29% 17

37.14% 26

35.71% 25

70

The benefits of the partnership outweigh any difficulties

1.43% 1

1.43% 1

18.57% 13

41.43% 29

37.14% 26

70

There is a history of good relations between DEC & Health

0% 0

4.29% 3

45.71% 32

42.86% 30

7.14% 5

70

The roles, responsibilities & expectations of each partner is understood by the other

1.43% 1

5.71% 4

24.29% 17

51.43% 36

17.14% 12

70

Administration, communication & decision-making are clear & simple

0% 0

11.43% 8

21.43% 15

47.14% 33

20% 14

70

Both DEC & Health are involved in planning & priority setting for Got It!

0% 0

2.86% 2

28.57% 20

41.43% 29

27.14% 19

70

The contribution of time, personnel & resources by both DEC & Health is sufficient for Got It!

5.88% 4

13.24% 9

25% 17

38.24% 26

17.65% 12

68

Decision-making about Got It! is participatory, responsive & inclusive

0% 0

8.82% 6

19.12% 13

50% 34

22.06% 15

68

Managers in DEC & Health support the Got It! partnership

0% 0

4.41% 3

25% 17

50% 34

20.59% 14

68

DEC & Health staff together have the required skills for delivery of Got It!

0% 0

4.41% 3

14.71% 10

51.47% 35

29.41% 20

68

The partnership between DEC & Health adds value for children & families

1.47% 1

1.47% 1

8.82% 6

38.24% 26

50% 34

68

There are regular opportunities for contact between staff in DEC & Health

1.47% 1

10.29% 7

27.94% 19

32.35% 22

27.94% 19

68

Different priorities, goals & processes between DEC & Health are addressed

1.47% 1

7.35% 5

32.35% 22

42.65% 29

16.18% 11

68

There are ways to share information and resolve difficulties with Got It!

1.47% 1

5.88% 4

16.18% 11

45.59% 31

30.88% 21

68

Alternative views about Got It! can be expressed

0% 0

4.41% 3

16.18% 11

60.29% 41

19.12% 13

68

The partnership between DEC & Health in the Got It! program produces clear outcomes

1.49% 1

5.97% 4

17.91% 12

43.28% 29

31.34% 21

67


106

Summary of findings: Health–Education partnership

The quality of the Health–Education partnership, at state, regional and school levels, is pivotal to the success of the Got It! program. Attention to engagement strategies, communication and decision-making mechanisms is an ongoing requirement for success.

Rushing through the engagement phase can jeopardise good relationship building and communication, which will impact on the implementation of key components of the model of care.

Funding for release of teachers in schools to undertake tasks associated with Got It! has supported implementation and facilitated a partnership approach. The budgeted allocations were underspent in two of the three pilot sites during the evaluation period.

A total of 80% of Health and Education staff consider the quality of information provided on the Got It! program as ‘above average’ or ‘excellent’.

The continual process of engagement with new schools is demanding. Got It! staff require skills in generating enthusiasm and dealing with resistance. This includes demonstrating benefits and showing connections between the universal program and national curriculum.

Using an adaptation of the VicHealth Partnership Analysis Tool, it was found that ‘A partnership based on genuine collaboration has been established. The challenge is to maintain its impetus and build on current success.’

Professional development has included training for teachers to deliver packaged programs in K–2 classrooms, with Fun Friends (Pathways) being adopted as the program of choice in two of the three sites. Training sessions have also been developed by the Got It! teams, either as a series focused on classroom strategies or on specific topics requested by the school.

4.3.2 Selecting schools for Got It!

Schools involved in the Positive Behaviour for Learning (PBL) strategy have been the starting point

for engagement in the Got It! program across the three pilot sites. PBL provides a framework within

which Got It!, and other programs, can be incorporated into school plans and structures. Schools

who have adopted PBL have been regarded at regional steering committees as having the

appropriate orientation and values for Got It! implementation. Information has been distributed to

principals through DEC at regional level and calls for expressions of interest sought in order to

develop a potential list and subsequent program of schools for implementation. Got It! teams and

regional DEC staff have also spoken at principals’ forums and have hosted information sessions for

principals. Schools that are not part of PBL have not been excluded and there are several non-PBL

schools that have undertaken Got It! programs. Responses to questions about school selection

consistently pointed out that the view of the principal is the key deciding factor in whether Got It!

goes into a school. With changing principals and changing circumstances in schools, there is a need

to constantly keep principals informed about the Got It! program and how to get involved.

Principals experience a number of barriers to taking on the Got It! program, which were discussed in

interviews with DEC staff and in the Got It! staff focus group. Some of these barriers include:


107

Timing is not right with plans already confirmed for the year.

Not seen as a priority as there are other programs regarded as more suitable that teachers

can run.

There are many additional programs running in schools that require casuals to be brought in.

This is disruptive for classes as teachers are taken away for training and other activities.

Concern about focusing on bad behaviour rather than being a positively framed program.

Not regarding their school as being as needy as other schools.

Too big a commitment over a reasonably long period.

Now that Got It! is becoming known in the area and there is an existing relationship with some

schools, the Got It! teams have begun to be contacted directly regarding future offerings. As a

partnership program, it is important that school selection procedures, as negotiated between Health

and DEC, be followed for school selection. These procedures should be documented at the regional

level and communicated to schools if enquires are received directly by the Got It! teams.

On some occasions Got It! teams have found that the school they enter is not prepared for the

delivery of the Got It! program at that time, perhaps due to a recent principal change. This creates

difficulties as, given the preparatory work involved, it is generally not possible to quickly find an

alternative school in which to run the program. In such situations, the program has still been

delivered in the school but probably not with the levels of success that would have resulted from

better preparation in the school. A Got It! team member explains:

When we became aware of all of this information [changes in the school] we were too far

down the track to actually pull out. Maybe that’s what we needed to do. But the work up is so

extensive … When things change at the last minute, it’s really hard.

Based on current level of demand, the urban and regional Got It! teams supported a three-year cycle

of schools going through the Got It! program. At current capacity of eight schools per year, this could

provide the program in the same 24 schools every three years. Every child would therefore

experience the program when in K–2 at those schools. There remain, however, many schools that

have not taken up the Got It! program. If interest increases, there may not be the capacity to return

to schools in three years. In current circumstances, a Got It! team member assesses the capacity for

coverage of schools to be reasonable:

There’s always more you could do. You could always spend about 10 years in one school doing

well. But in terms of covering and balancing it, it feels okay to me.

The rural Got It! program was established a little later than the others and none of the original staff

remain in the team. This, together with the daunting distances faced in that region, made staff less

optimistic about the potential to get around all of the schools that have a need for and interest in

Got It!


108

Summary of findings: school selection

School selections for the Got It! program have been made on the basis of DEC regional processes and the interest of school principals.

Based on current levels of interest, a three-year cycle through schools appears to Got It! teams to be achievable. For the rural pilot site, however, the distances and number of small schools in the region limit realistic coverage.

Professional development has included training to for teachers to deliver packaged programs in K–2 classrooms, with Fun Friends (Pathways) being adopted as the program of choice in two of the three sites. Training sessions have also been developed by the Got It! teams, either as a series focused on classroom strategies or on specific topics requested by the school.

4.3.3 Systems for sustaining the impact of Got It!

Got It! teams have found that involvement in and ownership of the Got It! program by the school are

key to sustainable outcomes. Working on communication, relationship building, information

provision and involvement from the outset has been found to contribute to schools developing

ownership and responsibility for the program. Comments made by DEC staff in the interviews

indicate that many schools have implemented and are continuing strategies as a result of the Got It!

program. Some examples of outcomes and impacts that have continued beyond the life of Got It! in

schools include:

coffee and chat mornings to support parents

teachers being trained in and running Exploring Together group in school without the direct

involvement of Got It! team

continuing to run Fun Friends in K–2 classes

inter-school exchange and information forum for teachers delivering Fun Friends, with the

Got It! team providing background support if required

implementation of school policy and procedures that integrate ideas from Got It!

training/programs

adopting, promoting and reinforcing particular techniques, behaviour management

programs and/or slogans school wide

use of resources suggested or provided by the Got It! teams

improved knowledge about services and referral

less tangible shifts in understanding of socio-emotional issues amongst school staff that can

impact on how they approach children and families

Got It! teams continuing to give presentations on Got It! to make sure that schools have

current information on the program.

The aspects of Got It! that a school decides to sustain may be quite limited and focused, but these

are chosen because of the impact they are perceived to have for children in the school. For example,

an assistant principal comments:

There are certainly aspects that we will revisit and incorporate across the board at school. For

example, the Stop-Think-Do.


109

In the online survey, respondents were asked to suggest strategies needed to sustain the impact of

the Got It! program in schools. Suggestions have been synthesised in the following comments.

Follow-up contact with families who did the small group program. This could include

informal support group (e.g. coffee morning) at the school, support by school counsellor and

follow up by a Got It! team member a couple of months down the track to reinforce

strategies and outcomes and to assist with linking into other services if required.

School staff member involved during the program as a group facilitator to continue key role

in coordinating ongoing strategies related to Got It!.

Teacher-release funding for school staff to sustain involvement and attend training.

Got It! teams continue to be available in consultative role.

Opportunities to connect with the Got It! teams and other schools through a Got It! network

of schools involved. Focus may vary between areas and over time, but could be to review

delivery of Fun Friends or other programs in class.

Develop policy and information relating to Got It! / social-emotional learning, for induction

of new staff.

Having a principal or assistant principal committed to sustaining the impact of Got It! was identified

by DEC staff and by Got It! teams as a driving force in ensuring that there is some follow-on impact

from the Got It! program in the school. The implementation of such strategies is also supported by

clear timeframes and systems within the school where implementation is monitored and evaluated.

It was also pointed out by the Got It! teams, however, that regardless of the relationships and

strategies that are put in place, the subsequent interventions in schools will not have the same

intensity as when the Got It! staff are present. Some contact with schools and a return to schools

every few years is needed to keep momentum going. A Got It! team member explains the

frustrations associated with running a time-limited program:

Across the two terms it’s awesome … They know that we’re going to be there week after week,

doing what we do, but once we’re done they don’t have it in their scope to continue that

program … Every school has asked can we run it again … You could spend your whole time in

the one school just redoing the program with different families again and again. They don’t

have the resources or the money or the time to do that in a school, post our intervention. That

can be a bit frustrating for them but also for us.

All of the Got It! teams reported that their attention to sustainability in schools has heightened

through the life of the pilot project. Funds allocated to DEC by the NSW Ministry of Health from the

Got It! budget for teacher release in schools could also be utilised to support sustainability efforts

and activities in schools following the end of the six-month Got It! screening and intervention. This

has commenced in one of the Got It! pilot sites.

Consideration for development of the Got It! program

Sustainability begins with establishing school ownership and responsibility at the outset of involvement with Got It!. Sustainability efforts are supported by Got It! teams being available for advice and assistance. Engagement and follow-through phases in the model of care are seen as key for sustaining the impact of Got It! in schools.


110

Summary of findings: sustainability in schools

The ongoing impact, after the Got It! team has finished in a school, has varied between schools. A variety of strategies relating to staff, children, families and school policy have been adopted as flow-on effects from Got It! in schools. In some schools, however, the ongoing impact appears negligible.

A sense of ownership and responsibility for Got It! in the school, and having a principal or assistant principal who is committed to the program, were identified as key factors contributing to sustainable impacts.

Got It! teams have attended more to supporting schools with sustainability strategies over the time of the pilot.

4.3.4 Organisational and management issues for Got It! teams

The location of Got It! teams in CAMHS was regarded by each of the three teams, CAMHS managers

and MoH staff as appropriate. Within CAMHS, staff have access to the clinical expertise and

supervision required to manage difficult and complex practice situations.

Those more difficult children do come up for clinical review. That is reviewed as a team with a

psychiatrist there and having some of that back up whether that offers more or whether that

just offers the reassurance to the clinicians that they’ve done as much as they possibly can,

exploring what they have done and being able to say, ‘Well, the rest is beyond my capacity’.

(Got It! team member)

Being in CAMHS provides access to professional development opportunities, which is important for

clinician skill development. For example, one process journal noted the need to better understand

the impact of maternal postnatal depression on child development and strategies for attending to

attachment issues later. The CAMHS Perinatal Infant Mental Health Service was enlisted to assist the

Got It! team with this. Location in CAMHS has also allowed for clinicians outside the Got It! team to

be co-opted to co-facilitate groups at times when there have been staff vacancies (e.g. Aboriginal

Health Worker in one team, psychologist in another), which also provides an interesting opportunity

for these other CAMHS staff. Location within CAMHS also facilitates referrals to other specialist

teams.

Some Got It! staff did, however, express concern at the difficulties in accessing LHD Got It! funding

for training and training resources.

Even though the finances are there and supposedly quarantined and supposedly we should be

able to say yes as a team, this is what the program needs and if their funds are there this is

what we will do. This has been a huge frustration for us. Because, yes, the program has come

with a lot of funding, it’s come with ample funding to be able to provide all this, but we just

haven’t been able to access it!

Given the challenges associated with distance, the differences between communities and associated

service systems, and the separate organisational structures, it seems appropriate to locate Got It!


111

teams within the same geographic boundaries as CAMHS teams and not to expect Got It! teams to

work across several CAMHS, as has been the case in the rural pilot site. This may involve smaller

teams and fractional appointments to cover Got It! model requirements.

One of the challenges of locating Got It! teams within CAMHS has been the different theoretical

basis and associated service delivery framework for Got It! in comparison to the clinical services that

dominate CAMHS and LHDs more broadly. Coming from an early intervention and preventative

perspective, the Got It! program contains many elements that sit outside the traditional approach to

clinical services where patients/clients with diagnosed conditions attend clinics for treatment. These

elements include community engagement, community education, health promotion, screening,

referral for preventative reasons, and implementation of strategies to facilitate behavioural and

systemic change. The dominance of a clinical model can lead to these approaches being

misunderstood, undervalued and under-resourced within CAMHS. This tension may explain some of

the difficulties in accessing resources and indicates the need for clear, strong leadership in the Got

It! teams.

The loss or absence of middle management positions in LHDs has impacted on Got It!. Teams

expressed feelings of isolation without a manager to advocate for their unique requirements and for

access to Got It! funding within their LHDs.

At the team level, delays in filling vacant positions and limits to accessing the resources intended for

the Got It! program have caused stress and raised concerns about sustainability. The process

journals completed by the three Got It! teams revealed just how busy they are and how they are

constantly problem solving challenges associated with this new and innovative partnership program.

For the rural program, in particular, the difficulties experienced in attracting and retaining staff in

the program and the constant demands of travel stretched the Got It! staff. All three of the pilot

sites experienced pressures of taking on extra workload to manage the program in the face of vacant

staff positions and having to engage in a ’battle‘ to access resources to pay for teacher training and

other requirements for program delivery. In order to manage such pressures and deliver a viable

program, the Got It! teams require a clear organisational status, transparent accounting, and

decision-making authority. Given the innovative nature of the program, strong leadership for the

program is important. There is a need for coordination at the LHD level and streamlined funding

approvals. Clinical leadership, supervision and team leadership have at times been absent or lacked

clarity for the Got It! teams. A designated leader with decision-making authority could:

oversee clinical, program implementation and team issues

oversee client data entry systems and requirements – as program is very different to

standard clinical services

oversee decision making and establish criteria for selection of families for groups

facilitate regular team meetings

facilitate decision making on key issues

ensure that agendas and records are maintained for accountability and efficiency

ensure effective communication procedures

negotiate and attend partner meetings / steering committee

represent Got It! at relevant forums

identify and respond to weaknesses and strengths in the team


112

initiate and coordinate professional development

monitor and advocate for appropriate use of resources

liaise with higher levels of management.

As discussed above in relation to the Got It! model of care, having a mix of skills and professional

backgrounds in the Got It! teams appears to have worked well, with a particular need for skills in

therapeutic group work, community engagement, school education and adult education. The need

for dedicated administrative assistance, particularly for data entry during the screening phase, was

also apparent from the evaluation. The appropriateness of including a staff psychiatrist within the

team profile was questioned by a Ministry of Health respondent, who regarded it as costly and

unnecessary for the type of service offered. Clearer requirements for the Got It! staffing profile may

be useful for LHDs; however, requirements that are too stringent could be thwarted by local

recruitment difficulties.

Strong state-wide leadership was identified by both DEC and Health staff as important for the future.

Recognising the innovative nature of the Got It! program, the availability of expert advice to Got It!

teams and schools and advocacy for the program within LHDs were identified as requirements

during the start-up phase. More forums providing opportunities for connections and interchange

between those involved with Got It! in the different geographical areas was also requested by the

Got It! teams. The opportunities that have been made available were valued, as indicated by the

following comment by a Got It! team member:

It’s certainly very valuable in terms of sharing our different experiences and expertise and

different ideas about how things work. I think every time we get together we come away going

‘Maybe we should try this and what about that?’ I think it’s really useful for the development

of the service.

Summary of findings: organisational and management issues

Got It! teams are appropriately located within CAMHS. Working across several CAMHS services, together with the distances involved, has presented challenges for the rural pilot site.

As an early intervention and preventative program, Got It! principles differ from the dominant clinical services model within CAMHS.

Each of the LHD pilot sites have faced difficulties recruiting suitably qualified staff to Got It! teams. With staff vacancies, all teams appear to be working to capacity with implementing the Got It! model of care and supporting the data gathering for the evaluation.

Got It! teams require a mix of clinical, group work, educational and community engagement skills.


Got It! is an innovative program within CAMHS. Strong leadership, advocacy, supervision and service development roles are required in LHDs to achieve an optimal level of success. State-level leadership and a state-wide approach to program development and the implementation of recommendations emerging from the evaluation are also indicated, should Got It! receive ongoing funding.


113

4.3.5 Conclusions on organisational features

Process and qualitative data collected for the evaluation indicate that the Health–Education

partnership structures that have been put in place and the location of Got It! teams in CAMHS are

appropriate for the implementation of the Got It! program. The partnership is progressing well in

light of the innovative nature of the program and very limited collaboration between schools and

mental health services in the past, but will require continued effort for success. State-wide program

development to address the issues identified in this evaluation would strengthen the program for

the future.

4.4 Findings: families’ experiences of Got It!

This section focuses on the experiences of families in 12 schools where the Got It! program ran in

the first half of 2013. Firstly, findings from the phone interviews with parents/carers who were not

part of the targeted group program are presented. These relate to experiences with and perceived

impacts from the range of Got It! program components. Secondly, the experiences and views of

parents in the group program are presented, including some insights into the experiences of the

children in the group program.

4.4.1 Parents/carers not in targeted group: experiences of the Got It! program

A phone survey was conducted with 40 parents/carers with children in the 12 schools where the Got

It! program ran in 2013. This was a purposive sample of parents of children with Abnormal or

Borderline scores on the ‘conduct’ sub-scale for the SDQ (parent or teacher) but who were not in the

targeted group program. Table 6 provides a summary of the nature of involvement that these

parents had with the Got It! program. Whilst 30% of these parents had no further exposure to Got It!

following the initial completion of forms (and some said they were left wondering what had

happened to the program), for the other 70% there was some involvement.

Table 6: Involvement in Got It! program by parents not in targeted group (n=40)

Nature of involvement No. %

Received phone call inviting to an interview 15 37.5

Attended assessment interview 13 32.5

Spoke with child about classroom program (e.g. Fun Friends) 9 22.5

Attended presentation / information seminar for parents 8 20.0

Read newsletter items 8 20.0

Spoke with teacher/school staff about Got It! 5 12.5

No involvement after completing initial forms 12 30.0

Parents/carers were asked if they were aware of any impacts for their child from the Got It!

program. Of the 40 parents/carers, 6 (15%) said that they had observed changes as a result of the

program. Whilst the impact on children appears small, with only 6 (15%) of parents reporting a

positive change for their child, it is worth noting that there are some positive impacts for families not

in the targeted groups that otherwise would go unseen or unreported to schools or other services.


114

For the six who thought that Got It! had made an impact for their child, four related this to the Fun

Friends program, as the following comments illustrate:

We talk each week about the Fun Friends activities. The Green thoughts and Red thoughts has

helped a lot.

Talking about feelings at home more. Understanding it.

In the last few weeks he’s been more tolerant. I don’t know if that’s been due to Got It!.

Social skills don’t come naturally to my son, but Fun Friends has given him some ideas on how

to deal with situations.

Two parents/carers related the impact on their child to other components of the Got It! program. For

one this was receiving the phone call about an interview. Whilst the parents could not attend, it

prompted them to focus more on their child and to interact more. The mother said that she had

noticed positive changes as a result. For the other it was in the assessment interview in which some

pointers were offered and subsequently implemented by the parent. She said:

It certainly has had an impact. We’ve noticed changes because of the different ways we handle

situations.

There were, however 34 of the 40 parents/carers (85%) who said that they were not aware of any

impact from Got It! for their child. Three (7%) of these said that their children did enjoy the Fun

Friends program, but that it did not have any noticeable impact.

In addition to the impact on children, positive impacts on parents/carers’ own understanding and

behaviours was reported by 12 parents, representing 30% of those interviewed. Despite a general

scepticism by many service providers about anyone reading the newsletter items, it appears that

these parenting tips do make an impact for some and, given the ease with which they are delivered,

are certainly worth continuing to pay attention to. The parents/carers also gained useful ideas from

the parent seminars, the assessment interviews and the Fun Friends ‘homework’ activities. The

following comments by parents point to some of the unseen impacts:


try not to lose my cool with them. I try to slow down and think.

It [parent seminar on ‘resilient kids’] makes me think about how I’m addressing issues … I lose

my temper too quickly … I coach the kids a bit … Do things together more now.

I love the items in the newsletter … I like doing the parents’ homework [for Fun Friends] … It’s

all learning.

I use the techniques suggested in the newsletter information … maybe some things from the

interview … talking about emotions. I hadn’t really thought about that before.

The other 70% of those surveyed could identify no impacts from the Got It! program for them as

parents/carers. For most, this was really of no consequence for them. They acknowledged that they

were too busy to take too much notice of what was going on at the school. Of some concern,


115

however, were the eight parents/carers (20%) who said they had expectations when they learnt

about the Got It! program as they were seeking some new ideas or guidance. They expressed

disappointment that they did not get this. Three of these parents/carers expressed a strong desire to

attend a parenting group or program. The Got It! teams were contacted by the evaluation team in

order to organise information about other local services to be passed on to these families (one from

each of the three pilot sites). These findings reinforce the importance of attending to the provision of

information on parenting programs and referral services at all points in the Got It! process.

Parents/carers were also asked about any impacts of the Got It! program that they had noticed for

the wider school community. There were 7 (18%) of the parents who said that there was a positive

impact and made comments that related either to knowing someone in the targeted group program

for whom there had been changes or more general statements about valuing any programs that

assist children and families.

Respondents were asked for their comments on the process of assessment / selection for the group

program. One third (13) of the respondents from the sample had attended an assessment interview.

Most said that they did not know what the group selection process involved but they were confident

and trusting that the children that ’needed it the most‘ were being selected. Some parents/carers

said that they felt a bit disappointed at the time that their family was not chosen, but at the same

time felt relieved that their child therefore ’was not as bad‘ as others in the school. On the whole,

respondents were content to accept the outcomes.

Six parents commented that they could not consider attending a weekly group program because of

work commitments. It was decided either before or through the course of the interview that the

group program was therefore not suitable. Some said that they were following up with other

services. One parent said that his child’s teacher had spoken with him about the group, but as his

wife does not speak English and he works, attendance was not feasible.

Five parents said that they had been interested in doing the group program but they missed out. Two

of these were not contacted for an interview and were left wondering why others were approached

and not them. The other three parents spoke about attending an interview and feeling positive about

attending the group, but then being told that they weren’t chosen. These three parents felt left in

the dark about why they weren’t chosen, the criteria for selection and what other services may be

available to assist them. All three were from the same school, which suggests that post-interview

follow-up for this school was lacking.

Most parents who attended an interview and were not selected for the group were quite satisfied

with this outcome and understood that participation in the group needed to be prioritised. Some

were given information on other services (e.g. FRS). Two parents commented that the interview was

useful for finding out more about the program and also for them to decide that the group was not for

them / their child. One said that this decision was made because she did not want her son associated

with the ’naughty kids‘ as he would copy them and this would have a detrimental rather than a

positive impact for him. One parent said that they were very surprised to have been asked to come in

for an interview as there were no problems or difficulties. They were subsequently not selected for

the group, which she said was appropriate.


116

There were a few comments that the screening and interview process was thorough and

professional. An assessment process that combined information from parent and teacher was well

received as a better approach to parents volunteering or requesting to attend a group. Only one

parent said that the interview itself was an unpleasant experience, saying that it was ’full on‘ and

’intrusive‘. This was, however, the minority view with most people expressing confidence in the

process. One parent expressed concern that the screening questionnaires were not accurate as

parents are unlikely to be truthful and could under-emphasise any behaviour difficulties. Again, this

was a minority view.

When asked for any suggestions for improvement to the Got It! program, the most common

comment was that families were not aware of what the Got It! program entailed. Parents said that

more information on the program was needed. The experience for many was completing the

screening forms and then not hearing anything more about it. For some there was a degree of

expectation and then subsequent disappointment when there were no developments or information

on where to seek assistance. For others they had simply thought no more about it. There were

suggestions for making the Got It! program more visible in the school, such as more outreach by

teachers, updates in the newsletter and information at assemblies. In particular, more information

was requested for those who missed out on the group, including clear information on criteria for

group selection, why they were not selected and what alternatives they might seek out. A number of

parents commented that they had not been aware that Fun Friends was a part of the Got It!

program. This indicates a need to explain that all of the components of the Got It! program are linked

together, so that it is not seen as the small group alone. A number of parents pointed out the

difficulties for families where parent/s work. It was suggested that groups could be run at different

times. Another suggestion was to provide online parenting resources for working parents. Finally,

there was a suggestion that parent seminars could be more practical, focusing on tips and strategies

for parents, rather than explaining what the screening and small group program is about. Most

parents don’t take part in that program.

Summary of findings: families not in targeted group

Got It! can have an impact on families not included in the targeted group program, but this impact is often not visible outside the family. From a sample of 40 parent/carers of children with elevated conduct scores who were not selected for the targeted group:

70% said they were exposed to other components of Got It! after initial screening

15% said that Got It! had a positive impact for their child

30% said that Got It! had a positive impact for them as parents/carers

20% said that they were hoping to get some new ideas or guidance from Got It! and were subsequently disappointed that they did not

33% (13) had attended an assessment interview, but were not selected for a group.

5 respondents (13%) had wanted to attend the group program (including 2 who were not invited to an assessment interview) and did not understand why they were not selected.

Most who had attended an assessment interview (10 of 13 assessed) were satisfied with the outcome of not being selected for the group and trusted that those in greatest need were selected.

General comments were that there was inadequate information on all the components of the Got It! program, that access was difficult for working parents and that providing practical parenting tips and guidance through a range of mediums (online, seminars, written) would enhance the impact for families.


117


Attention to all components of the Got It! model and communication on these, using a variety of media, to the whole school community will broaden the impact of Got It!.

4.4.2 Parents’ experiences of the targeted group program

Qualitative data on targeted group participants’ experiences was gathered in interviews with 13

parents from 12 targeted groups held in the first half of 2013. The sample comprised 11 mothers and

two fathers, selected on the basis of consent and availability from the 63 families who completed the

group program. At least one parent was included from each school group. (Note: participation data

on group attendance and withdrawals are presented above in the process findings on the targeted

group component of the Got It! model of care. The purpose of this data is not to quantify outcomes

but rather to illustrate the impact of the Got It! program with personal experiences and insights.)

Follow up phone calls were also made parents six to eight months after the conclusion of the group

program. Findings from these interviews are also reported below. These personal experiences of

impacts and outcomes add insights to complement the findings from standardised outcome

measures presented in the following chapter. Every effort has been made to present the full range of

views and experiences voiced by the parent respondents in the qualitative data and quotations

provided here.

This qualitative data provides valuable insights into the screening, assessment and group experiences

for these parents. Of particular interest were their comments on the impact that the group program

had for themselves and their child. All of the parents were able to relate some positive changes that

have come from the group program. For some these were quite small, but for others the changes

were significant and the program was regarded as a turning point for them. For some of the group

participants, the Got It! group followed on from past parenting programs and experiences with

services, but for many this was the first experience of any family intervention or parenting program.

The data presented here offers insights into personal experiences and the impact that a group

program can have for individual families. The commentary has therefore been kept to a minimum in

order to allow the words of the parents speak for themselves.

Parents were asked about the impact of the program for them. They spoke about impacts in terms of

techniques they had learnt, insights they had gained, differences they had observed in their children,

spending time and building a closer relationship with their child, strengthening connections with the

school, acknowledgement that their child does have difficulties, and a sense of satisfaction that they

were on the path to addressing these. Following are some comments from parents that illustrate

how the Got It! program has changed their approach to parenting:

I’ve taken a step back too. It’s not like, ‘X: do this’, you know, ‘I want you to do this and do it

now‘. It’s like stop and take the time to actually ask her … I take a little bit more time with her

and I think she appreciates it. It’s not all the time and like she hasn’t turned into a perfect child

but there is progress there … It’s routine, you know, a change to your lifestyle.


118

Not to sort of yell and scream all at once. Sort of give him time out like three warnings then

time out … I mean we’re still working on it, you know, trying to get things sort of into routine,

yeah, with it all. So we’re still learning.

The first part where I noticed a change was using the Feelings Detective and verbalising what

you’re feeling and what someone else is feeling, what I’m feeling. That was magic … so that

constant ‘What am I feeling? How is X feeling? Stop, let’s think about this. What am I feeling?’

it just really worked. He got pretty cranky at first to be stopped mid-anger like but then he kind

of got into it.

The program’s got us thinking differently about the way we communicate with our children …

You forget sometimes that it’s okay to play, you know? We’re in the business of raising children

and they’ve got to eat, they’ve got to be clothed, they’ve got to do homework, that sort of

thing. So, just getting communication working in a more dynamic way with the child. You know,

‘How are you feeling? What’s going on?’ That sort of thing. So that’s been big … and slowing

down and trying not to, I guess, short circuit the process of interacting with the children.

I guess we’ve got some tools out of this process to work with, you know. The ‘Stop, Think, Do’.

These sorts of tools which are good for helping verbalise ways to communicate about

important things with the little ones.

We’ve got a positive outlook now, rather than, you know, one that feels a bit desperate.

I’ve learnt ways to talk to my son. There used to be no conversation. It was very sad really. It

has given me pointers … We’d been living in silence and we actually talk to each other now.

I probably do more of that positive reinforcement with her … I’ve learnt about assertiveness …

I’m getting better at clearly asking for what I need and want so that is good.

I’ve just learnt to be more consistent with him and not give in so easily. Because it’s what I used

to do, was give in too easily and just cave in to him a lot when he used to cry … But they’ve

taught me now like you’ve got to stick with it and not cave in. But then if you come to the point

where you’re feeling like caving in, walk away for five minutes, 10 minutes and then go back

and try and do it again.

When he’s naughty like he gets the first warning, second warning and then time out and then if

he keeps being naughty then he gets sent to his room. If he comes out from his room and he’s

being naughty again then he gets something taken off him or he gets like not being able to play

with something … I suppose I’m more grounded with the fact that he knows what’s going to

happen if he’s naughty instead of like beforehand I didn’t really have a set thing … It also helps

me because I don’t have to think of what am I going to do if he does this sort of thing ‘cause

now I know what I’m doing. So it’s a lot less stressful and I think I have more patience.

I used to speak to him too much … now I’ve just got to keep it short and to the point so now he

gets it more.

We used to give him a biscuit or something … whereas now with our rewards like he gets stars

on his chart. We have a star chart and once he gets a certain amount of stars he gets …


119

something with one of us instead like I’ll read a book to him … or he’ll go with dad somewhere

like downtown.

I didn’t know anything about emotions … So that was a big deal for me that I got out of the

program … So what I learned is like how for me to behave when they are acting like that. Me be

a bit more patient in listening, understanding the emotion rather than me be expecting them to

be patient all the time. Because they’re just kids … I can patiently explain them the rules …

what supposed to do, what not supposed to do and what would be the reward after that if you

behave like that. So, it’s more talking as a family … I learned how to have a combination of

being big, strong, wise and kind … Just because you’re a father doesn’t mean you’re going to

have that authority … Yeah, a lot more towards communication.

To implement this is a very big part of life. So we have to implement on everyday basis, because

we live with them every day. We feed them every day. But just feeding food is not going to

help. Feeding love and compassion is very important …I want to do all the right for my kids.

I think just the acknowledgement of his difficulties and trying to take the first steps in dealing

with them.

Implementing the idea of him having a choice about how he behaves. So trying to get him to

realise well if he chooses to react this way then these will be the consequences. If you choose to

do this, then this is what can happen … Some days are good, some days are not so good but it’s

positive steps to be taking.

Parents’ need for patience to their children … Need to ask children what the problem. Ask.

Always ask first and then listen … I’m changing. The change was needed. I always ask, ‘What

you’re feeling? Why are you like this?’ … Yeah, he’s telling me.

I just give him a hug and a kiss and I say, ‘You’re a good boy’ or ‘Well done’ and say, ‘You can

play your game’.





how to change myself. How to behave … We need to stop and just think ‘Why are we fighting?’

Where before I’d just go, ‘Oh look, just shut up’. Like, you know, ‘Stop it, Stop!’… It’s just

changed my way of thinking.

Such comments about positive learning and changes made by parents dominated the interviews.

From a couple of the interviews it appeared that the group program had not changed some practices,

such using a hit as a behaviour consequence.

The parents also spoke about the changes they had observed in their children. Some parents had

noticed significant changes, whereas others saw some small shifts but were generally optimistic that

this would be built upon in the future. The following comments by parents provide some insights into

the perceived changes for the children in the targeted program:


120

A few key words they’ve used in the program have helped her think about her actions and

maybe adjust her actions … She would normally fight back at him … But now she has sort of

said, ‘Stop, X’ and like ‘What’s your problem?’ Like she uses it on him and on herself like to calm

down, calm the situation down. So, and she has used a lot of restraint in moving away from the

situation.

She’s had no other role model other than myself. So I think for both of us to calm down and chill

out for a while and see what’s bothering each other … and she’s like, ‘Now Mum, let’s just sit

down and take a minute’.

It depends on if he’s tired. If he’s tired you just can’t talk to him and you can’t do nothing with

him unless, only cuddle him. That’s all you can do if he’s tired, but at other times it works … It’s

pretty much the same.

[H]as been give strategies too and responsibilities for his emotions and that has been a huge

impact … So that’s been the biggest thing, him having like a toolbox … to bring out when he

needs it. Which he really needs all the time … He’s learnt to use these techniques to manage his

anger and from there, everything else is kind of falling into place … When he’s learnt to do the

‘Stop, Think, Do’ it’s been amazing. I cannot explain how changed, how much of a change in

our family has been. The harmony, the respect he has for our family … But we’re not having to

do this every 20 minutes now, because he’s actually either walking away or he’s talking. He’s

verbalising really well … I never realised you had to break down the steps for him so much … He

feels kind of empowered to control himself.

The team have identified that my son has a particular need for sensory input. … They’ve got

him wearing a bit of weighting on his shoulders and given him a sort of knobbly cushion to sit

on … It’s hard to quantify … one of the observations that was made to me today was that last

term … ten incidents in the playground where he was taken off line, if you like, and he has only

had three this term.

He’s built some friendships through the group as well, because they’ve got something in

common … He’s happier, more relaxed. He’s, you know, wanting to be engaged in whatever’s

going on, and his mum and I are feeling much more positive about that as well.

They’ve learnt how to describe how they feel in different situations and how they feel physically

in an emotional situation … She’ll go home and say, ‘I like that you do this for me and some

things that I don’t like … is that you roll your eyes when I talk to you’ … very diplomatically … I

think the things that she’s learnt, she hasn’t noticed that she’s learnt them and I think that’s

what’s going to work the most ‘cause they’re just everyday.

You get the first warning, second warning and then you get a ‘think about me’ chair. And he

doesn’t like the ‘think about me’ chair, so he’s trying to stop the warnings before he gets to

that part.

Now he’ll stop and think about what he’s doing and he’ll ask them nicely if he can play too or if

he can have a go … Since it’s similar at school now as it is at home, it’s easier for him to

understand … I think it’s helped him with the ‘Stop and Think’ thing, the traffic lights … Since

he’s been doing the stop and think thing he’s been sort of getting a lot better.


121

She has learned emotions and we have practised with her at home, like how to classify

emotions, how to deal with emotion. So before … she will get angry and she will get frustrated.

But now I can ask her ‘What are you feeling?’… ‘You want to talk to me about it?’ And then

straight away, all that you do is break the ice. And sometimes it’s little things in communication

that just makes it so much easier than having big lectures … then she can say, ‘I feel scared’…

She’s more open to me about it because she knows we were in the program together.

When I’m watching him in the group, like from the start he was moving around and making

noises and not sitting still and not wanting to get up and participate … by the end of it he was

wanting to … he was enthusiastic. So I saw those sorts of positive things … He learnt about his

emotions more. But getting him to speak about it and the language for it, I think is an ongoing

thing.

He can tell his feelings, but just this. He can’t control his temper.

Another change, maybe before he does homework he used to moan. Now he does it quickly …

He has more interest in learning words.

He was going cranky last night and I said, ‘We need to do the dragon breathing’… he changed,

like he just said, ‘Oh, okay’, because then he realised that he was being silly. So, he didn’t do it,

but I said, ‘If you’re going to keep going, you’re going to have to do the dragon breathing’. And

he’s just changed, because it’s clicked with him, well, yeah okay well I don’t need to do that or

act like that.

Most of the parents commented on the positive experience of spending time with their child at

school. This was identified as a particular highlight for their children. For some of the parents,

spending time at the school has also strengthened their involvement with the school in other ways.

It’s the one-on-one thing. He gets to spend one on one time with me, which he doesn’t usually

get very often.

She loved me being at the school.

I loved coming up to school and I love that breakfast club and I love that we were able to spend

so much time together.

He’s so excited. Fridays are a highlight. They’re an absolute highlight. It’s partly, ‘Yay, dad’s

coming to school’. But it’s also, I think he really enjoys the group because it’s so interactive.

It’s been nice to spend the time with X and doing things with her and that’s what she seems to

remember. She just thinks that she gets to play with mummy at school and that’s all really. But

it has been nice to see that, because she’s little.

Yeah, he loves it and I help out now. I help out here every second Monday as well in the

kindergarten room.

As an early intervention program, the parents’ reasons for their child being in the group and how

they had addressed this concern in the past are of interest. It was apparent that for most of these

families, the Got It! program was the impetus to confront a niggling or significant concern about their


122

child’s behaviour. For a couple of parents, however, it was a surprise to them that there were

concerns about their child. All of the parents identified their child’s anger outbursts and/or their

difficulties making positive friendships with other children as the reasons why they were in the

group. Some said that they had previously considered organising an assessment by a psychologist or

health professional, but they had not acted on this or the financial implications had been a deterrent.

One parent consulted a psychologist with their child prior to the Got It! group and continued this

whilst attending the group. A few parents had previously attended parenting programs. The majority

of parents interviewed said that without the Got It! program being offered they would not have

sought assistance in the short term.

The following comments give a sense of how Got It! impacted on the parents’ understandings of

their child’s behaviour and the need for some action to be taken.

I’m working full time, I’m never at the school. So I really had no concept of what was happening

at the school other than what I was being told at the parent-teacher interviews, and by the

time I got there I was shocked ‘cause I had no idea that anything was wrong or anything that

was happening. And I found that the communication with the school was a problem ‘cause they

wouldn’t tell me what had been happening until it got to a point where it was, where she was

in, stuck in the planning room … I didn’t realise she had a problem … which is what she was in

the program for, making positive relationships … It’s easier when she’s little rather than when

she’s older and complicated ... I was surprised that I got picked ‘cause I just didn’t really see the

problem as a big problem.

I was shocked. Yeah, because I honestly thought there would have been kids worse off than X

that needed it.

It’s hard to pinpoint one thing, but the main reason I went to Got It! was for him to manage his

anger … he was hitting his siblings twice daily in a really hard fashion … like violent … He’s such

an intense child. I feel like I’m running up beside him constantly trying to get in front. That’s

how I felt my life was before Got It! ... I felt like I was constantly on guard.

Part of the reason that he was a suitable candidate for Got It! was because of his poor social

skills. It’s not that he’s a bad kid or a mean kid. He just doesn’t quite understand the rules of

engagement … But he’s also got a need. You know. Sitting still and listening and joining in is

difficult for him … He had a strong perception of himself as a naughty child. To the point where

he’d go and hide so that he couldn’t be told that he was doing anything naughty … As a family,

we were all desperately unhappy about the fact that we had a boy who was so unhappy to go

to school and we really worried for his future.

When I did the survey, I thought, ‘Oh yeah, this is really great, I can finally talk to people about

how he is’. And when I got the phone call saying he got into the program, I was a lot happier

too because I thought, ‘Yes, I can get some help for who he, like who he is’. Like a lot of people

have told me, ‘You’ve got to take him to a nutritionist and everything ‘cause we think he’s got

ADD’. I don’t want to put him on medication. I don’t want him to have to take pills for the rest

of his life. I want help so I can work with him a bit more.


123

Like he gets frustrated if someone won’t share with him … ‘cause he’d like lash out and hit

them just ‘cause he was so frustrated. Like he wasn’t actually being nasty. It was just ‘cause he

was so frustrated that this kind of happened.

Before I started the program I was supposed to take, I was going to take X to a psychologist to

just get his behaviour assessed and stuff ‘cause I was just worried about him with his social

interaction and sharing and that sort of thing … I’ve still got the letter in my bag … We don’t

really have that much money.

From my experience, I’ve never done something like that and I felt that there was a need. There

was a big need for me to know … Because my daughter was really, to be honest, she gets

frustrated. She gets angry. She doesn’t know what’s the next step … We had an opportunity. A

big window to change.

I was on the verge of consulting psychiatrists and so forth and going down that track and then I

heard about this program. So I was relieved to get in … You’re just wondering, ‘Have I just got a

naughty child?’, because his problems are more at home than at school. So you just don’t know

whether it’s you just being ultra-sensitive or not seeing how they really are and just having a

wider scope of reference in being able to view your child.

The parents were asked about their experiences with the screening, assessment interview and

selection process for the Got It! program. Some immediately had seen the Got It! program as an

opportunity to address their concerns. Two respondents spoke about actively lobbying to get into the

program. For others, it was not until they went to the assessment interview that they understood

what the program was about. The screening forms appear to have been daunting for some. The

relationship between the screening forms and the group program was not always clear to the

parents.

The whole paperwork is very overwhelming … if you can … streamline it … I think people are put

off … if it’s behaviour issues … they go, ‘Oh no, my daughter or my son’s not like that’.

I pushed to be selected. In fact, I really went proactive about it … I sussed it out. I even went

online … I filled out his form and then I actually looked for it on the internet and did his score

and I kind of went, ‘Hopefully he’s in’ … I did that and then I kind of just really showed my

interest to the teacher and said, ‘I’d really like X to go for this. I think he would really benefit’.

The interview was very in-depth. It was probably more in-depth than ever anyone’s ever asked

anything like that … They took it from day one, you know, newborn. It was quite detailed … I

felt like they had a good handle of where we’d been and where we came from and so I was

really pleased when we got chosen.

I thought the first process and everything was great and I think the … newsletters and getting

people’s heads around it. Then you had the behavioural surveys, which are time consuming but

at the same time very enlightening. I think it’s worth every parent just to be forced to do,

encouraged to do that … The interview was very in-depth, and while it was confronting, it was,

you did feel like you were heard.


124

[Got It! staff] is very good at putting you at ease, you know, without sort of building a bunch of

expectations which may or may not be realistic. So that was very good … The engagement

process was really good. The information up front to say, ‘Look we’re going to try the Got It!

program. If you’re interested, register your interest’ … I thought all of that was handled

extremely well. I guess I feel lucky that we were part of that selection as well.

It seems like a very time-consuming approach, how to pick them with all those individual

activities of like hundreds of people … and then pick down the five … It’s pretty crazy but I don’t

know how you’d streamline that. The way you did it with the matching up of the results of the

teachers and the results the parents had done and seeing the differences probably is a good

sign.

You know the survey thing that they sent home, I didn’t fill that out. My other half filled it out

because I was worried that if we filled it out … they’d think we were bad parents or something

and like call DoCS or something … So I was a bit worried that it was going to happen to me … I

just don’t like calling attention to myself … plus I have anxiety so that doesn’t help … I get

irrationally worried about things.

With the survey, we didn’t get any information about it before we got it. Like we got it and then

you just pretty much filled it in … Maybe if you had someone come in and talk to parents or

maybe get the teachers to talk to the parents about the survey so that they knew what it was

about and there’s nothing to worry about … I’m pretty sure that some people probably just

filled it out … to make it sound like everything was fine … They need some sort of information

about it so that people are more willing to be involved … so they know that it’s just to help.

We got the first brochure … just 15, 20 parents came to the meeting … We agreed that day to

come and we wanted to implement it … We have an opportunity. A big window to change … I

actually persisted. We kind of begged if we can get into the group, because we were really

scared about X … We perceived it as something special and it’s only a few people in there … so

we don’t want to miss out … We had that opportunity. I felt really blessed, to be honest.

I think it was a bit confusing for people as to what was wanted. Because they wanted all these

people to fill in forms, but they were only wanting six in total to participate and they only

wanted ones with these kind of behaviours. I think a lot of people that went to that initial

session were a bit confused as to what was being offered … I don’t mind filling out forms as

long as they sort of, you could see that the results were being addressed in the course, which I

didn’t necessarily do.

Well I don’t remember actually filling out the form. I don’t know whether I got it but we got

some phone calls and we did surveys over that … and then I got into the program … I thought,

well, I really don’t know what it’s all about … I knew that it was a parenting, it was to help, but

I didn’t know what angle you were going to take. I had no idea what was going to be involved

but I came every week and I enjoyed it. I looked forward to it.

Some of my good friends just said, ‘Oh, how’d you get into that?’ and I said, ‘Well, we did that

form … Oh well, I had an interview and X was picked‘. I said, ‘So he’s one of the lucky ones’.


125

Once selected, the parents came with different expectations as to what the program would achieve:

I never approached this program thinking it was going to be a cure-all or anything like that.

I thought it’d be quite helpful. Well, I was certainly hoping it to be, because you don’t go into

something thinking that it’s going to be a waste of time … I was hoping that it would help X and

I … discover each other in a different way.

So I kind of went in from that point of view that I didn’t know what I was going to get out of it

but we’ll give it a go and we’ll see what happens … I guess it’s hard to think that your child

struggles.

The parents were encouraged to talk about what the group experience was like for them, over the

nine to 10 weeks of the targeted group program. On the whole the parents reported a positive,

engaging experience in which they valued the opportunities to share and interact with the Got It!

staff and the other parents. The groups appear to have reduced feelings of isolation and

strengthened the social network for some of the participants.

It was very therapeutical to talk to the other parents … It’s just the same old thing, isn’t it? You

think you’re the only one that’s battling … The other mums are really lovely … we haven’t gone

out for coffee of anything like that as yet, but certainly we can identify with each other.

Just learning what other families and all that, like what they’re going through, and there’s one

lady in particular which is going through exactly the same as what I am. So that’s really good to

know.

We have that kind of more in-depth knowledge of each other now and I think … we might be

keen to keep catching up too because it’s a good way for our children to be around each other

as well in that non-judgemental way and I’ve gotten to know the parents.

It’s been alright. I guess it’s been good to know that other people are going through similar

things to you … and it was good to meet with other parents. They were just as open … they

were honest and they were trying.

I was such a young mother and they’re a lot older than me, except for one of them. I’ve learnt a

lot from them. Like, they’ve given me ideas to work with him … I’ve only just moved here … last

year so I don’t really know anyone. So, being in the group with the other mothers, it’s helped a

lot.

Not too bad, like it’s been pretty good. I suppose we don’t really, the family members, we don’t

really speak that much with each other and that’s because most of us are shy so, yeah, we

don’t really speak to each other that much.

We learn about each other’s parenting. Like, common problems in our households, which gives

us another boost. You’re not the only one.

All the mothers, the fathers, the nans, were all from different levels of employment … and all

had the same problems … So it was good, all the mothers got on good … There was some, two


126

ladies there with a very high-up education and they were just as fun and helpful to what we

were.

A number of parents commented on the constructive group process that was created by the

facilitators, which covered the key content areas and balanced this with an exploration of personal

circumstances. Having the group structured to have parents and children together for part of the

program and also to have some time separately was experienced as useful. For some, however, the

need to attend to every group member’s concerns became a bit tedious. A degree of patience,

acceptance and open-mindedness was needed as a group participant. A couple of parents expressed

that they felt some uncertainty about who was to take control of their child’s behaviour: the parent

or the group facilitator.

It’s a gentle way of telling the kids how to react to situations and that it’s not unusual that they

feel rejected or upset or angry.

You have to be open-minded and accept change and try to change, rather than just think,

‘Well, I’m doing it my way and it’s not working but I don’t know how to do it any other way’.

It was really excruciating at times when we’d do the homework part and it would take 50

minutes but I understand it needs to be done … It showed me how they stuck with it. The staff

worked with each child. They didn’t force them in a punitive way but they really made it clear

that these are the expectations and ‘What are your choices and what are you going to do?’ and

that was brilliant. It was brilliant to see because they didn’t give up and they didn’t let them off

the hook.

Having that more interactive group style class approach is good, and of course it’s great for

them to have their parents involved.

Getting everyone comfortable and engaged and introduced to each other and making sure that

everybody is focusing on building a bit of a rapport, which was very good.

After an hour, to be very honest, doing activities together, and the kids do need their own time.

It was also slightly difficult in the sense that you were supposed to be disciplining your child,

but then you had other people there and were they disciplining them to the same degree? …

Initially the facilitators were giving out a lot of stickers to encourage them and some of the

parents had problems with that because it was like, ‘You’re jumping on one good thing, but the

rest of the time they’d been painful’. So that it was … ‘Am I in control of my child? Or who is

running this?’

They laid the rules down. Not in a formal way, like just saying, ‘Okay guys, this is it‘. So, it was

relaxed but okay … So it was a calming process, but also in a way that well, we’re not going to

take any rubbish.

We got a lot of say. They would ask us questions. We had homework. We’d talk about our

homework and that would be from activities … from time out, to spending time with kids, to

reward systems … All of us had a go and all of us helped each other. We got to vent about how


127

we grew up as well. So, ‘cause that they’re trying to work out how we react as a parent from

how we was brought up and things like that. Yeah, so it was good.

All of the parents said that their child enjoyed the Got It! group. Most of the parents commented on

how much their child looked forward to the group each week and several said that there were no

indications of any stigma attached to going to the group. One parent, however, expressed concern

that her child was missing other class activities during the Got It! group and that he got upset about

this.

She did like to share like her experiences at home and that with the other kids.

He absolutely loves it! … It’s hard getting him out of bed of a morning. ‘Come on. We’ve got to

go and see [Got It! staff]’.’Oh yes!’ And he’ll jump out of bed.

He looked forward to it so much … We didn’t actually miss it but he was worried he was going

to be sick … That was foremost on his mind that he really wanted to go to Got It!.

I don’t think that they felt any stigma about going to the program or being pulled out of class

or anything. Yeah, ‘cause it was an enjoyable, fun thing to do … I honestly don’t think anyone

noticed, mainly the kids didn’t notice that they were there for behaviour management purposes

at all.

He’s going to be quite upset about it ending though. He counts the days down to Wednesday …

He likes the activities and things they do too and also I think it’s helped him like with the ‘Stop,

Think’ thing. The traffic lights.

My son immensely enjoyed coming. He really did like it and he sort of blossomed a bit during it

as well so I’m grateful for that … They did art activities and games and things like that so I think

he enjoyed that part.

The level of staffing and having the opportunity to draw on the knowledge of different professionals

in the Got It! team was noted by some of the parents:

There was an OT and a psychologist … there are different ways of seeing it, an issue, and

dealing with an issue. So the more opinions the better.

There are four at any one time. There were three or four people with eyes on the children …

That’s a pretty impressive amount of resourcing … I was expecting one, maybe two, people and

probably more than eight children and parents.

One parent questioned the role of the teacher in the group:

There’s quite a bit of input from the teacher … She hasn’t been trained in this program, so I

don’t understand why she has so much to say. She would speak over the children … and it’s

nearly like, ‘Oh, I’m the teacher. You will listen to me’ … maybe a teacher needs to do a course

… just to understand where the Got It! group is going and what they’re trying to achieve. But it,

yeah, that bothered me.


128

The group size and composition generally regarded as appropriate. Groups ranged in size from three

to eight families. Whilst some of the parents in the smallest groups commented that it would have

been better for the group process to have more families involved, having a small group size was

recognised as important. As would be expected, there were some similarities and some differences

between the children and their behaviour patterns. No one expressed that the behaviour of the

children had a negative impact on group process or on what was learnt in the group.

It was a nice small group, eight children.

Most of the boys had similar behaviour problems and X’s wasn’t really like that at all … but I

still managed to pick up heaps of stuff. Her behaviour management … if you go back to basics,

it’s very similar … the others that had similar kids with similar problems, they probably did get

more of that bounce off each other, ‘This is working, this isn’t working’.

If we’d had more kids in there it would have been too much, like they would have played up

more. But because there was only the four in there, it was really good. They all pitched in and

helped each other … I think if there was more kids in there, it wouldn’t work as well.

They’ve all come together really well. Like I thought with two older kids there and the two

younger kids they wouldn’t mix too well but they’ve mixed in really well together and get along

really well.

Other parents no talk to each other. Just talk to the staff. Ask questions and answer some

questions … Just me and another lady always come and another just come a few times … I think

more is better.

A couple of parents suggested that the number of group sessions could continue be extended for

longer than 10 weeks. Several parents commented that the ‘real work’ does not begin until midway

through the term. On the whole, however, the feedback suggests that there was sufficient time to

learn, practice, reflect upon and reinforce the different techniques that were being learnt in the

group.

By week four or five is when you’re starting to get the real work coming out in the group and

then week eight is the last real week of that, so you really only get three sessions … The first

four or five weeks, everybody was on their best behaviour … So it took a little while particularly

for the parents to begin to be comfortable to, in the group sense, reveal their real parent.

The parents were asked about how they would continue or build on the changes that had begun to

occur through the Got It! group and whether any follow-up services were planned. Most expressed

that they had learnt enough to continue on their own without further input from professionals. They

had the resources handed out during the group program to refer to along with new techniques that

they had learnt and practised. Some said that they were expecting contact details for local services to

be provided by the Got It! team that they could follow up in the future if need be. Some other

parents felt that the Got It! program had helped them to identify the particular needs of their child

and as a consequence they would be seeking out referrals to other specialised services. The school

counsellor was also identified as a possible support in the future if needed. Having built up a

relationship with the Got It! team members, several parents said that these were the people they

would contact in the future if they needed assistance. Whether this was appropriate or not in light of


129

the nature of the Got It! program was not explored with the interview participants. At the time when

the interviews were conducted (within a few days of the final group session), none of the parents

had a firm arrangement for any follow-up referral or support service in place.

I’m intending to sit down and go through all the brochures and handouts … Well, I think it’s

quite beneficial so I will look for other services because I think at some stage there … I think it’s

probably good for X to vent.

I’ll keep going with what they’ve taught me.

I intend to keep our psychologist [in place before involvement with Got It!].

So for us particularly, the team are going to send me some information about how we might be

referred to specialists around this particular sensory issue. So they feel that X would benefit

from further professional engagement.

They keep saying that you should contact your school counsellor and work from there, but I

think I’d just call the [Got It!] team.

We can always ring [Got It! team member] and just say, ‘Look I’m having trouble with this, do

you have something like, any recommendations of what I can do?’

I’d say I’ll probably go okay … I think if I keep reminding myself about it … I think we have

numbers and stuff in our book, like heaps of different places … We have the teacher, a couple

of teachers that come in with us.

Now it is up to us. We have a great knowledge … I know there’s a lot more resources. A lot

more awareness and knowledge out there. If we fall back into the headspace we were before

and can’t manage again and things are getting out of hand, we can always look back to them

… They did tell us about a couple of books which I’m already looking into getting … because

knowledge is never enough.

They haven’t given it to me yet but they’re going to give me names about who I can go and see

with X for further professional help.

I: So, who would you contact if you needed some help in the future? P: I no idea.

I see myself needing to get him assessed and then whatever they sort of say is his issues, to

then be able to work on those.

The need for the principles and approaches being taught in the Got It! program to also be reinforced

and continued by the classroom teacher was identified by a few of the parents interviewed. Some

made the comment that this was happening, but others pointed out that there was a gap between

the Got It! program and the behaviours of teachers in the classroom.

The Got It! process is also working with the teachers … teacher is aware of this kind of sensory

need … I can’t speak for the other families as all of the particular goals and requirements will

be different.


130

I think to do this you need constant help. Children need parents and teacher to do this so

there’s some consistence … The teacher, I don’t know how he work … I think this program need

to connect to teacher. Connect teacher and communicate with teacher … This program is just a

short time … the teacher is important.

You can’t say to a child, ‘You’re meant to act like this’ but a teacher can act in a completely

different way … Well, this particular teacher was trying to shame X and I thought it was totally

inappropriate.

Most of the parents could not identify any areas for improvement in the group program as they

were content with how the program had run. Four parents did, however, have particular suggestions

for improvements that they had thought about in preparation for the interview. For two parents this

was a request for more individualised tailored plans, for another it was better coordination with the

school and for one it was more involvement by partners in the group program. Only one of the 13

parents interviewed expressed general disappointment in the program for them.

I just personally didn’t feel that it was targeted enough to meet these extra behavioural

problems … Because you’ve got four professionals there … and I would just think that they

would be able to offer something more than what we were given … There was only five in the

group and then one dropped out … and all these professionals that it should have been more

child-centred. I just did feel given that they wanted to interview me … that they would take

something from that to individualise it and I didn’t see that … a wasted opportunity … I have

still taken a lot from it and so I don’t discount it.

An individual session with parent and child and Got It! … to work through an individual plan or

talk through issues that you’re not prepared to talk about in front of the group… two or three

times during the term.

The only negative is … school pragmatics. I don’t think the school had enough communication

… They kind of were still organising things on the days that Got It! was on … for example, they

had the book parade and it was on at the same time … and so I think better communication

with the school would have been great.

I think the group generally feels that it would be really good to have this process include an

opportunity for the partners to have a group with the children. They had one partner evening,

which was, you know, sort of a light glimpse of what the group’s about … I’ve been able to

implement more workable strategies as a parent … and my wife’s sort of been not as fully

engaged … I would love it if there was another opportunity for a group style time with the child

where the partners could come along.

The general, overall statements from the parents to sum up their experiences in the small group

program were otherwise positive. Several parents had taken time off work, or reorganised their work

hours, in order to attend the Got It! group. All said that this had been worth it.

I thought it was good … It was difficult at first to make that time to go like when you work full

time but, no, it was definitely worth doing and everything was good. The venue was good and

the food was good and it was comfy. I thought that [staff] , they’re lovely so they made it really

cruisy and easy and the kids loved it.


131

I just thought it was really good and I think that lots of other people need to do it.

I loved the whole experience. Honestly, I loved it!

There’s nothing wrong with the program. It’s just up to parents to get involved and implement

the pointers … I feel very comfortable. It’s wonderful … the biggest thing is finding time to come

along. It’s difficult to fit in when you work.

Look, I think it’s very, very beneficial.

I give it two thumbs up.

Summary of findings: experiences of parents in targeted group

Qualitative interviews were conducted and audio-recorded with 13 parents who participated in 12 targeted groups at 12 schools across the three pilot sites.

Only one parent expressed disappointment in the group as she felt that it was not sufficiently tailored to individual circumstances. Otherwise parents were very positive about the groups, reporting that the group content was useful and the group process was enjoyable.

All parents said that their child enjoyed going to the Got It! group and having their parent with them at school.

A few parents reported that the quality of their relationship with their child had significantly improved.

All of the parents were able to identify particular things that they had learnt, changes they had made in their parenting practices and changes in their child.

The groups served to strengthen bonds between parents and with the school.

Got It! staff were generally very highly regarded as group facilitators.

The screening process was experienced by some as daunting and the relationship between screening and the group program was unclear to many of the parents, as were the criteria for selection for the group program.

4.4.3 Children’s experiences of the group program

The focus groups conducted with the children who attended the targeted small groups provided the

evaluation team with insights into the experiences of the Got It! group for these children. Given the

age group of participants, small number of participants and the nature of the data collected, the

findings presented here are intended as insights rather than as conclusive outcomes. It was very

useful for the evaluation team to meet with the children in their groups. This allowed the team to

experience and to better understand the school settings, child behaviours and group dynamics that

are encountered in the targeted groups.


132

In the focus group sessions, children were asked to think about the Got It! group and then circle a

face on a sheet of ‘emotions faces’ that looked like how they felt going to the Got It! group. Of the 16

children, 15 circled happy faces and one circled an indifferent face. Comments from parents,

teachers and Got It! staff also supported the view that the children enjoyed attending the group and

looked forward to coming each week. The other activities run with the children in the focus groups

encouraged them to talk about what they got out of the Got It! group (using picture cards and a

mock radio interview). Given the developmental stage of the children the ability to articulate this was

limited, but it was apparent that the children enjoyed going to the group. They enjoyed being actively

engaged in crafts and other activities, but in particular they enjoyed having a parent/carer there with

them, as the following quotes from the children illustrate (I=Interviewer, C=Child):

I like going to the Got It! group ‘cause it’s really nice to go there and we do lots of different

activities.

I: What’s the best bit? C: When it’s the mums and kids.

I liked the Got It! group because we get to learn new stuff and I really like going there ‘cause

the people there are really nice.

I liked doing the things and making the things.

It’s pretty awesome because I love my mum and you get to do stuff with them and you get to

spend more time with them.

You can come with your mum! We have so much fun.

We get to have lots of fun and play.

It was also apparent from the children’s comments that the Got It! groups were not just fun and

games, but that the children were also working on developing social-emotional skills in the groups.

Some of the children spoke about this learning and a couple of techniques taught in the group. The

techniques mentioned were the ‘Stop, Think, Do’ reminder to slow down and think and ‘The Dragon’,

which is a breathing exercise that encourages children to stop and relax. The children demonstrated

the dragon exercise for the focus group facilitators. Some of the social-emotional learning for the

children and their experiences of the learning process are illustrated in the following quotes:

We learned stuff like ‘Stop, Think, Go’. No, ‘Stop, Think, Do!’

Keep calm when you speak to somebody.

I yell … and hurt people when I get really, really angry I: Is that something that you’ve talked

about when you come to the Got It! group? C: If you stop, they stop.

It’s made me feel better and learn more about feelings.

When I first started, I was a little bit silly, but getting back to it from going to the Got It!

program, that’s making me feel very good and I’m starting to get happier, and if I get happier I

get more work done and I get to play more with people and I get to be nicer. It makes me feel

very nice.

You have to follow instructions, even though they might sometimes seem a bit boring.


133

You have to listen to the rules.

We’ve got mats. There’s a red one to calm down and a black one is to settle down if you be

naughty.

C: We like the Dragon. I: Why do you do the Dragon? What’s the point of the Dragon? C: To

keep you calmed down.

We just practised this sort of, where you get into something when we don’t know what to do

and learn how to do it.

About comfortable and uncomfortable feelings.

Summary of findings: children’s experiences of targeted groups

The children enjoy attending the targeted group program. In particular, they enjoy having a parent/carer with them at school and doing the various activities and crafts associated with the group learning.

The children were able to identify some learning and techniques that they got from the groups. Given the developmental stage of the children, their ability to articulate learning was limited and hence the results of the outcome evaluation has been important in determining that the group experience has translated into measurable changes in

behaviour.

4.4.4 Families’ experiences six to eight months after the targeted groups

Of the 12 parents interviewed by phone six to eight months after the targeted group ended, 11 said

that positive changes made as a result of the Got It! program were still evident. For one family,

however, upheavals had been so significant that any impact from Got It! was overshadowed. The

changes experienced by this family included domestic violence, moving interstate and taking shelter

in a refuge. The other parents reported that positive changes from Got It! were still noticeable. For

some, the impact of Got It! was regarded as life changing:

It’s had a significant impact. It’s changed our whole life … He can control his anger … He has

strategies and we have strategies … The harmony in our house is amazing. We feel like a

normal family now … He’s lovely to his sisters … He doesn’t hit and bully them any more … He’s

happy now. He used to be intense and sad and moody … We don’t have that any more … The

Got It! program was the single one thing that made the change.

I couldn’t have asked for a better child.

For three of the families, an important impact of Got It! was in providing an avenue through which

ongoing professional services were accessed. These included psychiatry, psychology and

occupational therapy services. For each of these parents, ongoing intervention was regarded as

necessary to maintaining positive change. For one parent, getting medication for her child was seen

as the key to the success of other strategies:


134

[Got It!] has made an impact because it was the first step in getting some further professional

help … The medication has calmed him down.

To varying degree, all except one parent regarded the Got It! program as having offered them,

and/or their child, new insights, understanding and strategies associated with positive changes in

their lives. The parents’ perspectives on the ongoing changes for the children included being calmer,

having insight into their own behaviour and its impact on others, understanding the consequences

of certain behaviours, being able to communicate feelings and improved relationship building skills:

It’s helped him to understand what’s going on better. He learnt about feelings and how to tell

how others are feeling from their actions.

It has helped us and helped her with making friends … She’s found more confidence and is

more outgoing.

It’s given her skills in forming new relationships.


can get them before the problems get too bad.

The parents also talked about how Got It! had changed their own approach to parenting. The

consistency and degree to which change had occurred again varied somewhat between families. The

changes included using warnings and consequences, stopping and thinking about strategies more

rationally, feeling more relaxed and ‘in control’ of parenting and attending to relationship building

skills with their child:

Giving warnings and having time out still works … We are trying to stick with the warnings and

then punishment if he doesn’t do it … Some days are better than others.

I haven’t used the strategies for a while … He’s still got some of the traits but there hasn’t been

as much need … We did practice the strategies quite a bit.

Having the knowledge at the back of my mind … It is making me stop and think ... Asking the

question ‘How are you feeling?’ It is an important question of life. Before the course, I didn’t

know about it … I’d ask a million [times] ‘Why?’… Sometimes I forget and have to remind

myself.

I’m calmer … and I’ve been made more aware of things … I give her encouragement before

things escalate too far.

It’s made me change too in how I go about things … We do more as a family because their

behaviour is good … and I’m thinking about their feelings more too … It helped me to help him.

I haven’t even had a phone call from the school to say that he’s acting up.

We have benefited from the program … We learnt a lot … I couldn’t have done it without the

program … Both of us have learnt to stop and think … talking face to face about problems

when he is calm.

We have playtime after school, rather than doing my emails.


135

It helped us spend time together and to build our relationship.

Still occasionally has emotional outbursts and we try to work the ‘Stop, Think, Do’ steps. But



of myself and my child. My daughter often says she wants to go back to Got It!. She really

loved it. It helped us both. Thank you very much. (Written comment on follow-up SDQp)

Apart from the one family that had experienced major upheavals, none of the other parents

reported deterioration in their child’s behaviour in the months following the end of the Got It!

group. On the whole parents saw that positive gains were maintained through the transitions to new

classes and teachers at the beginning of the school year, and for two children a change of school.

Lapses in behaviour were generally regarded as prompts for parents to refocus on parenting

strategies:

There have been some lapses, but it’s up to the parents … When I’m not using the methods

then things start to fall apart … They need constant attention … I need to stop and think.

He hasn’t reverted to how he was … He’s progressed beyond our hopes … It’s as though he’s a

different child.

He has a new teacher and he’s mucking up a bit.

The school year’s off to a great start.

He’s changed a lot … There’s been ups and downs … He didn’t change immediately … he has

continued to improve.

There will always be a struggle because he tries to take control … But I’m much more confident

that I’m in charge … We know what it takes to make an impact … we are more in tune about

what needs to be done to manage him.

All of the parents acknowledged the value of the Got It! program, but some parents also identified

other factors that had been contributors to improvements for them and their child. These factors

included a new teacher, a different school environment, peer/family relationships, a growing

maturity, counselling, medication and adjustment to changes in the family:

He’s grown up a bit … has a new teacher … has come to terms with us splitting up … and I’m in

a good place now after the separation.

He’s just getting a bit older.

It’s the medication that has calmed him down.

Just as these factors can influence positive behaviour changes for children, environmental factors

can also have a negative impact on child behaviours when external stressors challenge the coping

mechanisms in families. Whilst the Got It! program can develop new skills and insights to assist in

dealing with parenting demands and day-to-day stresses, families in crisis are likely to require

services and supports beyond what the Got It! program can offer. From this small group of 12


136

families, this was the case for the family living in a refuge after escaping from family violence. This

points to the need for Got It! teams to be attuned to potential violence in the homes of group

participants and to work together with other services in offering an integrated response and

specialised referrals. Other potential stressors in families can include mental illness, substance abuse

and grief and loss. The Got It! program is unlikely to be successful in families where factors such as

these are not adequately responded to. Attention to ongoing assessment and referral, as discussed

earlier in this report, is again indicated.

Summary of findings: families’ experiences six to eight months after the targeted groups

Qualitative feedback from parents indicate that the positive impacts on child behaviour and parenting practices are largely maintained at six to eight months following the completion of the targeted group program.

The existence of family violence is likely to limit the potential for positive impacts of Got It!. The need for Got It! teams to be attuned to potential violence, mental illness, substance abuse and grief and loss in families attending group programs, and to refer to appropriate specialised services, is indicated.

4.4.5 Conclusions on the impact of Got It! for families

The information gathered from direct contact with parents, carers and children who have

participated in the Got It! program in 12 schools has been very useful in contributing insights into

experiences of those for whom the program is designed to impact. The targeted group program has

made a substantial impact. All parents who were interviewed could identify changes in parenting

practices that have been implemented as a result of the program. The groups have also had a

positive impact on parent-child relationships, social networks between parents attending the

groups, and connections between parents and the school. The children enjoyed attending the

groups and participated in social-emotional learning activities. The Outcome Evaluation adds further

evidence for measurable changes in child and parent behaviours following the targeted group

program.

The impact of the program for families with a child with elevated conduct scores but who did not

participate in the targeted group program is lower. This is understandable given that the attention

and resources directed to the targeted groups are higher than other components in the Got It!

model of care. What is important, however, is the finding that Got It! can have an impact and make a

difference for these families who are in need, but are not selected for the group program. These

findings strengthen the position made throughout this report, that all components of the Got It!

model of care should be given balanced attention.


137


Whilst the targeted groups are a central feature of Got It!, the universal components, assessment, individual behaviour management and referrals have a bigger target group. The impact of these components could be strengthened. With appropriate leadership and a collaborative developmental approach, Got It! could be strengthened to better address the requirements of stakeholders.


138

5. Outcome Evaluation

5.1 Design and methods

5.1.1 Evaluation questions

The evaluation team designed a research project to address the following questions relating to the

impact of the Got It! program:

1. What are the levels of behavioural adjustment for children pre- and post-Got It! (K–2

children in universal and targeted group components of program)?

2. Are there reductions in behaviour problems on completion of targeted group?

3. Are any reductions in behaviour problems amongst targeted group sustained six to eight

months after the Got It! program?

4. Are there changes in parenting practices by parents/carers of children with conduct

problems on completion of Got It! targeted group?

5. Are any changes in parenting practices sustained at six to eight months follow up?

6. Are any demographic characteristics of children associated with measures of behavioural

problems or with program outcomes?

7. Are measures of parenting practices associated with measures of behavioural problems?

8. What is the impact of Got It! for parents/carers of children who are not in the targeted

program?

9. Do teachers report improved knowledge and skills in behaviour management following the

Got It! program?

Questions 1 through to 7 are addressed primarily through the use of standardised measures

administered before and after the Got It! intervention program. The findings have also been

enhanced with additional qualitative and survey data. The findings relating to questions 8 and 9 have

been addressed using qualitative and survey data only.

5.1.2 Research design

A quasi-experimental pre- and post-test design was used to determine changes in universal and

targeted groups from before to after the Got It! intervention on measures of child behaviour and

parenting practices. The Got It! intervention examined for the evaluation comprised six months of

program delivery across the three sites, encompassing the full intervention program delivered in 12

schools. This period was chosen in order to gain a representative sample of full program delivery in a

variety of schools across the three sites (four schools in each site). As the Got It! teams do not

commence programs in all schools at the same time, the six-month period in early 2013 does not fit

exactly with the calendar months. Rather, the intervention group of schools is equivalent to a six-

month load of full-time Got It! program delivery and incorporates the full intervention. This is


139

important to note in the calculation of costs for the economic analysis.

In addition to measures taken pre- and post-intervention, a follow-up post-test was conducted with

the targeted intervention group six to eight months after completion of the intervention. This gives

an indication of the degree to which any behavioural changes are sustained following the Got It!

program, with six to eight post-intervention being the longest follow-up period possible within the

constraints of evaluation reporting. Qualitative and survey data has been drawn upon to further

enhance understanding of the outcomes and program impact.

The pre- and post-test design does not yield findings that are as strong as those under randomised

controlled trial (RCT) conditions. An RCT was not, however, possible for this evaluation given the

constraints of the project specifications and the ‘real world’ context of program implementation. In

the absence of a control group, a sample from the universal intervention group, matched to the

targeted intervention group on behaviour scores and geographic location, was taken in order to

compare changes on child behaviour and parenting practice measures at the first post-test point.

Another limitation of the methodology relates to the lack of standardisation of the Got It!

intervention model. A pre- and post-test design relies on a clearly defined intervention. As described

in the Process Evaluation, the Got It! intervention has features that are common across the three

sites but, within the framework provided in the model of care, there is also considerable room for

choice, variation and adaption by individual clinicians and teams. Even with all teams using the

Exploring Together program for the targeted group intervention, it was not possible to ensure

program standardisation or fidelity.

5.1.3 Sampling

The 12 schools at which data was collected for the evaluation comprise all of the schools in the Got

It! program for the six-month period. Within each of these schools, baseline data was sought from all

K–2 children and a parent/carer. The total population of K–2 children across the 12 schools was

1,627. Scores were included in the evaluation data for the 1,061 who returned the questionnaires

and for whom a parent/carer provided consent to include data, representing a 65% response rate.

Selection for the targeted group programs was made by the Got It! teams in accordance with their

standard practices. Again, the full population of targeted group participants was approached for

inclusion of pre- and post-intervention data. From the 63 families who completed the targeted

group program across the 12 schools, data for 57 (90% response rate) were included in the

evaluation results. Further detail on response rates for the different measures is provided in Table

13.

At the first post-intervention data collection point, a non-randomised sample of children with

elevated conduct scores who were not included in the targeted group program was taken in order to

gather data from their parents/carers to compare to the targeted group parent/carers. All had

consented initially to follow-up contact. The sample of 33 represents parents/carers of 10% of the

333 children from the population with elevated conduct scores that were not selected for the

targeted groups. Participants were selected on the basis of parent/carer availability in response to

the phone survey request. Whilst this sample provided some opportunity for comparison to the

targeted group, the sample is small (10% of those with elevated scores and not in targeted groups).

All had elevated conduct scores on SDQ and were drawn from across the 12 schools, but there were


140

insufficient potential respondents available to match on a range of variables to the targeted group.

The usefulness of the comparison group has therefore been limited.

Apart from the comparison group, a universal population approach has been taken to the sampling

and the good response rates support findings for these 12 schools to be presented with reasonable

confidence. However, given the strategic selection of sites and schools for the Got It! program on the

basis of need and interest in the program, generalisability of these results to other schools and the

wider K–2 population is limited.

5.1.4 Selection and description of standardised measures

Several standardised measures were selected for pre- and post-intervention administration to

measure child behaviour and parenting practices. The selection criteria employed by the evaluation

team were as follows:

1. Validated scales to measure behaviours/variables shown to be associated with conduct

problems in children.

2. Inclusion of scales to measure both child behaviours and parent/carer parenting practices.

3. Inclusion of measures of child behaviours from different perspectives (i.e. parent, teacher,

clinician perspectives) in order to counter potential biases.

4. Presentation, length and language in scales acceptable and appropriate for the context and

participant groups and likely to maximise response rate.

5. Affordable within project budget.

In determining which scales best met these criteria, the evaluation team gathered information from

research literature and from consultation with clinicians and CASEA program managers prior to the

evaluation data collection period. A workshop was conducted by the evaluation team with Got It!

teams and DEC staff to gather information on how the different scales are likely to be received by

schools, teachers, parents and clinicians. Up to this point, the three Got It! teams had used a variety

of different measures in their internal clinical screening, assessment and evaluation practices. Apart

from the SDQ screening measure, there was no consistency in the measures used across the three

teams. The combined experience of the clinicians at the workshop offered very useful advice on the

utility of the respective measures within the school context. Having been through an evaluation of a

similar program in Victoria, the CASEA program managers offered insights into the usefulness of

different scales.

The standardised measures finally included in the Got It! evaluation are:

SDQt Strengths & Difficulties Questionnaire – Teacher version (Goodman 2001)

SDQp Strengths & Difficulties Questionnaire – Parent version (Goodman 2001)

ECBI Eyberg Child Behaviour Inventory (Eyberg 1998)

HoNOSCA Health of the Nation Outcomes Scale (Child & Adolescent) (Gowers et al. 1999)

PS Parenting Scale (Arnold et al. 1993)

APQ Alabama Parenting Questionnaire – (Shelton, Frick & Wootton 1996)


141

Table 7 provides detail on the standardised measures used, the participant groups, the data

collection point/s, the assessment perspective (who completes scale) and the rationale for inclusion.

Table 7: Standardised measures included in Got It! evaluation

Scale Group/s Administration points

Assessment perspective

Rationale

SDQt

Strengths & Difficulties Questionnaire – Teacher version

(Goodman 2001)

All K–2 Children

(incl. children in targeted groups)

Pre-intervention

Post-intervention

Teacher -

Based on classroom behaviour observations

Nationally endorsed Mental Health Outcome collection measure.

Well-established, free, internationally validated scale widely used in screening and clinical practice to assess a range of child behaviour problems. Useful online clinical support resources. Adapted for use with different age groups and by different assessor perspectives. Enables triangulation of results.

Completed in about 10 mins.

SDQp

Strengths & Difficulties Questionnaire – Parent version

(Goodman 2001)

All K–2 Children

Children in targeted groups

Pre-intervention (all)

Post intervention (sample of children with elevated scores)

Pre-intervention

Post-intervention

6–8 mths post

Parent / carer –

Assessment of child’s behaviour


See above

SDQ takes about 10 mins to complete

ECBI

Eyberg Child Behaviour Inventory

(Eyberg 1998)


Pre-intervention

Post-intervention

Parent / carer –

Assessment of child’s behaviour

Very well established and validated measure of child behaviour problems to assess clinical problems. Most widely used in international research studies examining changes in conduct problems following intervention. Cost prohibitive for use with universal intervention group.

HoNOSCA

Health of the Nation Outcomes Scale (Child & Adolescent)

(Gowers et al. 1999)


Pre-intervention

Post-intervention

Clinician –

Based on assessment interview and group behaviour observations


Very quick and easy for clinicians to complete following initial assessment process.

PS

Parenting Scale

(Arnold et al. 1993)

Parents of all K–2 Children

Parents of children in targeted groups



Pre-intervention

Post-intervention

6–8 mths post

Parent / carer self-assessment of parenting practices

Measures Laxness, Over-reactivity & Verbosity: 3 features of parenting style found in research to be associated with conduct problems in children. Reliability & validity established.

Takes about 10mins to complete.

APQ

Alabama Parenting Questionnaire:

Positive Reinforcement and Parental Involvement subscales only

(Shelton, Frick & Wootton 1996)

Parents of all K–2 Children

Parents of children in targeted groups



Pre-intervention

Post-intervention

Parent / carer self-assessment of parenting practices

Validated & reliable measure. Sub-scale constructs measure factors associated with conduct disorders. Inclusion of two of the independently validated subscale measures that focus on strengths-based parenting behaviours: Positive Reinforcement & Parental Involvement (which are not covered by PS). Two sub-scales take about 5 mins to complete.


142

Other measures reviewed during the design phase that were not eventually included in the

evaluation were: Child Global Assessment Scale, Child Behaviour Checklist, Sutter-Eyberg Student

Behaviour Inventory, Parenting Tasks Checklist, McMaster Family Assessment Device, Kansas

Parental Satisfaction Scale, Depression, Anxiety and Stress Scale and Parenting Stress Index. The

exclusion of these scales is not a reflection on the quality of the assessment tools, but rather that

the selected group of scales in combination best met the five selection criteria provided above.

Some of these other scales continue to be used by Got It! clinicians in their assessment and follow-

up treatment and referral practices, but scores have not been obtained by the evaluation team.

The SDQ is a widely used validated scale to assess child behaviour, with established UK norms. The

SDQ has been adapted and validated for use with different age groups and, within Australia, is a

nationally endorsed Mental Health Outcome collection measure. The version to assess behaviour of

children aged four to 10 years has been used in this study. It has two forms, one that is completed by

teachers (the SDQt) and one completed by parents (the SDQp). The SDQt and SDQp scales are

essentially the same, but the assessment of the child’s behaviour is made from the different

perspectives of parents and teachers. The questionnaires take about 10 minutes to complete and

include 31 questions in the SDQt and 33 questions in the SDQp. In addition to the Total Difficulties

score, the SDQ is comprised of five sub-scales that can be used to identify the domains where

difficulties occur. The five subscales are: Emotions, Conduct, Hyperactivity, Peer Problems and Pro-

social Behaviour. The Conduct sub-scale is of most direct relevance to the concerns of the Got It!

program.

The SDQ screens are useful in identifying children that would benefit from further clinical

assessment for early intervention. The score can be an indicator that behaviours fall outside a

normal range. Using UK norms, scores for each sub-scale and the total fall in one of three bands:

Abnormal (top 10% and indicating a potential clinical problem), Borderline (next 10%) or Normal

(remaining 80%). In addition, there is an Impact score where in SDQt teachers assess the impact that

the child’s behaviour has on the classroom and peer relationships. In the SDQp parents/carers assess

the impact of the child’s behaviour on home life, friendships, classroom learning and leisure

activities. The Impact scores can again be placed in Abnormal, Borderline or Normal bands. The SDQ

is free to use and has useful administration and scoring resources available online. The SDQ has been

used for screening purposes since inception of the program by all of the Got It! teams and was also

adopted by the evaluation team as the most appropriate measure of conduct problems across the

whole K–2 population for evaluation purposes.

The ECBI is also an internationally well-regarded and validated scale to assess disruptive behaviour in

children that has established American norms. A teacher version (SESBI_R) is also available but has

not been used for the evaluation. The ECBI rating forms must be purchased and not copied and

would be cost prohibitive for use in universal screening. It is most suited to assessment of children

who have been identified as having problems that could require clinical interventions. The ECBI is a

parent/carer rating form with 36 items that require parents/carers to rate the frequency of

particular behaviours and then to identify whether or not this presents a problem. The frequency

scale responses combine to form the ‘Intensity’ score and the sum of problems responses form a

‘Problem’ score. A cut-off point for the identification of a clinical problem is established from the

normative sample for both Intensity and Problem scores. The ECBI had previously been used by

some of the Got It! clinicians in their clinical assessment work and it was regarded as useful and

appropriate for the Got It! targeted group population. The ECBI has been incorporated into the


143

evaluation as a pre- and post-targeted group intervention measure.

Both the SDQp and ECBI offer parent/carers’ assessments of child behaviour and the SDQt give the

additional perspective of the teacher. For the purpose of the evaluation, a suitable measure was also

sought that offers a clinician assessment of the child. The HoNOSCA is a very quick, 15-item global

mental health scale for completion by clinicians following initial assessment of a child. It is one of the

nationally endorsed mental health outcome measures in Australia and is also used by the CASEA

program in Victoria to measure program impact. Whilst it does not focus as directly on child

behaviours as other measures, it is a useful global measure with established benchmarks. Most of

the Got It! clinicians were familiar with the HoNOSCA and, as a Mental Health Outcome and

Assessment Tool, there was support within LHDs for administration of the tool and recording of

results. The results of this evaluation suggest, however, that the HoNOSCA does not have the level of

sensitivity to measure Got It! program impacts and for ongoing evaluation, an alternative measure

could be investigated.

The scales discussed above measure child behaviour. Given the close association between parenting

behaviour and child conduct concerns evident from the research literature, it was important to

include measures of parenting behaviours as part of the Got It! evaluation. The evaluation team

sought to identify scales that measure parenting behaviours associated with the emergence of

conduct disorders. The PS and the APQ, both self-administered by parents/carers, were identified as

potential measures. One of the Got It! teams was using the PS with targeted group parents and

found it to be useful and well received. The PS has 30 Likert scale questions and it generally takes

about 10 minutes to complete. In three sub-scales, the PS measures ‘Laxness’, ‘Over-reactivity’ and

‘Verbosity’, which are three features of parenting style found in research to be associated with

conduct problems in children. Several international studies have supported the reliability and

validity of the scale in relation to parenting of young children. The PS focuses on how inappropriate

child behaviours are managed by parents/carers, but the evaluation team identified an absence of

attention in the scale to positive parenting behaviours of involvement and positive reinforcement

that are protective factors for children. The PS was included in the evaluation design for

administration during the screening phase across the universal sample, but concerns remained that

it didn’t fully capture all of the parenting behaviours of interest.

The APQ is also a validated and reliable measure of effective parenting behaviours relevant to the

identification and treatment of child conduct problems. It has five sub-scales that measure five

dimensions of parenting: positive involvement with children; supervision and monitoring; use of

positive discipline techniques; consistency in the use of discipline; and use of corporal punishment.

None of the Got It! teams had previously used the APQ. The scale is, however, used in the ongoing

monitoring of outcomes for the CASEA program in Victoria. Both during and after the workshop, Got

It! clinicians voiced strong concerns about the use of the scale because of the references to

’punishment‘ in some of the sub-scales, which was deemed inappropriate language for the

participant groups. Each of the APQ sub-scales are independently validated and it was therefore

decided that only two of the sub-scales be included. These two subscales measure strengths-based

parenting behaviours that are missing from the PS: ‘Positive Reinforcement’ and ‘Parental

Involvement’. The two sub-scales in the APQ comprise 16 Likert scale questions that can be

completed in about five minutes. It was decided that the PS in combination with the two sub-scales

from the APQ provides an encompassing measure of parenting behaviour well suited to the

evaluation of Got It!.


144

As can be seen in Table 7, a balance of parent/carer, teacher and clinician assessment perspectives

has been achieved in the selection of well-established, tested and appropriate measures of child and

parent/carer behaviour associated with child conduct problems. Additional measures are included

for assessment of the children and parents/carers participating in the targeted groups, as change in

these groups is of most interest to the evaluation of Got It!. Change resulting from the universal

intervention program was assessed by teacher SDQt scores pre and post intervention.

5.1.5 Design of additional data collection instruments

An important feature of the Outcome Evaluation has been the examination of access and equity

issues for different demographic groups (e.g. cultural, family structure, geographic location, age,

gender, employment etc.). Characteristics of the children and families who have elevated scores are

selected for the targeted group program or are associated with intervention outcomes are important

to examine as part of an evaluation. A brief questionnaire was designed to gather demographic

information for the purpose of such analyses. Demographic data was collected from parents/carers

with the initial pre-intervention questionnaires.

A range of other data collection instruments were designed by the evaluation team to collect

information on intervention processes and impact for the participant groups. Whilst these largely

relate to the process evaluation, some of the findings on perceptions of program impact and

outcomes are useful to consider in conjunction with the findings from the standardised measures.

5.1.6 Data collection process

Principals at each of the 12 schools participating in the Got It! program during the evaluation period

were contacted to explain the purpose and process of the evaluation prior to commencement of

data collection. Packages of information on the Got It! evaluation were compiled for distribution to

parents/carers of K–2 children in the 12 participating schools. Each package comprised an

information sheet, consent form and the PS, APQ and demographics questionnaires. The Got It!

teams in each of the three LHDs took responsibility for distributing these packages to parents/carers

of K–2 children through classroom teachers. This was done together with SDQp questionnaires,

which are used in the standard screening protocol by each of the Got It! teams prior to the

intervention program. Teachers took responsibility for the collection of completed forms and passed

these back to the Got It! teams. Teachers also completed the SDQt for all K–2 children as part of the

standard Got It! screening process and consent was sought from parents to share these results with

the evaluation team.

All baseline data was de-identified with a unique child identification number when entered into the

Got It! database. The local databases were managed by the Got It! teams, who entered baseline data

from returned questionnaires. Consent was required from parents for access by the evaluation team

to scores from any of the questionnaires. Some parents chose not to consent and in these cases the

results continue to be held by the Got It! team for clinical purposes only and are not included in the

data forwarded to the evaluation team. The Got It! teams managed much of the baseline data entry

for the evaluation, with guidance on the new database provided by the evaluation team. Some data

entry support was also provided by the evaluation team for the baseline PS, APQ and demographics.

Databases from the three sites were forwarded to the evaluation team for compilation into one

baseline database.


145

The evaluation team was reliant on Got It! team members to take part in the evaluation process as

research assistants in the baseline data collection phase as this was an appropriate and efficient

extension of their screening role in the schools. Whilst roles in relation to program delivery and to

the evaluation were clarified at each point in the process, the potential for role confusion should be

acknowledged. In addition to Got It! team members being research assistants, they are clinicians

delivering the intervention and they were also research participants sharing their experiences of

delivering the Got It! program with the evaluation team as part of the process evaluation. This

crossover of roles is not ideal, but was necessary within the logistical constraints of the evaluation

project.

Following initial analysis of the baseline data, the evaluation team visited each of the 12

participating schools to give a presentation to teachers on the Got It! evaluation and initial findings.

At this point post-intervention information and questionnaires were distributed to teachers for

completion at the end of the intervention period. Communication and data collection negotiations

at the two post-intervention points took place directly between teachers and the evaluation teams.

The Got It! teams did, however, collect the first round of post-intervention data for the targeted

group participants as their direct personal contact was regarded as the best way to maximise the

response rate. With contact details from parent/carer consent forms, the evaluation team was able

to contact parents in the comparison group post-intervention and the targeted group for the six to

eight month post-intervention data collection. The post-intervention SDQp was conducted by phone

with the comparison group, whilst the PS and APQ forms were posted with return envelopes. A low

response rate for the PS and APQ (eight returns) resulted in a decision to exclude data for the

comparison group on these parenting measures. Post-intervention data entry was managed by the

evaluation team.

5.2 Findings: baseline data

5.2.1 Baseline sample characteristics

Baseline data was collected for a total of 1,061 children for whom a parent/carer provided consent

for data to be included in the evaluation. These 1,061 children comprised 65% of children in

Kindergarten to Year 2 (K–2) in 12 schools. The K–2 population for the 12 schools was 1,627;

however, this number excludes Kindergarten children in one large school who, on school request,

were not included in the Got It! program and associated screening. As shown in Table 8, baseline

data relates to 66% of K–2 children in the four Dubbo site schools, 62% of those in the four Mount

Druitt site schools and 69% of those in the four Newcastle site schools. For most of the children not

included, a parent/carer did not return the SDQp form for the Got It! program screening or the

evaluation consent form which was attached to this. In addition, some families who did return the

screening form chose not to participate in the evaluation. Overall, for the 12 schools, SDQp

screening forms were returned to Got It! teams by parents/carers for 1,213 (75%) children. From

these families, 152 (13%) chose not to participate in the evaluation. The data presented here

therefore represents 65% of all K–2 children in the 12 schools and 87% of children who participated

in the screening for the Got It! program. Whilst a good response rate has been achieved, it needs to

be recognised that an inclination by parents/carers to return the questionnaires and consent to

participate in the evaluation could reflect biases in the sample which impact on findings. The total


146

sample of children comprises 21% (228) from the Dubbo site, 40% (423) from the Mount Druitt site

and 39% (410) from the Newcastle site.

Table 8: Baseline sample

K–2 children in 12 schools*

N=

Participated in SDQ screening

Baseline n =

Baseline as proportion of K–2

population

Baseline as proportion of K–2 screening

participants

Dubbo

347 241 228 66% 95%

Mount Druitt

687 545 423 62% 78%

Newcastle

593 427 410 69% 96%

Total

1627 1213 1061 65% 87%

*Kindergarten classes were excluded from one school.

For those standardised measures that included a formula for dealing with some missing scores,

these have been applied. Beyond this, cases with missing scores are excluded from the analyses.

That is, the statistics presented in this report relate to valid cases only for each variable.

5.2.2 Demographic characteristics of the sample

The sample comprised 50% (528) male children and 50% (532) female children. Children were aged

between four and eight years at the time of data collection and were in K–2 classes in 12 schools

(with Kindergarten classes excluded from one school).

The demographic and screening questionnaires were completed in 97% of cases by a parent and in

2% of cases by a grandparent. This provided some demographic information about the family, but as

some questions relate only to the respondent, it does not capture information on any other parent

or carer in the family. Eighty-eight percent (88%) of the respondents were female. As shown in

Figure 12, 57% (570) of parents/carer respondents were aged in their thirties.


147

Figure 12: Age group of parent/carer screening respondent

Responses to the question on the primary care arrangement for the child are depicted in Figure 13.

The majority of the children were cared for by two parents living together (68%, 676), 24% (238) by a

single parent, 4% (44) by two parents living separately and 4% (42) had some other care

arrangement. Respondents were asked to specify the ‘other’ care arrangement and grandparent/s

accounted for 30% (13) of the ‘other’ category. As Figure 14 shows, 86% (857) of the children had at

least one sibling.

There were 85 (9%) respondents who indicated that the family identified as Aboriginal or Torres

Strait Islander. A language other than English, as the main language spoken at home, was indicated

by 176 (18%) of respondents. A total of 47 languages were specified with Hindi, Arabic, Tagalog,

Punjabi and Gujarati each being specified by more than 10 respondents.

0

100

200

300

400

500

600

20s (15%) 30s (57%) 40s (25%) 50s (2%) 60s+ (1%)


148

Figure 13: Primary care arrangement for children screened

Figure 14: Number of children in the families of children screened

As shown in Figure 15, 39% (383) of respondents did not work, 35% (343) worked part time and 26%

(259) worked full time. Information on the employment status of any other parents/carers in the

family was not collected.

Primary Carer n=1000

2 parents together 68%

Single parent 24%

2 parents living separately 4%

Grandparent 1.3%

Other 3%

0

50

100

150

200

250

300

350

400

450

1 (14%) 2 (41%) 3 (26%) 4 (12%) 5 or more (7%)


149

Figure 15: Employment status of parent/carer screening respondents

There was a spread of educational levels of respondents (Figure 16). In addition, 20% indicated that

they were currently studying.

Figure 16: Highest education level of parent/carer screening respondents

0

50

100

150

200

250

300

350

400

450

Not working 39% Part time 35% Full time 26%

0

50

100

150

200

250

300

350

Less than Yr 106%

Year 10 -17%

Year 12 -11%

TAFE equiv -33%

Uni Degree21%

Postgrad -12%


150

5.2.3 SDQ measures of child behaviour: comparison to norms

The Strengths and Difficulties Questionnaire (SDQ) was used to measure child behaviour. Two

versions of the scale were used: the SDQp (completed by parents) and the SDQt (completed by

teachers). A higher score on the scale represents more behaviour difficulties. The scales each

produce a total score, together with five sub-scale measures: ‘conduct’, ‘hyperactivity’, ‘emotional

difficulties’, ‘peer problems’ and ‘pro-social’. There is also a measure of impact of behaviours on

environment (family, friends, classroom). In the absence of Australian norms for the scales, UK

norms have been used. Based on UK norms, the total score and each of the sub-scale results fall into

one of three bands: Abnormal (top 10%), Borderline (next 10%) or Normal (remaining 80%). The

total scores, conduct sub-scale scores and impact scores are of most direct relevance to the goals of

the Got It! program; hence, analysis has focused on these results.

A comparison of SDQ (p & t) results for our sample of 1,061 children was made with published data

on UK norms. As tests for normality found all distributions to be skewed, a non-parametric test

(Wilcoxon) was used to test for differences between the UK and Got It! samples. Results indicate

there was no significant difference between the Got It! sample and the UK sample for SDQp total

scores, but there was a difference for SDQt total scores (p<0.001). The Got It! sample scores for

SDQt total scores was significantly lower (mean = 5.91) than the UK scores (mean=6.6). This indicates

lower levels of behaviour problems reported for the Got It! sample than in UK population studies.

That is, whilst for the UK sample Abnormal represents the top 10% of SDQ scores, for the Got It!

sample it represents only the top 7.6%. Similarly, the means for both the SDQp conduct sub-scale

scores and SDQt conduct sub-scale scores were also significantly lower for the Got It! sample than

the UK sample (means of 1.46 against 1.6 and 0.82 against 0.9 respectively, p<0.001). A different

trend was, however, found in relation to impact scores for SDQt, which were significantly higher for

Got It! sample than UK sample (means of 0.47 against 0.4, p<0.001). Thus, whilst teachers scored

behaviour problems as lower than the UK norms, they scored the impact of these behaviours on the

classrooms as higher. No apparent reason for this has emerged from the study.

As the Got It! sample is not a representative Australian sample, we are unable to draw any

conclusions from these results regarding Australian norms. The results may, however, reflect a

number of possible scenarios regarding the Got it! sample. The results may be skewed by the

voluntary participation in the evaluation. Perhaps a higher proportion of parents/carers of children

without behaviour problems, than those with behaviour problems, consented to participate in the

screening and the evaluation. This would bring the overall scores down. Another explanation could

be that since most of the screening was done by teachers during the first few weeks of the school

year, teachers were only beginning to get to know the children and any behaviour problems may not

have been fully observed. This could also have brought scores down.

5.2.4 Demographic characteristics associated with elevated SDQ scores

Data relating the range of demographic variables to scores for the SDQ measures (total and conduct

sub-scale scores on both parent and teacher versions) were analysed in order to test for any

significant relationships. The results of Wilcoxon / Kruskal-Wallis tests, with significance levels, are

presented in Table 9. The results indicate that the gender of the child, age of parent, primary care

arrangement, language spoken at home, Aboriginality and the education, employment and study


151

status of parents are related to scores on the SDQ. This was not, however, the case from both

parents’ perspectives (SDQp) and teachers’ perspectives (SDQt), with age of parent and Aboriginality

having no association to SDQp scores and study status having no relationship to SDQt scores.

Supporting a large body of research in the field, the child being male has the strongest association

with higher scores on the SDQ.

Table 9: Relationships between demographic variables and SDQ scores

Note: Higher SDQ score means more behaviour difficulties

SDQp Total

SDQp Conduct sub-scale

SDQt Total


Gender of child Boys score higher than girls p<0.001

Same relationship as found for SDQp total p<0.001



Education level of parent/carer who completed form

Trend shows a lower education level of parents associated with higher scores for children p<0.001




Employment status of parent/carer who completed form

Children of parent/carer who was not employed scored higher than both children of part-time employed & those of full-time employed p<0.001



Children of parent/carer who was not employed scored higher than children of parent with part-time employment p=0.02

Primary care arrangement

Children in all other care arrangements scored higher than children cared for by 2 parents living together p<0.001

Children in all other care arrangements scored higher than children cared for by 2 parents living together p<0.003 Children in “other” care category scored higher than children in all other groups p<0.01

Children cared for by a sole parent and children in “other” care category scored higher than children cared for by 2 parents living either together or separately p<0.001

Children cared for by a sole parent and children in “other” care category scored higher than children cared for by 2 parents living either together or separately p<0.001

Main language spoken at home

No relationship Children who spoke English at home scored higher than those who spoke another language p=0.002

Same relationship as found for SDQp conduct p=0.04

Same relationship as found for SDQp conduct p=0.03

Aboriginal or Torres Strait Islander (ATSI)

No relationship No relationship ATSI children scored higher p=0.003

ATSI children scored higher p=0.001

Age of parent/carer who completed form

No relationship No relationship Children with parents in 20s score higher than those with parents in 30s or 40s p<0.015

Children with parents in 20s score higher than those with parents in 30s or 40s p<0.006

Study status of parent / carer

Children of studying parent scored higher than others p<0.001

No relationship No relationship No relationship


152

Key finding

A child being male, not being cared for by two parents living together, having parent/carer with lower education level, having a parent/carer who is not employed, being Aboriginal and having a parent/carer in their twenties are factors found to be associated with higher behaviour difficulties scores as measured by the SDQ. Speaking a language other than English was not associated with higher behaviour difficulties scores; in fact, the opposite was indicated.

Considerations for addressing child disruptive behaviour problems

The association of disruptive behaviour problems with social factors suggest the importance of wider social and community programs to address disadvantage in communities. Supportive programs and resources to assist Indigenous communities, address unemployment, offer pathways and options for education, and support young parents are likely to have flow-on effects in child behaviours.

5.2.5 SDQ bands: comparison between sites

The proportions of children in each of the three bands (A=abnormal, B=borderline and N=normal) on

SDQp and SDQt (totals and conduct sub-scale) were analysed using chi-square tests to determine

any associations between geographic site and the proportions in each category. Across the three

sites there were 11.7% of children with parent SDQp conduct sub-scale scores in the Abnormal band.

For teacher ratings of conduct on SDQt there were 7.6% of children in the Abnormal band. The

significant differences between the sites have been highlighted in bold in Table 10. Adjusted

residuals of greater than 2 or less than –2 have been used to identify where differences between

sites lay.

In the Dubbo site, there was a higher proportion in the Abnormal band on the SDQp and SDQt

(17.5% and 14% respectively). Mount Druitt had a lower proportion in the Borderline band on the

SDQp (5%). The analysis of actual scores using Wilcoxon / Kruskal-Wallis tests confirmed this trend,

with Dubbo scoring higher than average in comparison to Newcastle and Mount Druitt on total and

conduct subscale scores for both SDQp and SDQt (p<0.001 for all tests).

It should be noted that these findings relate to the 12 schools participating in the Got It! program

during the evaluation period, and do not necessarily reflect the situation across the sites more

generally.


153

Table 10: Proportions in SDQ score bands across sites

No. screened

No. baseline sample

SDQp Conduct Abnormal band (% within site)

SDQp Conduct Borderline band (% within site)

SDQt Conduct

Abnormal band (% within site)

SDQt Conduct

Borderline band

(% within site)

Dubbo

239 228 40 (17.5%)* 26 (11.4%) 32 (14.0%)* 14 (6.1%)

Mt Druitt

541 423 40 (9.5%) 21 (5.0%)* 19 (4.5%) 13 (3.1%)

Newcastle

423 410 44 (10.7%) 50 (12.2%) 30 (7.3%) 14 (3.4%)

Total

1203 1061 124 (11.7%) 97 (9.1%) 81 (7.6%) 41 (3.9%)

*Wilcoxon / Kruskal-Wallis test - Significant difference p<0.001

The higher proportion of children in the Abnormal SDQ bands from the Dubbo site is likely to reflect

the different demographic characteristics of the three sites. The demographic variables found to be

associated with elevated SDQ scores, discussed in the section above, are presented by site in Table

11. The demographic data reveals a higher representation of the demographic characteristics

associated with elevated SDQ scores amongst children in the four Dubbo site schools. This is the

case for all relevant variables, except having a parent who is studying, which is at a comparable level

across the three sites. Of particular note is that 50% of Dubbo site children have a parent with an

education level at Year 10 or less, in comparison to 15% in Mount Druitt and 17% for Newcastle; 30%

of families in the Dubbo site identified as ATSI, in comparison to 1% in Mount Druitt and 3% for

Newcastle; and 32% of parents in the Dubbo site were in their twenties in comparison to 12% in

Mount Druitt and 7% in Newcastle. These findings from the four schools in the Dubbo site fit with

the wider demographic profile of the site reported in the profile data above, that places Western

NSW at a higher level of disadvantage than the other sites on a range of parameters.


154

Table 11: Demographic variables associated with elevated SDQ by site

Dubbo

Mount Druitt Newcastle

Lower Ed of Parent Year 10 or less

n=220 109 50%

n=376 57 15%

n=397 66 17%

Unemployed parent

n=219 120 55%

n=374 156 42%

n=393 104 26%

Not living with both parents

n=222 105 47%

n=380 114 30%

n=398 107 27%

Parent studying

n=179 36 20%

n=323 70 22%

n=330 64 19%

ATSI

n=219 65 30%

n=374 4 1%

n=396 13 3%

Parent in 20’s

n=247 80 32%

n=384 45 12%

n=397 26 7%

Key finding

The schools serviced by the Dubbo Got It! team experienced higher levels of behaviour difficulties, as measured by the SDQ, than occurred in either Mount Druitt or Newcastle. This finding reflects the different demographic characteristics of the three sites, in particular that Dubbo was found to have a higher representation of ATSI children, parents with lower education levels and parents in their twenties.

5.2.6 Comparisons between SDQ scores of children selected for the Got It! targeted groups

and other K–2 children: unmet need

There were 68 children selected to commence the targeted groups across the three sites. Baseline

data was available for 64 of these children (94%). As described above, baseline data was also

collected for 997 children who were not selected for the targeted groups (64% of K–2 children not in

targeted groups). The main difference between the children selected for the targeted groups and

the other K–2 children is that all children in the groups have elevated conduct scores (p<0.001 for

both SDQp and SDQt). All of the children in the targeted groups had SDQ scores (either parent or

teacher) in either A (abnormal) or B (borderline) bands for either the total score or the conduct sub-

scale score as this was the first criterion for consideration for the targeted groups.

The proportions of children in each of the three conduct score bands for SDQp for the targeted

group program and those not in a group program are provided in Table 12. Overall, for the 1,061 K–2

children screened, 124 (11.7%) scored in the Abnormal band and 97 (9.1%) scored in the borderline

band. Thirty-four (34) children (54.7%) of the 64 in the targeted groups had SDQp conduct subscale

scores in the Abnormal band and another 11 (17.2%) scored in the Borderline band. Fifteen (15)

children in the targeted groups (23.4%) scored in the Normal band for conduct according to SDQp


155

questionnaires completed by their parent/carer. Eighty-nine (89) children, (8.9%) of those not

selected for the targeted groups, were in the Abnormal band and 86 (8.6%) scored in the Borderline

band. From those children with SDQp scores in the Abnormal band, 71.8% were not selected for the

targeted groups and from those in the Borderline band, 88.7% were not selected. Thus, in total, 21%

of children with elevated SDQp conduct scores were selected for the group program.

Table 12: Proportions of SDQp conduct bands for children in and not in targeted groups

Conduct bands SDQp Baseline Total

Missing Abnormal Borderline Normal

Total K–2

Count 16 124 97 824 1061

% of Total K–2 1.5% 11.7% 9.1% 77.7% 100.0%

Not in group program

Count 13 89 86 809 997

% of those Not in Group 1.3% 8.9% 8.6% 81.1% 100.0%

% of those in SDQp Conduct band 81.3% 71.8% 88.7% 98.2% 94.0%

% of Total K–2 1.2% 8.4% 8.1% 76.2% 94.0%

In group program

Count 3 35 11 15 64

% of those in Group program 4.7% 54.7% 17.2% 23.4% 100.0%

% of those in SDQp Conduct band 18.8% 28.2% 11.3% 1.8% 6.0%

% of Total K–2 .3% 3.3% 1.0% 1.4% 6.0%

Table 13 shows the proportions of children in conduct score bands for SDQt completed by teachers.

Overall, 81 (7.6%) of K–2 children were in the Abnormal band and 41 (3.9%) were Borderline.

Twenty-eight (28) of the children in the targeted groups (43.8% of the 64 children) had SDQt conduct

scores in Abnormal band and another 10 (15.6%) were in the Borderline band. There were 26

children in targeted groups (40.6%) in the Normal band for conduct as assessed by teachers. There

were 53 children with Abnormal band SDQt conduct scores that were not selected for the targeted

groups. These 53 comprise 5.3% of those not included in the targeted groups. Another 31 (3.1%) had

Borderline scores and were not included in the targeted groups. Thirty-one percent (31%) of the

children that teachers identified with elevated conduct problems (A or B bands on SDQ) were

selected for the targeted groups and 69% were not.

As teachers and parents do not always have the same assessments of the behaviour of individual

children, the SDQ scores and associated bands did not directly correspond for the SDQp and SDQt.

There was a correlation found between the SDQp and SDQt conduct scores (p<0.001). This was,

however, found to be relatively weak (correlation co-efficient 0.3224). A similar correlation was

found for the total SDQ (p) and (t) scores (p<0.001, co-efficient 0.3637).


156

Table 13: Proportions of SDQt conduct bands for children in and not in targeted groups

Conduct bands SDQt Baseline Total

Missing Abnormal Borderline Normal

Total K–2

Count 20 81 41 919 1061

% within total K–2

1.9% 7.6% 3.9% 86.6% 100.0%

Not in group program

Count 20 53 31 893 997

% of those Not in Group 2.0% 5.3% 3.1% 89.6% 100.0%

% of those in SDQt Conduct band 100.0% 65.4% 75.6% 97.2% 94.0%

% of Total

1.9% 5.0% 2.9% 84.2% 94.0%

In group program

Count 0 28 10 26 64

% of those in Group program 0.0% 43.8% 15.6% 40.6% 100.0%

% of those in SDQt Conduct band 0.0% 34.6% 24.4% 2.8% 6.0%

% of Total

0.0% 2.6% .9% 2.5% 6.0%

In order to identify the total number of children with elevated behaviour scores, a subset of data

was taken from the database. The subset comprised those children who had Abnormal or Borderline

scores on one or more of the following measures: SDQp total, SDQp conduct, SDQt total, SDQt

conduct. A total of 394 children were identified, comprising 37% of the total sample. Less than one

in five (17.3%) of the children with elevated behaviour scores were amongst the 68 children selected

for the targeted group program. This proportion is lower than the results for the SDQ conduct sub-

scale scores as elevated conduct score had a greater influence on group selection than elevated total

score, given the focus of the group intervention program.

Key finding

About 80% of children who, based on SDQ measure, are assessed by parents as having elevated conduct problems are not being selected for the targeted groups.

Around 70% of children assessed by teachers as having elevated conduct problems are not included in the group program. The identified children are not the same group, with a relatively weak correlation found between teacher and parent SDQ scores.

17% of children with an elevated Total or Conduct score on either the SDQ parent or teacher measure were included in the targeted group program.

Given these findings, teachers and parents may perceive the Got It! program to be inadequately reaching those in need.


157

5.2.7 Demographic characteristics of children selected for the Got It! targeted groups in

comparison to other K–2 children

Analyses of demographic characteristics were conducted to determine whether children in the

targeted groups differed from the other K–2 children on variables other than behaviour scores. Chi-

square tests were conducted to identify significant differences between the two groups, with a

Monte Carlo test being conducted when Chi-square assumptions were not met. No significant

differences between children in targeted groups and other K–2 children were found in relation to:

number of children in the family

identification as Aboriginal / Torres Strait Islander

education level, employment status or study status of the parent/carer who completed

the screening questionnaires.

In line with the K–2 population, the age of children in the targeted program ranged from four to

eight years. As shown in Figure 17, the targeted group trends a bit toward the younger end of the K–

2 population.

Figure 17: Age of children in targeted group program

Table 14 details the demographic variables on which the targeted group participants differed

significantly from the other K–2 children.

0

5

10

15

20

25

30

4 yrs 5 yrs 6 yrs 7 yrs 8 yrs

No. children in each age group n=62


158

Table 14: Demographic differences between children in targeted groups and other K–2

Targeted Group Other K–2

Primary Care

2 parents living together 48% 69%

p=0.001 Monte-Carlo 2-sided

2 parents living separately 12%

4%

Other 10%

4%

Age of Parent / Carer 20’s 24% 14%

p=0.045

Monte-Carlo 2-sided 50’s 7%

2%

Main Language Not English 2% 19% p=0.001

Chi-square

As shown in Table 14, a significant difference was found between the primary care arrangements

for those in the targeted groups and other K–2 children. Whilst no difference in the proportion of

single-parent carers was found between the two groups, there was a lower proportion of two

parents living and caring for children together for the targeted group children (48%) than was the

case for other K–2 children (69%). A higher proportion of children in the targeted groups were found

to be cared for by both parents who live separately (12% compared with 4% for other K–2 children).

For 10% of children in the targeted groups the ‘other’ category was selected (e.g. grandparent, aunt,

step-parent), which is significantly higher than for the other K–2 children at 4%.

A significant difference between the children in the targeted groups and other K–2 children was also

found in regard to the age of parent/carer who completed the questionnaire. Children in the

targeted groups had a higher proportion of carers in their twenties (24% compared with 14% for

other K–2 children), a lower proportion with carers in their thirties (42% compared with 58%) and a

higher proportion in their fifties (7% compared with 2%). The latter is likely to reflect the higher

proportion of grandparent carers for the children in the targeted groups.

Whether a language other than English was spoken as the main language at home was a significant

difference between the children in the targeted groups and other K–2 children. Whilst 19% of K–2

children spoke a main language other than English at home, this was the case for only 2% (1 child) in

the targeted groups.


159

Key finding

Children selected to participate in the targeted group program differ from the wider group of K–2 children on several variables apart from behaviour scores. The differences relate to:

primary care arrangement

age of parent/carer

language spoken at home.

Relatively fewer children in the targeted groups live with two parents and more children are cared for by separated parents or by another carer. More children in the targeted groups had a parent in their twenties and more spoke English at home than was the case for other K–2 children.

5.2.8 Comparison of demographics for children selected for targeted groups and other

children with elevated conduct scores

Data for the subset of children with elevated conduct scores, defined as A or B rankings for total

score or conduct subscale score on SDQp or SDQt (n=394, 37% of sample), were analysed to

determine demographic differences between those who were selected for the targeted groups and

those who were not. Chi-square tests were conducted to identify significant differences between the

two groups, with a Monte Carlo test being conducted when Chi-square assumptions were not met.

Amongst those children with elevated conduct scores, there was no significant difference for the

following demographic characteristics between those selected for the targeted group and those not:

Age group of parent/carer completing the screening form.

Primary caregiving arrangement (e.g. single parent, two parents together, parents separated

or other).

Number of children in the family.

Identification as Aboriginal or Torres Strait Islander.

Education level, employment status and study status of parent/carer respondent.

The only demographic variable on which a significant difference was found between those selected

for the targeted group and those not was the main language spoken at home (Pearson Chi-square

two-sided test, p=0.005). Whilst 15% of the children with elevated conduct scores spoke a language

other than English as their main language at home, only 2% of the children selected for the targeted

group spoke a language other than English.

Key finding

The children selected for the targeted group program are representative of the wider group of children with elevated conduct scores for all demographic variables measured, except for language spoken at home. Given that the targeted groups are conducted in English, it is understandable that language could present a barrier to participation for families that speak languages other than English. Other reasons for children with elevated scores not participating in the targeted groups include unavailability or lack of interest by parent/carer, parent/carer or child assessed as unsuitable by Got It! clinicians or demand too great for targeted program in which groups need to be kept small (See process evaluation section 4.2 for detail).


160


Examine intervention options within Got It! for engagement of families from culturally and linguistically diverse backgrounds; for example, group, individual, assessment, information/referral intervention options.

5.2.9 Associations between parenting practices and SDQ scores

Two self-administered scales were used to measure parenting practices of parents/carers, the

Arnold Parenting Scale (PS) and the Alabama Parenting Questionnaire PI and PR subscales (APQ). The

lower the score on the PS reflects more effective parenting practices, whilst the higher the score on

the APQ reflects more effective parenting practices. These scales were completed respectively by

968 and 997 parents/carers. Spearman’s rho correlation coefficients were calculated to test for any

relationships between the measures of parenting practices and measures of child behaviour.

Weak correlations were found between both measures of parenting practices and total SDQp scores.

The correlation between PS and SDQp was 0.2454 (p<0.001) and between APQ and SDQp it was -

0.2246 (p<0.001). The correlation between APQ and SDQt was very weak at -0.0880 (p=0.006). No

correlation was found between PS and SDQt.

Key finding

Parenting practices, as measured by Arnold Parenting Scale and Alabama Parenting Questionnaire, are not associated with teachers’ assessments of child behaviour on SDQ. These parenting measures do, however, have a weak correlation with parent assessments of child behaviour on SDQ.

5.3 Findings: post-intervention outcomes

5.3.1 Pre- and post-dataset

On completion of the targeted Got It! interventions, post-intervention data was collected. Using

individual identifiers attached to the 1,061 children with baseline data, the post-intervention scores

were matched with pre-intervention scores. For each measure, the distributions of the differences in

scores (post-score – baseline score) were analysed to identify significant changes in scores following

the intervention. Sample sizes and return rates for the pre- and post-dataset are provided in Table

15. Whilst 68 children commenced the targeted groups, results are included only for those who

completed the group program. Consent to include scores was obtained for 60 of the 63 who

completed the targeted groups.


161

Table 15: Pre- and post-datasets for child behaviour and parenting practice measures

Participant group/s N=

Baseline n=

(response rate)

Post-scores matched to

baseline n=

Pre to Post Return rate

Post difference

distribution**

Child Behaviour measures:

SDQt total

Children in universal All K-2* N=1627

Incl. children completed targeted groups (n=63)

1061 (65%)

60

(95%)

973

60

92%

100%

Skewed

SDQt Conduct As above

SDQp Total

All K-2* N=1627

Incl. children completed targeted program (n=63)

Incl. children with elevated baseline scores*** NOT in targeted groups (n= 333)

1045 (64%)

57 (90%)

33

(10% sample)

49

33

86%

100%

Normal

SDQp Conduct As above

ECBI Intensity

Children completed targeted program N=63

55 (87%)

45 82% Normal

ECBI Problem


50 (79%)

31 62% Skewed

HoNOSCA


60 (95%)

58 97% Normal

Parenting Practices measures:

Parenting scale

Parents of children in universal & targeted program All K-2* N=1627

Incl. those who completed targeted program (n=63)

968 (60%)

57

(90%)

48

84%

Normal

APQ

Parents of children in universal & targeted program All K-2* N=1627

Incl. those who completed targeted program (n=63)

997 (61%)

57 (90%)

45

79%

Skewed

*Kindergarten classes were excluded from one school.

**Nature of distribution informed choice of test for statistical analysis

*** Elevated score = A or B score on Conduct or Total scale for SDQp or SDQt.


162

5.3.2 Outcomes for participants in targeted group program

The targeted group program was completed by 63 families. Data for participants in the targeted

group programs were analysed using one-sample T-test for normally distributed data and the non-

parametric equivalent, Mann-Whitney test, for skewed distributions. Results for each of the

measures are provided in Table 16, with significant results highlighted in bold. Significant

improvements in child behaviour scores from pre- to post-intervention were found on the scales

that most directly measure disruptive behaviours in children (conduct sub-scales in SDQp & SDQt

and the ECBI). The SDQ total and HoNOSCA scales measure relevant but more global behaviours.

Whilst a significant result was not found for these other measures, results for all measures show a

trend toward improvement. The analysis of measures of parenting behaviour found a significant

improvement on scores for the Parenting Scale, but not for the APQ (a trend in the desirable

direction is, however, indicated). For those scales where a significant positive post-intervention

change was found, the percentage improvement in scores is provided in the final column of Table

16. Findings indicate significant positive benefits from the Got It! program, with up to 6.3%

improvement in scores. This is an encouraging outcome for the pilot phase of an intervention

program.

Table 16: Outcomes of Got It! intervention (targeted group program): Pre-Post differences

Note: Decrease in score indicates improvement for all measures except APQ sub-scales, where reverse is true.

Intervention group (n)

Baseline mean

Post – Pre Difference in mean scores

Standard deviation*

% Standardised mean change

Child Behaviour measures:

SDQt Total

60

13.5 -0.5 Not significant

6.1 No significant change after intervention

SDQt Conduct

60 3.2 -0.4 p=0.001

2.4 -4.3%

SDQp Total

49 16.1 -0.58 Not significant

6.0

No significant change after intervention

SDQp Conduct

49 3.8 -0.5 p=0.025

2.0 -6.3%

ECBI Intensity

45 140.9 -14.4 p=0.001

24.7 -0.4%

ECBI Problem

31 15.8 -3.1 p=0.04

7.9 -2.1%

HoNOSCA 58 9.5 -0.5 Not significant


Parenting Practices measures: Parenting Scale

48 99.5 -9.7 p=0.004

22.0 -0.5%

APQ 45 65.0 0.8 Not significant


*Standard deviation within, for paired data.

It was also useful to examine the proportions of children falling within the behaviour bands

(Abnormal, Borderline, Normal) for each of the behaviour measures, before and after the targeted

intervention. As shown in Table 17, there was an increase in the proportion of children in the Normal


163

band for each of the four behaviour measures following the intervention. Depending upon the

measure, there a reduction in the proportion of children in the Abnormal band of between 7% and

20%.

Table 17: Proportions of targeted group participants in child behaviour bands pre- and post

datasets

n=

Abnormal Borderline Normal

Pre Post Pre Post Pre Post

SDQt Conduct

60 25 (42%)

21 (35%)

10 (17%)

9 (15%)

25 (42%)

30 (50%)

SDQp Conduct

49 28 (57%)

20 (41%)

6 (12%)

10 (20%)

15 (31%)

19 (39%)

ECBI Intensity

45 27 (60%)

18 (40%)

NA 17 (38%)

28 (62%)

ECBI Problem

32 18 (56%)

13 (41%)

NA 14 (44%)

19 (59%)

Note: Only cases with both pre and post measures included.

The proportion of children in the targeted group program who individually had a positive shift in

assessment band following the intervention was also calculated. This included those that shifted out

of the ‘Clinical’ (ECBI) or Abnormal or Borderline (SDQ) bands and into the Normal bands, or from

Abnormal to Borderline (SDQ). Depending upon the measure, it was found that between 18% and

33% of children made a positive shift into the Normal or Borderline bands for behaviour following

the targeted group intervention. The results are shown in Table 18. Whilst there were also some

children who made a negative shift in band (between 3% and 16% depending upon the measure),

and the majority had no shift in band, this should be considered in the context of an overall

significant improvement in scores as described above. The ECBI outcomes for the Got It! evaluation

also parallel the findings of Bonin and colleagues (2011) who reviewed international research on

outcomes of parenting intervention programs for child conduct problems. They found an average

rate of 34% reduction in clinical cases of conduct disorder from pre to post intervention using the

ECBI measure.

Table 18: Targeted groups: shifts in child behaviour bands from pre- to post-intervention

n=

Positive shift in assessment band*

n (%)

Negative shift in assessment band

n (%)

No shift in assessment band

n (%) SDQt Conduct

60 11 (18%)

7 (12%)

42 (70%)

SDQp Conduct

49 16 (33%)

8 (16%)

25 (51%)

ECBI Intensity

45 14 (31%)

5 (11%)

26 (58%)

ECBI Problem

32 6 (19%)

1 (3%)

25 (78%)

* Clinical/Abnormal to Normal/Borderline or Borderline to Normal.

The post-intervention SDQ measure includes two global questions about the perceived improvement

following the intervention and the perceived helpfulness of the intervention. The SDQp results


164

provide the parents’ perspectives and the SDQt results provide the teachers’ perspectives. The

results are presented in Table 19 and Figures 18 and 19. The trend is that level of improvement and

helpfulness of the intervention are both reported more positively by parents than by teachers.

Nonetheless, 41% of children are regarded by teachers to have been helped ‘quite a lot’ or ‘a great

deal’ by the Got It! targeted intervention.

Table 19: Parent and teacher ratings of improvement in child’s behaviour and helpfulness of

intervention

N= Improvement n (%)

Helpfulness n (%)

SDQt

60 Much worse 4 (7%) Bit worse 2 (3%)

About same 19 (32%) Bit better 28 (47%)

Much better 7 (11%)

Not at all 9 (15%) Only a little 26 (43%)

Quite a lot 20 (33%) A great deal 5 ( 8%)

SDQp

48 Much worse 0 ( 0) Bit worse 2 (4%)

About same 6 (13%) Bit better 24 (50%)

Much better 16 (33%)

Not at all 0 (0) Only a little 8 (17%) Quite a lot 29 (60%)

A great deal 11 (23%)

Figure 18: Parents’ and teachers’ ratings of improvement in behaviour of individual children

following targeted intervention (%).

0

10

20

30

40

50

60

Much worse Bit worse Same Bit better Much better

SDQp n=48

SDQt n=60


165

Figure 19: Parents’ and teachers’ ratings of helpfulness of group intervention for individual

children (%)

Key findings

Significant improvements in child behaviour scores after the targeted intervention were found on the scales that most directly measure disruptive behaviours in children; that is, conduct sub-scales in SDQp & SDQt and the ECBI.

Findings indicate significant positive benefits from the Got It! program of up to 6.3% improvement in scores. This is an encouraging outcome for the pilot phase of an intervention program.

Depending upon the measure, it was found that between 18% and 33% of children made a positive shift into the Normal or Borderline bands for behaviour following the targeted group intervention.

41% of children are regarded by teachers to have been helped ‘quite a lot’ or ‘a great deal’ by the Got It! targeted intervention.

83% of parents indicated that their child had been helped ‘quite a lot’ or ‘a great deal’ by the Got It! targeted intervention

The analysis of measures of parenting behaviour found a significant improvement on scores for the Parenting Scale, but not on the APQ.

0

5

10

15

20

25

30

35

Not at all Only a little Quite a lot A great deal

SDQp n=48

SDQt n=60


166

5.3.3 Six-to-eight month follow up of targeted group participants

In order to assess whether changes observed at the end of the targeted group program were

maintained six to eight months later, SDQt, SDQp and Parenting Scale measure scores were collected

in March 2014. Teachers returned SDQt questionnaires relating to 50 children who completed the

targeted group program (83% return rate). The return rate from parents was unfortunately low, with

only 14 parents returning the SDQp and Parenting Scale, representing 23% of the 60 who completed

the group program.

At the six-to-eight month follow up, 84% (42) of children who completed the targeted group

program had maintained or improved the SDQt conduct score band they were in at the post-

intervention point. Of the 27 children with scores in the Normal band post-intervention, 85% (23)

had maintained this level at the six-to-eight month follow-up. A further seven children had moved

into the Normal band from either the Borderline or the Abnormal bands, making a total of 60% (30)

in the Normal band at the six-month follow up. In addition, a further four children improved their

scores and moved from the Abnormal into the Borderline band. There were eight children who

maintained their Abnormal band score and a further seven who moved into the Abnormal band

from either the Borderline (3) or Normal (4) bands. This represents 30% of children (15) in the

Abnormal band.

A similar pattern was found for the SDQp scores. At the six-to-eight month follow up, 93% (13) of

children maintained or improved their behaviour score band. Six children (43%) maintained a score

in the Normal band and another four children (25%) improved and moved into the Normal band

from either Abnormal or Borderline bands. Three children maintained an Abnormal or Borderline

band and one child moved down into the Abnormal band from the Normal range. In light of the low

response rate for the SDQp at the six-to-eight month follow up (23%), a sample bias toward those

families where improvements have been observed and maintained is likely.

For the 14 parents who returned the Parenting Scale, nine had a score that was the same or an

improvement on their first post-intervention score. For those five parents whose parenting scores

had deteriorated between the first and the second post-intervention data collection points, all had a

score that was still an improvement on their original baseline scores (excluding one with no baseline

score for this scale).

Key findings

The child behaviour improvements made post-intervention on the SDQt and SDQp were maintained by about 85% of children at the six-to-eight month follow-up point.

The majority of parents were continuing to improve parenting practices at the six-to-eight month follow-up point, as measured by the Arnold Parenting Scale.

5.3.4 Universal-targeted intervention nexus

The SDQt measure was completed by teachers on all K–2 children with consent to include scores,

both at the baseline/screening phase and soon after completion of the targeted group intervention.

The SDQ results reported above relate to children in the targeted group program only. There was


167

also the opportunity to examine the results for the targeted group in comparison to the wider group

of K–2 children. Whilst the mean reduction in conduct scores was greater for the children in the

targeted group program than for other children, there was also a reduction in the mean SDQt total

and SDQt conduct scores for K–2 children overall. The Mann-Whitney test showed no significant

difference between those in the targeted groups and those in the universal program on

improvement in SDQt scores (total and conduct). The ability to draw conclusions about the impact of

the intervention from this finding is, however, limited. The K–2 comparison group was not a control

group, as allocation was not randomised, there was a universal program that varied from site to site

and other variables were not controlled.

A sample of K–2 children with elevated conduct scores who did not take part in the targeted group

program was taken as a comparison group to those in the targeted program phone (n=33, 10% of

sub-group of children with elevated scores). Parents/carers from this group were contacted by

phone to complete the SDQp post-intervention measure. As for the SDQt, no significant differences

were found between the children in the targeted group program and those who were not in terms

of improvement in scores. Whilst for this sample there was some control for level of behaviour

problems, the sample size for the comparison group was small and a range of other variables were

not controlled. A research design with random selection of sites, schools, classes and participants,

random allocation to groups and control for relevant variables would be needed in order to draw

strong conclusions regarding the changes for the participants in the targeted group program, those

in a universal program and those who did not undergo an intervention.

In the real-world context of implementing a pilot program in three unique settings, the potential to

produce findings on outcomes from the universal program has been limited. Each site took a

different approach to the universal intervention. Within sites, schools have also taken on different

aspects of the universal program and on different time schedules. Organisational and funding delays

in the Dubbo site meant that the universal component had only just commenced at the second data

collection point. Some schools in this site had not yet begun to implement the classroom program

and other schools were only a week or two into the program. Whilst the demographics and

contextual factors vary considerably between the three sites, this situation did provide us with the

opportunity to compare scores for K–2 children in Dubbo site schools where there had effectively

been no universal intervention and children in the other two sites. In addition, the other two sites

had taken different approaches to the universal intervention. In Newcastle, teachers were trained in

the Fun Friends program that was implemented in all K–2 classes, whilst in Mount Druitt a series of

teacher seminars were tailored and delivered by the Got It! team. Analysis of SDQ scores at the two

data collection points (baseline and post-targeted intervention) provided some comparison between

the three sites.

The Kruskal-Wallis and ANOVA tests were used to see if any of the sites differed from baseline to

post-intervention with respect to the average scores on the SDQp and SDQt. For the SDQp, no

significant differences were found between the sites for either the total or conduct sub-scale

average scores. This was not surprising given the small samples involved. Difference between the

sites was, however, found with respect to SDQt, for both total and conduct average scores. The

Mann-Whitney test was conducted for each pair of sites in order to establish where the difference

lay. Both Newcastle and Mount Druitt were found to have had a greater reduction (improvement)

on both the SDQt total and conduct scores than Dubbo (p<0.001) for K–2 children. No significant


168

difference was found between Newcastle and Mount Druitt. Mean ranks for sites where significant

differences were found are presented in Table 20.

In the absence of controls for other variables, it is not possible to conclude that it was the universal

program that impacted on the different outcomes between the three sites. It is worthy of note,

however, that the one site that had not yet implemented the universal program was the site that

had a significantly poorer result, in terms of difference in scores between the two data collection

points for K–2 children.

Table 20: Site comparisons: differences in K–2 SDQ scores between two data collection points

(Post-baseline with lower score representing improvement.)

Site n = Test Significance difference

Mean Rank

SDQt Conduct

Dubbo Mt Druitt

207 373

Mann-Whitney

Yes p<0.001

327.26 270.04

Dubbo Newcastle

207 393

Mann-Whitney

Yes p<0.001

333.29 283.23

Mt Druitt Newcastle

373 393

Mann-Whitney

No NA

SDQt Total

Dubbo Mt Druitt

207 373

Mann-Whitney

Yes p<0.001

330.65 268.22

Dubbo Newcastle

207 393

Mann-Whitney

Yes p<0.001

336.32 281.63

Mt Druitt Newcastle

373 393

Mann-Whitney

No NA

SDQp Conduct

Dubbo Mt Druitt

24 25

Mann-Whitney

No NA

Dubbo Newcastle

24 33

Mann-Whitney

No NA

Mt Druitt Newcastle

25 24

Mann-Whitney

No NA

SDQp Total

Dubbo Mt Druitt

24 25

ANOVA No NA

Dubbo Newcastle

24 33

ANOVA No NA

Mt Druitt Newcastle

25 33

ANOVA No NA

Note: Significance level was adjusted for the multiple tests performed on the data.

Whilst the universal program may assist children who are not in the targeted group program, it also

has a potential role to play in engaging families in the targeted program. By presenting a whole-

school approach and a positive awareness of the Got It! program, it is likely that families in need will

be more inclined to take part in the targeted group program. This was indicated in process

evaluation findings.

The Newcastle site had the most families who completed the targeted groups, but when elevated

SDQt conduct scores are used as an indicator of level of need, the proportions participating in the

targeted groups are comparable across the three sites. Details are provided in Table 21.


169

Table 21: Targeted group completions as proportions of K–2

No. screened

No. in baseline sample

No. (% of sample) with elevated conduct

scores (Abnormal or

Borderline band on SDQt conduct*)

No. completed

targeted group

Group completions

as % of children screened

Group completions

as % of elevated

scores

Dubbo

239 228 46 (20.1%) 18 7.5% 39.1%

Mount Druitt

541 423 32 (7.6%) 18 3.3% 56.3%

Newcastle

423 410 44 (10.7%) 27 6.4% 61.4%

*This is one indicator of level of need for targeted group program, but other criteria are also used in selecting families for group program.

Key findings

Whilst significant differences in SDQ scores were not found between the targeted group and other K–2 children following the intervention, all trends were in a positive direction. Together with the single sample T-test results, this provides some support for the impact of the targeted program particularly in light of its pilot status, the early phase of implementation and limitations in study design

K–2 children in schools in the Newcastle and Mount Druitt sites had significantly greater levels of improvement on SDQt scores at the second data collection point than was found in the Dubbo site. The Dubbo site schools had not implemented the universal intervention by the time of the second data collection. Within the limitations of the study design, this could be taken as tentative support for the impact of the universal program in K–2 classes.

The Newcastle site had the highest number of participants in the targeted program, but the three sites were comparable in terms of targeted group participants as a proportion of total number screened and as a proportion of children with elevated SDQt conduct scores.

5.4 Findings: qualitative and quantitative perspectives on outcomes

The outcome evaluation questions regarding professional development outcomes for teachers and

the impact of the Got It! program for parents/carers who are not in the targeted groups (evaluation

questions 8 & 9 above) have largely been addressed in the Process Evaluation report. There were no

suitable standardised outcome measures to address these questions; hence, survey and qualitative

approaches have been employed to examine the impact of the Got It! program for teachers and

parents in the wider school community.

In the online survey of Health and Education staff involved with Got It!, respondents were asked to

rate the degree to which the Got It! program was observed to have made a positive change in the

following groups: K–2 children; children in targeted groups; parents/carers in groups; parents/carers

across school; and teachers. Results presented in Figure 20 are encouraging in that the majority of

respondents see positive change occurring for all groups, with the exception of ‘parents across the


170

wider school community’. As reported in the Process Evaluation, professional development for

teachers is well regarded and Got It! teams have been responsive to feedback in continuing to shape

professional development sessions so that they respond more effectively to school requirements.

Survey results presented in Figure 20 indicate that the vast majority of staff are observing changes in

teachers as a result of the Got It! program. This professional development for teachers is

fundamental to the implementation of the universal social-emotional learning program delivered by

them to K–2 children, where positive changes are also being observed by respondents.

Figure 20: Health and Education staff ratings of change in key groups following Got It!

(n=72 Online Survey respondents)

It is for the parents across the wider school community that most Health and Education staff

respondents either saw no changes or did not know of any changes. Hence, from the perspectives of

staff involved with Got It!, it seems that it is the parent information campaign that is having the least

observable impact. Comments added to the online survey indicate, however, that respondents are

modest in their expectations of the impact that Got It! could make on the wider school community.

Interviews with parents did, however, show that some assistance was being provided to families

facing challenging child behaviour. Whilst the universal program would not be expected to have as

big an impact on parents who are not in the targeted groups, the following parent comments reveal

that some impact was experienced:


171


try not to lose my cool with them. I try to slow down and think.

It [parent seminar on ‘resilient kids’] makes me think about how I’m addressing issues … I lose

my temper too quickly … I coach the kids a bit … Do things together more now.

I love the items in the newsletter … I like doing the parents’ homework [for Fun Friends] … It’s

all learning.

I use the techniques suggested in the newsletter information … maybe some things from the

interview … talking about emotions. I hadn’t really thought about that before.

Within the constraints of varied school contexts and uncontrolled variables, which were unavoidable

parts of the real-world nature of the evaluation, the measurement of outcomes from the universal

intervention program for all K–2 children has been limited. This is not to say that there is no

evidence of impact. The Process Evaluation and qualitative data are useful to refer to in order to

appreciate the impact that the universal program can have. The profile of the Got It! program within

the schools was clearly a means to engaging families with the targeted program, who would not

otherwise have sought out assistance. The program was generally regarded in a positive light and as

a desirable program within the school. Parents spoke about disappointment in not getting selected

and in lobbying to be selected. Such an engagement with a targeted intervention program is unlikely

to have occurred in the absence of a universal program.

I pushed to be selected. In fact, I really went proactive about it … really showed my interest to

the teacher and said, ‘I’d really like X to go for this. I think he would really benefit.’ (Parent)

It is also useful to revisit at this point the qualitative perspectives on outcomes provided by parents

who participated in the targeted group program. Whilst the outcome measures reported here in the

Outcome Evaluation indicate that the targeted program has led to some measurable changes, the

comments from some of the parents in the targeted groups further emphasise the extent of this

impact on family life. For example:

Still occasionally has emotional outbursts and we try to work the ‘Stop, Think, Do’ steps. But



of myself and my child. My daughter often says she wants to go back to Got It!. She really loved

it. It helped us both. Thank you very much.

I’ve just learnt to be more consistent with him and not give in so easily. Because it’s what I used

to do was give in too easily and just cave in to him a lot when he used to cry … But they’ve

taught me now like you’ve got to stick with it and not cave in. But then if you come to the point

where you’re feeling like caving in, walk away for five minutes, 10 minutes and then go back

and try and do it again.

It’s made me change too in how I go about things … We do more as a family because their

behaviour is good … and I’m thinking about their feelings more too … It helped me to help him.

I haven’t even had a phone call from the school to say that he’s acting up.


172

I used to speak to him too much … now I’ve just got to keep it short and to the point so now he

gets it more.

Implementing the idea of him having a choice about how he behaves. So trying to get him to

realise well if he chooses to react this way then these will be the consequences. If you choose to

do this, then this is what can happen … Some days are good, some days are not so good but it’s

positive steps to be taking.





how to change myself. How to behave … We need to stop and just think, ‘Why are we fighting?’

Where before I’d just go, ‘Oh look, just shut up’. Like, you know, ‘Stop it, Stop!’… It’s just

changed my way of thinking.


can get them before the problems get too bad.

It’s changed our whole life … He can control his anger … He has strategies and we have

strategies … The harmony in our house is amazing. We feel like a normal family now … The Got

It! program was the single one thing that made the change.

Whilst the children who attended the program are young, they too were able to share with the

evaluation team some of the changes they thought that Got It! had provided for them, as the

following comments from group participants indicate:

It’s made me feel better and learn more about feelings.

When I first started, I was a little bit silly, but getting back to it from going to the Got It!

program, that’s making me feel very good and I’m starting to get happier, and if I get happier I

get more work done and I get to play more with people and I get to be nicer. It makes me feel

very nice.

Comments such as these add weight to the findings from the outcome measures. It is useful to

consider both outcome and process components of the evaluation together to both enhance and

explain findings. For example, our baseline data indicate a weak relationship between parenting

practice measures and child behaviour measures, which is in contrast to prior research in the field

where a much stronger relationship has been observed. Qualitative findings from the process

evaluation, however, suggest that some parents felt threatened by a perception that their parenting

practices were being scrutinised at the school and that there may be imposed consequences. It

could be speculated that such concerns have influenced parents to answer questions in a way that

places them in a more positive light. This is a possible explanation for the results in relation to

parenting practices.

When the results of the Outcome Evaluation are considered alongside the findings from the Process

Evaluation, it can be reasonably concluded that the Got It! pilot program has demonstrated the

capacity to produce positive outcomes through a combination of universal and targeted


173

components. The targeted component is reaching families with children with elevated behaviour

problems and post-intervention improvements in both parenting practices and child behaviour are

evident. The positive changes in child behaviours were found to be maintained at the six-to-eight

month follow-up point for the majority of children. Considering these findings in light of the pilot

status of the program, and with attention to the strategies and principles for future development

that have emerged from the evaluation, it is proposed that Got It! could play an effective role in

reducing disruptive behaviours and conduct disorder in the future.


174

6. Economic Evaluation

In order to quantify the costs and potential benefits of the Got It! program, findings from the

Literature Review, Process Evaluation and Outcome evaluation have been drawn upon. Costs and

benefits associated with delivery of the program across the three pilot sites for six months in 2013

have been calculated for the economic analysis. Net economic gains were calculated for a base case

and a best case to estimate the overall return on investment in the Got It! program. Benefits (in cost

savings) were allocated and discounted over a 30-year period to calculate Net Present Value (NPV)

and benefit–cost ratio (BCR) for the two scenarios.

The following components of Got It! for the six-month period were included in the analysis:

Teacher training / consultation by mental health clinicians for 85 K–2 teachers.

Universal programs for 1,627 K–2 children and their families. (Note: Not all children/families

participated and the nature of the interventions varied between sites. Components included

classroom program delivered by teachers, seminars for parents, parenting information

campaign.)

Screening of 1,213 K–2 children for elevated conduct problems using SDQ (Strengths &

Difficulties Questionnaire).

156 assessment interviews for families with a child identified through screening as having

elevated behaviour problems to select for suitability for targeted group program .

12 x 10-week group programs completed by 63 children with elevated conducted problems,

each with a parent/carer also attending (one group program run in each of 12 schools).

(Note: an additional five children withdrew after commencing the group program.)

6.1 Program benefits

The economic benefits of the Got It! program are predominantly derived from long-term cost

savings in diverting children with early disruptive behaviours from developing conduct disorder. The

outcome data for the children in the targeted groups for the six-month evaluation period in 2013,

reported in the Outcome Evaluation, have been used to calculate the number of children likely to be

diverted from this trajectory. The figures are based on outcome data relating to the group of 63

children who were assessed as having elevated conduct problems and completed the targeted group

program with a parent or carer, within the context of a universal school program. The programs

were run in schools located in geographic areas of assessed need and where principals engaged and

supported the program. Any extrapolation from the economic data presented here needs to be

within the parameters of these inclusion criteria.

6.1.1 Benefits of preventing conduct disorder

The economic benefits of the Got It! program are predominantly in the avoidance of the costs

associated with defiant, aggressive and other behaviours linked to conduct disorder in the longer

term. In order to estimate such benefits, it is necessary to draw on research literature on the

patterns of behaviour associated with early conduct problems and with conduct disorder.


175

Around 5% of Australian children exhibit aggressive and defiant behaviours that could be given the

clinical diagnosis of ‘conduct disorder’. The median age of onset is around 11 years. Around twice

the number of boys as girls have conduct disorder (Bywater 2012, Foster et al. 2007, Hughes 2010,

Sawyer et al. 2000). There is wide agreement that if emerging conduct problems are not addressed

early, there is a strong chance that behaviours will develop into significant and costly longer-term

problems such as school failure, unemployment, substance abuse and criminality. Children with

early onset conduct problems tend to develop more severe behaviours in adolescence and

adulthood that are costly to individuals, families and society (Bywater 2012, Foster et al. 2007,

Hughes 2010, Murrihy et al. 2010, Raphael 2000, Sainsbury Centre for Mental Health 2009, Webster-

Stratton & Reid 2010).

The probability map in Figure 21 is based on the trends identified from research literature on

lifetime trajectories for children with early conduct/behavioural problems. The research evidence in

this field is limited; hence, a degree of uncertainty is inherent in the proportional allocations. The

allocations have however been informed by research findings and, on the basis of these,

conservative estimates made. Following is the summary of findings drawn upon in developing the

probability map in Figure 21.

It is estimated that 40% of children with early behaviour problems develop conduct disorder

(Hutchings et al. 2007).

When other factors are controlled, conduct disorder is a strong predictor of poor outcomes

in adulthood, including criminal behaviour, substance abuse, poor educational and

employment outcomes, mental illness, early pregnancy and relationship breakdowns (Bonin

et al. 2011, Bywater et al. 2009, Charles et al. 2011, Freidli & Parsonage 2007, Mihalopoulos

et al. 2007, Sainsbury Centre for Mental Health 2009, Scott et al. 2001, Thomas 2010).

A strong connection has been identified between criminal activity in adulthood and conduct

problems in childhood. Ninety per cent of repeat juvenile offenders were found to have

conduct disorder in childhood (Scott et al. 2001). An estimated 30% of crimes committed in

the UK have been attributed to those with conduct disorders in their childhood, with a

further 50% of crimes attributed to offenders exhibiting other conduct problems in

childhood (Sainsbury Centre for Mental Health 2009). Whilst it is not the case that all

children with conduct disorders go on to engage in criminal activity, it appears that

somewhere between 14% and 60% are later imprisoned (Sainsbury Centre for Mental Health

2009). Those children with early onset conduct disorder have been found to be two to three

times more likely to become chronic offenders in adolescence (Webster-Stratton & Reid

2010). For our model, it is assumed that 40% of children with conduct disorder will engage in

criminal behaviour.

A child with conduct disorder is four times more likely to have committed a violent criminal

offence by the age of 25 (Freidli & Parsonage 2007).

The current rate of school retention to Year 12 in NSW is 75% (ABS 2012). A child with

conduct disorder is, however, twice as likely to leave school or be unemployed before the

age of 18 (Fergusson & Horwood 1998).

As stated above, the relationship between risky substance abuse and conduct disorder has

been identified in international research. The individual and cultural factors contributing to

alcohol and drug abuse are, however, many and the reviewed research does not indicate the


176

degree to which substance abuse may be attributed to conduct disorder. A conservative

allocation for substance abuse has been made.

Likewise, mental illness, early pregnancy and relationship breakdown are outcomes

associated with conduct disorder. Due to insufficient research and costing data, these

additional cost impacts of conduct disorder are recognised by not costed in the economic

analysis.

Figure 21: Life trajectory probability map for children with emerging conduct problems

In order to reduce the costly behaviours associated with conduct disorder, the goal of the Got It!

intervention is to decrease the 40% proportion of children with emerging conduct problems who go

on develop conduct disorder (level 2 in Figure 21).

For the economic analysis, the avoided costs associated with conduct disorder are in three domains:

1. System and societal costs associated with criminal behaviour.

2. Productivity loss associated with not completing Year 12.

3. Health costs, productivity losses and burden of disease costs resulting from high-

risk substance abuse.

Mental illness, early pregnancy, relationship breakdown and the generational costs of poor

parenting associated with conduct disorder are acknowledged as contributing additional costs of

conduct disorder but, due to insufficient quantifiable data, these factors have not been costed in this

economic analysis.

100

5-8 year olds with

disruptive behaviours

40

Develop Conduct Disorder

16 engage in criminal behaviour

20 leave school pre Yr 12

Higher proportion engage in smoking, drug & alcohol abuse in adolescence & adulthood.

60

Do not develop Conduct Disorder

6 engage in criminal behaviour

15 leave school pre Yr 12


177

The lifetime costs for the three domains have been drawn from the Access Economics (2010) report

on benefits associated with Positive Family Functioning interventions. These values have been

adopted as the most current and relevant costing in the Australian health and community services

domain. Access Economics used a range of data sources to establish costs, including Australian

Institute of Health and Welfare, Australian Bureau of Statistics, National Health Survey, Productivity

Commission Report on Government Services and the Australian Institute of Criminology. The costs

published by Access Economics (2010) have been inflated here by 3% per annum to obtain 2013

values. The 2013 values used in the Got It! economic evaluation are provided in Table 22. The per

person costs for children with conduct disorder reflect the 2:1 males to females gender proportions

of children with behavioural problems and conduct disorder.

Table 22: Discounted lifetime costs per person of behaviours associated with childhood conduct

disorder (2013 dollars)

Cost type

Males Females Per person with CD

Criminal behaviour

543,267 467,626 518,053

Year 12 non-completion

104,754 60,059 89,856

Smoking, alcohol and drug abuse

846,540 850,787 847,956

6.1.2 Quantifying the benefits for six-month Got it! cohort

Benefits associated with the universal and targeted components of the Got It! program were valued

according to known and expected outcomes associated with a six-month intervention period in

2013. The universal benefits for K–2 teachers and students include the immediate educational and

skill development benefits. The universal program also has a role to play in engaging the families in

the targeted program. The benefits calculated below for the targeted group of children relate to the

outcomes from the program delivered in a universal context.

The benefits from the universal components of the program were calculated using the Willingness to

Pay (WTP) concept. The value given to teacher training within the Got It! program is based on the

2013 training charges of the Pathways organisation to train teachers in the Fun Friends program

(Pathways Health and Research Centre). This is the universal program that two of the Got It! teams

provide training in for teachers, through a ‘train the trainer’ agreement. The universal program to all

K–2 children and families (parenting information, seminars, classroom interventions etc.) has similar

features to other social-emotional skill development programs offered to schools (e.g. anti-bullying

programs). A comparable per child amount has been used for this component of the program. The

benefits associated with the universal components of Got It! are summarised in Table 23.


178

Table 23: Benefits from universal components of Got It! program using WTP values (2013 dollars)

WTP unit value

No. persons Total

Teacher training

600 85 51,000

K–2 children/families – education program

10 1,627 16,270

Total

67,270

Note: Same values relevant to Base Case and Best Case scenarios

The targeted group program for children with elevated conduct scores and their families is the

component of the Got It! program that is of greatest interest in potentially reducing long-term costs

of conduct disorder. Estimating the effect size of the targeted group program is the first step in

plotting a likely trajectory for participant children. There were 63 children who completed the

targeted group program.

As described in the Outcome Evaluation, a range of measures of child behaviours were taken prior to

the targeted group intervention and again on completion of the program. The changes in child

behaviour scores found using single sample T-tests are provided again in Table 24. (Note: Scores

were not available for all children.) As reported in the Outcome Evaluation, significant improvements

in child behaviour scores from pre- to post-intervention were found on the scales that most directly

measure disruptive behaviours in children (conduct subscales in SDQp & SDQt and the ECBI).

Findings indicate significant positive benefits from the Got It! program of up to 6.3% improvement in

scores. Given that the Got It! program is in the pilot phase of implementation, the significant results

from the single sample T-tests provide reasonable evidence for program impact. This is in the

absence of a control group, which would have enabled stronger conclusions to be drawn.

Table 24: Outcomes of Got It! Intervention (targeted group program): significant improvements in

scores

Measure Intervention group (n)

Difference in Means*

Post - Baseline

Standard deviation within

paired data

Standardised mean change

SDQt Conduct sub-scale

60

-0.40 p=0.001

2.4

-4.3%


49 -0.46 p=0.025

2.0

-6.3%

ECBI Intensity scale

Problem scale

45

31

-14.4 p=0.001

-3.1 p=0.04

24.7

7.9

-0.4%

-2.1%

* Decrease in score indicates improvement for all measures.

The SDQ and ECBI scales also place scores in one of three bands: Abnormal, Borderline and Normal.

As reported in the Outcome Evaluation, a child shifting into the Normal behaviour band is also a


179

useful indicator of intervention impact. As reported in tables 17 and 18 the proportions in the

Normal band were found to be higher post-intervention for each of the measures used. The increase

in proportion in the Normal band ranged from 8% to 24%, depending upon the measure, with up to

62% in the Normal band post-intervention. It was also found that between 18% and 33% of children

made a positive shift into the Normal or Borderline bands for behaviour following the targeted

group intervention, depending upon the measure. Whilst there were also some children who made a

negative shift in band (between 3% and 16%), and the majority had no shift in band, this should be

considered in the context of the overall significant improvement in scores described above.

Movement into the Normal range on the ECBI for around one-third of the children matches the

findings of Bonin and colleagues (2011), who reviewed international research on outcomes of

parenting intervention programs for child conduct problems. They found an average rate of 34%

reduction in clinical cases of conduct disorder from pre to post-intervention. On the basis of the Got

It! intervention outcome measures, and further informed by this international research, an

assumption has been made for the economic analysis that 22 of the 63 children (35%) from the

targeted groups in this six-month Got It! cohort have moved into or have been supported to stay in

the Normal band as a result of the intervention program.

A limitation of the outcome evaluation of the Got It! program, and of the vast majority of

evaluations of parenting programs, is that evidence regarding the sustainability of intervention

outcomes is very limited. As reported in the Outcome Evaluation for the Got It! cohort, post-

intervention scores were found to have been maintained for around 85% of children at the six-to-

eight month follow-up point. The longer-term outcomes are, however, unknown. For the purpose of

this economic evaluation, a best case scenario with 10% recidivism has been adopted and a base

case scenario with 50% recidivism to original level of behaviour problems. Thus, for the best case it is

assumed that 20 children who shifted into, or were maintained in, the Normal band for behaviour

following the Got It! intervention will continue along this track. For the base case, the assumption is

that 11 of 22 children will continue in the normal behaviour range.

As depicted in the probability map in Figure 21, 60% of the children with early conduct problems

would not go on to develop conduct disorder even if there were no intervention. Therefore, in the

absence of other predictive capacity, it is assumed for 40% of this Got It! cohort, ongoing

improvement can be attributed to the Got It! intervention. This has been applied to both the best

case and the base case scenarios. From this 40% diverted from developing conduct disorder, not all

would engage in the costly behaviours associated with the disorder. For the purpose of the

economic analysis, it is assumed that 25% of those who would have developed conduct disorder are

consequently diverted from each of the costed behaviours: criminality, early school leaving and

substance abuse.

These proportional allocations to base and best cases are summarised and applied to the six-month

Got It! cohort number in Table 25.


180

Table 25: Diversions from long-term behaviours associated with conduct disorder: base case and

best case scenarios

Base Case

Best Case

% impacted

Got It! 6 month cohort N=63

no. impacted*

% impacted

Got It! 6 month cohort N=63

no. impacted*

Improve due to Got It! program

35% ⇓

22 35% ⇓

22

Sustain improvements

50% ⇓

11 90% ⇓

20

Would otherwise have developed Conduct Disorder

40% ⇓

4 40% ⇓

8

Diverted from criminal behaviour

25%

25%

25%

1 25%

25%

25%

2

Stay at school to Year 12

1 2

Diverted from high risk substance abuse

1 2

*Rounded to whole no. of persons

The estimated long-term benefits of the six-month Got It! targeted group program, in terms of

avoided costs of conduct disorder, are based on the cost allocations from Table 22 and the

proportional allocations for base case and best case scenarios in Table 25. The associated costs are

presented below in Table 26.

Table 26: Estimated benefits from six-month Got It! targeted program gained by diverting children

from behaviours associated with conduct disorder (2013 dollars)

Lifetime costs averted per person by avoiding CD

Base Case Best Case

Criminal behaviour

518,053 518,053 1,036,106

Year 12 non-completion 89,856

89,856 179,712

Smoking, alcohol and drug abuse 847,956

847,956 1,695,912

Total

1,455,865 2,911,730


181

6.2 Program costs

The Got It! program cost used for the analysis is the actual expenditure on the program by each of

the participant organisations. The Ministry of Health allocated funding to three LHDs to implement

the pilot in three sites. Actual expenditure on the program during 2013 has been obtained from each

LHD for the period corresponding to the outcome data collection period. In addition, NSW Ministry

of Health provided allocations to DEC regions relating to each pilot site. This funding was in turn

allocated to schools to release teachers to participate in the Got It! program. Again, actual

expenditure reports were obtained from DEC for the relevant period in 2013. Finally, there is a state

administration budget for the overall monitoring and support of the program across the sites,

including ongoing evaluation and program development forums. This is provided by the Mental

Health – Children and Young People unit in the Ministry of Health. The costs included in the analysis

relate to the components that would continue beyond the establishment phase, if Got It! were to be

ongoing (forums, resources, quality assurance and process evaluation). Program delivery costs for six

months of program delivery during 2013 are provided in Table 27.

Table 27: Expenditure on Got It! program for six months in 2013

Site 1 Site 2 Site 3 Total

LHDs

377,731 360,826 171,452 910,009

DEC Regions*

49,275 32,869 23,366 105,510

State Admin**

NA NA NA 61,794

Total 427,006 393,695 194,818 1,077,313 Notes: * Actual expenditure (not allocated), reflecting under-spend in some regions.

** Based on 2103 expenditure, but total only includes anticipated state admin contributions if program were to be ongoing.

The total six month program expenditure of $1,077,313 financed Got It! program delivery to twelve

schools, incorporating a targeted group program for 63 families, a universal program targeting 1627

K-2 children, associated parenting information sessions and professional development for 85

teachers.

Another cost associated with the Got It! program relates to the flow-on use of specialist health and

family support services likely to follow on from early engagement with services through the

program. Referrals to follow-up services were made for a number of families participating in the

group programs during the evaluation period. Data on the proportion of families that took up these

referrals is not available. These ongoing service costs may, however, be offset by reduced usage of

mental health and other support services in the future resulting from the diversion from developing

conduct disorder. For the purpose of the economic evaluation a cost-neutral assumption regarding

specialist service usage is made.


182

6.3 Benefit–cost analysis

As detailed above, the Got it! program outcome data, together with findings from international

research, were used to propose a 35% improvement rate for children screened as having elevated

conduct problems and who completed the targeted Got It! group program with a parent.

International research and national economic data was also drawn upon to calculate potential

savings associated with avoiding criminal, early school leaving and substance abuse behaviours as a

result of the early intervention diverting children from developing conduct disorder. An incidence

approach has been taken to calculate the number of children, from a six-month cohort, who may

benefit in the long term. Base case and best case scenarios are provided. Benefits from the universal

components of the Got It! program were valued using a WTP approach with comparable products.

Actual expenditure for delivery of the Got It! program over six months in 2013 were used to cost the

intervention.

Within this framework of assumptions for the economic analysis, net economic gains were

calculated for the base case and best case to estimate the overall return on investment in the Got It!

program under these different scenarios. Benefits have been allocated over a 30-year period, with

criminality and substance abuse diversion allocations commencing seven years post-program and

Year 12 completion productivity allocations from 11 years post-program. The government standard

7% annual discount rate was used to calculate Net Present Value (NPV) in 2013 dollars, with 4% and

10% discount rates also calculated for sensitivity analysis. Spreadsheets with allocations and

calculations are included in the appendix. A summary of costs and benefits is provided in Table 28.

The NPV and benefit–cost ratio (BCR) are measures of social and economic savings or losses

attributable to an intervention in current dollar values. A positive NPV indicates that the benefits of

the program outweigh the costs and a BCR of greater than 1 indicates that program benefits

outweigh costs.

Table 28: Summary of modelled costs and benefits of six-month delivery of Got It! program across

three pilot sites (2013 $ prices)

Base case Best case

Program costs

1,077,313

1,077,313

Universal program benefits WTP

67,270

67,270

Targeted program Lifetime benefits

1,455,865

2,911,730

Net Present Value (NPV)

-277,250 (4%) -547,450 (7%)

446,318 (4%) -90,286 (7%)

Benefit–Cost Ratio (BCR)

0.74 (4%) 0.49 (7%)

1.41 (4%) 0.92 (7%)


183

On the basis of the conservative estimates used for this BCR analysis, the best case scenario (10%

recidivism) shows that the Got It! program would be a positive return on investment at a 4% annual

discount rate and comes very close to meeting upfront costs at a 7% discount rate (BCR = 0.92). For

the base case scenario (50% recidivism), a net loss for the program is indicated for each discount

rate, but is still a reasonable result in light of the pilot status of the program with BCR at 0.74 for 4%

discount rate.

It should again be acknowledged that certain potential benefits (e.g. avoiding early pregnancy and

generational parenting impacts) have not been included in the costing model. There are also the

likely flow-on effects of changes in parenting approach for other children in the family. These

potential benefits of the Got It! program have not been given dollar values but would strengthen the

bottom line in providing further value for money.

The BCR results also need to be considered in the context of the program being a pilot. With

attention to quality improvements over time, informed by the Process Evaluation, and with

monitoring of behaviour recidivism rates in the medium term, the program has the potential to offer

both value for money and the stimulus for changing the trajectory of young people’s lives. As an

early intervention strategy, Got It! focuses on the prevention of a specific and costly mental health

disorder. Findings from this evaluation indicate that the program has the potential to be of both

social and economic value.

Key findings

A best case scenario (10% recidivism) produced a positive NPV ($446,000, in 2013 dollars) at 4% p.a. discounting.

At 7% discounting for the best case scenario, the result came close to meeting upfront costs (BCR = 0.92).

The base case scenario (50% recidivism) did not produce a positive NPV (BCR = 0.74 at 4% discount rate).

In light of the pilot status of the program and the inability to put a dollar value on additional expected social benefits, results support an expectation that the Got It! program would provide value for money in the long term through diverting children from the costly behaviours associated with conduct disorder.


184

7. Conclusions

The Got It! program has developed in response to the growing body of research evidence on the

effectiveness of early intervention programs for children with disruptive behaviour problems.

Findings from this evaluation add to that body of research. The potential to divert children from the

path of conduct disorder and consequent costly anti-social behaviours in adulthood is promising;

however, sustainability of outcomes into adulthood remains an important issue for ongoing

research. The Got It! model of care builds on the positive outcomes that are associated with early

intervention, universal access, and the promotion of positive relationships between child, family and

school systems. As an early intervention program for K–2 children, the targeted component of Got

It! seeks to address emerging behaviour problems at a time when intervention is likely to be most

effective and in a context where children and families can be actively engaged within the school

setting.

The data presented here has illustrated the experiences of Got It! from the perspectives of the range

of stakeholders: children, families, schools, health staff and external organisations. The partnership

and organisational arrangements that have been set in place during the pilot phase offer a solid

basis for the development of the Got It! program. With appropriate leadership and a collaborative

developmental approach, Got It! could be strengthened to better address the requirements of

stakeholders by increasing the reach of the targeted group program, engaging more effectively with

families from culturally and linguistically diverse backgrounds and enhancing supported referral and

community networking processes.

The information gathered from direct contact with parents, carers and children who had

participated in the Got It! program in 12 schools during the evaluation period contributed very

useful insights into experiences of those for whom the program is designed to impact. The targeted

group program has made a substantial impact. All parents who were interviewed could identify

changes in parenting practices that have been implemented as a result of the program. The groups

have also had a positive impact on parent-child relationships, social networks between parents

attending the groups, connections between parents and the school and experiences of

family/mental health services. These are all protective factors for children and families. The children

enjoyed attending the groups and participated in social-emotional learning activities. The Outcome

Evaluation offers evidence for measurable changes in child and parent behaviours following the

targeted group program.

The impact of the program for families with a child with elevated conduct scores, but who did not

participate in the targeted group program, is lower. This is understandable given that the attention

and resources directed to the targeted groups are higher than other components in the Got It!

model of care. What is important, however, is the finding that Got It! can have an impact and make

some difference for these families who are in need, but are not selected for the group program.

Engagement with the whole school through universal programs also provides an effective platform

for engaging families into the targeted group program, who otherwise would not seek out services.

These findings strengthen the position made throughout this report, that all components of the Got

It! model of care should be attended to for effective outcomes.


185

The qualitative, process and outcome data presented in this report highlight successes of the

program and inform a range of considerations for the future development of the Got It! program, if

it is funded into the future. Whilst the targeted group programs have been well established during

the pilot phase, some other components of the model of care could be developed for greater

effectiveness and wider impact. It is also proposed that the Got It! model of care be developed as a

four-stage model in order to focus more directly on the important phases of engagement and

follow-through, alongside assessment and intervention.

When the results of the Outcome Evaluation are considered alongside the findings from the Process

Evaluation, it can be reasonably concluded that the Got It! pilot program has demonstrated the

capacity to produce positive outcomes through a combination of universal and targeted

components. The targeted component is reaching families with children with elevated behaviour

problems and post-intervention improvements in both parenting practices and child behaviour are

evident. The positive changes in child behaviours were found to be maintained at the six-to-eight

month follow-up point for the majority of children. Considering these findings in light of the pilot

status of the program, and with attention to the strategies and principles for future development

that have emerged from the evaluation, it is proposed that Got It! could play an effective role in

reducing disruptive behaviours and conduct disorder in the future.

The results of the Economic Evaluation indicate that on the basis of available financial and

intervention outcome data, and in the context of findings from international research, there can be

optimism that the Got It! program would provide value for money in the long term through diverting

children from the costly behaviours associated with conduct disorder. The best case scenario

produced a positive NPV ($446,000, in 2013 dollars) at 4% p.a. discounting. A break-even result at

7% was very close (BCR = 0.92), but the base case scenario did not produce a positive NPV (BCR =

0.74 at 4% discounting). This result is strengthened with recognition of additional likely benefits that

have not been given a dollar value in the analysis (e.g. flow-on effects for other children in the

family, avoiding early pregnancy, generational parenting impacts). The results of the economic

analysis are promising in light of the pilot status of the program.

With attention to ongoing evaluation and quality improvements, the program has the potential to

offer both value for money and the stimulus for changing the trajectory of young people’s lives. As

an early intervention strategy, Got It! focuses on the prevention of a specific and costly mental

health disorder. Findings from this evaluation indicate that the program has the potential to be of

both social and economic value.


186

References

Access Economics (2010). Positive Family Functioning. Final Report for Department of Families,

Housing, Community Services and Indigenous Affairs. Canberra.

American Psychiatric Association (2013). Diagnostic and Statistical Manual of Mental Disorders 5th

Edition: DSM-5. American Psychiatric Publishing, Washington.

Arnold, D. O’Leary, S. Wolff, L. & Acker, M. (1993). The Parenting Scale: A Measure of Dysfunctional

Parenting in Discipline Situations. Psychological Assessments, 5: 137–144.

Australian Bureau of Statistics (ABS) (2012). Australian Social Trends: Data Cube – Education and

training. Cat. No. 4102.0

Australian Bureau of Statistics. (2013a). Quickstats. Accessed 09/06/2013 from:

http://www.abs.gov.au/websitedbs/censushome.nsf/home/quickstats?opendocument&nav

pos=220

Australian Bureau of Statistics. (2013b). Census of Population and Housing: Socio-Economic Indexes

for Areas (SEIFA), Australia, 2011. Accessed 09/06/2013 from:

http://www.abs.gov.au/ausstats/[email protected]/Lookup/2033.0.55.001main+features100162011

Australian Government (2013). A Snapshot of Early Childhood Development in Australia 2012. AEDI

National Report. Australian Government. Canberra. http://www.rch.org.au/aedi/

Baker-Henningham, H., Scott, S., Jones, K., Walker, S. (2012). Reducing child conduct problems and

promoting social skills in a middle income country: cluster randomised controlled trial. The

British Journal of Psychiatry, 201:101–108.

Beauchaine, T. P., Hinshaw, S. P., & Pang, K.L. (2010). Co-morbidity of attention-deficit/hyperactivity

disorder and early onset conduct disorder: Biological, environmental and developmental

mechanisms. Clinical Psychology, 17(4): 327–336.

Bonin, E., Stevens, M., Beecham, J., Byford, S. & Parsonage, M. (2011). Costs and longer term savings

of parenting programs for the prevention of persistent conduct disorder: A modelling study.

BMC Public Health, 11:803.

Brann, P., Corby, D., Costin, J., McDonald, J., Hayes, L. & Turner, M. (2007). An evaluation of an early

intervention approach to disruptive behaviours in primary school children. Eastern Health

Mental Health Program, Victoria.

Bywater, T., Hutchings, J. Daley, D. Whitaker, C., Yeo, S. T., Jones, K., Eames, C. & Edwards, R. T.

(2009). Long term effectiveness of parent interventions for children at risk of developing

conduct disorder. The British Journal of Psychiatry, 195:318–324.

Bywater, T., Hutchings, J., Linck, P., Whitaker, C., Daley, D., Yeo, S. T., & Edwards, R. T. (2011).

Incredible Years parenting training support for foster carers in Wales: A multi-centre

feasibility study. Child: Care, Health & Development, 37(2): 233–243.

Bywater, T. J. (2012). Perspectives on the Incredible Years Programme: psychological management

of conduct disorder. British Journal of Psychiatry, 201: 85–87.

http://www.abs.gov.au/websitedbs/censushome.nsf/home/quickstats?opendocument&navpos=220

http://www.abs.gov.au/websitedbs/censushome.nsf/home/quickstats?opendocument&navpos=220

http://www.abs.gov.au/ausstats/[email protected]/Lookup/2033.0.55.001main+features100162011

http://www.rch.org.au/aedi/


187

Charles, J. M., Bywater, T. & Edwards, R. T. (2011). Parenting Interventions: A Systematic Review of

the Economic Evidence. Child: Care, Health & Development, 37(4): 462–474.

Council of Australian Governments. (2009). Protecting Children is Everyone’s Business: National

Framework for Protecting Australia’s Children: 2009-2020. Commonwealth of Australia,

Attorney General’s Department. Canberra.

Department of Education and Communities (2013). Rural and Remote Education: Effective

Partnerships and Connections. Department of Education and Communities, November 2013.

https://www.det.nsw.edu.au/media/downloads/about-us/our-reforms/rural-and-remote-

education/randr-epac.pdf

Department of Health (Victoria), (n.d.). CAMHS and Schools Early Action Program: Program

Description. Unpublished document.

Dretzke J., Davenport C., Frew E., Barlow J., Stewart-Brown S., Bayliss S., Taylor R., Sandercock J., &

Hyde C. (2009). The clinical effectiveness of different parenting programs for children with

conduct problems: a systematic review of randomised controlled trials. Child and Adolescent

Psychiatry and Mental Health, 3: 7–16.

Duncombe, M., Havighurst, S., Holland, K. & Frankling, E. (2012). The Contributions of Parenting

Practices and Parent Emotion Factors in Children at Risk for Disruptive Behaviour Disorders.

Child Psychiatry and Human Development, doi:10.1007/s10578-012-0290-5.

Edwards, B. & Bromfield, L. (2009). Neighbourhood influences on young Children’s conduct problems

and pro-social behaviour: Evidence from an Australian national sample. Children and Youth

Services Review, 31(3): 317–324.

Edwards, R., O’Ceilleachair, A., Bywater, T., Hughes, D. & Hutchings, J. (2007). Parenting programme

for parents of children at risk of developing conduct disorder: cost effectiveness analysis.

British Medical Journal, doi:10.1136/bmj.39126.699421.55: 1–6.

Eyberg, S. (1998). Eyberg Child Behaviour Inventory. PAR, Lutz, Florida.

Eyberg, S. M. & Bussing, R. (2010). Parent-Child Interaction Therapy for preschool children with

conduct problems. Chapter 6 pp. 139–162 in R. C. Murrihy, A. D. Kidman & T. H. Ollendick

(eds) Clinical Handbook of Assessing and Treating Conduct Problems in Youth. Springer, New

York.

Eyberg, S. & Pincus, D. (1999). ECBI Eyberg Child Behavior Inventory SESBI-R Sutter-Eyberg Student

Behavior Inventory-Revised. PAR, Lutz, Florida.

Fergusson, D. M. & Horwood, L. J. (1998). Early Conduct Problems and later life opportunities.

Journal of Child Psychology and Psychiatry, 39(8): 1097 – 1108.

Foster, E. M., Olchowski, A. E., Webster-Stratton, C. H. (2007). Is stacking Intervention Components

Cost-Effective? An analysis of the Incredible Years Program. Journal of the American

Academy of Child & Adolescent Psychiatry, 46(11): 1414–1424.

Friedli, L. & Parsonage, M. (2007). Building an economic case for mental health promotion. Journal of

Public Mental Health, 6 (3): 14–23.

https://www.det.nsw.edu.au/media/downloads/about-us/our-reforms/rural-and-remote-education/randr-epac.pdf

https://www.det.nsw.edu.au/media/downloads/about-us/our-reforms/rural-and-remote-education/randr-epac.pdf


188

Furlong, M., McGilloway, S., Bywater, T., Hutchings, J., Smith, S. M. & Donnelley, M. (2012).

Behavioural and Cognitive Behavioural Group-Based Parenting Programmes for Early Onset

Conduct Problems in Children Aged 3 to 12 Years (Review). The Cochrane Collaboration, John

Wiley & Sons.

Gibbons, J. & Plath, D. (2012) “Single Session Social Work in Hospitals” Australian and New Zealand

Journal of Family Therapy. 33 (1): 39-53.

Gibbons, J. & Plath, D. (2009) “Single Contacts with Hospital Social Workers: the Clients’

Experiences” Social Work in Health Care, 48(8): 721-735.

Gibbons, J. & Plath, D. (2005) “Everybody puts a lot into it! Single Session Contacts in Hospital Social

Work” Social Work in Health Care, 42 (1): 17-34.

Goodman, R. (1997). The Strengths and Difficulties Questionnaire: A research note. Journal of Child

Psychology and Psychiatry, 38, 581-586.

Goodman R. (2001). Psychometric properties of the Strengths and Difficulties Questionnaire (SDQ).

Journal of the American Academy of Child and Adolescent Psychiatry, 40: 1337–1345.

Gowers, S. G., Harrington, R. C., Whitton, A., et al. (1999). Health of the Nation Outcome Scale for

Children and Adolescents (HoNOSCA): Glossary for HoNOSCA score sheet. British Journal of

Psychiatry, 174: 428–433.

Gray, M., Joy, E., Plath, D. & Webb, S. (2013) “Implementing Evidence-Based Practice: A Review of

the Empirical Research Literature" Research on Social Work Practice, 23 (2): 157-166.

Hemphill, S. A. & Littlefield, L. (2001). Evaluation of a short-term group therapy program for children

with behaviour problems and their parents. Behaviour Research and Therapy, 39: 823-841.

Hilarski, C. (2007). Conduct Disorder. Chapter 5 pp. 97–116 in B. A. Thyer & J. S. Wodarski (eds)

Social Work in Mental Health. John Wiley & Sons, Hoboken, NJ.

Hughes, C. (2010). Conduct Disorder and antisocial behaviour in the under-5’s. Chapter 71 pp. 821–

827 in C. L. Cooper, J. Field, U. Goswami, R. Jenkins & B. J. Sahakian (eds). Mental Capital and

Wellbeing. Wiley-Blackwell, Chichester.

Hutchings, J., Bywater, T., Daley, D., Gardner, F., Whitaker, C., Jones, K., Eames, C. & Edwards, R.

(2007). Parenting interventions in Sure Start services for children at risk of developing

conduct disorder: pragmatic randomised controlled trial. BMJ online, doi:

10.1136/bmj.39126.620799.55

Jones, D., Dodge, K. A., Foster, E. M., Nix, R. & Conduct Problems Research Group (2002). Early

Identification of Children at Risk for Costly Mental Health Service Use. Prevention Science, 3

(4): 247–256.

Kazdin, A. E. (2010). Problem-Solving Skills Training and Parent Management Training for

Oppositional Defiant Disorder and Conduct Disorder. Chapter 14 pp. 211–225 in J. R. Weisz

& A. E. Kazdin (Eds) Evidence-Based Psychotherapies for Children and Adolescents (2nd

Edition).Guildford Press, New York.


189

Kazdin, A. & Wassell, G. (2000). Therapeutic Changes in Children, Parents, and Families Resulting

From Treatment of Children With Conduct Problems. Journal of the American Academy of

Child & Adolescent Psychiatry, 39(4): 414–420.

Keenan, K, Boeldt, D., Chen, D., Coyne, C., Radiah, D., Duax, J., Hart, K., Perrott, J., Strickland, J.,

Danis, B., Hill, C., Davis, S., Kampani, S. & Humphries, M. (2011). Predictive validity of DSM-IV

Oppositional Defiant and Conduct Disorders in clinically referred pre-schoolers. Journal of

Child Psychology and Psychiatry, 52 (1): 47–55.

KidsMatter (n.d.). Growing Healthy Minds. Australian Government, Department of Health and

Ageing. http://www.kidsmatter.edu.au/ accessed 15/10/2013.

Mihalopoulos, C., Sanders, M. R., Turner, K. M. T., Murphy-Brennan, M. & Carter, E. (2007). Does the

Triple P-Positive Parenting Program provide value for money? Australian and New Zealand

Journal of Psychiatry, 41: 239–246.

Mindful: Centre for Training & Research in Developmental Health (n.d.). Tuning in to Kids. University

of Melbourne. http://www.tuningintokids.org.au/ accessed 15/10/2013.

Morrison, M., Macdonald, G. & LeBlanc, T. (2000). Identifying conduct problems in young children:

Developmental Pathways and Risk Factors. International Social Work, 43 (4): 467–480.

Murrihy, R. C., Kidman, A. D. & Ollendick, T. H. (2010). (eds) Clinical Handbook of Assessing and

Treating Conduct Problems in Youth. Springer, New York.

NSW Government, Department of Premier and Cabinet (2013). Keep Them Safe Interim Review.

Department of Premier and Cabinet, Sydney.

http://www.keepthemsafe.nsw.gov.au/__data/assets/pdf_file/0008/153818/Interim_Revie

w_Report.pdf accessed 28/10/2013.

NSW Ministry of Health (2011). Getting on Track in Time – Got It! Program Description and Model of

Care. NSW Ministry of Health. Unpublished internal document.

Pathways to Health and Research Centre (2013). Fun Friends Program.

http://pathwayshrc.com.au/fun-friends-4-7-years/ Accessed 09/06/2013.

Phal, K. M. & Barrett, P. M. (2010). Preventing Anxiety and Promoting Social and Emotional Strength

in Preschool children: A universal evaluation of the Fun FRIENDS program. Advances in

School Mental Health Promotion, 3 (3): 14–25.

Plath, D. (2012) “Implementing Evidence-Based Practice: An Organisational Perspective” British

Journal of Social Work. Published online 30 October 2012. doi: 10.1093/bjsw/bcs169

Raphael, B. (2000). Unmet need for prevention. Chapter 9 pp 132–145 in G. Andrews & S.

Henderson (eds) Unmet Need in Psychiatry. Cambridge University Press, Cambridge.

Sainsbury Centre for Mental Health (2009). The Chance of a Lifetime. Sainsbury Centre for Mental

Health, London.

Sawyer, M. G., Arney, F. M., Baghurst, P. A., Clark, J. J., Graetz, B. W., Kosky, R. J., Nurcombe, B.,

Patton, G. C, Prior, M. R., Raphael, B., Rey, J. Waites, L. C. and Zubrick, S. R. (2000). Mental

Health of Young People in Australia: Child & Adolescent component of the National Survey of

http://www.kidsmatter.edu.au/

http://www.tuningintokids.org.au/

http://www.keepthemsafe.nsw.gov.au/__data/assets/pdf_file/0008/153818/Interim_Review_Report.pdf

http://www.keepthemsafe.nsw.gov.au/__data/assets/pdf_file/0008/153818/Interim_Review_Report.pdf

http://pathwayshrc.com.au/fun-friends-4-7-years/


190

Mental Health and Well-being. Commonwealth Department of Health and Aged Care,

Canberra.

Scott, S., Knapp, M., Henderson, J. & Maughan, B. (2001). Financial cost of social exclusion: follow up

study of antisocial children into adulthood. British Medical Journal, 323: 1-5.

Scott, S., Sylva, K., Doolan, M., Price, J., Jacobs, B., Crook, C. & Landau, S. (2010). Randomised

controlled trial of parent groups for child antisocial behaviour targeting multiple risk factors:

The SPOKES project. Journal of Child Psychology and Psychiatry, 51 (1): 48–57.

Shelton, K., Frick, P. & Wootton, J. (1996). Assessment of Parenting Practice in Families of

Elementary School-Age Children. Journal of Clinical Child Psychology. 25(3): 317–329.

Thomas, C. R. (2010). Oppositional Defiant Disorder and Conduct Disorder. Chapter 16 pp. 223–239

in M. K. Dulcan (ed) Dulcan’s Textbook of Child and Adolescent Psychiatry. American

Psychiatric Publishing Inc., Washington.

Trentacosta, C. J. & Shaw, D. S. (2012). Preventing early conduct problems and later delinquency.

Chapter 20 pp. 309–322 in E. L. Grigorenko (ed) Handbook of Juvenile Forensic Psychology

and Psychiatry. Springer, New York.

VicHealth (2011). The Partnerships Analysis Tool. Carlton South. Victorian Health Promotion

Foundation. http://www.vichealth.vic.gov.au/Publications/VicHealth-General-

Publications/Partnerships-Analysis-Tool.aspx accessed 24/10/2013.

Waddell, C., Hua, J. M., Garland, O. M., Peters, R. D. & McEwan, K. (2007). Preventing Mental

Disorders in Children. Canadian Journal of Public Health, 91 (3): 166–173.

Wade, C., Macvean, M., Falkiner, J., Devine, B. & Mildon, R. (2012). Evidence Review: An analysis of

the evidence for parenting interventions in Australia. Parenting Research Centre, Melbourne.

Webster-Stratton, C. H. & Reid, M. J. (2010). The Incredible Years program for children from infancy

to pre-adolescence: Prevention and treatment of behaviour problems. Chapter 5 pp. 117–

138 in R. C. Murrihy, A. D. Kidman & T. H. Ollendick (eds) Clinical Handbook of Assessing and

Treating Conduct Problems in Youth. Springer, New York.

Woolgar, M. & Scott, S. (2005). Evidence Based Management of Conduct Disorders. Current Opinion

in Psychiatry. 18: 392–396.

Zisser, A. & Eyberg, S. (2010). Parent– Child Interaction Therapy and the Treatment of Disruptive

Behavior Disorders. Chapter 12 pp. 179–193 in J. R. Weisz, John R. & A. E. Kazdin (eds).

Evidence-Based Psychotherapies for Children and Adolescents (2nd Edition), Guilford Press,

New York.

Zubrick, S. R., Silburn, S. S., Burton, P. & Blair E. (2000). Mental Health Disorders in Children and

Young People: Scope, Causes and Prevention.

http://www.vichealth.vic.gov.au/Publications/VicHealth-General-Publications/Partnerships-Analysis-Tool.aspx

http://www.vichealth.vic.gov.au/Publications/VicHealth-General-Publications/Partnerships-Analysis-Tool.aspx


191

Glossary of terms and acronyms

AEDI Australian Early Development Index

ANOVA analysis of variance

APA American Psychiatric Association

APQ Alabama Parenting Questionnaire – (Shelton, Frick &

Wootton, 1996)

ATSI Aboriginal, Torres Strait Islander

BCR Benefit–Cost Ratio

CAMHS Child and Adolescent Mental Health Service

CASEA CAMHS and Schools Early Action program (Victoria)

DEC NSW Department of Education and Communities

DSM-V Diagnostic and Statistical Manual of Mental Disorders 5th Edition

ECBI Eyberg Child Behaviour Inventory (Eyberg, 1998)

EIPPP Early Intervention and Placement Prevention Program

FRS Family Referral Service

Got It! Getting on Track in Time – Got It! – Early intervention program to reduce conduct

problems in children

HAC Health Administration Corporation

HoNOSCA Health of the Nation Outcome Scale – Children and Adolescents (Gowers et al. 1999)

HREC Human Research Ethics Committee

IRSD Index of Relative Socio-Economic Disadvantage

K–2 Kindergarten to Year 2 (Early and Stage 1 of Primary School

education)

LGA local government area

LHD Local Health District (NSW Health)

MH-CYP Mental Health – Children and Young People. Policy and planning unit in NSW

Ministry of Health

MoH NSW government, Ministry of Health

NEAF National Ethics Application Form

NGO No-Government Organisation

NPV Net Present Value

PBL Positive Behaviour for Learning (DEC policy framework)

PS Parenting Scale (Arnold et al. 1993)

RCT Randomised Controlled Trial

SEIFA Socio-Economic Index For Areas

SERAP State Education Research Approval Process (DEC)

SDQp Strengths and Difficulties Questionnaire

SDQp Strengths and Difficulties Questionnaire – parent version (Goodman 2001)

SDQt Strengths and Difficulties Questionnaire – teacher version (Goodman 2001)

WTP Willingness To Pay


192

Appendices

Appendix 1: Data Collection Instruments

Parents/carers in targeted groups Interview Schedule

6-8 month follow up phone interview

Children in targeted groups Focus group schedule

Parents/carers not in targeted groups Phone survey questionnaire

School profile completed by Got It! teams Template

Process journal completed by Got It! teams Template

Got it! staff & managers Focus group schedule

Got it! staff & managers, MoH staff Interview schedule

Health & Education staff Online survey questionnaire

DEC staff Interview schedule

External & referral services Interview schedule


193

Appendix 1: Data Collection Instruments

Parent in targeted group – Interview Schedule

Conducted Post-Group

Introduction

Thank you for making the time to do this interview. Your feedback on the Got It! program is a very

important part of the evaluation. I’ve got a number of questions to ask you, but I’m really interested

to hear about your experiences in some detail, and in particular, any changes that you have noticed

for yourself and your child because of Got It!

Before we start, I’ll just remind you that whilst we may use some quotes and examples from what

you say in the interview in our evaluation report, we will make sure that there is no information

included that could identifying you or your child personally.

So that I can capture all the detail of what you say, I would like to record our discussion. Nobody will

listen to the recording apart from the researchers. This is just because I can’t take notes as quick as

you can talk!

Are you happy to go ahead now?

Questions

1. What aspects of the Got It! program had the biggest impact for you?

2. What did you learn from the Got It! program, if anything, about parenting and child

behaviour?

3. Were there changes that you made as a result of what you learnt?

4. What do you think your child learnt from the Got It! program?

5. What changes have you noticed in your child, if any, because of the program?

6. Had you tried to get help with parenting before the Got it! program?

7. Thinking back to a few months ago, how did you feel when you were contacted about

participating in the group?

8. What were you expecting from the Got It! group before you started?

9. Are there any new services or activities that you or your child is involved with since you

started Got It!?

How frequently do you/child attend?

For how long do you anticipate attending?

10. Do you have any suggestions on how the Got It! program could be improved?

11. Any other comments?

Thanks for taking part in the interview. Your answers are valuable.


194

Parent in targeted group – Phone Interview Schedule

Conducted 6-8 months Post-Group

1. What has been the ongoing impact of Got It! for you and your child?

2. Has your child’s behaviour improved or continued to improve after Got It!?

3. Overall, what has helped and what hasn’t helped with your child’s behaviour?

4. Reminder to complete the SDQ and Parenting Scale questionnaires that have been posted.


195

Children in Targeted Group - Focus Group Schedule

Conducted Post-Group

1-2 Facilitators + teacher present to deal with disciplinary issues if necessary.

The Got It! program is new. You were one of the first groups of kids to go to Got It! group. We’re here to find

out how it all went. If it was good, then maybe lots more kids in other schools will have the chance to go along

to a Got It! program like you. So, that’s why we want to talk to you about Got It!

Check that each child is happy to take part and that we have signed parental consent.

Explain use of audio-recorder

Rules: Respect what others have to say – don’t talk about or make fun of what others have said after

our talk today.

Activity 1:

You’ve been going to the Got It! group for a few weeks now.

I’d like you to think about what going to the Got It! group has been like for you. How you felt about being in

the group and what you learnt.

I’ve got some sheets here with lots of different faces. I’d like you to pick a face that is like how YOU feel about

Got It! (Go through sheet and pick out what some of the emotions may be)

Put a circle around one face (or if you really can’t decide, pick 2)

Activity 2:

Now we’re going to talk a little bit about:

What you learnt from going to Got It! group

If Got It! has made things different for you at school or home

St Lukes “Bears” cards are scattered on floor.

Pick a bear you think is a bit like you and tell me if you think Got It! has helped you with this.

E.g. Nervous bear – Sometimes I feel nervous when I have to talk to people, but I’ve learnt that if I smile and

am interested in other people, then they are generally nice to me.

So, everyone pick a bear that is something like you and you can tell me if you have learnt anything from the

Got It! group about that.

Ask clarifying & elaborating questions as required.

Activity 3:

Now we’re going to pretend that I’m an interviewer on the radio. Pretend that this is a radio station that lots

of kids listen to. These kids are at schools we’re they’re thinking of having Got It! groups just like the one

you’ve been going to.

So, I’ve decided to interview kids who’ve gone to a Got It! group so that these other kids listening to the radio

can find out what Got It! is all about.

You’re the experts on Got it! So, let’s pretend that I’m interviewing you on the radio. What do you think you

would say to the kids at the other school about Got It!?

What are the best about the Got it! groups?

What were the difficult things about the Got it! that they will have to be prepared for?

What would you tell them about the things that they will learn?

How could Got it be made better for this next school?

Closure

It was really great that you came along to our discussion today. Thanks – Here’s some cupcakes!!


196

Parents/carers not in targeted group (with children with A/B scores) - Phone Survey

Conducted Post-Intervention

Introduction

I’m phoning about the Got It! program that was run earlier this year in your child’s school.

My name is Debbie and I’m one of the researchers evaluating the Got It! program in schools across the state.

Currently we’re phoning parents & carers to find out about experiences of families in schools where Got It!

was run, but who were NOT part of the Got it! small group program.

Do you remember that you signed a consent form earlier this year to say that you were interested in

answering a phone survey?

Is this a convenient time to talk for 20 minutes or would you like me to call back at another time?

Reminder: Your comments are confidential and neither you nor your child’s name will be used in any

reporting.

Just in case you’re not aware, the Got It! program is run in schools and includes:

Information sessions on managing child behaviour presented to all K-2 parents;

Training for teachers to run sessions in K-2 classes on managing emotions and behaviours;

Small group program for selected children & their parents / carers.

Questions

1. When the Got It! program was running in your school, did you do any of the following …

Attend a presentation for all parents on managing child behaviour? ⃝

Speak with a teacher about possibly attending a small group program? ⃝

Receive a phone call from Got It! staff about possibly attending small group? ⃝

Attend an interview about possibly attending Got It! small group ⃝

Speak with your child or teacher about the Got It! or Fun Friends

program that is run in K-2 classes? ⃝

Other _______________________________________________________ ⃝

2. Are you aware of any impacts that the Got It! program had for your child?

Not aware of any impacts ⃝

Yes. Details _________________________________________________________

__________________________________________________________________


197

3. Did the Got It! program make any impact on you as a parent/carer?

No ⃝

Yes. Details _________________________________________________________

___________________________________________________________________

4. Are you aware of any impacts that the Got It! program had on the school community?

Not aware of any impacts ⃝

Yes. Details _________________________________________________________

___________________________________________________________________

5. You don’t need to give any details in response to the next question, just yes or no.

Does your child have any diagnosis that has an impact on their behaviour?

No ⃝

Yes ⃝

No SDQ

6. Do you or your chid receive services to assist with child behaviour, parenting or family

support?

Again, just Yes or No is needed.

No ⃝

Yes ⃝

No SDQ

7. Do you have any comments on how families were selected for the Got It! small group

program in your school?

8. Do you have any suggestions on how the Got It! program could be improved?

NOTE: If answer is YES to either question 3 or 4 - END HERE

Those are all of the questions I have – Thank you for taking the time to answer them and helping with

the evaluation of the Got It! program.

If answered NO to BOTH questions 3 & 4, continue on.

9. I’m now going to read out a list of child behaviours and I’d like you to say whether each was

‘Not True’, ‘Sometimes True’ or ‘Certainly True’ for your child over the past 3 months.

Both sides of SDQ


198

Those are all of the questions I have. We are, however, also asking parents/carers if you would be

prepared to complete a written questionnaire on your own approach to parenting (You completed

one like this before the Got It! program at your school). Would you be prepared to complete the

questionnaire if it is posted out to you?

If Yes, Address: ____________________________________________________

Thank you for taking the time to answer the questions and helping with the evaluation of Got It!


199

School Profile Template – Completed by Got It! teams

Please complete 1 form electronically for each school – expand as required - & email when complete to:

[email protected] Thanks!

School:

__________________________________________________________________________________

What process led to the Got It! program running in this school now?

-

-

-

How Many?

Children in School:

K-2 children:

Classes with K-2 children:

SDQs returned by teachers:

SDQs returned by parents:

Comments on particular characteristics of school (location, demographics etc):

-

-

-

Apart from K-2 teachers, which other school staff are involved with Got It! and in what ways:

-

-

-

Overview of Got It! program in the school:

Got it! Component

Activities & Programs There may be several activities

in each program component

Timeframe Wk/term – Wk/term

No. involved Comments

Engagement with School community


Social-Emotional Learning programs run by teachers

mailto:[email protected]


200

Parent Information Campaign

Behaviour screening of children

Assessment of children

Targeted group intervention for children

Targeted group intervention for parents

Individual support for children & parents

Referral to specialised services

Selection for Targeted Group Program:

No. of children Inclusion / Exclusion Criteria or explanation of

clinical decision making

Comments – incl. who may be missing out & why?

Children identified as potential for group program based on teacher SDQs

Children identified as potential for group program based on parent SDQs

Children (& their parents) identified as suitable to interview for group program

Children (& their parents) interviewed for group program

Children selected for targeted group program

Partial completion of targeted group program

Completed all sessions of targeted group program

NA

Any other comments on process of getting into targeted group program and who may miss out:


201

Process Journal completed by Got It! teams - Template

Site:

Set 10-15mins aside during your weekly team meetings to discuss and record questions in the

following table. Some weeks you may not fill in every section.

We would like to capture insights into your work throughout the engagement, screening,

assessment, program and follow-up phases of Got It!

Entries can be either electronic or hand written.

Forward to the evaluation team monthly: [email protected] or 20 The Terrace, The Hill,

2300 Thanks!!

Date:

Key Action or Activity

Participants What did and /or didn’t work

What factors contributed

What was learnt

For kids For Parents

For Teachers and Schools

This week we struggled with …..

Important decision/s made & what lead to this ….

Our wins this week ….

mailto:[email protected]


202

Got It! Staff Focus Group Schedule – Presented in PowerPoint


203

The same questions were subsequently asked and appeared on a slide for each of the nine

components of Got It! model of care listed on slides 4 and 5.


204


205

Got It! staff & managers – Interview schedule

Note: Given the other opportunities for input, this schedule is therefore used to suggest topics and

questions, rather than as an interview guide.

These interviews are in addition to focus groups with staff and an online survey for Health and DEC

staff. The interviews allow key people involved with the implementation of Got It! to pick up on issues

that they would like to discuss in some detail. The interview is flexible and adaptive to what you think

are the key issues for the evaluation of Got It!.

Check consent form is signed

Remind that interview will be audio-taped or notes taken.

Here are possible topics that we can talk about ….

Components of Got It! model of care

DEC partnership / school selection & engagement process

Sustainability - maintaining positive impact in schools & with families

Organisational location Benefits & drawbacks of being in CAMHS

Statewide coordination & development

Case examples to illustrate experiences - Particular families

- Particular schools

Are there particular things that you would like to focus on in this interview?

********************

1. What do you see as the strengths and limitations of the Got It! model of care?

2. Do you have any comments on the balance between the universal, screening and targeted

components of the model?

How effectively have the 3 components worked together?

3. How would you describe the nature of the DEC-Health partnership in Got It! implementation?

What processes are required to make such a partnership work?

4. What have been the experiences with selecting participant schools and engaging with the

school communities?

5. How adequately does the Got It! model address sustaining a positive impact in schools and

with families? What supports sustained impact?

6. Do you have any comments on the organisational location of Got It! teams in local CAMHS

services? Do the teams receive adequate direction, guidance and development opportunities

from state and local levels?

7. Is there anything that you think could improve Got It! if it were to be expanded in the future?

8. Any final comments on the Got It! program

Thank you!

Health & Education staff involved with Got It! – Online survey questionnaire


206

Note: A demonstration version of the online survey on “Survey Monkey”, including introductory

statements, can be viewed at: https://www.surveymonkey.com/s/WM266YD

What is your position?

Answer Choices

K-2 Teacher

Teacher (not K-2)

School Counsellor

School Principal

Deputy Principal

Other school staff

DEC Regional or state office staff

CAMHS Got It! team member

CAMHS Manager

Other Health staff

Comments

When were you involved with Got It! and with which sites?

Answer Choices

2012

2013

Dubbo (Western NSW)

Mt Druitt (Western Sydney)

Newcastle

Which of the following components of Got It! were you involved with? Tick all responses that apply to you

Answer Choices

Completed SDQ child behaviour screening questionnaires

Member of implementation or steering committee/s at state, region or local area level

Member of school based Got It! implementation Action Group

Delivered parent information session/s

Attended parent information session/s

Delivered teacher information or training sessions

Attended teacher information session/s

Did training to provide "fun friends" or "emotion coaching" in classroom

Delivered "fun friends" or emotion coaching program to K-2 children in class

Co-facilitated small group program for selected children & parents/carers

Provided follow-up support to children assessed by Got It!

Other role with Got It! (please detail below)

Comments

How would you rate the impact of the Got It! program for the following target groups?

https://www.surveymonkey.com/s/WM266YD

https://www.surveymonkey.com/analyze/?survey_id=40007487&OPT=NEW



207

No evidence of positive change

Some positive change

Considerable positive change

Don't know

K - 2 classrooms (level of behaviour difficulties)

Children who attended the small group program

Parents / carers who attended the small group program

Parents / carers across the wider school community

Teachers (Understanding & management of conduct & emotional problems)

Comments

How would you rate the success of each of the following components of the Got It! program? From 0 = poor to 4 = excellent.

If you have additional comments, insert in the box below.

0 =

poor 1

2 = average

3 4 =

excellent Don't know

Information on Got It! provided to teachers & parents

Training/education for school staff

Parenting information provided to families in the school

Screening of children (SDQ questionnaire)

Assessment & selection of children for targeted small group program

Small group program for selected children and their parents/carers

Individual child behaviour management programs

Referral to other specialised services for individual children/family

Comments

Open Questions:

What are the best features of Got It! ?

What suggestions do you have for improving Got It! ?




208

Partnership between Health and Education in the implementation of Got It! Please

rate your level of agreement with the following statements.

From 0 = strongly disagree to 4 = strongly agree

0 Strongly disagree 1

Disagree 2 Not sure

3 Agree 4 Strongly

Agree

There is shared understanding & commitment to Got It! by DEC & Health

The need for a partnership is understood by both DEC & Health

DEC & Health are willing to share ideas & resources to fulfil Got It! goals

The benefits of the partnership outweigh any difficulties

There is a history of good relations between DEC & Health

The roles, responsibilities & expectations of each partner is understood by the other

Administration, communication & decision-making are clear & simple

Both DEC & Health are involved in planning & priority setting for Got It!

The contribution of time, personnel & resources by both DEC & Health is sufficient for Got It!

Comments

0 Strongly disagree

1 Disagree 2 Not sure

3 Agree

4 Strongly agree

Decision-making about Got It! is participatory, responsive & inclusive

Managers in DEC & Health support the Got It! partnership

DEC & Health staff together have the required skills for delivery of Got It!

The partnership between DEC & Health adds value for children & families

There are regular opportunities for contact between staff in DEC & Health

Different priorities, goals & processes between DEC & Health are addressed

There are ways to share information and resolve difficulties with Got It!

Alternative views about Got It! can be expressed

The partnership between DEC & Health in the Got It! program produces clear outcomes

Comments

Open Questions:

What ideas do you have on strategies to develop the Got It! program. In particular, how best to:

* Engage families to participate in Got It! (including different cultures & backgrounds):

* Sustain positive changes in families and schools:

Do you have any other suggestions?

I would like to take part in an interview (optional). Please provide details here, or email: [email protected]




209

DEC staff interviews

The staff volunteering to be interviewed may have a variety of different roles within DEC and in

relation to the Got It! program. Some questions may not be relevant to some participants.

Sign Consent form & Confirm that interview will be audio-taped or notes taken.

1. What has been your involvement with the Got It! program?

2. Following on from the online survey / presentation at the school, were there particular

issues that you wanted to comment on for the evaluation or would you rather I work

through the questions I have?

3. How would you describe the type of children and families that the Got It! program is best

suited to assist?

4. Do you have any comments on how successfully the partnership between DEC & Health

works in planning / coordinating / delivering Got It! ?

5. Do you have any comments on the school selection process for participation in Got It!?

6. How much a part of the school/s does the Got It! program become?

7. Are school staff generally aware and on-board with the program?

8. Do you have any comments on how Got It! can best reach families to participate in the

targeted groups (SDQ screening & interview selection process)?

9. Teachers are interviewed by Got It! staff as part of the process of deciding which children

are suitable for the targeted program (along with SDQ results and parent interviews). How

do teachers know which children are likely to benefit? How much weight do teacher

assessments carry? Additional comments on screening process.

10. Do you have any specific examples of positive changes for children and families as a result of

Got It!?

11. Are there changes are there in the way school staff understand and respond to conduct or

behaviour problems in the school/s as a result of Got It!?

12. Does the Got it! program have an impact on the classroom environment? Are less resources

and time required to manage behaviour problems as a result of Got It!? Specific examples

such as reduction in additional classroom support?

13. Do you have any other comments on the impact of Got It! for the school/s?

Prompts – positive impacts & challenging impacts

14. What is needed in order to sustain the positive impacts of the Got It! program?

15. Other comments on Got It! or ideas for improvement?


210

External & Referral Services – Interview Schedule

Note: Questions adapted to the nature of involvement that the organisation has had with

the Got It! program.

1. Name of organisation:

2. Role of respondent in the organisation:

3. What is your involvement with the Got It! program?

4. What is your understanding of what Got it! is about?

5. What are the positive features of Got It!?

6. Do you have any examples of positive changes for children and families as a result of work

that has involved Got It!?

7. Do you have any comments on the children and families that Got It! is targeting and how

effectively they are engaged in the program?

8. How would you describe the coordination and referral processes between your organisation

and Got It!?

9. What could be done to improve the coordination and referral processes between Got It! and

your organisation?

10. Is there anything else you can think of that could improve Got It! if it were to be expanded in

the future?

11. Other comments on Got It!?


211

Appendix 2: Cost-Benefit Analysis Spreadsheets

Base case and best case scenarios

Base Case Scenario - Cost-Benefit Analysis for 1 cohort of Got It! pilot program participants (6 mth program in 2013: 12 schools in 3 pilot sites)

( $,000 at 2013 prices) 0 1 2 3 4 5 6 7 8 9 10 11 12

2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025

Total Program Costs

Expenditure on 6 mth program - 12 schools in 3 pi lot s i tes -1077

Benefits

Universa l component: Teacher tra ining & community education - WTP 67

Targeted component: Divers ion from criminal i ty 22 22 22 22 22 22

Targeted component: Productivi ty ga ins Yr 12 retention 5 5

Targeted component: Divers ion from high risk substance abuse 35 35 35 35 35 35

Total benefits 67 0 0 0 0 0 0 57 57 57 57 62 62

Net benefits (2013 Prices) -1010 0 0 0 0 0 0 57 57 57 57 62 62

Discounted Net Benefit at 4% -1010 0 0 0 0 0 0 43.315 41.649 40.047 38.507 40.274 38.725



Discount rates 4% 7% 10%

Net Present Value (NPV) -277 -547 -705

Benefi t Cost Ratio (BCR) 0.74 0.49 0.35

13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Totals

2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043

22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22

5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5

35 35 35 35 35 35 35 35 35 35 35 35 35 35 35 35 35 35

62 62 62 62 62 62 62 62 62 62 62 62 62 62 62 62 62 62

62 62 62 62 62 62 62 62 62 62 62 62 62 62 62 62 62 62

37.236 35.803 34.426 33.102 31.829 30.605 29.428 28.296 27.208 26.161 25.155 24.188 23.257 22.363 21.503 20.676 19.88 19.116 -277.250

25.728 24.045 22.472 21.002 19.628 18.344 17.144 16.022 14.974 13.994 13.079 12.223 11.423 10.676 9.9777 9.3249 8.7149 8.1448 -547.450

17.959 16.327 14.842 13.493 12.266 11.151 10.137 9.2159 8.3781 7.6165 6.924 6.2946 5.7224 5.2021 4.7292 4.2993 3.9084 3.5531 -704.504


212

Best Case Scenario - Cost-Benefit Analysis for 1 cohort of Got It! pilot program participants (6 mth program in 2013: 12 schools in 3 pilot sites)

( $,000 at 2013 prices) 0 1 2 3 4 5 6 7 8 9 10 11 12

2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025

Total Program Costs

Expenditure on 6 mth program - 12 schools in 3 pi lot s i tes -1077

Benefits

Universa l component: Teacher tra ining & community education - WTP 67

Targeted component: Divers ion from criminal i ty 43 43 43 43 43 43

Targeted component: Productivi ty ga ins Yr 12 retention 9 9

Targeted component: Divers ion from high risk substance abuse 71 71 71 71 71 71

Total benefits 67 0 0 0 0 0 0 114 114 114 114 123 123

Net benefits (2013 Prices) -1010 0 0 0 0 0 0 114 114 114 114 123 123




Discount rates 4% 7% 10%

Net Present Value (NPV) 446 -90 -402

Benefi t Cost Ratio (BCR) 1.41 0.92 0.63

13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Totals

2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043

43 43 43 43 43 43 43 43 43 43 43 43 43 43 43 43 43 43

9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9

71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71 71

123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123

123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123 123

73.871 71.029 68.298 65.671 63.145 60.716 58.381 56.136 53.977 51.901 49.904 47.985 46.139 44.365 42.658 41.018 39.44 37.923 446.318

51.041 47.702 44.581 41.664 38.939 36.391 34.011 31.786 29.706 27.763 25.946 24.249 22.663 21.18 19.794 18.499 17.289 16.158 -90.286

35.629 32.39 29.445 26.768 24.335 22.123 20.111 18.283 16.621 15.11 13.736 12.488 11.352 10.32 9.3822 8.5292 7.7538 7.049 -402.290


213

hsnet - got it! · got it! final evaluation report debbie plath consulting & family action...

Documents