PowerPoint Presentation

January, 2014

Career DevelopmentProving our Value - Getting Much Closer

1

National research agenda to better understand what works in career services

3 calls for proposals from HRSDC over 6 years

Source of major international envy among Career Development Practitioners (CDPs) and Researchers

Evidence-Based Research in Canada

2

Ethics: Clients deserve interventions that have demonstrated success

Confidence: Practitioners should know that a given intervention will result in a predictable outcome

Security: Being able to provide evidence of success increases ability to obtain and keep funding for services and secure jobs for CDPs/Employment Counsellors

Why Evidence-Based Practice?

3

Outcome-focused intervention

Evidence of client change

contrast with

Client flow

Ease of access

Practitioner time use

Its easy to measure variables that dont address client change

Why Evidence-Based Practice?

4

5

Highlight results of 3 recent research projects: What have we learned?

Explore how a select number of LMI and career development resources could be integrated into everyday evidence-based practice

Explain where we are now and next steps

Direct you to additional research studies

Session Objectives

6

6

What evidence do you currently collect?

What is done with the evidence currently collected?

7

Before we start.

What did the research do?

What was learned?

What was most surprising?

What are the implications for practice?

Reflective Questions.

8

8

Research Project #1:Assessing the Impact of LMI on Career Decision Making and Work Search

Research Team

Canadian Research Working Group in Evidence-Based Practice (CRWG),

Canadian Career Development Foundation (CCDF)

In partnership with

New Brunswick Post-Secondary Education & Labour,

Saskatchewan Advanced Education, Employment & Labour

9

9

LynneB (L) -

What is the problem?

There is very little literature about the direct contribution of LMI to employment/career decision-making (Savard & Michaud, The Impact of LMI on Career Decision-Making Process: Literature Review, FLMM, 2005)

Several questions remain unanswered:

How do people actually use LMI?

What (if any ) assistance would be helpful?

10

10

Research Questions

If client needs are assessed and clients are given LMI consistent with their needs,

To what extent does assistance by a service provider enhance their effective use of LMI?

OR

To what extent is independent self-help a sufficient process for clients to use LMI effectively?

11

11

Method

We prepared guided LMI packages (take-home booklets and Resource Centre binders) on:

Career Decision Making:

Know yourself

Know the Labour Market

Put it all Together

Job Search:

Check for Fit

Get Ready

Search for Work

Get a Job

12

Question for panel:

What would you say was the general response of clients to the LMI packages?

Why do you think the response of clients was so positive?

How were these packages different from what clients usually are given?

12

All participants in the study:

Received a needs assessment interview & completed an initial, pre-program survey

Received an LMI package specific to their identified employability need

Were randomly assigned to either a self- directed intervention delivery method or an assisted self-directed intervention delivery method.

Were given an orientation to the Resource Centre which they could freely use on their own

Completed a pre and post-pre survey

Received a cash honorarium and certificate of participation

13

Intervention

13

The self-directed group:

worked independently for 3 weeks, making use of the materials and the Resource Centre

returned in Week 4 for their exit interview

14

Intervention (cont)

14

Intervention (cont)

The assisted self-help clients received:

two additional AIS (Advice and Information) interviews (20-30 minutes) in weeks 1 and 3 focused on helping them understand, interpret and apply the LMI to their own situations and /or access additional LMI

returned in Week 4 for their exit interview

15

Sample by Province

ProvinceIntervention TypeDeliveryTotalIndependentAssistedSaskatchewanCDM201535JS 232548Province Total434083New Brunswick EnglishCDM281543JS121325Province total402868New BrunswickFrenchCDM151732JS11920Province total262652Total for 3 groups10994203

16

What did we Measure

The Dependent Measures for the data analysis were:

General ability to use LMI

Knowledge

Clear vision of what I want in my career future

Knowledge of print and online resources

Skill

Have effective strategies for keeping myself motivated

Have a realistic action plan

Personal Attributes

Optimism about what lies ahead re meeting my career goals

Confidence in my ability to manage future career transitions

17

17

Examples of Questions used in pre and in post-pre surveys

Knowing what you know now, how would you rate yourself before the workshop, and how would you rate yourself now?BeforeAveAfterAveA clear understanding of what I need to do to move forward in my career.2157481691.57132866533.11A clear vision of what I want in my career future.24374039111.84172963513.03Reviewed my past work, education and experience so that I know what skills and strengths I have.11335735142.05232168563.15A list of possible options that may fit with what I want in my career future.2349472391.64142566543.12Knowledge of print and online resources that help me to research career/employment options.3255421651.38112153743.32Confidence that career-related employment opportunities actually exist that fit with what I want in my career future.1941582581.751102758553.03

18

Changes in competency self assessments:

Pre program:

30% and 58% of the respondents rated competency Not OK

5% rated their competency as Exceptional

Post program:

1% 9% rated competency Not OK

39% rated their competency as Exceptional

The amount of change was similar across all three dimensions of the survey, knowledge, skills, and personal attributes all demonstrated about the same amount of change.

19

Descriptive Results

Descriptive Results (Cont.)

Of particular note are items that suggest increased ability to self-manage their careers, such as:

A clear understanding of what I need to do to move forward in my career.

A clear vision of what I want in my career future.

Effective strategies for keeping myself motivated to achieve my career/employment goals.

Confidence in my ability to research career, employment, and training options that are available

Confidence in my ability to manage future career transitions.

20

Differential Results-Total Score

Both CDM and JS groups had significant increases across time

Change in CDM group was significantly larger than in JS group

Participants in the JS group had higher scores than participants in the CDM group, likely indicating that JS participants were more familiar with using LMI before the project began.

Participants receiving assistance demonstrated greater change across time than did those in the independent modeSimilar pattern for all subscales

21

For group as a whole:

significant increase in overall ability to use LMI

neither intervention was more conducive to one manner of delivery compared to the other

To what extent would you say that any changes in the ratings on the previous pages are a result of your participation in this research project, and to what extent were they a function of other factors in your life?

22

mostly other factorssomewhat other factorsuncertainsomewhat this program mostly this programEnglish33154780French 0012526

Program

Attribution for Change

22

Question to Panel:

The results are very positive. Did the results surprise you?

You saw the clients at the end of the study when they came to complete the exit surveys and collect their honorarium. What sorts of comments did clients make about the study? What impact do you think the honorarium had on them?

Overall,no significant differences in the results of the anglophone and francophone samples with the exception that assisted francophone clients reported increased gains over the anglophone clients although both groups reported more positive change in the assisted condition.

23

Study repeated in francophone new brunswick

Follow up interviews (1 week and 4 months after intervention)

I was surprised at how much information there was

Initially I thought I could not do this, but I found out I could

Theres a lot to learn

I got more focused on my goal

I got clearer about what I wanted to do

24

Client Criticisms

The program needed to be longer given the number of resources available. (222)

Working full time made it very hard []. With a little more help it would have been much easier (242)

There was too much information and that made making a career decision more difficult [] very hard to do this alone (209)

25

What did practitioners say?

The structure helped me be more focused and concrete

I never really thought about intake of clients as assisted or self-helpnow its built into every intake I do

I used to be maternal and forget that it was the clients decision and I got more involved than I needed to. Now I ask the clients to be more responsible

Now I give more homework tasks and Im more specific. The clients felt more focused and so do I

26

What did the Research do?

Isolated LMI as an intervention

Isolated giving information and advice from employment counselling

Organized LMI into a career development learning sequence

Tested self-help versus a combination of self-help, information giving and advice

27

What was learned?

Tailored LMI embedded in a learning process results in knowledge and skill acquisition as well as the capacity for self-management

LMI appropriate for a clients specific need (opposed to general LMI) appears to support engagement and action

For many clients, a little (or no) professional LMI support is enough

Structure and timelines appear to motivate action and a sense of progress

Giving clients hands-on tools appears to motivate more than money

28

What was most surprising

The extent of positive client change in knowledge, skill and personal attributes in both conditions of self-help and supported self-help

The value clients placed on the LMI materials and the degree to which they used them

The positive impact of organized LMI and LMI specific to client need versus generalized LMI

29

What are the implications for practice?

To be discussed..

30

Drs. Dave Redekopp and Bryan Hiebert

Alberta Human Services

Manitoba Entrepreneurship, Training and Trade

Canadian Career Development Foundation (CCDF)

Research Project #2:Assessing the Impact of Career Resource and Supports Across the Employability Dimensions

31

31

There is very limited research about the direct contribution of career development learning resources on client progress toward or achievement of employability.

There are many assumptions made about self-help resources but very little actual evidence of their impact.

There is much talk about labour market attachment but on investigation it remains a very wooly construct.

32

What is the Problem?

Developed comprehensive career development coaching guides (way beyond LMI) covering all employability dimensions excepting career growth:

CAREER DECISION MAKING: Not yet clear about work/learning goal; needs to understand personal skills, interest, values and personality and more about different potential occupations in order to develop a vision for a preferred future

SKILLS ENHANCEMENT: Has career goal, but needs to explore skills and qualifications needed for the work they want and how to acquire them

JOB SEARCH: Has career goal and necessary skills/qualifications for desired work; needs skills, strategies and tools necessary to conduct a successful job search

JOB MAINTENANCE: Has difficulty keeping work; may need to address personal/life issues (e.g. health, behaviour, transportation, child care) or may need to work on the skills/attitudes required for success at work. The last part of this Guide is appropriate for someone already working, but wanting to improve their chances of keeping the job.

33

This Study

Randomly assigned clients to one of two treatment conditions:

Practitioner launched (in which they had their needs assessed, were oriented to the appropriate Guide and then worked independently with their Resource Guide for 4 weeks)

Practitioner launched and supported (in which they also worked with their Career Consultant for 4 weeks of regular services (i.e. 2-4 sessions on average) using the Guide as appropriate but not restricted in their interventions

Examined differential impact across treatment conditions on client knowledge, skills, personal attributes and labour market outcomes

34

This Study

Self Help Index(self-help-o-meter) to see if:

The propensity for self-help can be measured

The ability to self-help affects client outcomes

Labour Market Attachment Index to see if:

Specific variables related to the clients work history and life situation can be measured

These variables affect client outcomes

Developed Two New Indices

If clients are given a comprehensive needs assessment to determine their employability need(s), what is the differential effect of independent and consultant-supported career development resources on clients who are weakly attached to the labour market versus those who are strongly attached to the labour market?

Will all clients demonstrate positive change?

Will clients who are strongly attached to the LM show more positive change?

Will clients who are supported by a consultant show more positive change?

Will weakly attached clients struggle more in the independent condition?

36

Research Question

ProvinceDimensionDeliveryAlberta = 115CDM = 35Independent = 17Supported = 18SE = 25Independent = 16Supported = 9JS = 42Independent = 21Supported = 21JM = 13Independent = 10Supported = 3Manitoba = 113CDM = 44Independent = 21Supported = 23SE = 19Independent = 10Supported = 9JS = 43Independent = 23Supported = 20JM = 7Independent = 4Supported = 3TotalsTotal sample = 228CDM = 79 (35%)SE = 44 (19%)JS = 85 (37%)JM = 20 (9%)Independent = 122 (54%)Supported = 106 (46%)

37

Sample Composition

DeliverynBeforeAfterTotalPLR3423.71(11.38)45.50(10.93)34.60PLSR3824.92(11.41)49.05(8.86)36.99Total7224.35(11.98)47.38(9.98)

Key FindingsCareer Decision Making Total Score

Summary: Main Effect for Delivery: F(1,70)=1.32; p=.26Main Effect for Time: F(1,70)=253.72; p