Research in Action Series

Effectiveness of Mentoring Program Practices

Overview of Research in Action Series

• MENTOR has developed an innovative series - Research in Action

• Translates the latest research on mentoring into tangible strategies for mentoring practitioners.

• Makes the best available research accessible and relevant to the mentoring field.  

• Using the Research in Action series, programs can ensure their practices are based on current research, resulting in improved services and better impacts for young people.

• 10 issues

• Each issue provides:  – Research– Action – Resources

Overview of Research in Action Series

Research in Action Issues:

1. Mentoring: A Key Resource for Promoting Positive Youth Development

2. Effectiveness of Mentoring Program Practices3. Program Staff in Youth Mentoring Programs:

Qualifications, Training, and Retention4. Fostering Close and Effective Relationships in Youth

Mentoring Programs5. Why Youth Mentoring Relationships End6. School-Based Mentoring7. Cross-Age Peer Mentoring8. Mentoring Across Generations: Engaging 50+ Adults

as Mentors9. Youth Mentoring: Do Race and Ethnicity Really

Matter?10. Mentoring: A Promising Intervention for Children of

Prisoners

Issue 2: Program Effectiveness

• Download this issue by visiting http://www.mentoring.org/downloads/mentoring_383.pdf

• The research section of this issue was written by David L. DuBois, Ph.D., University of Illinois at Chicago

Objective of Mentoring

The central objective of any youth mentoring program is to establish and support beneficial relationships between youth and their identified mentors.

Objective of Mentoring

• High-quality programs establish and support relationships that are:

– Effective

– Safe

– Efficient

– Sustainable/growth-oriented

Effectiveness vs. Quality

Program Effectiveness• Impacts on mentoring

relationship quality• Youth outcomes

Program Quality• Safety• Efficiency• Sustainability

Program Effectiveness Problem

0

20

40

60

80

100P

erce

nta

ge

Effect Size

Negative

Close to Zero

Positive

DuBois, Holloway, Valentine, & Cooper, 2002

Mentoring Best Practices: Circles of Evidence

Research

*Professional Expertise & Experience

*Local Resources & Needs

*Client/Stakeholder Preferences &

BeliefsBest Practices

* Includes evidence obtained from sources other than formal research, such as clientsatisfaction surveys, program participant outcomes, and community demographic trends.

Conceptual and Methodological Issues

• Effects of any practice on program effectiveness will be dependent on a range of other factors

• “Local adaptation-fidelity debate”• Clear guidelines/benchmarks needed to

assess presence or absence of evidence-based practices

Sources of Evidence

Both formal research literature and internal analyses of information by mentoring programs are potentially relevant to evaluating the effectiveness of a practice.

Research on Program Practices

Descriptive Studies

• Highlight important trends that merit further investigation

• Provide information concerning the extent to which evidence-based practices are being adopted and adapted

MENTORING INAMERICA 2005

Researching Program Practices

Typical program evaluation is designed to estimate cumulative effects that are produced by the multiple practices that comprise a given program rather than the unique contribution of any one practice

Meta-Analysis

Program practices predictive of stronger positive effects on youth outcomes

Procedures for systematic monitoring of program implementation

Use of community settings for mentoring

Using mentors with backgrounds in helping roles or professions

Clearly established expectations for frequency of mentor-youth contact

Ongoing (post-match) training for mentors

Structured activities for mentors

and youth

Support for parent involvement

(DuBois et al., 2002)

Pre-Match Training

• Available evidence regarding the effectiveness of pre-match training for mentors is mixed– Meta-analysis found that the youth outcomes

were not significantly greater– A 2002 study found that mentors with less than

two hours of training reported the least close relationship; those with more than six hours reported the closest relationships

– A 1993 study found indicators of quality relationships did not vary significantly between those who received training and those who did not

Conclusions

This issue provides practitioners with a framework to examine research in the field of mentoring. This research is in the early stages and this makes it difficult to make informed decisions in regard to program practice. There is still much to study.

Action: Evaluation

• Program evaluation can measure– Youth outcomes– Program processes– Specific program practices

Evaluating Program Practices

• Step One: Identify problem, challenge, or opportunity

• Step Two: Develop a hypothesis or prediction

• Step Three: Determine which practice to assess

Evaluating Program Practices

• Step Four: Formulate the research question

• Step Five: Design the evaluation

• Step Six: Collect and analyze the data

• Step Seven: Use the results

Resources

Information Technology International (ITI)

ITI provides professional, programmatic, and technical support to government and industry.

www.itiincorporated.com

Evaluating your Program: A Beginner’s Self-Evaluation Workbook for Mentoring Programs.

www.itiincorporated.com/_includes/pdf/SEW-Full.pdf

Resources

MENTOR/National Mentoring Partnership

www.mentoring.org

The leader in expanding the power of mentoring to millions of young Americans who want and need adult mentors.– The Elements of Effective Practice– How to Build a Successful Mentoring Program

Using the Elements of Effective Practice™– How to Select a Survey to Assess Your Adult-

Youth Mentoring Program

Resources

Northwest Regional Educational Laboratory: National Mentoring Center

www.nwrel.org/mentoringNational organization that provides training and technical assistance to youth mentoring programs.

– Frequently Asked Questions About Research and Evaluation

– Measuring the Quality of Mentor Youth Relationships: A Tool for Mentoring Programs

Resources

Search Institute

www.searchinstitute.org

Provides leadership, information, and resources to promote positive healthy development of youth.

What's Working? Tools for Evaluating Your Mentoring Program – Now available as a free download

Resources

Michigan Resources

Mentor Michiganwww.mentormichigan.org

MSU Extension

http://web1.msue.msu.edu/cyf/youth/mentor/

This presentation provides an overview of Research in Action Issue 2:

Effectiveness of Mentoring Program Practices

This tool was produced by MENTOR/The National Mentoring

Partnership