Process Use:

Intentional Practice or Just Good Practice?

anzea 2013 Conference 22–24 July 2013

Alexandra Park, Epsom, Auckland

Michael BlewdenMassey University

Overview

• Background

• Research question/approach

• Case study of findings

• Implications for practice

Process Use

• Learning and development from stakeholder participation in evaluation

• Influence/consequence of evaluation processes

• Distinct from/independent of findings use

e.g. Stakeholder participation…

…enhances willingness to use findings

…develops evaluative thinking or action

…develops shared understanding

…influences the evaluand

Patton (1997) says:

“Evidence of process use is represented by the following kind of statement after an evaluation:

“The impact on our program came not just from the findings but also from going through the thinking process that the evaluation required”

Shaping this study

• Process use:

- enhances value and utility- often an unintentional side-product- more likely if we purposefully seek it

Patton (1997) again

“...the possibility and desirability of learning from evaluation processes as well as findings can be made intentional and purposeful…”

“…instead of treating process use as an informal offshoot, explicit and up-front attention to the potential impacts of evaluation logic and processes can increase those impacts and make them a planned purpose for undertaking the evaluation…”

The question of ‘intent’

Patton infers evaluators may

•choose to deliberately seek process use

•adopt specific practices to achieve it

•increase the value and utility of evaluation

The question of ‘intent’

• …historically, process use more typically regarded as “...an informal offshoot” (Patton, 2007)

• …few methodologies intentionally seek process use…rarely an integrated goal of practice (Morabito, 2002)

• Observations of NZ practice and context

The (research) question

• Why and for what purpose do evaluators seek process use?

• Why do they choose the practices they do to achieve it?

Research Approach

• Process use as ‘sensitising concept’

• Process use as a ‘construction’

• Interpretivist explanation

• Importance of context

Interpretivist explanation

Meaning + beliefs + desires = behaviour

e.g. action X was done because person held belief Y according to which doing X would fulfil desire Z

Process use intent and practice

Traditions

Values

Beliefs explaining beliefs

Practice setting

Beliefs about evaluation

Beliefs about role

Evaluation theory

Evaluation practice

Project setting

Beliefs about

outcomes

Journey to practice

Cultural context

Beliefs about practice

Intent and practice is

‘understandable’

Meaning

Why important

Justifications, reasons Point and

purpose

Expectancies

Embedded ‘rules’

General awareness

and experience of process

use

Process use

examples considered important

and intentional

Participants

• 24 practicing evaluators

• Eligibility criteria

• In-depth face to face interviews

• Auckland and Wellington location

Assumptions • Desirable for evaluators to seek process

use but not necessarily always

• Pursuing process use may have risks

• Understanding, use, relevance or appropriateness of the term not assumed

Evaluation as process

Evaluation as

development

Evaluation as findings

use

Evaluation as capacity

building

Intent and practice

Beliefs about evaluation

Evaluation as intervention

Social betterment

Enabling

Equality

Knowledge is experiential and constructed

Intent and practice

Beliefs about practice

Collaborative, transparent, understandable, trustful

Accountable to relational ethics

and morals

Should address issues of power and

inclusion

Tools and procedures as learning

Intent and practice

Beliefs about role

Should facilitate mutual learning, development, improvement

Responsibility to give back/return value

Should act in the interests of those with less power

Intent regarding process impacts

Intent and practice

Beliefs about outcomes

Data quality

Accept evaluative conclusions/findings

Capacity development and learning outcomes

Critical engagement

Attitudinal/affective change

Equality

Explanations • Findings are ‘ideographic’ - however….

• PU integral and inevitable

• Intent/practice understandable when evaluators are understood as thinking, ‘meaning makers’

• Enhancing process use about debating the evaluator’s mandate, role, responsibility

Reflections

• Do these evaluator beliefs have implications – positive or negative?

• Are there risks to evaluation?

• Could there be process use misuse?

• How should we respond?