Conference on Monitoring and Evaluation for

Responsible Innovation, Wageningen 19-20 March 2015

M&E for responsible innovation AND scaling

By: Seerp Wigboldus , Centre for Development Innovation,

Wageningen UR, the Netherlands

In opening

Our innovation abilities increasingly exceed our ability to foresee longer-term effects of the outputs and outcomes of innovation processes

This is a core concern which inspires ideas regarding responsible (research and) innovation

Scaling (up) processes put a multiplier effect on this concern

However, research and innovation processes are ill-equipped to engage responsibly with scaling processes

Increasing demand for ‘impact at scale’

Research and innovation often considered as a “finding out what works” after which the idea is to “do more of the same” (replication, multiplication, spreading, adoption, dissemination, transfer, etc.)

“What works”, or what is considered “responsible”, will be defined within certain system boundaries, domain boundaries, geographic boundaries, etc.

Scaling processes (doing more of the same) usually escape such boundaries leading to impact across scale levels, domain boundaries, etc.

The ‘scaling (up)’ part of responsible

innovation is what connects strongest to

societal processes

Addressing societal concerns

“When pathways to scale become roads to hell”

Scaling processes are essentially about societal processes

– who decides what should go to scale and not? Who benefits from scaling (up) and who doesn’t? Who is responsible for negative impact at scale? Who decides what is responsible?

Responsible innovation and responsible

scaling are not necessarily the same

If innovations are considered to be ‘responsible’ (relating to

e.g. inclusiveness, sustainability), will they also remain

being ‘responsible’ when they are scaled up?

Scaling up means entering new scale levels, new domains,

new geographical areas, etc. – how will we know

innovations will remain ‘responsible’ when they cross those

boundaries?

The use of asbestos (at the time an innovative product)

went to scale in an enormously successful way. We now

pay the price for lack of foresight by having to scale down

the presence of asbestos in older buildings

A need to expand perspectives on responsible

innovation

Little or no explicit reference to scaling processes in the concept of responsible innovation, while these are the very processes which raise concerns

Environmental degradation is not the result from some innovation process that went wrong – it is the scaling up of the use of certain products and the application of certain practices that really got us in trouble

E.g the plastic in oceans is a result of the scaling up of the use of plastic (packaging) materials

Defining “responsible”

At what scale level do we define responsibility?

● Product level?

● Process level?

● Production system level?

● Locality level?

● Landscape level?

Cumming, Cumming and Redman (2006) hypothesize that many of the problems encountered by societies in managing natural resources arise because of a mismatch between the scale of management and the scale(s) of the (ecological) processes being managed

Testing logics

The often-assumed logic

One application of best-practice, best-bet, best-fit innovation is good news, more applications of the same is better news

Critical questions

Is this always true?

Is this true in most cases?

Is this rarely true?

From innovation to scaling – beware of

fallacies

Menter (2004) on fallacies along the lines of scaling up:

The ecological (inference) fallacy (or cross-level fallacy): what

works at one level will work at another.

The composition (inference) fallacy: what is good for one

person is good for everyone (if one village was able to increase

income through growing a new crop, all villages in the region

could do the same).

The exception fallacy , which is sort of the reverse of the

ecological fallacy, such as in stereotyping: If one researcher

does something, all researchers assumed to be like that.

The argument can be extended to what we consider to be

“responsible”

Scaling up in agricultural development and

innovation cannot escape complexity

Implications of scaling up the use of a new crop goes beyond cropping system boundaries

Implications of scaling up in agriculture goes beyond agricultural system boundaries

Understanding systems ≠ understanding success and failure of scaling processes

Scaling up may seriously distort equilibria, distort proportions, distort wider context

We may want innovation to be disruptive, but scaling up means putting a multiplier on disruption – can we anticipate implications?

Processes of scaling (up) will tend to move

anything into the domain of the complex and

‘chaotic’

Even introducing a (simple) device may lead to unanticipated (e.g. health) consequences

Scaling (up) moves things into new domains, new geographies, and new scale levels

Experimental scaling (up) is hard to do

Need to get an idea about cross-scale, cross-level, cross-domain, cross-mandate, cross-temporal implications

↑Level

ofnet

benefit/value

Level of scaling up →

?

?

?Deserving label of being

“responsible”

Level of scaling up →

?

This is serious stuff

Inertia is a concern for innovation, but also a concern once scaling up innovation (s) has led to undesirable situations

Once farmers have started using agrochemicals and have seen economic benefits, it is hard to get them to reduce its use after environmental damage is found out

But also, once we have lost biodiversity as a result of scaling up agricultural production, it is hard to restore it

It is scaling processes which may race the effects of ‘innovations’ toward planetary boundaries and beyond

Scaling processes race us toward tipping points (Scheffer, 2009, 2011)

A vision for responsible innovation and scaling

Responsible innovation is a relative concept, relative to scale levels, relative to context, relative to other products and processes

Scaling (up) is not a mere extending of that which emerged from (responsible) research and innovation

Water is good, but too much water is not good. Food is good, but too much food is not good. Use of soils for agriculture is good, but too much... Etc.

It is through processes of scaling (up) that things which may have even started off as something good, become critical and even harmful

Hence: From design, (responsible) research and innovation needs to have potential future scaling (up) in mind

Late lessons from early warnings – improving

theories of innovation and theories of scaling

European Environmental Agency, 2013

Toward responsible theories of scaling (ToC)

Wrongly assumed scaling mechanisms, an example

It used to be (and perhaps still is) widely believed that the accumulation of wealth by the rich would be good for the poor as some of the increased wealth of the rich would trickle down to the poor.

We may see this as an assumed scaling mechanism. In 2011, however, OECD, in the report “Divided we stand. Why inequalities keep rising”, challenged the trickle-down theory (which relates to scaling processes) as unfounded in many cases.

Articulating “theory of scaling”

Along the same lines as the “theory of change”, articulate how scaling is expected to happen and what its effects are expected to be

Articulate assumptions related to this, including related risks

Develop critical reflection questions to be used regularly to assess not just the “success” of scaling, but also unanticipated effects across domains, scales and time

M&E to inform strategic guidance

● In design: activating foresight analysis

● During implementation: activating critical reflection and adaptive management

An illustration: Rubber in Xishuangbanna, China

Monoculture rubber was introduced some 50 years ago as a

way out of poverty and as a “green” development option

(apart from other reasons)

This massively went to scale, now covering more than a

quarter of the entire prefecture (half the size of Bhutan)

With the increase in scale of rubber production and further

(scientific) research, gradually more and more concerns were

voiced

Now, rubber cultivation in Xishuangbanna is considered to be a

big problem, mainly because of its negative environmental

effect

But, the economic benefits have made people addicted to it –

it is very hard to change to more responsible rubber cultivation

If only this had been foreseen 40 years ago...

This example is rather typical for many agricultural

development processes around the globe

Some labels we may want to put on this:

Rollercoaster development

Tunnel-vision innovation

Monorail scaling

Suggested description of what scaling-inclusive

vision for responsible scaling may involve

Responsible in terms of process:

- Well-informed ex ante and ex durante about relevant cross-

scale, cross-domain, and cross-temporal influences and effects;

- Meaningfully involving stakeholders in design, decision-making

and M&E of the scaling initiative;

- ‘Potential negative side-effects can be mitigated and or adapted

to.

Responsible in terms of effect:

- Positive outcomes can be sustained across domains, scale

levels and time (sustainability and environmental integrity);

- Disadvantaged groups share in positive outcomes

(inclusiveness);

- Effects and side-effects do not undermine system/ stakeholder

resilience.

Taking scaling processes seriously

If we are to take responsible (research and) innovation seriously, it also implies a need to take scaling processes seriously

This means treating them more seriously than as mere processes of adoption, transfer (of technology), rolling out, replication, etc.

Are conventional research and innovation processes equipped for this?

No!

Are conventional M&E processes equipped for this?

No!

Upcoming papers

Taking scaling processes seriously in agricultural research, innovation and development (part 1) - Toward an integrative approach

Taking scaling processes seriously in agricultural research, innovation and development (part 2) - Application of an integrative approach in case studies on green rubber in China and agroecology in Nicaragua

Probably to be published later this year in Agricultural Systems

Suggested M&E to keep an eye on whether responsible innovation (as process) remains being responsible as products, services, processes are scaled up

Strategic design and implementation phase

Design of innovation (both in terms of process as well as in terms of

outputs) needs to have future scaling in mind

Scenarios for potential scaling up need to be developed early on to

assess potential risks and limitations which may be involved if “this”

were to go to scale

M&E needs to check to which extent such processes are in place (on

paper, functional, or actively used in decision-making/selection

processes)

M&E design and implementation phase

Development and use of indicators and critical reflection processes

related to the extent to which the label of “responsible” keeps

applying when things go to scale

Effective M&E processes to assess the above

1. How can M&E responsibly support the management and governance of innovation processes towards a sustainable and equitable future?

2. How can M&E contribute to deeper reflexivity and transparent decision making?

- Effective (responsible) M&E of scaling up (that which emerges from “responsible” innovation), starts with including related issues/questions already at the design of innovation processes

- This involves articulating strategic questions regarding what will make us consider innovation (including scaling) to be “responsible” - this is to be translated to information needs and needed M&E (reflection) processes.

3. What are the prerequisites for taking responsibility for systemic change in terms of: *M&E professional’s roles & responsibilities; values and principles; competencies, *M&E process design, focus and approach, *institutional changes needed to support M&E for responsible innovation.

Develop agreement on

- What is understood by “responsible” and translate this to concrete,

monitorable categories per case

- Who is considered to be responsible for what and what this means

for quality requirements they need to comply with

Strengthen foresight capacities (incl. scenario analysis) in relation to

an ability to foresee longer-term/cross-scale/cross-domain effects of

innovations going to scale

Monitor the abovementioned categories in relation to what is meant

by “responsible”

Regulatory frameworks will often need to back this up since there is

usually a short-term price tag attached to being “responsible”

Questions, comments, additions, further

ideas, related experiences?

Key question: To what extent does the argument make sense?

If relevant,

● what other lines of thinking/experiences might support the argument?

● what are implications for views on responsible innovation?

● what are implications for the role which M&E needs to play?

● what may need to change in conditions for M&E to make this possible?