cookstove adoption measured with sums in darfur: a ... · discussion of algorithms, baseline...

1

Cookstove Adoption Measured with SUMs in Darfur: A Discussion of Algorithms, Baseline Adoption, and Effects of Intervention on Adoption

ETHOS, 2015 Daniel Wilson1, Mohammed Idris Adam2, Omnia Abbas3, Jeremy Coyle1, Angeli Kirk1, Javier Rosa1, Ashok Gadgil1,4 (1) University of California - Berkeley, Berkeley, California (2) El Fasher University, Darfur, Sudan (3) Potential Energy, Berkeley, California (4) Lawrence Berkeley National Laboratory, Berkeley, California 24 JAN 2015 ETHOS, Kirkland, Washington

2

Acknowledgements: with gratitude and thanks to our funders and sponsors:

2

Additional thanks to: A.J. and Catherine Orselli Fund, the Trussell Fellowship in Environmental Engineering, and the Joseph A. Dias Scholarship Fund.

Wilson, D.L., 24 JAN 2015

3

Motivation: Impact is a strong function of use, but we do not measure use well

•  Evaluation almost always relies on social surveys to collect user-reported data.1,2

•  Surveys are known to suffer from social desirability (courtesy) bias and recall errors.3-6

1.  Jessica J Lewis, S. K. P. Environmental Health Perspectives. 120, 637 (2012). 2.  Burwen, J. Energy and Resources Group, University of California, Berkeley, (2011). 3.  Edwards, A. L. (Dryden Press, 1957). 4.  Nunnally, J. C. & Bernstein, I. H. Psychometric Theory. (McGraw Hill, 1994). 5.  Das, J., Hammer, J. & Sánchez-Paramo, C. Journal of Development Economics 98, 76–88 (2012). 6.  Thomas, E. A., Barstow, C. K. & Rosa, G. A. Environmental Science and Technology (2013).

3

Roasting coffee in Meki, Ethiopia. © Daniel Wilson 2013

3 Wilson, D.L., 24 JAN 2015

4

Platform: We mounted commercially-available sensors on Berkeley-Darfur Stoves.

•  As of Jan ‘15, 41k efficient Berkeley-Darfur Stoves assembled and distributed in Darfur

•  Following the work of others1-6, we used temperature loggers (Maxim iButton) to monitor heat as a proxy for adoption.

1.  Thomas, E. A., Barstow, C. K. & Rosa, G. A. Environmental Science & Technology (2013). 2.  Ruiz-Mercado, I., Masera, O., Zamora, H. & Smith, K. R. Energy Policy 39, 7557–7566 (2011). 3.  Ruiz-Mercado, I., Canuz, E. & Smith, K. R. Biomass and Bioenergy 47, 459–468 (2012). 4.  Ruiz-Mercado, I., Canuz, E., Walker, J. L. & Smith, K. R. Biomass and Bioenergy 57, 136–148 (2013). 5.  Burwen, J. & Levine, D. I. Energy for Sustainable Development 16, 328–338 (2012). 6.  Berkeley Air Monitoring Group. Monitoring and Evaluation of the Jiko Poa Cookstove in Kenya (2013) 4


5

Research Questions: We looked for answers to questions about users’ behaviors and survey methods. Novel questions in this research include:

Ø  Is a large sample (n=180) of freely-distributed cookstoves adopted in a humanitarian crisis situation?

Ø  How does self-reported adoption of a technology correlate with sensor-measured adoption in an internally-displaced peoples’ camp?

Ø  Do enumeration activities impact cookstove adoption?


6

Top left: Nada Abdalla Mohammed, Afaf Adam

Abdallha, Mohammed Idris Adam, Eissra Hamid Gamer

El ddin, Abel Rahman Abdalla Addoma, Aziza

Mohammed Tugod, Fatima Adam Ibrahim, Om Alhosein

Ali Garbo

Bottom left: Idris Ibrahim Adam, Abdalla Mohammed

Suleiman, Adam Abdalla Amin 6

SUMs Team: Darfur


7

Sampled Population:

•  180 stoves were distributed in El Salam IDP Camp.

•  Follow up surveys were conducted 1-3 months after baseline surveys.

•  For 60% of the sample, we conducted a second follow up survey.


8 !

Surprise failure mode: Sensors were extensively tested on BDS stoves in the Lab before field work, however, we had a a surprise in the field

•  Some users inverted the stove and cooked with charcoal causing the sensor location to overheat

•  28% of sensors failed, mostly due to thermal damage.

•  This likely biases the study towards underestimating adoption (by an unknown quantity).

Testing SUMs mounting locations at LBNL with an infrared camera. Summer, 2013.


RECOVERED LOST

9

Results: We developed an ad hoc event-detection algorithm to process data

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●25

50

75

100

125

04:00 10:00 16:00 22:00

Tem

pera

ture

(C

)

● ● ●maximum minimum none(a1)

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●25

50

75

100

125

04:00 10:00 16:00 22:00

Tem

pera

ture

(C

)


●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●25

50

75

100

125

04:00 10:00 16:00 22:00

Time

Tem

pera

ture

(C

)

● ●not cooking cooking(a3)

0.00

0.25

0.50

0.75

1.00

0.00 0.25 0.50 0.75 1.00

Proportion of Days Used

Quantile

non-u

sers

users (b)

0

10

20

30

40

50

0 2 4 6

Number of cooking events per day

Particip

ants Use Status

users

QRQïXVHUV

(c)

0

10

20

30

40

50

0 1 2 3 4 5

Hours cooked per day

Particip

ants Use Status

users

QRQïXVHUV

(d)3.  Consecutive minima and maxima consolidated into cooking events.

1.  Local minimums and maximums labeled

2.  Incorrect minima and maxima removed


10

Visualization of Data

(for the entire experiment population)

sensor not activated

low average use heavy average use

cookstove currently in use

0 hr/day 5 hr/day

10 Wilson, D.L., 24 JAN 2015

11

Results 1: 73% of participants use the free Berkeley-Darfur Stove

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●25

50

75

100

125

04:00 10:00 16:00 22:00

Tem

pera

ture

(C

)


●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●25

50

75

100

125

04:00 10:00 16:00 22:00

Tem

pera

ture

(C

)


●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●25

50

75

100

125

04:00 10:00 16:00 22:00

Time

Tem

pera

ture

(C

)


0.00

0.25

0.50

0.75

1.00

0.00 0.25 0.50 0.75 1.00


Quantile

non-u

sers

users (b)

0

10

20

30

40

50

0 2 4 6


Particip

ants Use Status

users

QRQïXVHUV

(c)

0

10

20

30

40

50

0 1 2 3 4 5


Particip

ants Use Status

users

QRQïXVHUV

(d)

•  Arbitrary definition of “users:” “users” operate the stove on >10% of ownership days; the rest are “non-users”

•  73% (89 participants) are categorized as “users” and 27% (33) as “non-users.”

11 Wilson, D.L., 24 JAN 2015

12

Results 2: “Users” of the Berkeley-Darfur Stove typically spend 1.5 hours per day cooking 2 meals

•  Among users, average daily stove use was 1.5 hours (SD = 0.9), and number of daily “cooking events” was 2.0 (SD = 1.3)

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●25

50

75

100

125

04:00 10:00 16:00 22:00

Te

mp

era

ture

(C

)


●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●25

50

75

100

125

04:00 10:00 16:00 22:00

Te

mp

era

ture

(C

)


●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●25

50

75

100

125

04:00 10:00 16:00 22:00

Time

Te

mp

era

ture

(C

)


0.00

0.25

0.50

0.75

1.00

0.00 0.25 0.50 0.75 1.00


Qu

an

tile

no

n-u

se

rs

use

rs (b)

0

10

20

30

40

50

0 2 4 6


Pa

rticip

an

ts Use Statususers

QRQïXVHUV

(c)

0

10

20

30

40

50

0 1 2 3 4 5


Pa

rticip

an

ts Use Statususers

QRQïXVHUV

(d)

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●25

50

75

100

125

04:00 10:00 16:00 22:00

Te

mp

era

ture

(C

)


●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●25

50

75

100

125

04:00 10:00 16:00 22:00

Te

mp

era

ture

(C

)


●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●25

50

75

100

125

04:00 10:00 16:00 22:00

Time

Te

mp

era

ture

(C

)


0.00

0.25

0.50

0.75

1.00

0.00 0.25 0.50 0.75 1.00


Qu

an

tile

no

n-u

se

rs

use

rs (b)

0

10

20

30

40

50

0 2 4 6


Pa

rticip

an

ts Use Statususers

QRQïXVHUV

(c)

0

10

20

30

40

50

0 1 2 3 4 5


Pa

rticip

an

ts Use Statususers

QRQïXVHUV

(d) 12 Wilson, D.L., 24 JAN 2015

13

●●

●●

●

●● ●

● ● ●●●

● ●

●

● ●●

●

●●●●

●

●

●● ●● ●

●

●●

●

●●

●●● ● ●

●

● ●●

●

●● ●●● ●

●●●●

●●●

●●

●●●

●● ●

●●

●

●● ●●●●

●

●● ● ●● ●●

●

●

●●

●

●● ●

●●

●

●

●●

●

●●●●

●● ● ●●

●

● ●● ●

●●

●●●

0

2

4

0 2 4Average events per day, by SUM

Aver

age

even

ts p

er d

ay,

by s

elf−

repo

rt

●

●

usersnon−users

!

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●● ●

●

●

●

●

●

●

●

●

●●

●●

●

●● ●

●

●

●

●●

●

●●

●●

●

●●

●

●

●●

●

●●●

●●

●

●

●

●

●●

●

●

●

●●

●●

●

●

● ●●●

●●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●●

●●

●

●●

●

●

●

●

● ●●

●

0

2

4

0 2 4Average hours per day, by SUM

Aver

age

hour

s pe

r day

,by

sel

f−re

port

●

●

usersnon−users

!

Results 3: Participants over-report use in surveys by roughly 2X •  ~85% of participants

overestimate cooking hours and events.

•  Average participants over-report daily cooking hours by 1.2 ± 0.2 hours, and daily cooking events by 1.3 ± 0.2 events (p=0.05). ~2X overestimation

13 Wilson, D.L., 24 JAN 2015

14

0

1

2

ï�� ï� 0 � 10Days before/after followup

Coo

k ev

ents

per

day

Use Statusboth

QRQïXVHUV

users

use spike in non-user

popoulation just before follow up

0.0

0.5

1.0

1.5

2.0

ï�� ï� 0 5 10Days before/after followup

Hou

rs c

ooke

d pe

r day

Use StatusbothQRQïXVHUV

users

use spike in non-user

popoulation just before follow up

Results 4: A follow up survey causes courtesy use of the BDS but also results in dramatic and sustained increases in adoption among “non-users”

•  Just before the follow up survey, “non-users” adopt likely due to social pressure.

•  After the follow up, “non-users” behave indistin-guishably from users.

14 Wilson, D.L., 24 JAN 2015

15 15

Conclusions 1.  28% of SUMs lost causing bias.

2.  At least 73% of recipients were classified as “users.”

3.  Among users, average cooking hours and events per day were 1.5 hours and 2.0 events, respectively.

4.  On average, adoption is over-reported (p=0.05). 85% of participants over-report adoption. Average self reports are roughly 2X sensor-measured values.

5.  The first follow up causes a significant (p = 0.05) increase in adoption among the “non-user” group.


16 16

EXTRA SLIDES

17

The Situation in Darfur •  UNHCR estimates that there are 2.8 million IDPs,

refugees, and asylum seekers in Sudan.1

•  Most of these people are in large IDP camps (>100k residents) like those surrounding Al-Fashir, North Darfur.

1.  2013 UNHCR country operations profile – Sudan. http://www.unhcr.org/pages/49e483b76.html (accessed October, 2013) 17 17

18

173 respondents to the follow up survey

In surveys: •  100% self report that they use the BDS •  94% self report the BDS is their primary stove for making

food or for making tea •  About one third of women with working SUMs are not

using their BDS.

Yes 100%

Do you currently use the BDS?

Yes 94%

No 6%

Is the BDS a primary stove for making food

or drinks?

Stove Use Self-Reported in Survey Use Measured By SUMs

18

yes

no

Has the BDS been used more than once in the last two

weeks (measured by SUMs)

18

19

Other studies use SUMs data for detailed evaluation

•  Kirk Smith’s group measured variation in adoption and use of cookstoves in Guatemala.1,2

•  SweetSense and Evan Thomas are developing SUMs that are remotely accessible, transmit data over the internet.

•  Berkeley Air are using SUMs and UCB-PATS (particle sensor) used to evaluate/compare Philips vs. Oorja stove usage in India.3

•  A new study by Thomas et. al demonstrates disparities between surveys and sensors, but using cost prohibitive custom sensors (~$500 with free labor) and a small sample (n = 27)

1.  Ruiz-Mercado I, Canuz E, Smith KR. (2012). Temperature data loggers as stove use monitors (SUMs): Field methods and signal analysis. Biomass and Bioenergy 47 (2012).

2.  Ruiz-Mercado I, Canuz E, Walker JL, Smith KR. (2013). Quantitative metrics of stove adoption using Stove Use Monitors (SUMs). Online / Article In-Press (final version forthcoming). Biomass and Bioenergy (2013)

3.  Mukhopadhyay, Rupak, et al. "Cooking practices, air quality, and the acceptability of advanced cookstoves in Haryana, India: an exploratory study to inform large-scale interventions." Global Health Action 5 (2012).

19

20

08/01 09/01 10/01 11/010

50

100

150

date (mm/dd)

tem

pera

ture

(C)

SUMs Heavy User

08/01 09/01 10/01 11/010

50

100

150

date (mm/dd)

tem

pera

ture

(C)

SUMs Medium User

Sampling bias problem

•  Roughly one-fifth of sensors overheated and were lost during the experiment (likely due to inverted stoves).

08/01 09/01 10/01 11/010

50

100

150

date (mm/dd)

tem

pera

ture

(C)

SUMs Non User over represented

20

~1/3 of users

~1/3 of users

~1/3 of users

? yes no

20

21

0

20

40

60

80

100

120

140

160

180

200

0 2 4 6 8 10 12

Qua

ntity

of S

UM

s

Weeks Since Deployment

4.9 minutes"

SUMs collection and 1st follow up

Al-Fashir Rural Administrative Unit

Dar Zagawa Administrative Unit

Jabl Si Administrative Unit

Tawilla Administrative Unit

Korma Administrative Unit

9.8 minutes

12.3 minutes

14.7 minutes

3 min independent SUM 1 min “piggy back” 2nd SUM

7.4 minutes All units have 36 stoves: - 30 stoves: 1 SUM & 1 dummy - 4 stoves: 2 SUMs - 2 stoves: 2 dummies

4.9 minutes

Deployment Methods: deploy sensors in 5 groups at different sampling rates and follow up at different times; for 3 groups, a second follow up occurred

21

cookstove adoption measured with sums in darfur: a ... · discussion of algorithms, baseline...

Documents