MONITORING  &  MEASURING  CHANGE  IN  MARKET  SYSTEMS  

Systemic  M&E  Principles  in  the  context  of  Kenya’s  Market  Assistance  Programme  

 

SEVEN  PRINCIPLES  FOR  M&E  OF  INCLUSIVE  MARKET  DEVELOPMENT  

 •  USAID-­‐funded  SEEP  Network  Case  Study  based  on  Kenya  Market  Trust’s  DfID-­‐funded  Market  Assistance  Programme  

•  hIp://www.slideshare.net/MarketDevelopmentEastAfrica/systemic-­‐me-­‐case-­‐study  

•  Markets  are  complex  adapQve  systems  (conQnually  evolve  and  adapt)  

•  M&E  frameworks  for  inclusive  market  development  must  embrace  systemic  thinking  and  complexity  science  as  well  as  culQvate  flexibility  and  organizaQonal  learning  

PRINCIPLE  1:  INDIRECTNESS  OF  IMPACT  

Systemic  change  cannot  be  directly  delivered    by  a  development  iniQaQve  from  outside    -­‐  needs  to  emerge  from  within  the  system  

 

Issues:  •  Difference  between  direct  and  indirect  beneficiaries  becomes  pracQcally  useless    

•  InformaQon  needed  for  management  and  learning  is  different  from  the  informaQon  required  by  donors  

PRINCIPLE  2:  DEPTH  OF  IMPACT  •  Changes  in  a  system  are  different  depending  on  how  structural  and  long-­‐

lasQng  (how  “deep”)  they  are  •  Superficial  changes:  variaQons  in  incomes  and  number  of  jobs  •  Deeper  changes:  self-­‐awareness  -­‐  when  the  system  quesQons  its  own  nature  and  

evoluQon;  creaQon  of  new  relaQonships,  increased  access  to  informaQon,  shi^s  in  behavior  and  power  dynamics  

•  Closer  aIenQon  must  be  paid  to  deeper  changes  •  Consider  range  of  indicators  and  signals  that  indicate  that  change  is  occurring  at  

deeper,  more  structural  levels  of  the  system  •  Look  beyond  the  individual  market  actors  and  their  business  models  to  find  paIerns  

that  are  emerging  in  the  wider  system  •  Put  a  strong  emphasis  on  behavioral  aspects  of  the  market  actors    

   •  Issues:  

•  Pressure  from  donors  to  focus  on  reporQng  impacts  of  superficial  changes  •  All  types  of  change  provide  useful  informaQon,  but  an  exclusive  focus  on  the  most  

superficial  ones  will  have  negaQve  consequences  on  sustainability  and  scalability    

z  

PRINCIPLE  3:  NETWORK-­‐DRIVEN  CHANGE  •  Work  with  collaborators  who  have  potenQal  to  produce  

changes  that  “reverberate”  throughout  the  system  •  Understand  structures  and  dynamics  of  networks    •  Create  condiQons  for  self-­‐selecQon  of  key  collaborators  •  Promote  disseminaQon,  copying,  and  adaptaQon  by  collaborators  of  the  successes  or  lessons  generated  

•  Change  is  driven  by  the  collaborators’  networks,  not  by  them  as  individuals  

•  Unresolved  issue:  •  M&E  framework  does  not  generally  have  procedures  or  tools  to  measure  network  structures  and  dynamics  

•  Remains  in  the  subjecQve  or  intuiQve  domain,  linked  to  the  people  who  interact  with  the  networks  on  a  regular  basis  

PRINCIPLE  4:  UNPREDICTABILITY  

Behavior  and  evoluQon  of  complex  systems    resist  predicQon  

 •  Need  flexibility,  rapid  learning,  and  effecQve  

collaboraQon    •  OrganizaQonal  culture  that  rewards  faster  learning,  course  

correcQon,  adaptaQon  to  unexpected  circumstances,  promotes  trust,  values  mistakes  and  problems  as  opportuniQes  for  success  

•  Movement  away  from  long-­‐term,  staQc  strategic  analysis  and  planning  of  fixed  indicators    

•  Allow  teams  to  discuss  and  re-­‐evaluate  intervenQons,  exchange  lessons  quickly  and  reflect  about  progress  and  challenges  

PRINCIPLE  5:  SENSITIVITY  TO  EXTERNAL  SIGNALS  •  When  an  NGO  or  a  donor  declares  intenQon  to  intervene  in  a  

market  system,  public  and  private  actors  adapt  their  behavior  and  strategies,  seeking  to  maximize  benefits  in  many  different  ways  (eg  for  economic  gain,  reputaQon,  and  influence)  

•  Facilitators  must  be  careful  about  the  signals  they  consciously  or  unconsciously  send  when  they  engage  with  market  actors  

•  Successful  facilitators  manage  the  signals  they  send  to  promote  self-­‐selecQon  and  determine  the  honesty  and  commitment  of  strategic  actors  who  are  willing  to  collaborate  

•  How  a  program  uses  its  own  idenQty  is    very  important  and  needs  to  be  tailored  to  the  specific  situaQon  (challenging  the  hypothesis  that  minimizing  the  possibility  of  being  perceived  as  an  influenQal  subsidy-­‐giver  or  helper  is  the  only  recommended  strategy  for  facilitators)  

PRINCIPLE  6:  INFORMATION  DEFICIT  No  maIer  how  much  we  know  about  a  market  system,  there  will  always  be  informaQon  that  we  ignore  that  can  have  important  

consequences  on  how  the  system  performs  and  evolves    

 •  QuesQon  how  much  we  should  know  before  we  intervene  in  the  system  •  Need  for  conQnuous  learning,  effecQve  communicaQon  and  collaboraQon  

among  key  stakeholders  and  partners,  opportunism  and  flexibility  throughout  the  design,  implementaQon,  M&E  

•  M&E  Framework  co-­‐evolves  with  the  staff’s  knowledge  and  understanding  of  the  system  

•  Do  fewer  detailed,  hands-­‐off  analyses  (“cold”  analyses)  and  more  “full  immersions”  in  the  system  (closer  and  more  frequent  interacQons  with  stakeholders  to  experiment  with  new  ideas)  

•  AIach  a  high  value  to  informaQon  from  field  observaQons  

PRINCIPLE  7:  SUSTAINABILITY  AS  ADAPTABILITY  •  Sustainability:  efforts  to  increase  the  capacity  of  market  actors  to  conQnue  changing  their  own  systems  in  the  eventuality  of  shocks  or  new  trends  

•  M&E  frameworks  need  to  detect  whether  a  system  is  building  its  capacity  to  adapt  to  shocks  and  new  trends  

•  Require  shi^  in  understanding  of  “sustainability,”  from  the  permanence  of  given  outcomes  through  Qme,  to  the  capacity  of  a  system  to  benefit  from  new  opportuniQes  and  minimize  negaQve  impacts