8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 1/20
Monitoring and evaluation ofcommunity systems strengthening
Ms. Nathalie Zorzi
Manager, Monitoring and Evaluation Support TeamThe Global Fund18/19 March 2010
8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 2/20
CSS indicator development process
• August 2008: The Global Fund commissioned a review exercise in Pretoria to defineindicators for community level service delivery and systems strengthening;
• January 2009: UNAIDS developed a guidance tool for including Community SystemsStrengthening in Global Fund proposals which includes a number of CSS indicators;
• November-December 2009: A multi-partner Technical Working Groups oversaw theimplementation of 9 field exercises which aimed at capturing best practices in
community level service delivery and systems strengthening including the set of CSSindicators used at country level;
• February 2010: a harmonization workshop was organized in Geneva where M&Eexperts from various organizations reviewed existing CSS indicators and developed anupdated list which was aligned with the proposed building blocks and SDAs of the newCSS framework;
• March 2010: An internal review and further definition of the indicators was done by theGlobal Fund. Comments from the Technical Working Group and other stakeholderswere integrated into this process.
8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 3/20
Challenges - Review of CSS case studies
• Challenges at community level and CSOs- Community level workers have limited language
skills, multiple reporting, and high turnover.
- Electricity & internet have limited availability
- Ability to analyze the collected data for decision makingis limited – resulting in a perception of “purposelessdata collection”
- When you can’t make sense of your own data how can
you explain it to others in the community for theirmeaningful involvement?
8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 4/20
Challenges
- Limited capacity at sub-national & national level for written feedback &supervision to show purpose in data
- Poor planning for trainings for M&E at all levels
- Poor correlation of program performance, logistics and finance (morepurposelessness)
- Thus, loss of opportunity of learning at community level
- Poor data analysis and strategy at national level for systematicinvolvement of CSOs
- CSOs are registered with departments other than health causingcounting difficulties
8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 5/20
Approach
• Include both M&E for service delivery and CSS• Community level M&E means community level of bothCSOs and public health system
• Large CSOs follow M&E for both community level andhealth facility level according to the level of application
• Community level M&E to be integrated with national levelat the planning stage itself – Design jointly by various stakeholders – Harmonized list of indicators – Common reporting formats and data collection tools
8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 6/20
Approach
• MoUs between disease program and CSOs – Avoids double counting – Gives baseline for calculation of % for indicators – Better understanding of roles & responsibilities – Facilitated by provision of guidelines from the national level for
involvement of CSOs
• Build capacity of community level in sharing information with thecommunity• Include meetings at community level and program review at various
levels• Include operational research at community level• Include CSOs in evaluations and joint reviews and - learning
opportunities for health facilities and CSOs
8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 7/20
Approach
• Assuring data quality – Publication of M&E guidelines giving standardized data collection
tools and reporting formats
– Simple forms with minimal information at peripheral level
– Training & re-trainings of staff on M&E
– Guidelines on frequency and content of written feedback andsupervision
– Ensuring dedicated trained M&E staff at all levels for supervision(and their travel); ensure data entry staff
– Standardization of data base
8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 8/20
Current status of CSS indicators (March 2010)
• Currently we have a list of 21 indicators with a description of the rationale,definition, numerator, denominator, proposed method of measurement, datasources and data collection frequency defined
• Page 21 of the CSS indicator document contains a number of additionalindicators for consideration
• The list of CSS indicators should be updated and finalized to be ready forGlobal Fund Round 10 (May 2010)
8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 9/20
Building Blocks, SDAs and indicators (1)
BB1: Monitoring
& Evaluation
and Planning
SDA 1:
Monitoring
& evaluation
# and % of organizations working at the community level with a staff
member responsible for monitoring and evaluation (CSS-ME1.1)
# and % of organizations working at the community level using
standard data collection tools and reporting formats that enable to
report to the national reporting system (CSS-ME1.2)
# and % of organizations working at the community level submitting
timely, complete and accurate program reports according tonationally recommended guidelines (CSS-ME1.3)
# and % of organizations working at the community level conducting
reviews of their program performance in the past 3/6 months (CSS-
ME1.4)
SDA 2: Strategic
planning
# and % of organizations working at the community level with a costed
annual work plan in place which includes monitoring and
evaluation activities (CSS-SP2.1)
# and % of organizations working at the community level with at least
one staff member who received training or re-training in planning
and M&E according to nationally recommended guidelines in the
past 2 years (CSS-SP2.2)
8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 10/20
Building Blocks, SDAs and indicators (2)
BB2: Resources
and capacity
building
SDA 3: Community
actors: skills
building for
service delivery
# and % of community health workers currently employed who
received training or re-training in HIV, TB or malaria service
delivery according to national guidelines in the past 2 years (CSS-
SB3.1)
# and % of currently employed staff members of organizations
working at the community level that have been recruited more
than 1 year ago (CSS-SB3.2)
# and % of organizations working at the community level that receivedsupervision in accordance with national guidelines in last 3/6
months (CSS-SB3.3)
SDA 4: Financial
resources
# and % of organizations working at the community level with a
budget coming from more than one source (CSS-FR4.1)
# and % of organizations working at the community level that submittimely, complete and accurate financial reports according to
nationally recommended standards and guidelines (CSS-FR4.2)
# and % of organizations working at the community level that are
planning to implement resource mobilization activities (CSS-
FR4.3)
8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 11/20
Building Blocks, SDAs and indicators (3)
BB2: Resources
and capacity
building
SDA 5: Material
resources -infrastructure;
information;
essential
medical and
other products
& technologies
# and % of organizations working at the community level with the
minimum infrastructure in place to deliver quality HIV, TB and
malaria services (CSS-MR5.1)
# and % of organizations working at the community level that have not
experienced stock-outs of essential commodities in the last 3/6
months (CSS-MR5.2)
% of organizations working at the community level that keep adequate
logistics data for inventory management (CSS-MR5.3)
# and % of organizations working at the community level with staff
trained or re-trained in stock management in the past 2 years
(CSS-MR5.4)
# and % of organizations working at the community level that maintain
acceptable storage conditions and handling procedures (CSS-
MR5.5)
8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 12/20
Building Blocks, SDAs and indicators (4)
BB3:
Organisational
strengthening
SDA 6:Management,
accountability
and leadership
# and % of organizations working at the community level with staff
who have received training or re-training in the past 2 years on
management, leadership and accountability (CSS-MA6.1)
BB4: Community
networks,linkages &
partnerships
SDA 7: Building
community
linkages andcollaboration
for service
delivery
# and % of organizations working at the community level that are
implementing activities in partnership with other organizations
working at the community level or with the national healthsystem (CSS-CL7.1)
BB5: Enabling
environment
SDA 8:
Communicatio
n and advocacy
for enabling
environments
and community
engagement
# and % of organizations working at the community level that are
planning to implement communication or advocacy activities to
address barriers to equitable access for target populations
(CSS-CA8.1)
# of legal arrangements amended or put in place to build an enabling
environment for community engagement (CSS-CA8.2)
8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 13/20
Issues for discussion
• Are the indicators useful for actors involved with CSS that aim at strengtheninga limited number of organizations at the community level?
• Are there any considerations regarding the feasibility of data collection for theproposed indicators?
• If the list of building blocks and SDAs will be amended will this have an impacton the proposed indicators and how do we address this?
• How do we ensure that all actors involved with CSS have access to theindicators?
• Are there any suggestions on the next steps in the CSS indicator developmentprocess?
8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 14/20
Asia feedback1. Make the definitions more concise
1. Change from Community Systems Strengthening toCommunity SECTOR Strengthening (treat CSS as a strategy& not as a system)
2. Community Actors be changed to mean Key AffectedPopulations and grass-root community members.
2. The framework should focus on GF programming3. A new set of 7 building blocks with 14 SDAs have been proposed
4. The framework should be included in the ‘expanded’ version ofthe M&E toolkit
5. Technical support & assistance should be more local & peer-
based
8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 15/20
Proposed BB & SDAs
BB1:Sustainable funding
SDA1: Sustainable financial resources
BB2:
Community sectorworkforce
SDA 2: Build community workforce andstrengthen overall capacity of new and
existing staff and volunteers
SDA 3: Build technical skills for servicedelivery (of CBO staff and volunteers)
8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 16/20
Proposed BB & SDAsBB3:
Organizational
& management
Strengthening
SDA 4:
Organizational/management support andtraining for small NGOs/CBOs
SDA 5: Transparent financialmanagement structures
BB 4:
Planning and M&E
SDA 6: Needs assessment, gap
analysis, mapping
SDA 7: Evidence-based strategic planningamong community groups, linked to M&Esystems. (including OR)
8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 17/20
Proposed BB & SDAsBB 5:
Communication,
Information &
Networking
SDA 8: Partnership building & collaboration,
including networks of KAP
SDA 9: Information and
knowledge sharing (including
mechanism for information
sharing, material development
and resources)
BB 6:
Accountability and
transparency
SDA 10: Models & tools for community accountability & transparency to itsconstituency
8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 18/20
Proposed BB & SDAsBB 7:
Social
Mobilization &
Advocacy
SDA 11: Monitoring and
documentation (of both
community and government
interventions)
SDA 12: Advocacy, Campaigning &Community mobilization
SDA 13: Enabling environment
(policy, legal, governance)
SDA 14: Reducing stigma & discriminationin all settings
8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 19/20
Thank you
8/2/2019 Monitoring and Evaluation of Community Systems Strengthening
http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 20/20
Covered in this presentation
1. Indicators for community systems strengthening
2. Challenges of community level monitoring and evaluation
3. Approaches to overcome these challenges
4. Feedback on the CSS framework from Asia