main doc rom applied to grant implementation

Results Oriented Monitoring Review of Implementation of Projects and

Programmes Financed by the European Union for the Latin American and

Caribbean regions, including Cuba and OCTs

Effective Monitoring of EU Grant-

Implemented Projects

Prepared for Integration GmbH by TM Kowal, Key Expert for Climate Change,

Green Growth, Environment and Energy

Results Oriented Monitoring Review of Implementation of Projects and Programmes Financed by the European

Union for the Latin American and Caribbean regions, including Cuba and OCTs 2

Objectives of this training

Clarify understandings of the Grant Contract and of ‘PRAG’

Guidelines which decide how EU funds are spent in external actions

SESSION 1 – about grant funding projects

BREAK

SESSION 2 – about effective ROM and EOPRR

PRACTICAL EXERCISE – application of information

SUMMING UP AND POINTERS TO SOURCES



SESSION 1

SETTING THE CONTEXT

Main features of the grant aid modality in terms of the key differences with other aid funding methods

the attributes of the grant-funding project

development and approval process i.e. Calls for Proposals

main implementation methods and issues

key documents generated over the project lifecycle

related to project financial management

activity/output/outcome monitoring



What is PRAG, what is it for, and what does it contain?

PRAG Practical Guide to Contract Procedures for EU External Actions

PRAG English: April 2014; French, Spanish, Portuguese, Italian, German

Provides Practical Guidelines for contractual & procurement procedures

Guides how we can spend EU funds financed from the EU general budget and European Development Fund (EDF)

Rules: how we need to be accountable, transparent & fair: for ‘best value’

Explains procurement or grant contracting procedures for external actions

Set of key tools for project approvals, awards, signature, implementation



Why PRAG can be a challenge

PRACTICAL GUIDE TO CONTRACT PROCEDURES FOR EU EXTERNAL ACTIONS

Grant recipients are not generally structured to manage Public Funds and

often have weak capacity for managing finances

Language of PRAG is “not always easy” to understand

Extreme formulation: heavily framed in legalisms, acronyms, exceptions,

inclusions, all written out in dense “Euro-speak” so hard to disentangle

As PRAG offers ‘guidelines’ rather than ‘rules’, different stakeholders may

interpret aspects of the guidelines in various ways

BUT, these challenges must be overcome, otherwise recipients may be in

danger of not having the grants approved, fail to comply and not have

their grant-project costs recovered



Main features of the grant aid modality

A Grant or Call for proposals is a public invitation by the Contracting Authority, addressed to clearly identified categories of applicants, to propose operations within the framework of a specific EU programme.

Grants are direct financial contributions from the EU budget or from delegated funds.

Awarded as donations to third parties engaged in external aid activities.

Contracting Authority awards grants that are used to implement projects or activities that relate to the EU’s external aid programmes.

Two categories:

Grants for actions: to achieve objective as part of external aid programme

Operating grants: finance the operating expenditure of an EU body that is pursuing an aim of general European interest; or an objective that forms part of an EU policy.



Characteristics of a ‘grant’

Payment of a non-commercial nature by the Contracting Authority to a

specific beneficiary to implement an action intended to help achieve

an objective forming part of a European Union policy

But it is not a gift!!!

Funds support activities of the beneficiary

Funds cover only eligible incurred costs

Based on the reimbursement of the eligible costs: costs effectively

incurred by the beneficiaries deemed necessary for activities in question.

Results of the action remain the property of the beneficiaries

Conditionality of the grant award is set out in the Grant Contract as

a written agreement signed by the two parties.

Require co-financing by the grant beneficiary:

beneficiary's own resources (self-financing)

income generated by the action and

financial or in-kind contributions from third parties.



What are grants?

Summary:



Differences with procurement

Procurement

"Buying things“ via

Tenders

Grants

"Giving money“ via Calls for proposal

Purchase of services,

supplies or worksObject

Proposal from an applicant to contribute to the

achievement of a policy objective through:

- action grant: a project

or

- operating grant: applicant functioning costs

Contracting AuthorityOwner of

Results

Grant beneficiary: falls within the normal framework

of the beneficiary's activities

100% of the costFinancial

contribution

The Union finances a part of the costs, which are

eligible for Union-financing.

The grant beneficiary (or another donor) finance the

other part.

Allowed Profit Not allowed as it applies to the action



Differences with Contracts for Services, Supplies and Works:

Service Contracts Supply Contracts Works Contracts

Contracts of €150.000 or moreInternational open tender procedure

Contracts of €5.000.000 of more International open tender procedure

Contracts of €200.000 or moreInternational restricted tender procedure

Contracts between €60.000 and €150.000Local open tender procedure

Contracts between €300.000 and €5.000.000Local open tender procedure

Contracts under €200.000 but more than €10.000

Framework contractsCompetitive negotiated procedure(Beneficiary consults at least 3 service providers)


Competitive negotiated procedure(Beneficiary consults at least 3

service providers)


Competitive negotiated procedure(Beneficiary consults at least 3

contractors)

Contracts with a value of €10.000 or lessSingle tender





Actors

Three kinds of actors that may receive funding under a grant contract:

Contracting Authority filled by:

Commission itself

A national body

A beneficiary country

An international organisation.

Applicant

If awarded the grant contract, the applicant will become the beneficiary

Identified as the coordinator in special conditions of the grant contract

The coordinator is the main interlocutor of the contracting authority.

Represents and acts on behalf of any other co-beneficiary (if any)

Coordinates the design and implementation of the action

Co-applicants (if any) - who will become the co-beneficiaries

Co-applicant(s) participate in designing and implementing the action

Costs they incur are eligible in the same way as those by the applican.

Affiliated entities (if any)

Having a structural link with the applicants, a legal or capital link, considered affiliated



Special context set by NGOs and municipal/local level actors

Partner-owned and partner

managed

Designed/ decided by the

partner with donor input

Embedded in the local

institutional context

Balanced programme

autonomy vs. integration.

Results may be achieved

with inputs/ resources of

other organisations within

private, CSO, public and

INGO sectors

Identifying Stakeholders

STAKEHOLDERS - Individuals or institutions that

may – directly or indirectly, positively or negatively –

affect or be affected by the project or program.

14

Private Sector

and TA Provider

Companies

Donors,

Public Sector

Multi-stakeholder

alliances

Civil Society Organisations

and Local actors

(municipal-level) Watchdog of industry

Providing expertise on local situation

Implementation partner

Finding a common language between actors:



‘Macro’ Projects

With objectives of national

importance

Maybe cross-sectoral

Managed from the Centre

Often complex with multiple

components

E.g. Modernisation of a

Ministry; Promoting an

active labour market

‘Micro’ Projects

With objectives of local importance or ‘pilot’ in nature

Usually sector-based

Managed locally, but may be part of a ‘macro’ project

Usually limited to one type of ‘action’

Eg.s Improving drinking water supply of municipality; improving services for internal migrants

CONVERGENCE OF ACTORS’ INTERESTS



Overall life-cycle

Project Stages:

1) Proposal Preparation based on Call for Proposals

2) Evaluation

3) Negotiation

4) Project Start & Implementation

5) Project Reporting

6) Project Closure

7) Post-Project Tasks

Pro

ject S

tages

18



Strict rules governing the way in which grants are awarded

Transparency

Availability of grants publicised; accessible

Observance of the requirements of

confidentiality and security

Equal treatment

Grant award process completely impartial.

Evaluated by an evaluation committee

Advice of assessors; published criteria.

Programming:

annual work programme

Procedures and rules regarding grants

are outlined in the General Conditions

and Special Conditions. These vary

according to the source of money for the

grant: EU budget or EDF and the year



Types of programming contexts / genesis via financing instruments

DIRECT MANAGEMENT:

Work programme is adopted by EC & published on the

EuropeAid website

Chairs evaluation committees

Decides the results of calls for proposals and signs the contracts

INDIRECT MANAGEMENT WITH EX-ANTE CONTROLS:

Work programme is adopted by the contracting authority

Published on its Internet site and on the EuropeAid website

Contracting authority submits the work programme to the EC for

approval

Chairs evaluation committees; decides results of calls for

proposals

European Commission is represented as an observer

INDIRECT MANAGEMENT WITH EX-POST CONTROLS:

Work programme is adopted by the contracting authority

Published on its Internet site (or in any other appropriate media)

and on the EuropeAid website

Decides on the results and signs contracts without prior

approval of the EC.



Award procedures calls for proposals



Key characteristics and trends in calls for proposals:

Competitive – which tendencies ?

Right to proponent's initiative

Procedural obligations – in order to maximise equality of chances of all

applicants and transparency

Budget obligations: no core financing, but activity related financing;

obligation of co-financing

Technicalities are important: HQ or local calls? Amounts? Eligibility?

Pooling of funds



Step 1: Preparation and Guidelines



Typical call for proposals:



Calls for proposals

Grants must be awarded following the publication of a call for proposals

Open or restricted call for proposals

Calls for proposals are by default restricted

Two-step procedure where applicants may ask to take part

Concept note in response to a call launched through published guidelines

Applicants who have been shortlisted invited to submit a full proposal

In exceptional cases: calls for proposals may be open

Applicants are free to submit a grant application

Concept note must be submitted together with the full application

Evaluation process is still carried out in two steps

Launch of an open rather than a restricted call must be justified by the particular technical nature of the call, the limited budget available, the limited number of proposals expected or organisational constraints



Step 2: Potential Applicant Data On-Line Registration

PADOR is an on-line database in which organisations register as

potential applicants to calls

Intended to improve service aimed at non-state actors and local

authorities (not individuals) through calls for proposals.

Organisations are invited to regularly update their information, and

EuropeAid uses it to evaluate their operational and financial

capacity, as well as their eligibility.

PADOR eases the application process for organisations, as they are not

obliged to re-submit this information separately

Get a username and password from the Authentication Service (ECAS)

Indicate their unique identification number, called EuropeAid ID (PADOR

number), in their application forms.



Step 3: Evaluation of the Proposals



Role of EC in endorsement:



Eligible and non-eligible costs

Costs effectively incurred by the beneficiaries that are deemed necessary

for carrying out the activities in question

PRAG explains the rules and procedures to be applied for projects

Staffs of the partners taking part in the Action incur eligible costs

Non-retroactivity

Grants may, as a rule, only cover costs incurred after the date on which

the grant contract is signed.

Exceptionally, a grant may be awarded for an action which has already

begun only where the applicants can demonstrate and justify the need to

start the action before the contract is signed.

No-profit rule

Grants may not have the purpose or effect of producing a profit within the

framework of the action or the work programme



Eligible costs: summary



Parts of the Grant Contract

Annex I: General and Administrative Provisions:

Article 1 - General provisions

Article 2 - Obligation to provide financial and narrative reports

Article 3 - Liability

Article 4 - Conflict of interests

Article 5 - Confidentiality

Article 6 - Visibility

Article 7 - Ownership/use of results and assets

Article 8 - Evaluation/monitoring of the action

Article 9 - Amendment of the contract

Article 10 - Implementation

Article 11 - Extension and suspension

Article 12 - Termination of the contract

Article 13 - Applicable law and dispute settlement

http://ec.europa.eu/europeaid/companion/document.do?nodeNumber=19.2.1















Annex III : Budget for the Action

Budget is a breakdown of all eligible project related

costs.

Important to have a clear, and easy to understand

budget attached to the contract

Directly impacts smooth implementation of the project.

Operating grants: operating budget

Estimate Only!!!

Final disbursement will depend upon project outputs



Modifying the Contract

Making major changes to contract should be handled carefully, if possible, avoided!

Minor Changes

Changes in activities or budget that do not affect the basic purpose of the project

Transfer between budget headings involving a variation of 15% or less

Changes of address or phone number

Changes of bank account

Change of project manager / key experts

Changes to HR unit rates

Major Changes

Significant changes in activities

affecting basic purpose / indicators

Transfer amongst budget headings

outside the limits (15%)

Addition of a new budget line or

spending zero against a budget line

Change of Project partners

Extension of Contract

Changes in grant beneficiary status

or name



SESSION 2

Applying the information

Based on the contextual information, participants will be engaged in the

implications of grant-funding aid modality for ROM and EOPRR and

understand

main risks and assumptions affecting grant-funded projects

commonly-encountered errors & irregularities

good practices found useful in effective monitoring

recommended information-gathering methods



Contract preparation

Special Conditions:

Options for payments

One payment per year, one report per payment

Annexes:

Description of the Action

General Conditions

Budget

Contract award procedures

Standard request for payment and financial identification form

Templates for narrative and financial reports

Expenditure verification report

Financial guarantee form



Managing the Contract

Managing the grant contract should be an integrated

part of the overall project management

Grant partners must pay special attention to:

Making any changes to the contract

Procurement of any goods/services above 10,000 euros

Keeping records & reporting



Reporting: Why is it (so) important ?

Why is it (so) important ? Contractual obligation!

Criticism from CoA / Parliament on the quality of

reporting and DEVCO commitment

Competitive environment: actors to demonstrate

value for money / concrete achievements.

Full information on the implementation of the action

No set format/template, but the structure of reports

should match Annex I (narrative report) and Annex III

(financial report) with the same level of detail

Templates provided for Grants



Evaluation and Monitoring: EU’s right to command ROMs and/or evaluations on any action it funds:

EU’s right to command ROMs and/or evaluations

on any action it funds

Operational principles:

Collaborative manner

Planned ahead

Procedural matters agreed by both parties

Draft of the report provided for comments



Why M & E?

Identify the impact and sustainability of project

Record and share lessons

Account to stakeholders

Improve future project design

Ensure funds used effectively and efficiently

Enable EC to monitor performance of projects

they have funded



Keeping Records & Reporting

During the implementation period Grant Beneficiaries need to:

Manage their own internal monitoring and administration of the project

Facilitate external monitoring of the project

Provide obligatory reports to the Contracting Authority

Systems must be in place to ensure that there is compliance to the

Contract and PRAG

Manage data to show progress (or otherwise) towards project’s objectives



Types of monitoring-Evaluation-Audit:



Approaches to ROM Monitoring

The Spectrum of ApproachesTechnocratic Pluralist

Scientific approach Mixed perspectives

Objectivity Non-neutral

Standardisation Non-standard

Judgemental / Managerial Facilitates learning

& further dialogue

Importance of Combining Different Data

Collection Techniques

A skillful use of a combination of qualitative and quantitative

techniques will give a more comprehensive understanding of the

topic

Qualitative Techniques

(Flexible)

Quantitative Techniques

(Less Flexible)

VS.• Produce qualitative data that is often

recorded in narrative form

• Useful in answering the "why",

"what", and "how" questions

• Typically includes:

– Loosely structured interviews

using open-ended questions

– Focus group discussions

– Observations

• Structured questionnaires

designed to quantify pre- or post-

categorized answers to questions

• Useful in answering the "how

many", "how often", "how

significant", etc. questions

• Answers to questions can be

counted and expressed

numerically

Final Reports

AnalysisROM Monitoring Process

The ROM Process starts wide and finishes narrow



Narrative report shall include at least:

EC criteria

Efficiency

Effectiveness

Sustainability

Replicability

Does the information in your ROM report provide evidence for

the above?



Reporting pre-requisites

Logical Framework as basis for M & E

Baseline studies

Monitoring plan

Financial reports (minimum quarterly)

Annual reports

Case studies

Mid-term reviews and evaluation



Keeping Records & Reporting: Financial Reports

Financial Reports include:

Project costs

Standardised table format

Copies of supporting

documents

Copies of Time-sheets.

Checking of financial reports against contract and supporting documentation:

Eligible expenditure

Consistent with application

Correct exchange rates being used

Appropriate supporting documentation

VAT is not included

Significant deviance from planned expenditure



Keeping Records & Reporting:

List of some of the questions that an external project

monitor may want answers to:

Were the correct procedures used for the tender value?

Are the equipment/supplies there?

Is it being used for the purpose it was intended?

Do the serial numbers match the invoice?

Is the EU being given appropriate visibility?

What will happen at the end of the project?

Check for a list of equipment.



Key documents generated over the project lifecycle:

Project financial management and activity/output

monitoring;

Roles of ROMs

Baselines/reviews/evaluations; logframes

Narrative reports: model and major elements

Financial report : elements (expenses and expenditure

verification reports)

Modifications and addenda etc



Supporting Documents for grant-funded projects

For Human Resources

Term of reference/job profile

Timesheets

Copies of the outputs

Activities reports

For Travel

Tickets (paper tickets and boarding passes)

Any other proofs

For Trainings

Description of the training – training modules, training hours per subject, methods, etc.

Training schedule

List of participants / Attendance sheets

List of trainers / lecturers

Feed-back questionnaires

Training evaluation report

For Surveys /studies

Description of the methodology

Reports

For Publications

Copies of publications

Distribution lists

For working meetings

Programme

List of presentations / Handouts

Minutes (if applicable)

Press clippings

Feed-back questionnaires (if applicable)

For subsidiary supplies or wWorks

Invitation to tender (all documentation)

All offers received

Tender Opening Report

Administrative Compliance Grid

Technical Evaluation Grid

Evaluation Report

Shipping bills



Some specific useful techniques used to gather information:

Documentation review

Questionnaires, surveys, checklists

Case studies: interviews with selected stakeholders

using pre-set questions for comparison purposes

Focus groups: facilitated stakeholder group

discussions around specific issues which ensure that

all voices are heard

Individual interviews: in-depth exploration using non-

directive questioning




Summary and context of the Action

Activities carried out during the reporting period (i.e.

directly related to the Action description and activities

foreseen in the Agreement)

Difficulties encountered and measures taken to overcome

problems

Changes introduced in implementation

Achievements & results by using the indicators included in

the Agreement

Report on outcomes on final beneficiaries

Work plan for following period inc. objectives & indicators

of achievement.




Sustainability issues; reports on gender equality,

disabilities etc

Summary of any controls carried out and available

audit reports in line with the Organisation’s policy

on disclosure of such controls and audit reports.

Where errors and weaknesses are identified,

analysis of their nature and extend as well as

information on corrective measures should be

provided;

Control measures carried out and their impacts.

Lessons learned and how disseminated.



What factors can affect the efficiency of a Project?

Interacting issues:

1) Having to apply the Budgetary and Planning Guidelines

2) Application of PRAG

3) Existence of the deadline for contracting out fund UE (n+3)

Deficient planning processes in the PMU

Staff do not account for PRAG complexities

Do not plan in time to meet the milestones of each component or overall

Teams do not have enough training in planning by results

Team members are not flexible enough to re-orientate the annual or

three monthly plans to better deal with risks stemming from the context

Mismatches between Project plans & annual budgetary planning processes

Major disequilibrium between rational, measured budgetary planning in

planning by period, instead concentrating majors flows of funds in the

periods just before the conclusion of period n+3.



MAJOR GENERIC ISSSUES AFFECTING GRANT FUNDED PROJECTS

PRAG application involves the implementation of UE

procedures and can:

i) to reconcile PRAG with national procedures,

administrative staff require a double effort to

simultaneously address different reporting

requirements and processes.

ii) Exceeding the installed capacity of the PMU

(knowledge of the assigned personnel, volume of

human resources, estimated time of dedication).

iii) subject to change during ongoing operating

periods, requiring a readjustment of advanced

procurement procedures and subtracting efficiency



MAJOR GENERIC ISSSUES AFFECTING GRANT FUNDED PROJECTS

Subject to a management contract deadline (n + 3) may

cause the following risks:

i) Contracts made in haste and poor quality in the near

future to n + 3.

ii) Excessive pooling procurement / contracting under unique

bidding, can result in the loss of quality of goods and

services contracted.

iii) Project start dates from date of signing do not necessarily

coincide with the actual start date of the project: implies

excessively accelerated implementation.

iv) Priority of achieving procurement of funds to the

detriment of the strategic vision of the project.

ROM projects with problems – how on earth do we

tackle these?

Stakeholder

Analysis

Problem

Analysis

• Identifying key problems

affecting the project

• Constraints &

opportunities?

• Determining cause & effect

relationships

• Who are the main

causatives, and why are

problems occurring?

• Who has an interest in

problems not being solved?

• Identifying &

major

stakeholders;

• What are their

roles and

capacities?

PROBLEM ANALYSIS TOOLS

Type of Analysis Illustrative Tool

Stakeholder Analysis

Venn Diagrams, Stakeholder

Analysis Matrix, SWOT Analysis,

Spider Diagrams

Problem Analysis

Problem Trees

Force Field Analysis

Mapping

Objectives Analysis Objective Trees

Alternatives Analysis Organize and Analyse information

Questions to Ask in Defining the Problem

Who is causing the

problem?

Who says this is a

problem?

Who are impacted by

this problem?

Etc.

Who What Where

When Why How

• What will happen if this problem

is not solved?

• What are the symptoms?

• What are the impacts?

• Etc.

• Where does this problem occur?

• Where does this problem have

an impact?

• Etc.

• When does this problem occur?

• When did this problem first start

occurring?

• Etc.

• Why is this problem occurring?

• Why?

• Why?

• Etc.

• How should the process or

system work?

• How are people currently

handling the problem?

• Etc.

Work out what are the main Stakeholder Relationships

Project Partners

Beneficiaries

Stakeholders

Individuals or institutions that may –

directly or indirectly, positively or

negatively – affect or be affected by the

project or program.

Those who implement the projects (who

are also stakeholders, and may be a

target group.)

Those who benefit in whatever way

from the implementation of the

project.

Distinction may be made between

target groups; final beneficiaries and

indirect beneficiaries.

Generate an informal Venn Diagram

Relevance to core project

problems

Not

Powerful

Close

Powerful

Remote

Venn Diagram - Stakeholders(From the perspective of fishing families)

Key Components of Issue Diagram

Issues

Hypotheses

Key Questions

Questions which need to be answered or

topics which need to be explored in order

to solve a problem

Speculative answers for issues that are

phrased as questions and/or areas of

exploration for issue phrased as topics

Questions that probe hypotheses and

drive the primary research required to

solve the problem

Objectives Analysis

Stakeholder

Analysis

• Identifying &

characterizing

potential major

stakeholders;

• assessing their

capacity

Problem

Analysis

• Identifying key

problems,

constraints &

opportunities;

• determining

cause & effect

relationships

Objectives

Analysis

• Developing

solutions

from the

identified

problems;

• identifying

means to end

relationships

Alternatives Analysis

Stakeholder

Analysis

• Identifying &

characterizing

potential major

stakeholders;

• assessing their

capacity

Problem

Analysis

• Identifying key

problems,

constraints &

opportunities;

• determining

cause & effect

relationships

Alternatives

Analysis

• Identifying

different

strategies to

achieve

solutions;

• selecting most

appropriate

strategy.

Objectives

Analysis

• Developing

solutions

from the

identified

problems;

• identifying

means to end

relationships

Stakeholder Analysis Matrix

Stakeholder and

characteristics

related to the main

project problems

Interests and how

affected by the

project

implementation

problem

Capacity and

motivation to

bring about

change

Possible actions to

address stakeholder

interests:

ROM Report

Recommendations



Key ROM questions

How does the ROM process encourage participation?

Does it develop capacity and equity?

How does it empower people?

How does it affect and take account of gender disparity?

What indicators will you use to measure:

Participation

improved capacity

Accountability

empowerment

sustainability?



Information required from ROM and M&E

What significant difference/changes have been made to the lives of those involved?

Are these sustainable?

How have cross-cutting issues been mainstreamed (Human rights, gender equality, democracy, good governance, children’s rights, environmental sustainability, combating HIV/AIDS etc)

Partnership, management and capacity building

Participation of beneficiaries in M&E and their feedback

Learning from the action, how this has been utilised and disseminated



Partnerships:

Points covered:

• The importance of relationships

• Principles for partnership

• Capacity building

• Practical partnership issues

• First steps

• Formalised agreements with partners

• Project management structures



Lesson learning and dissemination:

Points covered:

How to identify innovations and support lesson learning

Means of dissemination of lessons learnt

These will include:

The project/sector/theme

General development context for the innovation

How work proceeded with partners

Attributes of the best practices, innovation

Features of the lesson or innovation that enable it to be widely adopted

Limitations on the recommendation domains

Programme management requirements

Capacity building needs to ensure replication



EU’s Communication & Visibility Manual for EU External Actions

Contractors or implementing partners or international organisations

should use their normal stationery in letterheads or fax headers

sheets, but should add the phrase:

"This project/programme is funded by the European Union" as well as the

EU flag

when communicating on matters related to the action.

Graphic identity of the EU must enjoy an equally prominent place

and size as that of the contractor or implementing partner