MONITORING & EVALUATION FOR RECIPIENTS OF EU FUNDS...

Ministry of Planning

European Union

MONITORING & EVALUATION

FOR RECIPIENTS OF EU FUNDS

Does this picture ring a bell?

Participatory Development Programme in Urban Areas

We ask participants to draw an

image which reflects what

monitoring means to them !!!


A quote to remember….


“ I keep six honest serving men, They taught me all I

knew. Their names are

What, Why, When, How, Who and Where”

(Rudyard Kipling)

What is monitoring


It is not only about observing and listening, but also about

assessing information, learning from it and adjusting one's

approach if necessary.

It tracks the actual performance or situation against what

was planned or expected according to pre-determined

targets

It provides an ‘early warning system’, which allows for

timely and appropriate intervention if a project is not

adhering to the plan

Internal and External Monitoring


` Internal Monitoring External monitoring

By Implementing body

itself (GB)

EUD, BMZ, PDP, MOP

Sources of

information

Grant contract e.g.

project log frames,

project’s monitoring

tool (covering out,

results, specific and

overall objectives),

procurements, GB

reports

Grant contract e.g.

project log frames,

project’s monitoring

tool (covering out,

result and impact),

procurements, GB

(internal) and PDP

(external) reports

Comparative

objectivity

More subjective More objective

Why do we monitor and evaluate?


• To generate specific information required by CA/donors about

the progress being made on NGO-implemented projects;

• To ensure that funds and resources are used properly;

• To keep project activities on schedule, and to review and

update the project plan as necessary;

• To review projected costs, timelines and deliverables and to

explain variances from the original plan;

• Positive outcomes help to secure continued or new financial

support: demonstrating the success attract more funds.

When do we monitor?


Monitoring has to be planned from the start, not tacked

on later as an afterthought.

Monitoring hence takes place throughout the whole

lifetime of the project.

A final evaluation at the end of the project (done by GB

and PDP) as well as a post evaluation months/years

after the completion of the projects (PDP)

When do we monitor GB projects?


GB Tri-monthly report

GB Final report

GB Tri-monthly report

GB Interim report

PISU 1st Monitoring

visit

PISU Monitoring

visit

PISU Monitoring

visit

PISU Spot Monitoring

visits

1 2 3 4 5 6 7 8 9 10 11 12

PISU Closure

visit report

PISU Post Evaluation

visit report

How do we monitor?


• Monitoring visits • Scheduled Regular visits

• Spot visits

• Project Reports

• Grant Beneficiaries’ reports

• Monitoring actors’ reports (as results of monitoring visits)

• Support activities (from PISU when required/recommended)

Monitoring visits


1. Scheduled Regular visits • First monitoring visit:

GB is informed prior to the visit

within 45 days after contract signatures

• Regular monitoring visits:

Conducted in line with the schedule agreed on with GB

2-3 per project.

2. Spot visits:

GB is not informed prior to the visit

Project Reports


Project reports can be classified into two groups:

1- Grant Beneficiaries’ reports

a) Tri-monthly reports

b) Interim report

c) Final report

2- Monitoring actors’ reports

a) Monitoring Visit Reports

b) Early Warning Report

c) Project Irregularity Report

Grant Beneficiaries’ reports


a) Tri-Monthly Progress Report :

Tri Monthly Progress Reports (see Annex 8) present

information about the current situation of the project

activities and achievement of results, current and

anticipated problems, remedial actions and planned

future activities.


b & c) Interim and Final Reports

The Grant Beneficiary must draw up Interim report and

a Final report.

These reports shall consist of narrative and financial

sections as in Annex 1(i,ii,iii) of the Monitoring Manual,

including supporting documents (if requested).

Grant Beneficiaries’ reports

Monitoring actors’ reports


a) Monitoring Visit Reports

Monitoring visit reports are the documents which indicate the actual

performance of the project and are used for analysis of the

problems and the needs of Grant Beneficiary.

b) Early Warning Report

The Early Warning Report serves as a tool to detect projects with

high risk on fulfilment of contractual obligations and to take

measures to solve the problem before getting into an intricate

situation.

c) Project Irregularity Report

Irregularity report is the last resort if there is nothing to correct the

intricate situation for fulfilling the contractual obligations. In case of

suspected or detected irregularities, it should be reported to the CA

immediately.

Support activities


Support activities are mainly carried out by PISU and UUU

to Grant Beneficiaries in the following fields:

• Facilitation between various stakeholders inside the target

areas

• Results and impact monitoring.

• Technical and financial documentation

• Procurement procedure and documentation

• Eligibility of costs, accounting and record keeping

• EU visibility and publicity rules

• Schedule of reporting

• Preparation of reports

How does it work?


A systematic process of continually improving what we do by

learning from what we have done and achieved in the past. This

approach emphasizes the need to learn by doing and to evolve

alongside changes in the environment.

This way of management requires a series of different steps:

1. Clarify the goals and purpose of the project

2. Plan actions and monitoring

3. Implement actions and monitoring

4. Gather data/evidence and feedback

5. Reflect on the findings and evaluate them

6. Modify/review actions based on the new knowledge/learning


Who is involved?

Who is involved?


• GB themselves

It is imperative that the project team members, including

management, go through all the project monitoring steps.

The team may need to get some advice or assistance

from outside sources, but ultimately they need to do the

work themselves.

What do we monitor?


Project Financing & Contractual

Structure (Grant Contract)

Financial Records including the

project Bank account

Compliance with contractual

and Procurement Procedures

Project Activities, Outputs,

Outcomes (Impact)

Financial

Bank

account

Procurement

Activities

,

outputs,

outcome

Monitoring – important issues:


• What activities have been completed, versus what should

have been done

• How long it has taken, versus how long it should have

taken

• What it has cost, versus what it should have cost

• The quality of outputs produced and delivered, versus

what was planned

• Outside:

• How people are reacting to our activities, versus what was

assumed

• The benefits people are drawing from the activities, versus

what was planned

Identify Sources of Information


1. Pre Assessment situation (Baseline data) In order to observe changes resulting from project activities, it is often

necessary to establish bench- marks against which indicators can be

judged.

2. Post Assessment Situation (Formative Evaluation)

Formative evaluation is concerned with identification of gaps in the

implementation process which could be overcome by the implementation

team early enough so as not to affect the end-of-program results. To that

end, the following measures are planned for closely monitoring and

formatively evaluating training interventions:


a) Trainees' Satisfaction Questionnaire: It aims to measure

trainees’ satisfaction with training content, delivery, trainer’s

skills and applicability to workplace. Trainees’ satisfaction

questionnaire is administered by a program coordinator at the

end of each training program. Results of those questionnaires

will be shared with respective trainers and program managers

so that identified trainee’s dissatisfaction with any part of the

training process could be overcome.

Post Assessment Situation


b) Trainees' Learning Gain: To assess learning gains, a

set of pre and post assignments are designed to ensure

trainees’ development of key knowledge and skills essential

for their on-the-job performance. Trainees’ scores on those

assignments will be used to select a sample of youth to

follow up with them on their application of training on the

job.



c) Follow-up Field Observations: Post training field visits will

be arranged to a sample of youth who attended the training

sessions. Those field visits will be conducted jointly by a

project coordinator and a representative from the respective

NGOs that delivered training to the visited youth. An

observation log will be developed to capture observers’ notes

during the field visit. At the end of their observation, observers

will conduct focus group meetings with observed youth to

discuss their observation notes.



D) Focus Group Meetings: During those meetings, issues

that affect trainees' on-the-job application will be discussed

with respective youth in order to highlight factors that

minimize the feasibility and applicability of the training

modules for youth.


Listing/Tracking End Beneficiaries


A development of a web application data base

on http://egypt-urban.net/grants/ Grant

beneficiaries/user/password/database/beneficiaries .

The use of such data base, help us;

• Involve grant beneficiaries from the very beginning in monitoring

themselves (Decentralized Monitoring)

• Ensure that our grant beneficiaries documenting their work

regularly

• The output of this data base will guide and give us alert and also

the GB of the number of end beneficiaries who are benefiting

from more than one project.

• The end beneficiaries should not be benefited twice

http://egypt-urban.net/grants/



Prepare a Detailed Monitoring Plan


We recommend not jumping into

implementation and data collection

without first having a clear idea of

what information you need, for

whom, and why.

For that purpose you will need a

monitoring plan for each project that

you will implement.

There are several key questions to consider in order

setting up a monitoring plan for your project


Grant Contract

Number

Title of the Grant

Contract

Project goal: Indicators of

Effectiveness:

What

information

to collect:

How to collect the

information:

Frequency: Who will

collect

information:

How will it be

analysed:

How will

information be

used:

Project specific

objectives:

Indicators of

Effectiveness

What

information

to collect:

How to collect the

information:

Frequency: Who will

collect

information:

How will it be

analysed:

How will

information be

used:

Project

Outputs:

Indicators of

Effectiveness

What

information

to collect:

How to collect the

information:

Frequency: Who will

collect

information:

How will it be

analysed:

How will

information be

used:


Impact Monitoring

What is impact monitoring?


• Impacts are intended or unintended, positive or negative

changes in a situation or in behaviour that occur as the

direct or indirect consequence of an intervention

• Impact monitoring allows steering complex programme

approaches, supports reporting requirements, builds the

cornerstone for improving aid transparency & mutual

accountability and is hence the basis for possible further

funding

Impact monitoring of GB projects


• The impact of each project (that is receiving funding from

the PDP Grant Scheme) on its respective target group and

area is to be monitored through the Grant Beneficiaries

own monitoring activity according to project indicators

(logframe) that indicate project performance.

Impact Chain


The direct measurable results/outputs (tangible products

and services) of your project which are largely under the

projects management control

High aggregated Impact

Overall objectives

Specific objectives

Expected results

Activities

These are the tasks as described in detail in the

application, which have to be undertaken in order to

produce the expected results of the project

The development outcomes expected at the end of your

project

This is the overall development aim towards which your

more specific project objective should contribute to

The long term benefits resulting from the action

Logical framework: Example on Lot 1



Overall objectives

Specific objectives

Expected results

Activities

The global objective of this Call for Proposals is: To

enhance the socio-economic situation of residents in the

eligible informal areas (employment, income generation,

improved services)

Lot 1: Employed residents

The unemployed residents are better skilled

100 unemployed residents trained

Train unemployed residents




Overall objectives

Specific objectives

Expected results

Activities

The global objective of this Call for Proposals is: to



improved services)

Lot 2: Improved income generation

Women setup and start rooftop farming

100 women trained on rooftop farming

Train women on rooftop farming




Overall objectives

Specific objectives

Expected results

Activities

The global objective of this Call for Proposals is: to



improved services)

Lot 3: Improved services (e.g. at a health center)

Staff is qualified

20 staff members trained

Train staff (e.g. of a health center)

How do we monitor? Sources of verification!



Overall objectives

Specific objectives

Expected results

Activities

PDP (Survey)

GB (Support PDP in

surveys) PDP (Ex-post Survey)

GB (Baseline Data & Final

project assessment)

PDP (Survey at the end of

the project)

GB (reports; internal

monitoring documents e.g.

training documentation)

PDP (Visits, GB reports

and on the spot visits)

GB (reports; internal

monitoring documents e.g.

training agendas)

PDP (Visits, GB reports

and on the spot visits)

Most important sources of verification


1. Pre Assessment: Baseline data

In order to observe changes (on the level of the specific and overall

objective) resulting from project activities and results, it is necessary

to collect baseline data. Baseline data functions as a bench-mark

against which the project objectives can be judged at the end of the

project.

2. Post Assessment Survey

After the conclusion of each project, a survey has to be conducted

asking the end beneficiaries questions in order to assess whether

the provided results have an impact on their situation/qualification.

Impact Monitoring tool


Description Indicator/

Milestone

Due Date Responsi

bility

Frequen

cy

Source of

Verification

Specific

information

Status (Text) Status (Colour) Comment

The indicator or

milestone (each

indicator is supported by

several milestones that

‘guide’ towards indicator

achievement).

The date the

indicator/

milestone is to

be achieved

latest.

Who is

responsible to

collect the

information

The frequency

the respective

indicator is

monitored

The source providing

the information to

verify the respective

indicator’s/milestone’

s achievement.

What specific information to

be collected

The current status information

regarding the achievement of the

respective indicator.

A ‘traffic lamp’ that categorises the

information given in the status info into

four categories: achieved, on track, at

risk, failed.

Additional information regarding the

monitoring of the respective

indicator/milestone

Level of Overall Objectives

Objective of lot

x

O1 Indicator

Level of Specific Objectives

SO1 Indicator

Milestone 1

Milestone 2

Milestone 3

SO2 Indicator

Milestone 1

Milestone 2

Milestone 3

...

...

Level of Expected Results Indicator 1.1

Milestone 1

Milestone 2

Milestone 3

Indicator 1.2

Milestone 1

Milestone 2

Milestone 3

Indicator 2.1

Milestone 1

Milestone 2

Milestone 3

...

...

...

Q ? & A:


Mahmoud Abou Zeid:

012 719 66610

[email protected]

Mohamed Ammar:

012 719 666 75

[email protected]



Monitoring visits are not

auditory, but Supporting

and advising !!

"You need to be ready

to taste the food while

you cook :)

MONITORING & EVALUATION FOR RECIPIENTS OF EU FUNDS...

Documents

Transcript of MONITORING & EVALUATION FOR RECIPIENTS OF EU FUNDS...