Monitoring and Evaluation Workshop... · Monitoring and Evaluation ... Part IV - Inserting Graphics...

27
Monitoring and Evaluation Marcos Ghattas for Stakeholders' Workshop on Qatar's NDS Implementation June 34, 2012 1

Transcript of Monitoring and Evaluation Workshop... · Monitoring and Evaluation ... Part IV - Inserting Graphics...

Monitoring and Evaluation

Marcos Ghattasfor

Stakeholders' Workshop on Qatar's NDS Implementation ‐ June 3‐4, 2012

1

TABLE OF CONTENTS1. Overview:

Why Moniotoring and Evaluation (M&E)?What is Monitoring?What is Evaluation?

2.  Managing for Results:Results Frameworks

3.  M&E ‐ Lessons Learned4.   Relevance to Implementation of QND

2

1.  Introduction 

• Why M&E Systems? ‐ M&E can provide information about the performance of government    policies, programs and projects at the national, sector, and subnational  levels. It can identify what works and what does not, and why.‐ M&E can also provide information about the performance of government; individual ministries and agencies; and of managers and their staff.

• What is Monitoring?‐ Monitoring is the routine daily assessment of ongoing activities and progress.

• What is Evaluation?‐ Evaluation is the periodic assessment of overall achievements.

3

2. Managing for Resultsis often referred to as performance‐based management, the objective of which is to improve efficiency, effectiveness, and accountability in government.• The Results Framework:‐ Effective M&E requires clear, logical chain (pathway) of results, in which results at one level are expected to lead to results at the next level, leading to the achievement of the overall goal.‐ If there are gaps in the logic, the chain (pathway) will not lead to the required achievement of the overall goal.‐ The major levels are: inputs, activities, outputs, outcomes and impacts.‐ This process requires the use of performance indicators to evaluate how much of the intended results are being achieved.‐ Figures 1 and 2 below provide two schematic illustrations of the results chain framework

4

Figure 1.  Managing for Results: The Results Chain

micro/managerial level                                                                               macro/influence level (national budget/(sector/agency)                                                                                                              planning office)‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐

inputs activitities outputs outcomes goals (impact)

efficiency effectiveness

resources to produce outputs

tasks undertaken to transform inputs into outputs

productsservicesproduceddelivered

intermediate effects of outputs on beneficiaries

long‐term improvement in beneficiaries

Source:  adapted fromManuel Fernando Castro ‐PREM notes

Figure 1:• The Results Chain can be applied at the national, agency, sector, 

program or project levels. The process involves the use of performance indicators.

• When setting indicators at the national level, must differentiate between macro/high level and micro/low level performance indicators.

• Macro level includes goals (impacts),  outcomes and outputs, which are mainly of interest to individuals involved in national planning and budgets.

• Micro level includes output, activities and inputs, which are largely of concern to the sector and agency budget and program officials.

• Output is the link between macro and micro levels of the government's (implementation) chain.

6

Performance indicators should be SMART ‐ Specific (precise and unambiguous), Measurable (appropriate to the subject), Achievable (of a reasonable cost), Relevant (serve to assess performance), and Trackable (easy to validate or verify) [Peter Drukner, 1954]. 

OR

They should be CREAM ‐ Clear, Relevant, Economic, Adequate, Monitorable [Sciavo‐Campo & Tommasi, 1999)  

7

Figure 2. The Policy Cycle: Linking Policy, Planning, Budgeting, Management, and M&E

                6.                   Review Policy

Review the past planning and budget period

                  1.                    Begin planning,Set objectives, 

policies, strategies, and 

priorities

                 2.                   Prepare Budget,

Allocate resources

3                  .Implement and 

Manage Planned Activities

                    4.                      Monitor Activities and Account for 

Expenditure

                5.                  Evaluate and 

Audit the efficiency and 

effectiveness of activities

Source: Keith Mackay ‐ PREM notes 

Figure 2: • Describes different parts of the policy cycle. The first step is to begin planning, 

develop policies, set objectives, strategies and priorities (box 1). In doing so, government takes into account  evidence as to what has worked and what has not worked (evidence‐based policy making), derived from reviews of past policies (box 6) [Figure 2 is sometimes referred to as the Learning Loop. A critical part for conducting planning and policy reviews is to measure performance targets and measure the degree of progress toward achieving them.

• Preparation of budgets and its allocation (the second stage in the policy cycle), also benefits from the reviews (box 6) of existing programs and expected performance of new programs.

• The third stage (box 3) is where performance indicators play a vital part as they help managers to monitor their activities, learn what is working and what is not and take corrective actions. Performance indicators are key to Evaluation of experience and progress towards achieving goals/targets/objectives.  

• The fourth and fifth stages are the accountability relationships as they reveal the extent to which the government has achieved its objectives. Strong accountability can provide the needed incentives to improve performance.                         9

3.  M&E ‐ Lessons Learned

• Good M&E requires both internal self‐assessment and external verification.‐ Implementing partners collect own internal data and external agency verifies the completeness and accuracy of the data collected by those implementingg partners.‐M&E systems must be as simple as possible. The more complex an M&E system, the more likely it is to fail.

• M&E systems must include standardized core.‐ if each implementing partner uses different systems or tools, the data cannot be analyzed or summarized effectively.‐ The need for a standardized core does not preclude individual implementing partners from collecting additional, situation specific information.                                                                                                         10

M&E ‐ Lessons Learned

• Specialized entity is required to collect, verify, enter and analyze primary M&E data from each partner.‐Without such an entity, data collection, verification and analyses are unlikely to happen.

• Linking program reporting to financial management and further disbursements will improve program reporting.

• M&E must be built into the design of a program, and should be operational before funding is provided, rather than being added later.‐ It is much harder and less effective to 'retrofit' M&E after funding has already been given.                                                  11    

M&E ‐ Lessons Learned

• No matter how sound an M&E system may be, it will fail without widespread stakeholder 'buy‐in'.

• M&E must be adequately resourced.• Data collection

It may be more helpful to move from baseline and follow‐up analysis to trend tracking, following trends over several time periods.

12

4.  Relevance of Presentation  to Implementation of QNDS

1. Performance indicators, targets/goals run throughout the NDS 2011 Report, and the last few pages are dedicated to M&E. The presentation attempts to clarify some technical jargon that specialists in the field of M&E tend to use, lay down some of the challenges. In doing this, a few key points have been identified and are the subject/objective of this workshop.

• First is the need for Executive Plans at the agency, sector and project levels. These plans need to benefit from a review of past experience (policies, strategies), and set priorities/goals for the next period.

• Second is the need to identify at the outset performance indicators that are clear, relevant, economic, adequate and monitorable.  And that enable the evaluation (measuring the impact) of policy, programs, as well the performance of managers and staff.          13

• Third is the importance of learning from past  evidence derived from reviews of past plans, policy, strategy etc. (Seeking truth from facts)

• Fourth is the need for highly trained and specialized staff in the art/science of M&E. While, the concept can be simplified, implementation is complex. Hence, the need for capacity building, which is emphasized in the NDS Report, and in reporting to the SC.

• Fifth is the need to report, and communicate effectively findings and progress, which are also subjects of this workshop.

14

Final Comments: M&E systems are very difficult to implement and sustain. Successful M&E systems require ownership so that they become instututionalized (and are not abandoned when  governments change), need to be adequately funded, and be implemented by dedicated units and highly trained staff. In the case of Qatar, government is stable and committed to the adoption of M&E systems, funding is not a constraint, and the authorities have indicated their willingness to support and fund HR requirements and capacity building at all levels. As public officials (civil servants), you have no choice but to own this initiative. The sooner you do, the better is everyone involved and the better off is the State of Qatar.                                                                      15

Editing Presentations

Part I - Filmstrip and Slide Management

You can edit and save PowerPoint formatted presentations such as PPT and PPTX of all available versions like 2003, 2007 and 2010

• Tap and drag vertically to scroll the filmstrip on the left side of the main slide

• Tap the to add a slide exactly like the one that you are viewing.

• Tap the on the left slide of the selected slide to delete that slide

• Tap, hold and drag to move a slide

• Tap additional slides once you have tapped and held on a slide to select multiple slides

Quickpoint® How ToQuickoffice® for iPad

Power Edit mode is available when you have selected an image, shape, or textbox. Tap the to open the Power Edit Toolbox. To dismiss the toolbox, tap the on the top right above the toolbox.

Part III - Power Edit Mode

For textboxes and shapes you can change:

• Bold

• Italics

• Underline

• Font size, style, and color

1. Alignment

2. Insert and remove bulleted lists

3. Insert and remove numbered lists

4. Fill color

5. Arrangement (Bring to Front, Send to Back)

• Tap to open the Add Graphic menu• Choose a shape category

• Tap from the Basic shapes category to add a textbox

• Tap or drag and drop any of the shapes to insert one of them• Tap the Images tab to change to the insert image menu• You can insert images from the iPad photo gallery, as well as your local repository. • Navigate to the desired directory and tap or drag and drop to insert the desired image.• You need to allow Quickoffice to use Location services in order to insert images from your Photo gallery.

Part IV - Inserting Graphics

• Single tap or a tap and drag (right to left) will advance to the next slide

• A tap and drag (left to right) will go back to the previous slide

• Tap on the left side of the screen will show the filmstrip and allow you to jump to a slide

• A double tap will exit slideshow mode

You can show your 2003, 2007 and 2010 presentations in slideshow mode both on device and on an external monitor or projector.

• Plug a video out connector into the dock

• Tap the to start slideshow mode

• Select the desired output location: “Main screen”or “External Monitor”

Presenting Slideshows

Opening Presentations

Viewing Presentations

Quickpoint® can open and edit Microsoft PowerPoint formatted presentations such as PPT and PPTX files. Tap on the desired filename in the File Manager to open the presentation.

Quickpoint® provides you a new, optimized way to view presentations in both Portrait and Landscape modes on your iPad®.

The main slide can be zoomed in and out using standard pinch zoom. When the slide is zoomed in, you can move the slide so you can focus on the desired part of the slide.

Part II - Changing GraphicsYou can do several things to images, shapes, and textboxes inside the presentation. Tap once to select the image, shape, or textbox and enable edit controls.

• Tap to rotate the image, shape, or textbox

• Tap and drag to move the image, shape, or textbox

• Tap a second time on the selected item to seethe delete dialog

• Double tap on the shape of textbox to begin text editing

• Tap to resize the image, shape or textbox

When presenting on an external monitor or projector, Quickpoint® gives you additional controls in a control panel

• Single tap or a tap and drag (right to left) or a tap on the will advance to the next slide

• A tap and drag (left to right) or a tap on the will go back to the previous slide

• Tap on the will show the filmstrip and allow you to jump to a slide. A second tap will dismiss the filmstrip.

• A tap and hold anywhere on the screen will activate a virtual laser pointer

• A double tap or a tap on the will exit slideshow mode

Video Out to External Monitors and Projectors