© Commonwealth of Australia 2003 The Quality Assessment Framework

The 16 th APEC Ministerial meeting “Ministers approved a new Quality Assessment Framework, which will replace the ECOTECH Weightings Matrix, as a tool to foster good quality project proposals and an efficient allocation of APEC resources.”

2005 Senior Officials Meeting (SOMI), Seoul, Korea Senior Officials endorsed the Small Group on Evaluation’s 2005 Work Program The Work Program further develops the approach to strengthen the quality of APEC projects and implements the key directive from the 16 th AMM Joint Statement.

Three key recommendations for implementation in The evaluation frameworks be adopted and implemented for Operational Account Projects from 2005 onwards. 2. The Guidebook on APEC Projects be updated to incorporate the evaluation frameworks, including replacing the ECOTECH Weightings Matrix and updating the project application form 3. APEC Working groups continue to receive technical assistance and support in implementing frameworks

Steps to achieving recommendations Projects being proposed for funding through the Operational Account from 2005 onwards will be assessed against the Assessment Framework prior to BMC Continuation of the trial process of the Assessment, Monitoring and Evaluation Framework (AM&E) focussing on M&E The Secretariat will complete the revisions of the Guidebook and project application form AM&E coordinator (Michelle Lowe) will coordinate support to the working groups. Australia will continue to fund an evaluation consultant.

The Quality Assessment, Monitoring and Evaluation Frameworks (AM&E) The Assessment Framework Will the project contribute to APEC’s mission? The Monitoring Framework Is the project on track to achieving its objective? The Evaluation Framework Have the objectives been achieved?

How to use the Quality Assessment Framework (QAF) Each working group establishes a small group to assess project proposals 16 design criteria to be assessed Each design criterion is scored from 3 to 0. -Good practice (3) -Satisfactory (2) -Marginally satisfactory (1) -Weak (0) -Not applicable (0) All criteria are of equal value, there is no weighting system

Comments made against each criterion can be a simple yes or no. Add scores together to get a final score. Projects can be ranked in order of merit. Lead shepherds ensure the framework is objectively filled in and are either kept informed or participate in the group. They should send the filled out QAF to the proposal proponent and project overseers to allow them to make revisions if required. Submit final project proposal, with QAF attached, to BMC. This will clearly indicate the strengths and weaknesses of the proposal to the Committee.

Examples 1.Does the proposal clearly state which one of the APEC priority themes of the 1996 Manila Declaration will be addressed? One, and only one, priority themes in the 1996 Manila Declaration should be chosen as the project goal. For example: Safeguarding the quality of life through environmentally sound growth. Comments (if any) Assessment Yes 2

9.Is the implementation methodology described in the proposal both clear and effective? Have the main risks to implementation been identified? Explanation: Implementation should be well-planned and logical, and based on a breakdown of functions over time. Main risks or assumptions should be identified and practical strategies prepared to manage them. CommentsAssessment No, needs more detail1

13. Does the proposal have a strategy to make project benefits sustainable? Explanation: Project benefits should last well beyond project completion. For example, regular up-dates of documents and training, ensuring that manuals are in local language, etc. CommentsAssessment Yes3

16.Final statement of suitability for APEC funding If implemented, is the project likely to make a sufficient contribution to APEC’s mission to justify its funding? Considers all relevant factors plus any others specific to a working group. CommentsAssessment Yes, high priority 3

Phase 2- Assessment, Monitoring and Evaluation The Monitoring and Evaluation Frameworks will be trialled and refined in  With a view to being adopted at the 17 th AMM and implemented in Final AM+E Frameworks will be reviewed annually. Benefits: Ensures projects on track, helps achieve the desired outcomes, enhances APEC credibility and supports regional development

Next steps for Working group projects Lead Shepherds should ensure projects being put to the BMC for Operational Account funding have been assessed using the Quality Assessment Framework (QAF) The Working Group should consider trialling the M&E Frameworks in 2005 (particularly those projects involved in the initial trial of the QAF)

Contacts For any further advice on the AM&E Frameworks please contact:  Michelle Lowe Ph (65)