1. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Why use a hammer when you need a wrench: Results-based monitoring and evaluation of projects Joy Gumz, CPA, CISA, PMP Project Auditors LLC "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only

2. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Objectives Explain differences among –Monitoring –Evaluating –Managing projects Clarify what results-based management is Describe tools and approaches that can be used for monitoring and evaluating Provide some tools!

3. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Format Case Studies Terms Process of Results Based Monitoring A Sampling of Tools Q & A Please: set your handy to “off” or “vibrate”!

4. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only About the session attendees Does anyone here perform monitoring and evaluation (M&E) of projects? What are your goals on attending this session? Is anyone here considering performing M&E in the future?

5. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Terms Managing Evaluating Results-based monitoring

6. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Implications Monitoring and evaluating projects is a different skill set Results-based monitoring is a new way to look at a project Understanding differences between roles is critical

7. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Scotland - The National Stadium Budget: £58million 6 years Status –In service Problem: –Cost overrun and funding gap

8. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only U.K. Courts – IT System £148m over 11 years £319m over 14 years £232 million 8 ½ years Status –Implementation Phase “the shoddiest ever”

9. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only U.S. Project Aspire – IT … US$88+ million 3 ½ years Status –Under evaluation

10. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only U.S. People First: Florida US$350+million 9 years Status –Under evaluation

11. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only U.S. Government: Deepwater US$24 billion 25 years Status –“Moving forward”: Primary contractor removed from program manager role

12. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Deepwater - 8 Ships – $100M

13. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Imagine you are building a vacation house in Spain…but you live in London A Project Monitor could Evaluate change orders Conduct onsite visits Provide proactive problem identification Review the schedule periodically Evaluate monthly progress Verify periodic progress payments are in order Assess the critical path Take photos of progress Be your advocate

14. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Implementation monitoring and results based monitoring compared What does RBM do? Why is it different from Earned Value? IMPLEMENTATION MONITORING RESULTS-BASED MONITORING ACTIVITIES Steps taken to implement project INPUTS Resources, money, staff OUTPUT What the project produces OUTCOME/ IMPACT Change in the “real world”

15. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only M&E process Looking at the effect of project activities and deliverables

16. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Tools: A Sampling Standards –Swiss Evaluation Society –20 page PDF www.seval.ch

17. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Tool: Logical Framework Agreed upon framework of how results will be assessed Created collaboratively

18. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Tool: Reporting Schedule Agreed upon schedule of reporting Monitor is to verify periodically the reporting is useful

19. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Tools: Opportunities Canada site lists –Evaluation contracts open to tender offer –Opportunities for collaboration –Career employment opportunities www.evaluationcanada.ca –Contracts, collaboration and jobs

20. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Tools: Networking Publish your profile as an evaluator known in the evaluation community Search systematically –Evaluators –Commissioners of evaluations –Research partners See who is involved in evaluation Over 180 names http://www.seval.ch/en/evaluator/index.cfm

21. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Success in Scotland Park and apartment complex King’s Gait - Glasgow

22. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Nine Step Process 1.Conduct an inventory 2. Agree on what to monitor 3. Gather baseline data on non-deliverable indicators 4. Build the monitoring system 5. During project gather data on project/product deliverables 6. Analyze and report monitoring information 7. Verify quality of monitoring process 8. Use the data 9. Reflect

23. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Organisations are looking for results Drive to improve governance and increase accountability –A project audit is a recognized QA tool –But it can be like hitting the project with a hammer! when what you need is a wrench

24. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Why Results Based Monitoring? Results Based Monitoring provides –Framework to avoid the confrontation that can result from an audit –Demonstrated success in construction –When organisation needs assurance – risk reduction –Especially useful when specialty expertise is needed

25. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only What does it take to be successful? Clearly defined roles Clear and verifiable objectives and indictors An organisation ready for bad news as well as good Subject matter expertise is vital - RBM is hands-on Understand influence Make the project manager see the benefit

26. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Implications for Project Managers Know what M&E using RBM is Know when it might be beneficial Understand steps for successful RBM Possible area of work for PMPs –Subject matter expertise in project area critical

27. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only A Summary RBM is a tool to improve project success Used by government, NGOs and public/private companies Requires –Definition in advance of what results will look like –Monitoring to see whether those results are achieved

28. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Evaluation Societies in Europe Austria – 16 organisations Denmark – 170 members Finland - Not available France – 250 members Germany - 460 members Italy - 340 members Spain -150 members Sweden - 200 members Switzerland - 300 members UK - 150 members UN - 36 UN agencies EU-Europe - Not available

29. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Questions?

30. "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only Contact Information Joy Gumz / Project Auditors LLC Phone: +001 949 452 0578 Email: jgumz@projectauditors.com Session #IND01 "PMI" is a registered trade and service mark of the Project Management Institute, Inc. © 2007 Permission is granted to PMI for Congress attendee use only