1. 1 Through the Looking Glass - How to do a Post Project Review Barbara Purchia February 20, 2001 Rational Software bpurchia@rational.com

2. 2 Overview What is a Post Project Review? When to do it How to do it Post Project Review Meeting Real life problems What have we found so far Summary Agenda

3. 3 A Rose by Any Other Name Post Project Review (PPR) Project Retrospective Project Legacy Project Summary Post Mortem Post Partum What Are They?

4. 4 What is a Post Project Review? A summary of project development, start to finish A brief description and analysis of –Project activities –Results –Lessons learned –Recommendations An improvement tool –NOT a record of failures What Are They?

5. 5 Why Should We Do a Post Project Review? Learn from project experiences Retain knowledge for future projects Understand how your project was done Identify areas to improve Understand successes Provide team closure What Are They?

6. 6 When Do I Do One? For completed, terminated, or cancelled projects –No more than 30 days after project end –NOT before project release –When all critical participants can attend During the project –After a major project milestone –At designated schedule points –When things are not going well ??? When to Do Them

7. 7 What’s Involved? Collect the information –Technical –Managerial –Metrics –Schedules Anything kept during the project –Manager or project logs –Emails –... How To Do Them

8. 8 What’s the Deliverable? Concise documented information summary –Development activities –Project metrics –Estimates and actuals –Technical and managerial lessons learned –Recommendations for future projects Focused only on the project and the processes How To Do Them

9. 9 How Do I Start? Hold a meeting including –ALL participating functions –Project management team –Affected functions AND/OR Request input via email –Remote sites –Managerial lessons learned –Filtered to be unattributable How To Do Them

10. 10 What Is a PPR Meeting? A tool for reviewing and analyzing the project Promote discussion on improvements Involve a critical mass of those directly involved Promote discussion of all project areas –Deliverables –Team dynamics –Processes Provide input for the PPR document Provide closure for the project team How To Do Them

11. 11 What Doesn’t the PPR Meeting Do? Assign blame to any individual or group Provide an exhaustive study of the project Provide solutions to all issues Shoot the messenger How To Do Them

12. 12 How to Hold a PPR Meeting Use a facilitator Set up a firm time Use an agenda Ensure Senior Management support/sponsorship Keep the focus on the process not the people!!! How To Do Them

13. 13 How to Plan for the PPR Meeting Pick the meeting date Identify attendees and ensure availability Book the room in advance Prepare an agenda If using a questionnaire, review the questions and tailor it for the project Send meeting announcement How To Do Them

14. 14 Roles and Responsibilities Senior Management –Ensure that a PPR is done PPR Sponsor –Call the PPR meeting –Ensure that information gathered –Produce and post the PPR document –Use the data Project team members –Attend the meeting and participate –Use the data Facilitator –Non-project member –Help plan and organize meeting –Ensure meeting runs smoothly Administrator –Capture the issues and recommendations How To Do Them

15. 15 PPR Meeting Attendees Managers Team Members Project Program Product QA/QE Development Documentation International Beta Service UI Manufacturing... All Project Participants How To Do Them

16. 16 Facilitator Preparation For each group, work with the sponsor to –Understand the environment –Understand the culture –Understand the people –Understand any previous PPR activities How To Do Them

17. 17 Sponsor Preparation Produce a brief project refresher –Time line of key milestones and/or significant activities Review the PPR questionnaire with the Facilitator Identify anticipated issue categories with the Facilitator (optional) Review any previous PPRs –Documents –Improvement activities How To Do Them Start 75 MPH Milestone

18. 18 Participant Preparation Think about the project Review the questionnaire Review meeting minutes Review emails, documentation,.. How To Do Them

19. 19 Room Preparation Done by the facilitator Ensure that meeting aids are in the room –Flip charts –Markers –Post-its –Tape or tacks –Pens Bring candy, donuts, or have lunch available after the meeting (optional) How To Do Them

20. 20 PPR Meeting Logistics Split into two parts –Looking backwards –Looking ahead The PRR Meeting

21. 21 Example Two Hour Meeting Agenda Introduction and Groundrules (5 minutes) Project refresher (10-15 minutes) What Went Well and Issues Gathering (40 minutes) Prioritization (10 minutes) Brainstorm solutions for top 3 issues (40 minutes) Meeting Closure (5-10 minutes) The PRR Meeting

22. 22 Groundrules Be polite and gracious Focus on problems and process No finger pointing No assigning blame No “You’s” Let each person have a turn Avoid defensiveness Respect everyone’s perspective The PRR Meeting

23. 23 Project Refresher Example The PRR Meeting

24. 24 Barb’s Lessons Learned What Went Well and Issues Gathering Split into separate activities Use Post-its What Went Well –Use flip chart sheet(s) –Put “What Went Well” Header on flip chart sheet –Write down positives on Post-its –Read in round robin fashion until done –Facilitator puts on flip chart sheet –Group any duplicates The PRR Meeting

25. 25 Barb’s Lessons Learned The PRR Meeting What Went Well and Issues Gathering (Continued) Issues –Use flip chart sheets –Write potential issue categories on top of blank sheets –Have several blank sheets –Tape to wall –Write down issues on Post-its –Read in round robin fashion until done –Facilitator puts on flip chart sheet –Group any duplicates

26. 26 Post-It Rules One item per Post-It Write as many Post-Its as you want Write enough so that each idea stands on its own Read 1 Post-It per person until we are done Identify duplicates and pass them to the facilitator One item per Post-It The PRR Meeting

27. 27 Issue Prioritization Everyone checks issues –Ensure grouped correctly –Ensure duplicates collected correctly Everyone gets 3 votes –Used however person feels Total all issues Identify top 3 issues –Check in with group to see if agree The PRR Meeting Last minute changes to requirements (8) Finish critical components sooner (4) More Beta Test sites (4)

28. 28 Brainstorm Solutions for Top 3 Issues Technique 1 Use flip chart sheets –Identify the issue and page number on top right –Write issue on top of flip chart sheet Write down all solutions –Get clarification as needed –Check in to see if captured correctly Spend 10 - 15 minutes per issue Check in if energy low The PRR Meeting

29. 29 Brainstorm Solutions for Top 3 Issues Technique 2 Identify volunteer facilitators for the top issues –Include similar items Break up into groups Volunteer facilitators should use flip chart sheets –Identify the issue and page number on top right –Write issue on top of flip chart sheet Write down all solutions –Get clarification as needed –Check in to see if captured correctly The PRR Meeting

30. 30 Meeting Closure Thank people for attending Answer any questions Re-iterate the next steps Thank people again The PRR Meeting THANKS!!

31. 31 Facilitator Tips Protect each speaker Pay attention to the speaker Keep the meeting positive Don’t filter the information Let people finish their thoughts Help people continue the discussion, not sidebars Check-in with group Ensure that senior or vocal person does not shut down meeting Make it fun The PRR Meeting

32. 32 Candidates for Questionnaire and Issues Project objectives Project requirements Project planning, tracking, and risk management Localization Training, Knowledge, Tools Functional specifications Development strategy/process Lessons learned Communications ??? The PRR Meeting

33. 33 Sample Questionnaire Project Objectives What were the objectives? Were the objectives clearly stated? Were the objectives of the project met? Did everyone work from the same set of objectives? Project Requirements Was there a Requirements document? Were the requirements known, understood and adhered to? Did the requirements change? If yes, why and how often? Project Planning and Risk Management Was there a project plan? Was it maintained throughout the project? At the outset, was the schedule realistic? Did the project plan change? If so, why and how often? The PRR Meeting

34. 34 What Does the Sponsor Do After the PPR? Collect metrics –Estimated and actual Staffing Milestones and schedules Project cost Code size Quality (e.g. defects) Any other useful metric information Collect managerial lessons learned Write the PPR document –Use an administrator to help –Have team review document Post or distribute the PPR document After the PRR Meeting

35. 35 After the PRR Meeting What Else Does the Sponsor Do After the PPR? Follow-up on action items Use information in the next or similar projects Communicate improvements

36. 36 Outline for PPR Document Introduction Results vs Plan –Status –Staffing –Milestones –Cost –Quality Lessons Learned –What went well –What didn’t go well –Technical lessons –Managerial lessons Recommendations for future projects After the PRR Meeting

37. 37 Real Life Problems The data is not used The document is not produced People feel that nothing will come from the meeting The document takes forever to complete Meeting gets out of control Key people don’t provide input Problems

38. 38 What Have We Found Consistent issue areas Issues moving further downstream in the process –Requirements and specifications –Communications –Quality –Project planning and tracking –Process People expect follow-through A focus on “management” instead of what “I” can do Department and organizational project similarities Discoveries

39. 39 How Have PPRs Helped Us? Used the schedule information to plan the next project Used actual cost information for project prediction Organization metrics –Time between phases –Time to market –Bottleneck areas Opportunities to share solutions Discoveries

40. 40 Summary Valuable tool for capturing and learning from past projects Valuable tool for predicting future projects Good mechanism for project closure Great vehicle for tracking organizational information Good way for the facilitator to learn about the team Try it. You’ll like it!!

41. 41 Any Questions??