1. Streamlined Audit/Evaluation Report Writing Presentation to 2014 FAEC Conference Andy Patchan, AIG for FRB/CFPB 1 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

2. My Background – 10 years at GAO, learned report writing from the ground up, with lots of supervisor revisions and little guidance – 2 years as an attorney at DOJ, where I gained experience in drafting legal opinions and case decisions – 13 years at GSA OIG, from audit manager to AIGA, where I began identifying ways to streamline report writing/development and helped develop report writing courses – 2 years as SES at Department of Education OIG, where I experienced other processes for report writing/development, but still similar challenges – At the FRB/CFPB OIG since 2008 2 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

3. The Challenge of Report Writing Focus of audits/evaluations is on the fieldwork – identifying and developing findings Report Drafting and fieldwork require different skill sets Lots of revisions by Managers 3 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

4. Problems in Audit Report Writing  No existing methodology for auditors to draft reports – same tools as 1980s. They have to look at past reports to try and understand how to draft the findings, conclusions and recommendations. Reviewers provide comments at meetings but less experienced auditors are unsure how to proceed Provided a methodology for auditors to follow in drafting findings, linking findings into a report message, and flowing the message into conclusions and recommendations  Long Timeframes to Develop the Initial Draft Decreased timeframes 50% in initial pilot projects; discussion draft in 1 month  Multiple Rounds of Reviewer Comments and Revisions Only 2 rounds of reviewer comments in initial pilot projects  Frustration and low morale on the part of auditors drafting the report On initial pilot projects, auditors took ownership of draft report, and were proud and excited about their draft  Long Timeframes for Completion of the Audit and draft report (12 months or more) About 6 months total for initial pilot projects 4 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

5. Focus of Streamlined Report Writing  Not instruction on how to write better grammar or use of Chicago style manual  Not discussion of auditing and how to identify findings Focus on a Methodology for organizing findings into a coherent, unified, and persuasive message Tools to more expeditiously develop a draft report and reduce frustration and management revisions 5 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

6. Key Areas for Focus Linking Findings to Audit Objectives Fully Developing Findings Before Drafting Message Development Conference Report Conference Report Message and Charge Paragraphs Conclusions and Recommendations Finalize the above before Beginning Drafting © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author. 6

7. Link Between Audit Objectives, Findings, and Report Message Audit Objectives define the main questions posed on your audit, and the boundaries for your areas of inquiry (audit scope), methodology, data to be collected, conclusions, and recommendations Audit objectives can address compliance, program effectiveness, adequacy of controls, economy/cost-effectiveness, etc. The report must answer the questions posed by the audit objectives All findings must tie-in to the audit objectives. 7 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

8. Examples of Objectives (1) Determine the extent to which the Pell grant program is properly controlled and administered to prevent fraud and abuse. (2) Assess the adequacy of the process and controls for effectively reviewing the Bonus Program applications to ensure eligibility requirements are met. (3) Assess how the information technology system will meet select requirements within planned timeframes. 8 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

9. Link Between Audit Objectives and Findings (cont.) How does a differently worded objective change the area of inquiry for potential findings and potential report message? Example Objective 1: Determine whether adequate procurement controls are in place to prevent fraud, waste, and abuse. Example Objective 2: Determine whether adequate procurement controls are in place to prevent fraud, waste, and abuse and assess whether additional costs are incurred. 9 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

10. Linking Findings to the Objectives (cont.) Objective 1: Determine whether adequate procurement controls are in place to to prevent fraud, waste, and abuse. Finding 1: Criteria: Federal Acquisition Regulations require an independent government cost estimate before soliciting offers from contractors for space alterations, to help evaluate proposed prices. Condition: Building managers are not always developing an independent government cost estimate (IGE). Effect: The contracting officer may not be able to negotiate lower prices Cause: Building managers were not aware of existing regulations and were not exercising proper oversight to ensure compliance. 10 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

11. Linking Findings to the Objectives (cont.) Objective 2: Determine whether adequate procurement controls are in place AND assess whether additional costs are incurred. Finding 1: Criteria: Federal Acquisition Regulations require an IGE before soliciting offers from contractors for space alterations, to help evaluate proposed prices. Condition: Building managers are not always developing an IGE. Effect: We reviewed 10 contracts with an estimate and found in 9 cases, the contract award cost met the IGE. In 10 contracts where there was no IGE, the contract cost was on average 20% higher than IGEs for other similar work. In addition, in some of the latter 10 contract files, notes by the contracting officers indicated they were unsure whether prices were reasonable and how to negotiate further. Cause: Building managers were not aware of existing regulations and were not exercising proper oversight to ensure compliance. 11 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

12. Link Between Audit Objectives and Findings (cont.) Thus, two slightly different objectives but with VERY DIFFERENT MESSAGES. The objectives define not only your areas of inquiry, scope, and methodology, but also the message of your findings and report.  using the 1 st example objective, the reader is expecting a message of whether controls are operational, and instances of where they’re lacking.  using the 2 nd example objective, the reader is expecting a message of whether the agency is incurring excess costs, and by how much, and that corrective actions are needed as soon as possible. 12 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

13. Fully Develop Findings Before Drafting Audit Findings Are the Building Blocks for Your Audit Report Much like constructing a house, if we don’t finalize the basic floorplan, our construction efforts will have to be redone, at significant effort and expense, as the floorplan is changed Not fully developing the findings during fieldwork results in redrafting over and over again as you attempt to develop the finding while you’re writing 13 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

14. Fully Develop Findings Before Drafting (cont.)  Establish the 4 elements of each finding Sample Finding Objective: Determine the Effectiveness of the Stock Program in Filling Client-agencies’ Requirements for Supplies 14 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

15. Fully Develop Findings Before Drafting (cont.) Audit Objective: Determine the Effectiveness of the Stock Program in Filling Client-Agencies’ Requirements for Supplies Criteria: To optimize the Stock Program operations, inventory managers need to ensure adequate stock is ordered to fill customer orders (Inventory Mgt. Handbook) Condition: We analyzed 100 stock items and found some items had insufficient stock available to meet customer orders, while others had lower customer demand and excess stock. Effect: Where insufficient stock, customer orders cannot be filled. Cause: The Stock Program computer system inadequately projects amounts of stock to be ordered 15 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

16. Fully Develop Findings Before Drafting (cont.) Analyze each element to determine what additional information is needed. What further questions need to be asked? Ask yourself “so what;” do the elements separately, and together, tell an important message? What is the message? 16 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

17. Fully Develop Findings Before Drafting (cont.)  Your findings must convey why the deficiency is important and needs to be corrected.  It is not enough to just indicate that procedures vary from what is required (not persuasive to the reader).  Keep in mind the cause, and what part that plays in the message of the finding. 17 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

18. Fully Develop Findings Before Drafting (cont.) For example, the following points convey the importance of a finding: – Key goals of the program are not being achieved – Excess funds are being used unnecessarily – Key controls are lacking/can be improved 18 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

19. Fully Developing Findings Before Drafting (cont.) Example of Fully Developed, convincing Finding: Criteria: To optimize Stock Program operations, inventory managers need to ensure that adequate stock is ordered to fill customer orders (Inventory Management Handbook) Condition: We analyzed 100 stock orders and found amounts ordered are inconsistent with customer demand in 80% of the cases. In 60 cases, insufficient stock was ordered. In 20 cases, excess stock was ordered. Yearly stock orders under the program represent $600 million. 19 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

20. Fully Developing Findings Before Drafting (cont.) Effect: Where insufficient stock was ordered, customer orders could not be filled, causing customers to go elsewhere and a risk of losing them as continuing clients. Where stock was in excess, holding costs increased an average of 10%, and depending on the item, such as paint or glue, could result in spoilage and waste. Yearly holding costs are $50 million. 20 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

21. Fully Developing Findings Before Drafting (cont.) Cause: The Stock Program’s computer system contributed to the problems in that the system’s recommended actions to the inventory manager are not based on actual customer demand and vendor lead-time data. Instead it relies on historical customer data and computer models from the 1970s, and inventory managers lack experience to make adjustments. 21 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

22. Message Development Conference Now that you have developed the 4 elements of each finding – you need to present your findings to OIG management Typical attendees: AIC, audit staff, audit manager, senior manager/regional IG/AIGA, Deputy IG (depending on organization setup and size) Audit team presents findings and associated elements Attendees raise any questions about the impact, significance, or facts about the finding or its elements to ensure sufficiency Length of conference – 1 to 2 hours 22 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

23. But what follows the Message Development Conference? How do we ensure the AIC and audit team know how to go forth in drafting? What is the starting point for the audit team in drafting? How does the audit team divide up report writing responsibilities? © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author. 23

24.  Reach consensus among the audit team on the report message and logic flow of the findings  Answer audit team questions on drafting  Help the audit team to “hit the ground running” as the start the drafting  Attendees: AIC, audit team, audit manager, senior audit manager, Regional IG/AIGA (depending on the organizational setup)  Length of time – ½ to 1 day, depending on scope of audit © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author. 24 Report Conference Can Bridge the Gap

25. Determining the Report Message and Charge Paragraph  Primary task is to “synthesize” the findings into the report charge paragraph, to create your overall message Only select elements of each finding may be presented, such as effect or condition Some findings may be “nested,” such as where the cause of the finding also functions as a finding with its own elements 25 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

26. Determining the Report Message and Charge Paragraph (cont.) This part of report development is the most tedious: The team is trying to craft the raw message of the report Have to use trial and error to figure out the best fit Need to try fitting elements of findings with others and combine the flow of the findings to achieve the best logical flow and message 26 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

27. Determining the Report Message and Charge Paragraph As you proceed in this work, what message or story line jumps out at you? Which are the strongest findings/elements and which are the weakest? What are the linkages between the findings Which points depend on others to make the case (which are primary versus supporting points)? 27 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

28. Synthesizing the Main Points of the Findings Into a Report Message and Charge Paragraph (cont.) Let’s Revisit the Previous Fully Developed Finding: “Inventory Orders Are Not Adequately Managed to Minimize Out-of- Stock and Overstock Conditions” (condition-based with effect introduced) In 60 of the cases sampled, stock ordered was insufficient to meet customer orders - may lose the customer - Handbook 999.99 requires adequate stock to fill customer orders (condition, effect, and criteria) In 20 of the cases, stock was excessive, which could increase program holding costs by several million dollars (condition and another effect) Stock Program information systems inadequately determined amount of stock to be ordered; the systems don’t use actual data, and the computer logic is from the 1970s (cause) 28 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

29. Synthesizing the Main Points of the Findings Into a Report Message and Charge Paragraph (cont.) Let’s assume another team member has identified an additional finding about problems with a new computer system under development to replace the old 1970s inventory management system and improve the ordering of adequate stock. You’ll need to integrate this additional finding into your Charge Paragraph and in conjunction with your previous finding about problems with the stock program. 29 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

30. Synthesizing the Main Points of the Findings Into a Report Message and Charge Paragraph (cont.) How do you then craft these points together in one report charge paragraph?  Logical ordering of the points?  Flowing story?  Unified message? 30 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

31. Synthesizing the Main Points of the Findings Into a Report Message and Charge Paragraph (cont.) Example of Report Charge Paragraph with Combined Report Message “Inventory Orders Are Not Adequately Managed to Minimize Out-of-Stock and Overstock Conditions”  In 80 of the cases, ordered amounts was not sufficient to meet customer orders  In 20 of the cases, excess stock was ordered, increasing holding costs by several million dollars.  The 1970s stock program computer system contributes to this problem by relying on older methods for determining amount of stock to be ordered (cause links to the additional finding)  Officials are developing a new replacement information system to better order optimal stock amounts, but the system is already undergoing design before all requirements have been detailed.  Making additional changes after the system is designed could result in costly retrofits. 31 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

32. DRAFTING REPORT SECTIONS FROM THE COMBINED REPORT MESSAGE AND CHARGE PARAGRAPH Once you have determined the main points of each finding and ordered/flowed them in a report message/charge paragraph, you have accomplished Phase 1 of Report Drafting Phase 2 is the actual “pen to paper” (finger to keyboard) of fleshing out the details of each point of your report message/charge paragraph It cannot be over-emphasized how important completion of Phase 1 and the prior steps we discussed is prior to detailed report drafting, or else you will waste time drafting multiple different versions trying to pitch findings in different ways, and relating different findings to each other in various combinations. If Phase 1 is completed well, Phase 2 can actually be the easier part of the drafting process. 32 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

33. DRAFTING REPORT SECTIONS FROM THE COMBINED REPORT MESSAGE AND CHARGE PARAGRAPH (cont.) In Phase 1 you determine how the pieces fit to complete the puzzle. In Phase 2 you craft the details for one piece (section or subsection) at a time. Each piece tells a specific part of the story. But the ordering and content of the sections and subsections need to flow from one another and each need to tell a convincing story or argue a specific point in a persuasive manner. 33 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

34. DRAFTING REPORT SECTIONS FROM THE COMBINED REPORT MESSAGE AND CHARGE PARAGRAPH (cont.)  Have to decide how to structure/map out the flow of the main points of the findings into sections and subsections?  Which are separate sections, or subsections of other sections?  Do some points get paired with other points in the same section?  Which points are stronger than others?  How do the weaker points fit in? 34 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

35. LINKING THE FINDINGS TO THE CONCLUSIONS AND SETTING UP THE RECOMMENDATIONS Conclusions are a very different part of the report; they bring together the message and tell the reader what it all means – the bottom line  Not just a summary of the main points  You need to stand back from the details in the draft  Determine the “bottom line” message(s) at a higher level than the details of the findings  Need to flow reader directly into recommendations The reader should expect them. 35 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

36. Final Thoughts “The time to begin writing an article is when you have finished it to your satisfaction. By that time you begin to clearly and logically perceive what it is you really want to say” (Mark Twain) “I learned that the only way you are going to get anywhere in life is to work hard at it. Whether you’re a musician, a writer, an athlete or a businessman, there is no getting around it. If you do, you’ll win - - if you don’t you won’t” (Bruce Jenner) 36 © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

37. Questions? Andrew.patchan@frb.gov © 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author. 37