1. Structuring Performance Data to Effectively Drive Management Decisions Tim Gribben, Josh Bull, & Mat Pascarella O FFICE OF P ERFORMANCE M ANAGEMENT U.S. S MALL B USINESS A DMINISTRATION December 5, 2012 1

2. Embedding Performance Management  Cascading Data-Driven Decision-Making  Framework of SBA’s Training Course  Permeation & Permutation of Training 2

3. Embedding Performance Management  Cascading Data-Driven Decision-Making  Framework of SBA’s Training Course  Permeation & Permutation of Training 3

4. Performance Management as Organizational Culture Senior Leadership must use data-driven quarterly performance reviews of CAP and Priority Goals (GPRA Modernization Act of 2010) Many agencies run this exercise out of their respective Performance Management Offices However, for performance management to effect dramatic change, the more difficult task will be to integrate the same data-driven decision-making throughout the agency, especially among program managers and mid- level analysts. Senior Leadership Program Managers Mid-level Analysts 4

5. Recognizing a Problem & Finding the Solution I DENTIFIED G AP SBA not effectively analyzing the wealth of data available nor using data effectively to drive management decisions L EADERSHIP B UY -I N Office heads select interested managers or analysts with basic experience in data analysis and willing to commit 3-4 hours per week to training PMO R UNS T RAINING PMO develops six week training course to teach the skills via lecture, activities, and homework as well as “real-world” final presentation 5

6.  ‘Lesson Learned’ continually improves course content & format  Final presentations raised by course participants turn into agency-wide projects  Future presentations to mgmt incorporate class teachings Making The Training “Stick” Goals Expectations M ANAGERS I NSTITUTIONALIZATION Getting buy-in from managers for maximum impact on participants, from executive leadership is key for institutionalization  Invest in personal & professional development of employees  Surface actionable, interesting analytic questions for participants to explore  Required to meet with their participating employees to discuss training goals and expectations  Asked to check in on participants to proffer help or feedback during training  Including phrase into SBA’s new strategic plan – “Investing in SBA’s employees through business, technical, and analytical training” 6

7. Summary of SBA’s Path  Identified skills gap related to data analysis / analytical presentations  Relatively few used data effectively to inform management decisions  Reliance on data summarization without insights WHERE WE WERE IN 2009  Created a training course to build new skills  Included it as an objective & strategy in the new strategic plan  Senior leaders and managers were also trained in the process WHAT WE DID  Data is being presented more effectively to inform management decisions  Senior leaders are using data to address management challenges WHERE WE ARE NOW 7

8. Embedding Performance Management  Cascading Data-Driven Decision-Making  Framework of SBA’s Training Course  Permeation & Permutation of Training 8

9. Structure the Presentation Communicate the analysis Synthesize & Develop Recommendations Session Key Lessons  Structuring a management presentation  Developing a story-line  Presenting and communicating with data  Creating visual charts and graphs  Synthesizing findings into management recommendations  Developing data-driven ‘so-what’ insights P URPOSE OF T RAINING : To help participants develop and structure analytic presentations that inform SBA management decisions Training Purpose & Format 9

10. WHAT THE TRAINING ISWHAT THE TRAINING ISN’T  Training to help develop presentations that inform management decisions Structuring Communicating Data Synthesis  Way to EMPOWER employee to drive SBA business decisions Χ Excel 101 Χ Powerpoint 101 Χ How to Report Data Χ Oral Presentation Coaching Χ Understanding SBA’s Programs  Risk-free environment Χ Formal Assessment of. Employee’s Abilities  Interactive with assignments  Group oriented Χ US lecturing THEM Χ Rigid Curriculum Data Analysis Training Ground Rules 10

11. Structure the Presentation Communicate the analysis Synthesize and Develop Recommendations Two step process – “Structure the Presentation”: Structure the management question (using an Issue True)  Organize the issues into buckets  Anticipate the data and analyses that will answer the questions Build a story line (using a “Ghost Deck”)  Plan the work to be done  Align with managers on direction before investing the time 1 2 S ESSION #1: Structure the Presentation 11

12. While developing the ‘Issue Tree’ to structure issues, we might consider two principles to answer ALLRELEVANT ALL and the most RELEVANT questions 1 …and keep digging deeper ( ask “So-What?” or “Why?” ) to ensure the issues are relevant Example: Why is my credit card bill increasing? Be “Mutually Exclusive, Collectively Exhaustive” (MECE) to identify all issues… Example: How can I save money for a vacation? Structure the Presentation 12

13. How can I save money for a vacation? Spend LESS money Earn MORE money Questions and analyses to consider Can I earn more interest on existing income? Can I invest in stocks/funds instead of checking account Can I earn new income? Can I sell garage items by having a yard sale Can I buy cheaper items? Coffee machine, not Starbucks; Shop at discount stores Can I buy fewer items? Spend less on gas by driving at 55 mph on the highway Be “Mutually Exclusive, Collectively Exhaustive” (MECE) to identify all issues 1 Structure the Presentation: Organize the Issues 13

14. SituationComplications Options to Consider Depending on the purpose, a senior management presentation might include the following components Inform a Decision Problem(s) to address Recommendations & Rationale Implementation Plan Build a Proposal PurposePotential Considerations Reason for Action (or context) Actions Taken Results Achieved Showcase a Success 2 Build the Story-line 14

15. 15 Build an Issue TreeHow to Build an Issue Tree (MECE)Mutually exclusive, collectively exhaustive (MECE)  Start as MECE as possible. It’ll help as you build. Relevant QuestionsHow to Turn an Issue Tree into Relevant Questions “so-what” “why”  Keep asking “so-what” and “why” story lineApproaches for building a story line  Situation, complication, resolution “ghost deck”Constructing a “ghost deck” Key Lessons: Structure the Presentation 15

16. Synthesize and Develop Recommendations Structure the Presentation Communicate the analysis Determine your message 1 Identify the comparison 2 Select the best chart type 3 MATERIAL ADAPTED FROM GENE ZELAZNY’S “SAY IT WITH CHARTS” S ESSION #2: Communicate the Analysis 16

17. The chart type you choose depends on the specific point or message you want to convey Therefore, the first step is to identify what message you want to share 1 Determine Your Message 17

18. 5 TYPES OF COMPARISON COMPARISON TYPEPURPOSEEXAMPLE ComponentDisplaying the size each component as a percentage of the total X% of government contracts go to small businesses ItemCompare how things rank Center X ranks fourth among all regions for program activity Time SeriesTrends or changes over time Student loan debt increased significantly Frequency Distribution Number of items that fall into numeric ranges Most grants fall into the $15 million -$25 million range CorrelationIdentify the strength of a relationship between two variables There is a strong relationship between employee satisfaction and employees’ awareness of responsibilities Source: Gene Zelazny, “Say it with Charts” 2 Identify the Comparison 18

19. K EY TYPES OF CHARTS Pie chart Bar chart Column chart Line chart Dot chart ComponentItemTime series FrequencyCorrelation Pie Bar Column Line Dot 3 Select the Best Chart Types 19

20. S CENARIO C OMPARISON T YPE C HART T YPE Percentage of government contracts by business size classifications (i.e., small business) Relationship marketing outreach & attendance at national parks Change in student debt levels from 2007-2010 Breakdown of the value of goods exported by region, as a percentage of the total Breakdown of grant awards by size buckets (i.e. 0-$500k; $500k-$1M, etc.) Frequency of small business government contracts by NAICS code Employee satisfaction vs. AWS participation rate Speed of disaster assistance for a given disaster by time buckets (i.e., 1 week; 2 weeks; etc.) 2009-2010 weekly count of White House press releases 3 Exercise!!! 20

21. 21 Determine the message Identify the comparison  Component, Item, Time Series, Frequency Distribution, Correlation Select the chart type  Pie, Column, Bar, Line, Scatter, or Other Key Lessons: Communicate the Analysis 21

22. Communicate the analysis Structure the Presentation Examples of Synthesis How to Synthesize & Develop Management Recommendations Synthesize and Develop Recommendations 1 2 Session #3: Synthesize & Develop Recommendations 22

23. Don’t just summarize. Summary re-states all the facts briefly; synthesis includes insights that help managers make decisions Ask “so-what”, “why”, or what does this mean for my manager, office, team, constituent, etc.? Lead with the answer (but it depends on the context) Foreshadow: Your initial synthesized statement should map to the remainder of the actions/recommendations/analysis Develop actionable recommendations that are supported by the facts 1 Tips for Synthesis 23

24. 1 Structure/highlight the major issues 2 Spell out the recommendations 3 Present the action plan (or path forward) 4 Ask for agreement/commitment (if appropriate) 5 Close off with “next steps” 1 Potential Components of a Synthesis 24

25. Dear Elizabeth W., How are you? I hope this note finds you well. I must say, I have not been doing too well— everything seems to be going wrong. My life is in shambles. For example, I had to pull two all- nighters last week for a very big report that was due last Friday. My boss said he told me about in two months earlier, but I totally forgot about it until he asked for a semi-final draft last Wednesday. I scrambled and just barely got it done—though I am so exhausted now. OMG. And then this weekend was really sad because it was my 30th birthday on Saturday, and nobody came to my party!!!  On Wednesday, I sent people a text message Friday night saying “come to my house tomorrow for dinner at 8 pm”. Half of my friends already had plans, and the other half went to my old house—because I forgot to tell them I moved!! Then they got mad at me for all the hassle. So my dog and I alone ate the 5 trays of meatloaf I cooked. Ugh. And to top it off, I got a call from my bank, and they are going to repossess my car! Apparently I missed three months of payments (I totally forgot to send them my new address for the bills!). And when I tried to use my credit card at the grocery store, it didn’t even work, as it had been shut off after reaching my credit limit. Now I’m hungry and can’t drive to work. What is wrong with me? Why do I have the worst luck? Sincerely, Josh 275 Words 2 E MAIL E XAMPLE : Before synthesis… 25

26. 2 E MAIL E XAMPLE : After Synthesis… Dear Josh, It seems you have a lot going on in your life. You may need some personal training in the areas of: Time Management Communication Organizational Skills Have you tried using a Life Coach? Let me know if I can help. Good Luck! Sincerely, Elizabeth W. 49 Words 26

27. 2 E MAIL E XAMPLE : Model Structure for an Answer 96 Words Dear Josh, You’re disorganized—you need to better plan/schedule your work activities, social gatherings, and finances. At work, you haven’t anticipated deadlines or managed workload, causing work to build. Keeping a better team calendar may help. In social settings, you don’t plan events with advance notice and clear instructions, leading to damaged relationships. You should communicate better with friends, and send formal party invitations. Financially, you have missed several due dates, damaging your credit standing. Consider on-line ‘bill pay’ and other tools to easily update your records and manage your expenses. Sincerely, Elizabeth W. 27

28. Dear Josh, You’re disorganized—you need to better plan/schedule your work activities, social gatherings, and finances. At work, you haven’t anticipated deadlines or managed workload, causing work to build. Keeping a better team calendar may help. In social settings, you don’t plan events with advance notice and clear instructions, leading to damaged relationships. You should communicate better with friends, and send formal party invitations. Financially, you have missed several due dates, damaging your credit standing. Consider on-line ‘bill pay’ and other tools to easily update your records and manage your expenses. Sincerely, Elizabeth W. Actionable recommendations Track the options What is the action? Lead with the answer Maintain a structure Highlight implications 2 E MAIL E XAMPLE : Model Structure for an Answer 96 Words 28

29. Because larger SBA Express loans are better performing, raising the caps on SBA Express above $350K may have a modest impact on program cost 2 Although SBA Express loans have relatively higher purchase rates, larger SBA Express loans are higher performing Note: Normalized purchase rate is defined as SBA purchase amount as a percent of SBA-guaranteed portion of the total disbursement amount. PowerPoint Example Example 29

30. Because larger SBA Express loans are better performing, raising the caps on SBA Express above $350K may have a modest impact on program cost 2 Although SBA Express loans have relatively higher purchase rates, larger SBA Express loans are higher performing Note: Normalized purchase rate is defined as SBA purchase amount as a percent of SBA-guaranteed portion of the total disbursement amount. HIGHLIGHT THE INSIGHT THE SO-WHAT A DESCRIPTIVE LEAD HIGHLIGHT THE INSIGHT PowerPoint Example Example 30

31. 31 Synthesize EVERYWHERE Ask “so-what”, “why”, or what does this mean Lead with the answer (depending on the context) Foreshadow Develop actionable recommendations Don’t just summarize. Highlight the insights K EY L ESSONS : Synthesize & Develop Recommendations 31

32. Teams of 2-4 participants Judges will be “VIP” SBA panel Who? Develop and present a 5-10 page management PowerPoint presentation on any SBA-specific topic. Work should: o Exhibit command of structuring and story-lining o Use data-driven charts and graphs effectively o Contain clear, actionable management recommendations and synthesis What? To show everyone what they have learned To have fun Why? S ESSION #4: Applying the lessons…THE CONTEST Prizes for best weekly assignments GRAND PRIZE for best final presentation Lunch at White House with Administrator What can I win? 32

33. C OMPETITION C HECKLIST : Criteria to Consider for Data-Driven Presentations Mastery AreaFeature Structuring the Presentation Does the presentation build a compelling, coherent story? Does the presentation identify (and prioritize) the most appropriate issues for analyzing a specific management question? Communicating the Data Does the presentation incorporate relevant data/evidence to inform a management decision? Is the data presented in a visually-compelling format that sends a clear message? Synthesizing Findings & Developing Recommendations Does the presentation surface interesting, high-level insights supported by the data? Does the presentation provide actionable recommendations (or options) for management to consider for improving SBA’s operations/programs? 33

34. Embedding Performance Management  Cascading Data-Driven Decision-Making  Framework of SBA’s Training Course  Permeation & Permutation of Training 34

35. P ERMEATION : Cultural shift created in the SBA R ESULTS OF HQ C LASSES o 1 st Class Winner: Senior level task force formed that developed a comprehensive plan to address retirements and led to a VERA/VSIP action. o 2 nd Class Winner: Analysis being used to inform Agency efforts for boosting volume of loans going to underserved markets. 35

36. Performance data highlighted a management question that needed exploration SBA’s Average Time-to-Hire was Worse than Agency Target 36

37. An Issue Tree was created to ask relevant questions and inform the data analysis How can SBA reduce time to hire? Cut # of steps Cut time per step SBA Steps Candidate Steps Program Office OHCM Utilize hiring flexibilities Pre- Interview InterviewOffer … Direct Hire Authority Excepted Service Veterans Part-Time Reemployment (2010 NDAA) STEPPMF Questions to Assess How many employees are eligible to retire? Ease of execution? Cost implications? Continuity in critical missions? Mentorship opportunities ? Infusion of new talent? 37

38. Further Data Analysis Led To: RISK In 5 years, Retirement & VERA eligible employees will outpace ineligible employees by 66% (2 out of 3 GS will qualify for an immediate annuity through voluntary or early retirement) CURRENTLY + 3 Years + 5 Years How can SBA mitigate effects of impending retirement tsunami? 38

39. How can SBA loan dollars more effectively go to underserved communities? Questions and analyses to explore Increase Demand for Loan Programs by Underserved Community Increase outreach to underserved business communities Increase the number of businesses that qualify for loans What loan programs do banks use most in underserved communities? What are the characteristics/criteria for these loan programs? Increase Supply of Loan to Underserved Community Increase the # of SBA programs targeting underserved businesses Increase the number of banks lending money in underserved communities How many banks are serving small businesses in underserved communities? What are characteristics of banks in underserved communities? (Size, national branch, local bank, etc)? What percentage of SBs are approved for loans through targeted underserved programs? Are SBA loan programs evenly distributed to underserved communities, by Region? Are SBA loans programs reaching the high concentration of underserved small businesses that they are created for? 39

40. Lender Outreach can be tailored to be region specific. Region 9’s Best Practices should be shared Highest concentration of Lenders USM Need not aligned with results A chart was used that highlighted insights… 40

41. P ERMUTATIONS : The Training Course at SBA BASICPrinciples  4 hours  Covers basic topics much like this presentation  Example Audience: Senior Leadership CONDENSEDSeminar  1.5 days  More in-depth look at principles  Built-in homework  Pre-defined options and data sets for the final presentations  Example Audience: regional/national conference attendees PRIMARY Full Course  4 to 6 weeks, each 2-3 hrs  Individual & Group homework  Deep dive into agency’s data sets & team-defined ACTIONABLE projects  Incentives for final presentation  Example Audience: Headquarters staff or dedicated field office 41

42. Interested in Finding Out More? Tim Gribben Director, Office of Performance Management Timothy.Gribben@SBA.govTimothy.Gribben@SBA.gov (202) 205-7565 U.S. Small Business Administration 409 3rd Street, SW, Ste 6000 Washington DC 20416 Josh Bull Program Analyst & Budget Analyst, Office of Performance Management and Office of CFO Joshua.Bull@SBA.govJoshua.Bull@SBA.gov (202) 205-7603 Mat Pascarella Program Analyst, OCFO Mathew.Pascarella@sba.govMathew.Pascarella@sba.gov (202) 205-6102 42