USING DATA TO IMPROVE PROGRAM PERFORMANCE 2013 TIG CONFERENCE Presenters: Michael O’Connor, Prairie State Legal Services (IL) Rachel Perry, Cleveland Legal Aid Society Jonathan Pyle, Philadelphia Legal Assistance Bristow Hardin, Legal Services Corporation (moderator)

The Presenters Will Highlight: The value of effective data compilation, analysis and reporting for assessing and improving program management and program performance Concrete examples of: 1. How the use of data can improve specific program operations and activities, e.g., identifying client needs, targeting resources, strengthening advocacy strategies, and improving management oversight 2. Major components of systems designed to compile and use data to accomplish these purposes 3. Data sets and metrics for assessing and improving program operations 4. Key operational steps and major challenges involved in the development and implementation of data compilation and analysis systems.

Effective Use of Data is an Essential Management Tool Appropriate data are necessary to meaningfully assess program implementation and results. Without such data, how effectively can program managers: Identify client needs and target program resources? (U.S. poverty population grew by 42% from 2000 to 2011; significant changes in geographic location and demographics of client population; economic trends and Great Recession had diverse regional effects – How has program responded?) Assess the effectiveness and efficiency of programs operations (e.g., intake systems, legal work supervision)? Evaluate the results of advocacy and other client services (e.g., client groups served/not served, impact of advocacy strategies)? Most broadly: How effective is our work? How can we improve?

Different Types of Data for Different Audiences Program management & board: assess and improve operations Program staff: evaluation and feedback, improve advocacy and client services Funders: demonstrate effectiveness of work, support funding requests Partners: assess the potential value of and develop joint initiatives Public / Media: “tell the story” and demonstrate the value and effectiveness of legal services Client community: articulate what we’re doing, enhance dialogues about how we can improve

One Size Does Not Fit All Data systems must be tailored to a program’s particular needs, resources and capacities. Key questions: What operational components do you want to examine? What do you want to find out? What are the appropriate data sets? Will they tell you what you need to know? What type(s) of evaluation system(s) are appropriate and feasible? What resources – financial and staffing – will be available? Who are appropriate partners?

Effective Metrics and Data Sets Are Critical “Knowing it when we see it” is not an acceptable evaluation criterion Metrics/data sets must be tailored to and appropriate for: 1. Specific systems / activities 2. Purposes 3. Audiences Many useful data sets are readily available or can be obtained cost-effectively: 1. Administrative data, e.g., case management system, intake data 2. Surveys of / interviews with staff, clients, stakeholders (however, value of many “client satisfaction surveys” is limited) 3. Census Bureau data and data from other public entities Universities and other policy/research entities can provide valuable assistance

LSC Data Initiatives LSC is undertaking initiatives to improve its use of data to: Enhance its effectiveness and efficiency Enable it to more effectively: 1. assess and improve grantees’ services 2. demonstrate the results grantees achieve for clients 3. better measure the extent to which grantees are unable to meet clients’ needs because of limited resources. LSC Grantee-Focused Data Initiatives TIG Data Projects LSC Data Collection and Reporting Project:

LSC Grantee-Focused Data Initiatives TIG Data Projects An “area of interest” in the 2011 and 2012 TIG Grant Cycles was the “Use of Data to Analyze Service Delivery and Develop Advocacy Strategies to Data Analysis” Total of six (6) data projects were funded in 2011 and 2012 LSC Data Collection and Reporting Project: Improved system for collecting and analyzing data from LSC grantees Tools and resources that enhance LSC grantees’ ability to collect and use data to: 1. Design, assess and improve their delivery strategies and program operations 2. Demonstrate the need for and benefits of the services they provide clients in their communities Provide training and technical assistance that fosters LSC grantees’ effective use of the tools All LSC data initiatives informed by and responsive to Grantees’ particular delivery models and local circumstances Other funders’ oversight and reporting requirements