Presentation on theme: "Program Evaluation and Metrics for SBA Regional Innovation Clusters Presented by: Berna Demiralp, Ph.D. Optimal Solutions Group, LLC M Squared Research."— Presentation transcript:
Program Evaluation and Metrics for SBA Regional Innovation Clusters Presented by: Berna Demiralp, Ph.D. Optimal Solutions Group, LLC M Squared Research Park | University of Maryland 5825 University Research Court, Suite 2800 College Park, MD 20740 301-306-1170 | info@OptimalSolutionsGroup.com
Introduction SBA’s Regional Innovation Clusters Initiative provides funding to ten clusters with the goal of strengthening clusters and enhancing their ability to assist small businesses; increasing opportunities for small business participation and growth within the clusters; stimulating regional economic development and growth.
What are clusters? “Clusters are geographic concentrations of interconnected companies, specialized suppliers, service providers, firms in related industries, and associated institutions (e.g., universities, standard agencies, and trade associations) in particular fields that compete but also co- operate” (Porter, 1998).
SBA Regional Innovation Clusters Clusters selected for the SBA’s Regional Innovation Clusters Initiative represent a broad spectrum of geographic areas and industries.
Agricultural Innovation Cluster (Monterey – Santa Cruz – San Benito, CA): agricultural innovation and leading-edge agriculture research. Carolina Nuclear Cluster (NC, SC): improving nuclear energy supply chain. Northeast Electrochemical Energy Storage Cluster (CT, NY, MA, ME, VT, NH, RI): developing, manufacturing, and deploying fuel cell and hydrogen technologies Advanced Power and Energy Cluster (MN, ND, SD, WI): power generation, storage, and alternative Illinois Smart Grid Regional Innovation Cluster (IL): Smart Grid innovation improving electric transmission infrastructure. Enterprise for Innovative Geospatial Solutions (MS, LA): geospatial businesses Upper Michigan Green Aviation Coalition (MI): expanding green aviation industry. NorTech FlexMatters (OH): advanced energy and electronic devices printed on flexible substrates. Huntsville Advanced Defense Technology Cluster Initiative (Huntsville, AL): aerospace technology San Diego Advanced Defense Technologies Cluster (San Diego, CA): autonomous systems, cyber security, and other technologies
SBA’s Regional Innovation Clusters Small businesses-oriented activities/events: Business counseling; Workshops, training, and mentoring; Matchmaking and networking activities; Assistance in obtaining financing; Commercialization and technology transfer; Export services; Intellectual property and patent services; Marketing services; and, Contract counseling and other legal services.
Evaluation Components Implementation evaluation Are clusters reaching the targeted small businesses in their regions? Are the cluster activities / events meeting the needs of the targeted small businesses? Impact evaluation Are the small businesses participating in cluster activities / events attaining their desired goals?
Program Parameters 1.Variety of cluster configurations and business models; 2.Varied small business recruitment strategies; 3.Varied inclusion and exclusion criteria for selecting small firm; and, 4.Clusters were free to choose the types and frequency of services/activities.
Data Collection Both quantitative and qualitative data: Cluster Administrator Survey Small Business Survey Large Organization Survey Interviews with Cluster Administrators Interviews with selected small businesses Cluster Quarterly Reports Quality Indicators Survey (to be completed by both the cluster administrator and the evaluator)
Data Collection Secondary data sources: Dun & Bradstreet database Quarterly Census of Employment and Wages (BLS) ADP Small Business Report Longitudinal Business database
Program Metrics and Outcomes Proximal: Access to capital; New alliances formed; Exports; Development of new products and services; Patents; Access to cleared secure facilities; Acquiring a contract or subcontract; Participation in industry chain; and Commercialization of new technology.
Program Metrics and Outcomes Distal: Number of businesses created; Revenue; Number of employees; and, Total compensation.
Metrics and Methodology How do the performance outcomes of small businesses participating in cluster activities/events change during the first year? Non-experimental: Performance metrics (levels) Growth models
Metrics and Methodology How do the changes in outcomes experienced by small businesses participating in cluster activities/events compare to the well-known benchmarks? Compare the changes in participating firms’ performance metrics (revenue, number of employees and total compensation) to similar small businesses in the same industries and geographic areas using secondary data sources.
Metrics and Methodology What impact did cluster participation have on small businesses’ performance metrics (revenue, number of employees, total compensation) during the first year? Quasi-experimental: Difference-in-Difference models Matched pair models
Metrics and Methodology What impact does cluster participation have on regional economies? Utilize input-output models for each metropolitan area. Input-output models take into account inter- industry interactions and transactions between producers, intermediaries, and consumers. Alternative input-output models: RIMS II (Census Bureau) IMPLAN REMI
Metrics and Methodology Input-output models estimate the impact on the regional economy: direct effect (e.g. 10 new jobs created in firm A); indirect effect (i.e. increase in output and employment in industries that supply firm A); and induced effect (i.e. the effect of increased household income and spending).
Metrics and Methodology What is the direct effect that will be analyzed in the input-output model? The change in participating firms’ performance outcomes (revenue, number of employees, total compensation) over and above the change in the outcomes experienced by the comparison group in the quasi-experimental models.
Conclusion Producing more cluster activities/events do not necessarily yield innovation and key outcomes; Investing in clusters do not necessarily yield key outcomes in the short-run (“not shovel ready”); Literature suggests that clusters can produce sustainable improvements in innovation and key outcomes; and, However, there are few rigorous evaluations of innovative clusters that provide generalizable findings.