1. National Science Foundation
Division of Science Resources Statistics
Defining an Economic Contributions Module to NSF’s Higher Education R&D Survey
John Jankowski
November 19, 2009
National Science Foundation
Division of Science Resources Statistics

2. Current Academic R&D Survey
National Science Foundation
Division of Science Resources Statistics
Current Academic R&D Survey
Currently a census of all universities and colleges with minimum of $150,000 of R&D spending (N = 708)
Conducted annually since Fiscal Year (FY) 1972
Web-based survey used by 99% of respondents
Voluntary survey (response rates regularly over 95%)
Requests expenditures for all separately budgeted R&D performed at institutions during previous FY
Detailed statistical tables published at the institution level

3. Current Academic R&D Survey (cont’d)
National Science Foundation
Division of Science Resources Statistics
Current Academic R&D Survey (cont’d)
Collects R&D expenditures by
Source of funds (federal, state/local, industry, institution, other)
Character of work (what percent is basic research?)
Field of science & engineering (S&E)
Federal agency sponsor and S&E field
Amount expended on research equipment, by S&E field
Amount passed through to subrecipients or received as a subrecipient
Non-S&E field

4. National Science Foundation
Division of Science Resources Statistics
Why Redesign?
1970s
Government largest source of R&D $$$; state government and
institution own funds ~10% each
Research at the bench
Single discipline focus
Life sciences ~50%;
physical sciences ~11%;
social sciences ~8%
Individual PI research
Individual labs
Basic research and publications
2000s
Government still largest source; decline in state government (~6%) but increase in institution own funds (~20%)
Computer & IT assisted research
Increase in interdisciplinary research
Life sciences ~60%;
physical sciences ~8%;
social sciences ~4%
Increase in collaborative research
Growth in centers

5. Academic R&D Survey Redesign Project
National Science Foundation
Division of Science Resources Statistics
Academic R&D Survey Redesign Project
Overview of Activities
Activities completed:
Recordkeeping interviews and cognitive testing of new questionnaire with 16 institutions
Consulted data users and academic expert panel
Multiple iterations developing and testing questions (~20 institutions) including possible IP and commercialization module
Final revisions to draft questionnaire
Web survey development - usability testing (15 institutions)
Selection of 40 institutions for FY 2009 pilot test
Pilot test launched November 16

6. Redesigned Higher Education R&D Survey
National Science Foundation
Division of Science Resources Statistics
Redesigned Higher Education R&D Survey
Changes to Questions: Results of Activities
All fields of R&D (both S&E and non-S&E) included in all survey items (previously only S&E R&D was included in totals)
Clinical trials and research training grants are now included in definition of R&D
Sources of funds: Separate categories have been created for nonprofit organizations and for institutional cost sharing
Basic research: question now asks for expenditures associated with basic research, applied research, and development
Expenditures by field and source: information on expenditures is collected by field of R&D for EACH individual source of funds

7. Redesigned Higher Education R&D Survey
National Science Foundation
Division of Science Resources Statistics
Redesigned Higher Education R&D Survey
New Questions
Total R&D expenditures of funds from foreign sources
Total R&D expenditures within a medical school
Total clinical trial R&D expenditures
R&D expenditures by type of funding agreement: contracts vs. grants
Listing all individual federal agency sources for R&D expenditures

8. Redesigned Higher Education R&D Survey
National Science Foundation
Division of Science Resources Statistics
Redesigned Higher Education R&D Survey
New Questions (cont’d)
Total R&D expenditures within interdisciplinary research centers
Specific cost elements of R&D expenditures (salaries, software, equipment, etc)
Headcounts of R&D personnel
R&D proposal and award counts, including ARRA awards
Nothing on IP or commercialization

9. What Data Users Want on IP/Tech Transfer
National Science Foundation
Division of Science Resources Statistics
What Data Users Want on IP/Tech Transfer
Results from a workshop held June 27, 2007
Many felt NSF survey should become gold standard for tech transfer measures
Data on patents and licenses ranked highest priority
Intellectual property/technology transfer should be broadly defined to include consulting and software products
Survey should measure university/industry collaboration, more complete startup data, seed funding and incentification resources, student involvement in IP, and funding sources of technology transfer
Not necessary to collect annual data

10. Academic Expert Panel Concerns
National Science Foundation
Division of Science Resources Statistics
Academic Expert Panel Concerns
Findings from meetings held in 2007 and 2008
Patents and licenses alone inadequate to represent academic R&D productivity. Research publications, disclosure documents, and copyrights also indicators of productivity
Concern over potential for misuse of data and reinforcing idea that sole purpose of academic R&D is to generate $$
Panel recommended against addition of technology transfer module to HERD survey. If considered, should address broader array of outputs, not just IP

11. Support from Senior NSF Management
National Science Foundation
Division of Science Resources Statistics
Support from Senior NSF Management
Nov NSF Director and Deputy Director Briefing
Drs. Bement and Olsen briefed and provided status report on HERD Survey redesign
IP/Tech Transfer module discussed
User demand and perceived importance of data to National Innovation System and as an output metric
Expert Panel concerns and objections
Senior NSF management affirmed need and supported SRS collection of data

12. IP Questions Tested Types of intellectual property activities:
National Science Foundation
Division of Science Resources Statistics
IP Questions Tested
Types of intellectual property activities:
Invention disclosures
Licenses executed
Option agreements
Active licenses
Patent applications and patents issued: utility, plant, design
Foreign patents filed and issued

13. IP Questions Tested (cont’d)
National Science Foundation
Division of Science Resources Statistics
IP Questions Tested (cont’d)
US patents filed and issued, by technology area or S&E field
Number of licenses executed, by technology area or S&E field
Revenue received from:
Exclusive vs. non-exclusive licenses
Sales of royalty rights and sales of equity
Number of start-up companies:
With university ownership stake
By technology area or S&E field

14. Cognitive Interview Findings on IP
National Science Foundation
Division of Science Resources Statistics
Cognitive Interview Findings on IP
Considerable overlap with AUTM questions and i-Edison reports
New questions not “different enough” from AUTM
Questions not meaningful: “99% of the value is in 1% of the stuff” (similar objections to AUTM questions)
Totally new questions (e.g., by technology areas) difficult to answer, burdensome, and subject to varying interpretations
Little concern about confidentiality or sensitivity
Respondents suggested new question areas: patents by foreign country; breakdown of licenses by revenue generation and lifetime revenue stream; info on unlicensed technologies; etc.

15. New R&D Personnel Questions
National Science Foundation
Division of Science Resources Statistics
New R&D Personnel Questions
R&D personnel headcounts
How many Principal Investigators were paid from the R&D salaries and wages you reported?
How many other personnel were paid from the R&D salaries and wages you reported?
Of the headcount total reported above, how many are postdocs?

16. Issues with R&D Personnel Questions
National Science Foundation
Division of Science Resources Statistics
Issues with R&D Personnel Questions
R&D personnel headcounts
Expenditures and personnel data systems not linked
Difficult to identify all personnel working on R&D: some institutions pay investigators’ salaries during school year; external salary support only requested for summer work
Manual tabulation required
Research faculty by field
Problematic definition of “faculty” (e.g., tenured vs. non-tenured)
Can be identified by department but not by R&D discipline
Problem of classifying faculty with dual (or multiple) appointments

17. Issues with R&D Personnel Questions
National Science Foundation
Division of Science Resources Statistics
Issues with R&D Personnel Questions
(cont’d)
Full-time Equivalent R&D personnel
Difficult to calculate: staff with appointments of various terms (9-month, 10-month, and 12-month)
Varying availability of data for different types of nonfaculty R&D personnel (e.g. postdoctorates, graduate students, etc.)

18. Next Steps: SRS-Funded APLU Workshop
National Science Foundation
Division of Science Resources Statistics
Next Steps: SRS-Funded APLU Workshop
From Outputs to Outcomes: A National Workshop to Identify Additional Measures of University Contributions to Regional Economic Growth and Innovation – grant award to the Association of Public and Land-grant Universities
Potential new measures to be considered:
Start-ups/new business formation
Community engagement
Faculty consulting
Human transfer
University application of resources