PSI Online The Socioeconomic Effects of Public Sector Information on Digital Networks: Toward a Better Understanding of Different Access and Reuse Policies 5 February 2008, OECD Headquarters Conference Centre, Paris Chairs: A) Eivind Lorentzen, B) Jean-Jacques Sahel Rapporteurs: A) Juan Carlos de Martin, B) Tilman Merz Summary of Session Three: Measuring the Economic and Social Costs and Benefits of PSI: Evaluation of the Existing Approaches and Suggestions for Future Work

Overview I.Questions for Discussion II.Break Out Session A– Government and PSI Supply-Side Experts III.Break Out Session B – Industry Representatives and Users IV.Summary

I. Questions for Discussion 1.What are the commonalities and differences among the various analytical methods identified and presented in Session two and in this Session? 2.What are the most effective metrics/indicators that can assess particular kinds of information/policies? What approaches and metrics/indicators effectively measure the network effects of the use of PSI online? 3.What are the main strengths and weaknesses of these approaches, e.g., their accuracy, comprehensiveness, relevance, validity and reliability? 4.What still needs to be known about the application of these methods to the evaluation of public information policies in the online environment? 5.What are theoretical frameworks, models and best practices used in assessing other information products or services that can be applied to the assessment of different policies of access to and reuse of digital PSI? 6.What are some future directions for the better study and measurement of access to and reuse of PSI online? 7.What other questions or issues should be raised in this context?

II. Break Out Session A – Government Representatives and PSI Supply-Side Experts Session A

Session A – Overview Revised Questions for Discussion 1.What are the main strengths and weaknesses of these approaches, e.g., their accuracy, comprehensiveness, relevance, validity and reliability? 2.What are theoretical frameworks, models and best practices used in assessing other information products or services that can be applied to the assessment of different policies of access to and reuse of digital PSI? 3.What are possible future directions for the better study and measurement of access to and reuse of PSI online? Session A

Question 1 – Existing Approaches 1. What are the main strengths and weaknesses of these approaches, e.g., their accuracy, comprehensiveness, relevance, validity and reliability? At this stage, no methodology seems clearly superior or general enough to be singled out. Need to proceed with more research using multiple methodologies. Session A

2. What are theoretical frameworks, models and best practices used in assessing other information products or services that can be applied to the assessment of different policies of access to and reuse of digital PSI? Existing frameworks could facilitate/cross-fertilise PSI analysis: o Open Access / Scientific Publishing o Open Source / Free Software o Assessing experience in specific sectors (meteorological services, geo-information, etc.) Existing research on the role of the public sector Question 2 – Relevant Frameworks Session A

3. What are possible future directions for the better study and measurement of access to and reuse of PSI online? a)OECD PSI Manual b)PSI Repository c)Academic Research Question 3 – Future Directions Session A

a) OECD PSI Manual Progress in measurement of PSI policies needed Project would ideally involve statisticians, national accountants as well as PSI experts A similar model was successfully used by OECD DSTI in co-operation with Eurostat in 1999 to produce a manual on data collection and analysis in the environmental goods and services industry Working towards a manual for PSI data collection and analysis of PSI policies Session A

a) OECD PSI Manual An OECD PSI Manual would… take into account the OECD PSI Policy Principles; address different audiences, such as Practitioners Re-users PSI Holders National accountants; clarify the extent and kinds of PSI; make measurement practices more comparable; promote involvement of additional stakeholders (regional, global, statistical entities, etc.). Session A

a) OECD PSI Manual Stakeholders from the following groups need to be identified and involved PSI Holders PSI Users and Re-users NGOs Libraries Publishers Sectoral bodies/communities (meteorological, geo- information, etc.) Session A

b) PSI Repository Requires Clarification of Several Dimensions Content Format (portal, database, forum, etc.) Ideally not a passive repository but a platform for proactive knowledge creation (e.g. Wiki, forum, etc.) Link to existing EPSIplus Project (EU-specific, future funding issues) Language How can a general PSI portal serve specific communities? Session A

c) Future Academic Research Considerations 1. Specific topics (see following slide) 2. Research funders: EC, foundations, etc. 3. Workshop/conferences 4. Online journal 5. Awards for theses and dissertations 6. The PSI manual Session A

c) Future Academic Research Specific Topics Special attention should be paid to research on individual users and on social effects of PSI use/re-use Network effects and network externalities (both positive and negative) need new assessment methods Role of automated knowledge extraction and re-use Pilot projects to test different approaches Promoting involvement of young researchers Session A

III. Break Out Session B – Industry Representatives and Users Session B

Session B – Overview 1.General Points on Future Directions and Measurement of PSI Policy Impacts 2.OECD PSI Principles 3.PSI Definitions 4.Repository for Mutual Support in Measuring PSI 5.Data Collection 6.Mapping the Value Chain 7.Potential further Studies Session B

1. General Points on Future Directions and Measurement of PSI Policy Impacts Session aims to generate ideas on next steps for research, measurement and analysis; also for follow-up to the OECD policy principles Need to produce facts and analysis for policy makers, businesses and users Need policy evaluation over time or across countries, using the same methodologies Possibly focus on top-5 PSI sectors across a number of countries – although these may differ by country Impact of PSI policy can depend on different political and institutional contexts – including at the sub-national level Broad support for looking at comparative experiences, e.g. telecoms liberalisation, environmental goods Session B

2. OECD PSI Principles Broad interest in OECD PSI principles Theme is recommendation to disseminate widely at lowest cost possible Importance of preparing framework and gathering information for any future review (once approved) Pay attention to the immediate obstacles which the application of the principles might pose for public finances Session B

3. PSI Definitions Importance of understandable and comparable definitions, e.g. product definitions, to go forward with the manual (on data collection and analysis) Important to make research more easily comparable and more focussed Potentially an approach that classifies types of PSI by function (end-use) is useful, rather than or in addition to, a product or activity approach Session B

4. Repository Elements could include Coordination of the different levels of surveys Exchange of approaches to, and tools for, data collection (questionnaires; model questionnaires; how to contact users) Exchange of survey results Database of best practices for PSI policy (e.g. on low-cost diffusion of PSI) Efforts towards a classification of types of PSI (e.g. by product or by function) Function as reference for survey design, classifications, and other activities Needs structuring and long-term management; not discussed how it would be made operational Session B

5. Data Collection (1/2) Accounting practices, different product/activity/functions classifications make analysis of PSI market development difficult Some bodies can more easily collect data (e.g. national competition bodies compared with academics) Care must be taken in designing and conducting surveys Surveys OECD successfully develops model questionnaires (e.g. use of ICT in businesses, use of ICT in households) Ensure consistency and adherence to best practices in surveys Survey design depends on understanding how people use PSI What information to gather? E.g. obstacles to re-use and use Grouping survey respondents: E.g. in analysing survey data, OFTs contractor grouped PSIHs by pricing regime Session B

Sampling methods – Survey points of contact with users and re-users Must take into account potential (currently unsatisfied) demand for PSI use and re-use Potentially make public announcement about PSI research in fora used by current and potential data requesters Publicly funded libraries serve as distributors of much government information in North America and Scandinavia – good opportunity to learn about users, particularly citizen users Data from applications based on Freedom of Information legislation may provide information on PSI use and (potentially unsatisfied) demand 5. Data Collection (2/2) Session B

6. Mapping the Value Chain Value created via re-use depends very much on the product Map PSI supply and demand, building on previous work Analyse value chains for re-use: Where are costs incurred, where is most value created Model value chains in terms of activities/business processes Attach costs to value chain components Can reveal at which step of the value chain obstacles to value creation lie How to value social and economic impacts: Obstacles to re-use and use can be ranked by the number of people affected or by the magnitude of the economic cost Session B

7. Further Studies a)Transition costs of changing policy regimes (e.g. from/to UK trading fund model) b)Substitution of PSI by non-government information: Pay attention to where and why this is happening and quality of created data c)Examine PSI licensing costs across sectors/access regimes/countries d)Network externalities of PSI-based user-created content e)Draw parallels to other areas and comparative situations: e.g. impacts of telecommunications liberalisation f)Include PSI policy in measuring government performance Session B

IV. Summary – Sessions A & B Session B Session A

IV. Summary – Sessions A & B Importance of OECD PSI Principles and follow-up Manual to improve measurement: strong case Repository: strong case but some issues to be clarified Academic research: strong case, new topics and approaches proposed: Clarify PSI definitions, further data collection, mapping the value chain Session B Session A