1. AGN Seminar 11 May 2006 The Economic and Social Data Service (ESDS) Louise Corti UK Data Archive AGN Seminar 11 May 2006

2. What is the ESDS? national data archiving and dissemination service running from 1 Jan. 2003 – 2012 national data archiving and dissemination service running from 1 Jan. 2003 – 2012 Support social science -related research, learning and teaching Support social science -related research, learning and teaching jointly supported by: jointly supported by: Economic and Social Research Council (ESRC) Economic and Social Research Council (ESRC) Joint Information Systems Committee (JISC) Joint Information Systems Committee (JISC)

3. Partners UK Data Archive, Essex UK Data Archive, Essex Manchester Information & Associated Services (MIMAS), Manchester Manchester Information & Associated Services (MIMAS), Manchester Cathie Marsh Centre for Census and Survey Research (CCSR), Manchester Cathie Marsh Centre for Census and Survey Research (CCSR), Manchester Institute for Social and Economic Research (ISER), Essex Institute for Social and Economic Research (ISER), Essex

4. ESDS Overview provides access and support for key economic and social data provides access and support for key economic and social data distributed service, bringing together centres of expertise in data creation, dissemination, preservation and use distributed service, bringing together centres of expertise in data creation, dissemination, preservation and use provides seamless and easier access to a range of disparate resources for UK Higher and Further Education sectors and beyond provides seamless and easier access to a range of disparate resources for UK Higher and Further Education sectors and beyond

5. Sources / type of data supported Data for research and teaching purposes and used in all sectors and for many different disciplines official agencies - mainly central government official agencies - mainly central government international statistical time series international statistical time series individual academics - research grants individual academics - research grants market research agencies market research agencies public records/historical sources public records/historical sources qualitative and quantitative qualitative and quantitative links to UK census data links to UK census data access to international data via links with other data archives worldwide access to international data via links with other data archives worldwide

6. Collections held UKDA 5,000+ datasets in the collection 5,000+ datasets in the collection 200+ new datasets are added each year 200+ new datasets are added each year 18,000+ datasets distributed worldwide per year 18,000+ datasets distributed worldwide per year History data service in-house (AHDS) History data service in-house (AHDS) Census Registration Service in-house (CRS) Census Registration Service in-house (CRS) databanks databanksMIMAS 30+ series of international macro databanks 30+ series of international macro databanks

7. International network

8. ESDS overview core archiving service (UKDA) plus four specialist data services core archiving service (UKDA) plus four specialist data services ESDS Government (CCSR/UKDA) ESDS Government (CCSR/UKDA) ESDS International (MIMAS/UKDA) ESDS International (MIMAS/UKDA) ESDS Longitudinal (ISER/UKDA) ESDS Longitudinal (ISER/UKDA) ESDS Qualidata (UKDA) ESDS Qualidata (UKDA) These provide: value-added data and documentation service-specific web pages and help support and training teaching datasets/online samplers discussion lists/newsletters promotion publicity and promotion

9. Core Service run by UKDA overall integration and management of ESDS acquiring, processing, preserving and disseminating data central registration service operating across the ESDS central 'first stop' help desk service front line user support cataloguing and describing data maintaining and developing web presence publicity and training

10. Data In Data acquisition Data acquisition offers and proactive scoping offers and proactive scoping data evaluation data evaluation Data ingest Data ingest checking, verifying checking, verifying converting, formatting, processing converting, formatting, processing documenting and contextualising documenting and contextualising Data preservation Data preservation long-term data management long-term data management Preservation Policy Preservation Policy UKDA is a TNA (National Archives) Recognised Place of Deposit UKDA is a TNA (National Archives) Recognised Place of Deposit

11. Data out registration using Athens including agreement to an End User Licence, fine-grained access control registration using Athens including agreement to an End User Licence, fine-grained access control download service (SPSS, STATA, ASCII, RTF etc) download service (SPSS, STATA, ASCII, RTF etc) request data on CD request data on CD online data analysis, including online data analysis, including Simple data analysis, visualisation, downloading and subsetting via Nesstar Simple data analysis, visualisation, downloading and subsetting via Nesstar ESDS Qualidata Online – interview transcripts ESDS Qualidata Online – interview transcripts ESDS Government Vital Statistics online ESDS Government Vital Statistics online International macro data via Beyond 20/20 and visualisation interface International macro data via Beyond 20/20 and visualisation interface Census data Census data

12. Access to data End User Licence End User Licence data can only be used by registered users (some exceptions relating to teaching) data can only be used by registered users (some exceptions relating to teaching) to preserve the confidentiality of, and not attempt to identify, individuals, households or organisations in the data to preserve the confidentiality of, and not attempt to identify, individuals, households or organisations in the data to use the correct methods of citation and acknowledgement in publications to use the correct methods of citation and acknowledgement in publications to supply the bibliographic details of any published work based on the data collections to supply the bibliographic details of any published work based on the data collections to offer for deposit any new data collections which have been derived from the materials supplied to offer for deposit any new data collections which have been derived from the materials supplied

13. Government data General Household Survey/Continuous Household Survey (NI) General Household Survey/Continuous Household Survey (NI) Labour Force Survey/NI LFS Labour Force Survey/NI LFS Health Survey for England/Wales/Scotland Health Survey for England/Wales/Scotland Family Expenditure Survey/NI FES Family Expenditure Survey/NI FES British/Scottish Crime Survey British/Scottish Crime Survey Family Resources Survey Family Resources Survey National Food Survey/Expenditure and Food Survey National Food Survey/Expenditure and Food Survey ONS Omnibus Survey ONS Omnibus Survey Survey of English Housing Survey of English Housing British Social Attitudes/Scottish Social Attitudes/Young Peoples Social Attitudes/NI Life & Times British Social Attitudes/Scottish Social Attitudes/Young Peoples Social Attitudes/NI Life & Times National Travel Survey National Travel Survey Time Use Survey Time Use Survey Vital Statistics Vital Statistics for England and Wales

14. Benefits of the large-scale government datasets good quality data good quality data produced by experienced research organisations produced by experienced research organisations UK/GB - usually nationally representative with large samples. Interviewers all over country UK/GB - usually nationally representative with large samples. Interviewers all over country good response rates good response rates well documented, NISRA, ONS, UKDA, QB etc. well documented, NISRA, ONS, UKDA, QB etc. continuous data continuous data e.g. Continuous Household Survey 1983, allows comparison over time e.g. Continuous Household Survey 1983, allows comparison over time data is largely cross-sectional data is largely cross-sectional hierarchical data hierarchical data individual and household intra-household differences intra-household differences household effects on individuals household effects on individuals

15. Nesstar service browse detailed information (metadata) about these data sources, including links to other sources browse detailed information (metadata) about these data sources, including links to other sources do simple data analysis and visualisation on microdata do simple data analysis and visualisation on microdata bookmark analyses bookmark analyses download the appropriate subset of data in one of a number of formats (e.g. SPSS, Excel) download the appropriate subset of data in one of a number of formats (e.g. SPSS, Excel)

19. ESDS Longitudinal Data five main studies, that are primarily UK Research Council: five main studies, that are primarily UK Research Council: British Household Panel Survey (BHPS) British Household Panel Survey (BHPS) British Birth Cohort studies: British Birth Cohort studies: National Child Development Survey (NCDS) National Child Development Survey (NCDS) British Cohort Study 1970 (BCS70) British Cohort Study 1970 (BCS70) Millennium Cohort Study (MCS) Millennium Cohort Study (MCS) English Longitudinal Study of Ageing English Longitudinal Study of Ageing possible forthcoming Medical Research Council population study datasets possible forthcoming Medical Research Council population study datasets – 1946 Birth Cohort – ALSPAC

20. Longitudinal data longitudinal surveys involve repeated surveys of the same individuals at different points in time longitudinal surveys involve repeated surveys of the same individuals at different points in time allow researchers to analyse change at the individual level allow researchers to analyse change at the individual level more complex to analyse more complex to analyse

21. British Household Panel Survey collected and deposited by the ULSC at Essex collected and deposited by the ULSC at Essex follows the members of 5500 households first sampled in 1991 follows the members of 5500 households first sampled in 1991 interviews conducted annually interviews conducted annually become a major resource for understanding the dynamics of British households become a major resource for understanding the dynamics of British households coverage includes: coverage includes: income, labour market behaviour, social and political values, health, education, housing and household organisation income, labour market behaviour, social and political values, health, education, housing and household organisation recently large new samples were introduced in Scotland, Wales and Northern Ireland recently large new samples were introduced in Scotland, Wales and Northern Ireland

22. British Birth Cohort Studies impact of childhood conditions on later life and understanding children and families in the UK impact of childhood conditions on later life and understanding children and families in the UK national Child Development Study follows a cohort born in a single week in 1958 - data collected at birth & ages 7, 11, 16, 23, 33, 42 national Child Development Study follows a cohort born in a single week in 1958 - data collected at birth & ages 7, 11, 16, 23, 33, 42 1970 British Cohort Study follows a cohort born in a single week in 1970 - data collected around birth & ages 5, 10, 16, 26 and most recently at age 30 1970 British Cohort Study follows a cohort born in a single week in 1970 - data collected around birth & ages 5, 10, 16, 26 and most recently at age 30 Millennium Cohort Study focuses on children born in 2000/ 2001 - first sweep at 9 months, second sweep at 3 years Millennium Cohort Study focuses on children born in 2000/ 2001 - first sweep at 9 months, second sweep at 3 years wide range of social, economic, health, medical and psychological issues wide range of social, economic, health, medical and psychological issues

23. ESDS International data portfolio regularly updated macro-economic time series datasets from selected major international statistical databanks that collectively chart over 50 years of global economic, industrial and political change: regularly updated macro-economic time series datasets from selected major international statistical databanks that collectively chart over 50 years of global economic, industrial and political change: the International Monetary Fund the International Monetary Fund the OECD the OECD the United Nations the United Nations the World Bank the World Bank Eurostat Eurostat the International Labour Organisation the International Labour Organisation the UK Office for National Statistics the UK Office for National Statistics access for UK HE/FE only access for UK HE/FE only access to micro data surveys access to micro data surveys Eurobarometers Eurobarometers International Social Survey Programme International Social Survey Programme other social data via other national data archives other social data via other national data archives

24. International data themes economic performance and development economic performance and development trade, industry and markets trade, industry and markets employment employment demography, migration and health demography, migration and health governance governance human development human development social expenditure social expenditure education education science and technology science and technology land use and the environment land use and the environment

25. ESDS Qualidata access and support for a range of qualitative datasets, hosted by the UK Data Archive access and support for a range of qualitative datasets, hosted by the UK Data Archive data from National Research Council (ESRC) individual and programme research grant awards data from National Research Council (ESRC) individual and programme research grant awards data from classic social science studies data from classic social science studies other funders/sources other funders/sources focus on DIGITAL Collections, but also facilitate paper-based archiving focus on DIGITAL Collections, but also facilitate paper-based archiving

26. Types of qualitative data diverse data types: in-depth interviews ; semi-structured interviews; focus groups; oral histories; mixed methods data; open-ended survey questions; case notes/records of meetings; diaries/ research diaries diverse data types: in-depth interviews ; semi-structured interviews; focus groups; oral histories; mixed methods data; open-ended survey questions; case notes/records of meetings; diaries/ research diaries multimedia: audio, video, photos and text (most common is interview transcriptions) multimedia: audio, video, photos and text (most common is interview transcriptions) formats: digital, paper, analogue audio-visual formats: digital, paper, analogue audio-visual data structures - differ across different document types data structures - differ across different document types

27. Classic study datasets Peter Townsend – Poverty, old age and Katherine Buildings Peter Townsend – Poverty, old age and Katherine Buildings Paul Thompson – oral history and Edwardians Paul Thompson – oral history and Edwardians Ray Pahl –Hertfordshire Villages studies Ray Pahl –Hertfordshire Villages studies National Social Policy and Social Change Archive National Social Policy and Social Change Archive

28. Online access to data

29. Other data Many other sources across disciplines Many other sources across disciplines Can do catalogue search or browse by subject Can do catalogue search or browse by subject Major studies page Major studies page New releases New releases

30. Catalogue record metadata

31. Study documentation

32. Help for users Help desk and dedicated data pages Help desk and dedicated data pages JISCmail list JISCmail list regularly updated web-based FAQs regularly updated web-based FAQs programme of training courses and publicity events programme of training courses and publicity events news bulletins and articles news bulletins and articles resources (links to other sites) resources (links to other sites) teaching datasets and/or exemplars teaching datasets and/or exemplars enhanced documentation e.g. enhanced documentation e.g. dataset and software guides dataset and software guides statistical guides (SPSS, Stata, weighting) statistical guides (SPSS, Stata, weighting) variables consistent over time on specific surveys (ESDS Government) variables consistent over time on specific surveys (ESDS Government) thematic guides thematic guides

33. The Census Registration Service the Census Registration Service provides one-stop registration and support for access to: the Census Registration Service provides one-stop registration and support for access to: Census Dissemination Unit from MIMAS – aggregate tables/Casweb Census Dissemination Unit from MIMAS – aggregate tables/Casweb Census Geography Data Unit (UKBORDERS) from EDINA – boundaries data Census Geography Data Unit (UKBORDERS) from EDINA – boundaries data Census Interaction Data Service (Universities of Leeds and St Andrews) - flow data Census Interaction Data Service (Universities of Leeds and St Andrews) - flow data Samples of Anonymised Records from CCSR – micro data Samples of Anonymised Records from CCSR – micro data CHCC - Historical Census Collection from AHDS History CHCC - Historical Census Collection from AHDS History

34. AHDS History AHDS History is one of the five Subject Centres of the Arts and Humanities Data Service. ahds.ac.uk AHDS History is one of the five Subject Centres of the Arts and Humanities Data Service. ahds.ac.uk AHDS History collects, preserves, and promotes the use of digital resources, which result from or support historical research, learning and teaching AHDS History collects, preserves, and promotes the use of digital resources, which result from or support historical research, learning and teaching

35. The AHDS History collection particularly strong in 19th and 20th century economic and social history particularly strong in 19th and 20th century economic and social history census data (1881 100% sample; 1851 2% sample; lots of local census returns) census data (1881 100% sample; 1851 2% sample; lots of local census returns) Great Britain Historical Database online Great Britain Historical Database online taxation materials taxation materials large-scale datasets of Welsh and Irish historical statistics large-scale datasets of Welsh and Irish historical statistics electoral data (poll books for local areas) electoral data (poll books for local areas) criminal court records (e.g. a collection of datasets on violent crimes 1600-1900) criminal court records (e.g. a collection of datasets on violent crimes 1600-1900) agricultural statistics (prices, output) agricultural statistics (prices, output) surveys of Scottish witchcraft surveys of Scottish witchcraft state finance data state finance data economic indicators/industrial production data economic indicators/industrial production data

36. Secondary analysis potential descriptive materialdescriptive material comparative research, restudy or follow-up studycomparative research, restudy or follow-up study re-analysis/secondary analysisre-analysis/secondary analysis research design and methodological advancementresearch design and methodological advancement replication of published statisticsreplication of published statistics teaching and learningteaching and learning

37. Using ESDS data for teaching tutor registers with ESDS and downloads data tutor registers with ESDS and downloads data registers a usage (for micro data) registers a usage (for micro data) tutor makes data available to students (if online only via secure network) tutor makes data available to students (if online only via secure network) tutor asks students to register or sign access agreement for teaching (not always necessary, but to be encouraged tutor asks students to register or sign access agreement for teaching (not always necessary, but to be encouraged

38. Data repurposed for teaching and training teaching datasets (available for download and via Nesstar) based on ELSA, NCDS, BHPS, HSE, BCS, LFS, GHS) teaching datasets (available for download and via Nesstar) based on ELSA, NCDS, BHPS, HSE, BCS, LFS, GHS) X4L Survey Data in Teaching X4L Survey Data in Teaching CHCC Learning and Teaching Materials CHCC Learning and Teaching Materials Practical Exemplars on the Analysis of Surveys Practical Exemplars on the Analysis of Surveys Understanding Social Statistics Understanding Social Statistics Teaching Resources and Materials for Social Scientists (TRAMSS) Teaching Resources and Materials for Social Scientists (TRAMSS)

39. Questions?

40. Repurposing data – issues to consider UKDA is guardian for the data/does not own the data UKDA is guardian for the data/does not own the data data can usually only be accessed by registered users data can usually only be accessed by registered users most data can be incorporated into teaching packages/courses but access restricted to the relevant students and not made publicly available e.g. on the web (some exceptions) most data can be incorporated into teaching packages/courses but access restricted to the relevant students and not made publicly available e.g. on the web (some exceptions) any derived data must be offered for deposit with UKDA any derived data must be offered for deposit with UKDA permission of data owners – where data are repurposed and to be made publicly available, permission of data owners must be sought permission of data owners – where data are repurposed and to be made publicly available, permission of data owners must be sought copyright e.g. LFS teaching dataset Crown copyright held jointly with the ESDS copyright e.g. LFS teaching dataset Crown copyright held jointly with the ESDS IPR of authors of repurposed material IPR of authors of repurposed material information on how to cite and acknowledge data information on how to cite and acknowledge data

41. Contactwww.esds.ac.ukhelp@esds.ac.uk ESDS FAQ: www.esds.ac.uk/about/faq.asp