1. Using PIAAC Data to Help Unemployed Adults Transition into Postsecondary/Workforce National College Transition Network November 14, 2017
Emily Pawlowski
Keren Hendel
American Institutes for Research
Copyright © 2017 American Institutes for Research. All rights reserved.

2. Outline Skills to Pay the Bills What is PIAAC?
Demographics of the Unemployed
Education and Unemployment
Implications for Policy and Practice
Resources

4. What Is PIAAC?
PIAAC is an international large-scale assessment first administered in in 24 countries including the U.S. (results released in 2013)
Organized by the Organization for Economic Cooperation & Development (OECD)
It assessed 16 - to 65-year-olds, non-institutionalized, residing in each country, irrespective of nationality, citizenship, or language status
Administered to a sample of about 5,000 U.S. adults on either laptop computer or paper-and-pencil
In , the U.S.:
Supplemented its original sample with 3,660 more adults with a focus on:
Unemployed (age 16-65), Young adults (age 16-34), Older adults (age 66-74)
Collected representative prison sample of 1,300 inmates,18-74, in state, federal and private prisons

5. Participating Countries
2012
2015
Australia
Austria
Belgium
Canada
Cyprus
Czech Republic
Denmark
Estonia
Finland
France
Germany
Ireland
Italy
Japan
Korea, Rep of
Netherlands
Norway
Poland
Slovak Republic
Spain
Sweden
United Kingdom
United States
Chile
Greece
Indonesia
Israel
Lithuania
New Zealand
Singapore
Slovenia
Turkey

6. PIAAC administers a background questionnaire and a direct assessment of literacy, numeracy and problem solving in technology-rich environments
Background questionnaire
Demographic
Education
Employment
Direct assessment of key information-processing skills

7. What Does PIAAC Measure?
PIAAC’s cognitive assessment measures:
Literacy
Numeracy
Problem solving in technology-rich environments (“digital problem solving”)
Literacy and numeracy are given either on a computer or in paper-and-pencil format

8. Definitions of PIAAC direct assessment subjects:
Literacy is understanding, evaluating, using and engaging with written texts
to participate in society,
to achieve one’s goals, and
to develop one’s knowledge and potential.
Numeracy is the ability to access, use, interpret, and communicate mathematical information and ideas, in order to engage in and manage the mathematical demands of a range of situations in adult life.
Problem solving in technology-rich environments involves using digital technology, communication tools and networks to acquire and evaluate information, communicate with others and perform practical tasks.

9. The PIAAC Background Questionnaire collects data on use of the following skills:
Interaction and social skills
collaboration and co-operation, work and time planning, communication and negotiation, and customer contact
Cognitive skills
reading, writing, mathematics, and use of information and communication technologies at work and at home
Education, Training, Work
Past and present education and work experience
Learning skills
coaching, formal/informal learning and updating professional skills
U.S. Additions
Basic skills training, political efficacy, health, race, language, adaptations to formal education, informal training, occupation, and earnings
Physical skills
use of gross and
fine motor skills

10. US National BQ Items: additions and adaptations
Education (current/past/required)
Country of origin/ Language/ethnicity
Training courses
Occupation
Economic Sector
Earnings
Health
Health:
Health status
Health insurance coverage
Sources of information about health issues
Preventive health practices
US National BQ Items: additions and adaptations included questions related to
Education (current/past/required)
Country of origin/ Language/ethnicity
Training courses
Occupation
Economic Sector
Earnings
Health
US Health questions were more extensive and included several questions on each category related to
Health status
Health insurance coverage
Sources of information about health issues
Preventive health practices
In general, would you say your health is excellent, very good, good, fair, or poor?
Do you have any difficulty seeing the words and letters in ordinary newspaper print even hen wearing glasses or contact lenses if you usually wear them?
Do you have any difficulty hearing what is said in a normal conversation with another person even when using a hearing aid if you usually wear one?
Have you ever been diagnosed or identified as having a learning disability?

11. PIAAC results are reported in two ways:
As Average scores on a scale
As Proficiency levels anchored by a specific set of concrete skills:
6 levels for Literacy and Numeracy
4 levels for Digital Problem Solving

12. Descriptions of the PIAAC proficiency levels for literacy defining what adults can do at each level
( )
Level 4
( )
Level 3 ( )
Locate single piece of information in familiar texts.
Read relatively short digital, print or mixed texts to locate single text.
Make matches between text and information that may require low level para-phrasing and drawing low-level inferences.
Identify, interpret, or evaluate one or more pieces of information and often require varying levels of inference.
Perform multiple-step operations to integrate, interpret, or synthesize information from complex texts, and may require complex inferences.
Integrate information across multiple, dense texts; construct syntheses, ideas or points of view; or evaluate evidence based arguments.
Level 2
( )
Level 1
( )
Below Level 1
(0-175)

13. Descriptions of the PIAAC proficiency levels for numeracy defining what adults can do at each level
( )
Level 4
( )
Perform basic tasks: counting, arithmetic operations with whole numbers.
Perform one-step tasks: count; sort; arithmetic operations; understanding simple percent (ex. 50%).
Perform 2 or more calculations, simple measurement; spatial representation; estimation; and interpret simple tables, graphs.
Understand & work with mathematical patterns, proportions, basic statistics expressed in verbal or numerical form.
Perform analysis, complex reasoning, statistics and chance; spatial relationships; and communicating well-reasoned explanations for answers.
Understand complex abstract mathematical and statistical ideas, embedded in complex texts, draw inferences; arguments or models; justify, reflect on solutions or choices.
Level 3
( )
Level 2
( )
Level 1
( )
Below Level 1 (0-175)
Mention that there are the same type of proficiency level definitions for PSTRE

14. Literacy example item: Below level 1
The test-taker is presented with a report on the results of a union election.
Results of Union Election
The test-taker is asked to compare the number of votes.
Candidate
Votes
Candidate A
4,857
Candidate B
5,423
Candidate C
3,563
So which candidate received the fewest votes?
90% of U.S. population are
likely to get this answer correct.
The test-taker needs to identify 3,563 as the lowest number of votes.
Note: Not an actual replica of test item

15. Numeracy example item: Below level 1
The test-taker is presented with four labels from common grocery items.
PEANUT BUTTER
$3.36
Weight: 28 oz
Packaged on:
3/24/12
TOMATO SOUP
$1.15
Weight: 3.5 oz
Packaged on:
5/17/12
Which product was packed first?
The test-taker is asked to determine which grocery item was packed first.
ORANGE JUICE
$2.99
Weight: 64 oz
Packaged on:
11/8/12
BEEF JERKY
$6.29
Weight: 16 oz
Packaged on:
2/14/12
86% of U.S. test-takers got this item correct
The test-taker needs to compare the date on each label to determine the earliest date.
Note: Not an actual replica of test item

16. Numeracy example item: Level 2
71% of household population are likely to get this answer correct.
The test-taker is presented with factory data for two years and a bar graph.
Month
Year 1
Year 2
Jan 20 17
Feb 21 22
Mar 34 31
Apr 30 36
May 35 33
Jun 28 23
Jul 24
Aug 25 19
Sep 14
Oct 18
Nov
Dec
Total
300
273
Number of workers absent In Year 2
The test-taker is asked to find two mistakes in the bar graph.
Which two bars are incorrect?
The test-taker needs to compare the table to the bars in the chart and find two bars that do not match the data table.
Note: Not an actual replica of test item

17. Numeracy example item: Level 3
The test-taker is presented with information about a shoe sale.
The test-taker is asked to calculate total amount one would need to pay for both shoes during the sale.
Running Shoes
SALE! Buy one pair – get the second
(of equal or lesser value) for half price!
38% of household population are likely to get this answer correct.
How much would you pay if you bought both pairs on sale? $
48.95
The test-taker needs to calculate and enter the correct dollar amount for the two pairs of shoes on sale.
Note: Not an actual replica of test item

18. International Comparison
Overall Findings
International Comparison

19. Literacy: U.S. average score was not measurably different than the international average.
Level 3
Level 2
Level 1 7

20. Numeracy: U.S. average score was lower than the international average.
Level 3
Level 2
Level 1

21. Problem Solving in Technology-Rich Environments(PS-TRE): U. S
Problem Solving in Technology-Rich Environments(PS-TRE): U.S. average score was below the international average.
Level 2
Level 1
Below Level 1

22. The U.S. has a higher percentage of unemployed adults than the international average.
Emphasize the 2014 release/2013 data collection because US unemployment looks different now

23. Average literacy scores for U. S
Average literacy scores for U.S. adults who were unemployed or out of the labor force were lower than the international average, while employed U.S. adults did not perform differently than their international peers

24. Average numeracy scores for U. S
Average numeracy scores for U.S. adults were lower than their international peers across all employment status groups

25. U.S. Unemployed Findings
Focus on Numeracy Skills
Keren starts here

26. Profile of the Unemployed U.S. Adults (age 16-65)
Characteristic
Percentage
Age
16-24 33
25-34 18
35-44 17
45-54
55-65 14
Gender
Male 47
Female 53
Race
White 52
Black 21
Hispanic 19
Other 8
Characteristic
Percentage
Born in the US
Yes 88 No 12
Education
Below high school 23
High school credential 54
Associate’s degree 8
Bachelor’s degree 10
Graduate or professional degree 5
Parental Education
Neither parent attained high school degree 16
At least one parent attained high school degree 49
At least one parent attained college degree 35

27. Numeracy Skills of the Unemployed: Employed adults scored higher in numeracy than unemployed adults.
Level 3
Level 2
Level 1
These numbers show the percentage distribution of the characteristics within the Household populations. In this case, 74% of the adult population is employed.
Overall performance—unemployed are doing worse
(74)
(7)
(19)
* Significantly different from “Employed”

28. A larger percentage of U. S
A larger percentage of U.S. adults who were unemployed or out of the labor force performed at the bottom levels in numeracy than those who were employed
42% of unemployed are in the lowest levels

29. Age of the Unemployed: There is no difference in average scores for unemployed adults, but employed adults ages scored higher than other adults. L3 L2 L1
As age increases, skills lower. Unemployed don’t get the same bump though at younger ages
Students are included if they are searching for work
(33)
(18)
(17)
(17)
(14)
(15)
(23)
(21)
(23)
(18)
* Significantly different from “16-24”

30. Gender of the Unemployed: Males scored higher than females for both unemployed and employed adults.
Mention that there is no difference between men and women in literacy. Important policy implication for women
(47)
(53)
(52)
(48)
* Significantly different from “Male”

31. Educational Attainment of the Unemployed: Average numeracy scores increase as educational attainment increases for both unemployed and employed, although scores of unemployed adults are lower than those of employed adults.
(23)
(54)
(8)
(10)
(5)
(10)
(49)
(10)
(19)
(12)
‡ Reporting standards not met.
* Significantly different from “Below high school”

32. Educational Attainment of the Unemployed: Over 75 percent of U. S
Educational Attainment of the Unemployed: Over 75 percent of U.S. unemployed adults have a high school diploma or less education. About half of these adults performed at the bottom proficiency levels in numeracy
Explain that higher ed corresponds with higher numeracy for both unemployed and employed

33. Race/Ethnicity of the Unemployed: White adults scored higher than adults of all other races across both employment status categories. L3 L2 L1
(52)
(21)
(19)
(8)
(67)
(14)
(11)
(7)
* Significantly different from “White”

34. Nativity Status of the Unemployed: Native-born adults scored higher than non-native-born adults in both the unemployed and employed categories. L3 L2 L1
(88)
(12)
(85)
(15)
* Significantly different from “Born in U.S.”

35. Language Status of the Unemployed: Native English speakers scored higher in numeracy than those who learned English as a second language among both unemployed and employed adults. L3 L2 L1
(84)
(14)
* Significantly different from “English was first language”

36. Parental Education Levels of the Unemployed: For both unemployed and employed, adults with lower levels of parental education scored lower in numeracy than those with higher levels of education. L3 L2 L1
(16)
(49)
(35)
(16)
(44)
(41)
* Significantly different from “Neither parent has attained high school degree”

37. Health Status of the Unemployed: Numeracy scores increase with health status for both unemployed and employed adults, but employed adults perform better than unemployed at each level. ‡ ‡
‡ Reporting standards not met.
* Significantly different from “Fair” health status.

38. Demographics of the Unemployed
About half of unemployed adults were ages years old, and about half were white.
About 75% of unemployed adults have a high school education or less.
About two thirds have parents with less than a high school credential or less.
U.S. unemployed adults, on average, have lower numeracy scores than employed adults across most race/ethnicity, age, gender, education level, race/ethnicity, nativity, language, parental education and health status.
Among the unemployed, in numeracy:
There is no difference in average scores between age groups.
Females average numeracy scores were significantly lower than males.
Those born in the U.S. scored significantly higher than those born outside the country.
Last slide

39. Time out of work: Unemployed U. S
Time out of work: Unemployed U.S. adults who have been out of work for less than one year have higher numeracy scores than those unemployed adults who have been out of work for longer periods of time
Level 2
Level 1
Emily starts here
Percentages are the percent of unemployed adults 16-65
Use it or lose it
(55)
(30)
(9)
(6)
* Significantly different from “<1 year”

40. Since 1970, there has been a shift in the U. S
Since 1970, there has been a shift in the U.S. economy away from routine and manual tasks and toward more analytic and interpersonal tasks in occupations.
Mean task input in percentiles of 1960 distribution
Source: Autor, D. H. and B.M. Price (2013), "The Changing Task Composition of the US Labor Market: An Update of Autor, Levy, and Murnane (2003)", MIT Mimeograph, June.

41. Previous Occupations of the Unemployed:
More adults with recent employment in semi-skilled and elementary occupations are unemployed compared to skilled occupations.
Percentage of adults age 16 to 65 who are unemployed, by previous occupation type
Previous Occupation Type
Percentage Unemployed in Each Occupation Type
Skilled 4
Semi-skilled white-collar 8*
Semi-skilled blue-collar
Elementary
12*
Adults in elementary occupations have lower average numeracy scores than adults in other occupations for both employment status groups
NOTE: Totals do not sum to 100 because “Percentage Employed in Each Occupation Type” not shown.
* Significantly different from “Skilled”

42. Previous Occupations of the Unemployed: Adults in elementary occupations have lower average numeracy scores than adults in other occupations for both employment status groups. L3 L2 L1
(17)
(19)
(37)
(27)
(8)
(16)
(28)
(48)
* Significantly different from “Elementary”

43. Participation in Formal Education: Among unemployed U. S
Participation in Formal Education: Among unemployed U.S. adults, those who participated in formal education in the past year had higher numeracy scores. However, among employed adults, there was no difference in numeracy scores between those who did and did not participate. L3 L2 L1
Use it or lose it
* Significantly different from “Participated in formal education”

44. Reason for Participating in Formal Education: A similar percentage of employed and unemployed adults participated in formal education for a job-related reason (72%).
We should explain this one…and
* Significantly different from “Unemployed”

45. Participation in Non-formal Education: Those who participated in non-formal education in the past year scored higher in numeracy than those who did not in both the unemployed and employed categories. L3 L2 L1
Talk about the percentages
(42)
* Significantly different from “Participated in non-formal education”

46. Reason for Participating in Non-formal Education: Looking at more specific reasons, the most common reason for participating in non-formal education for both unemployed and employed adults is improving job performance or career prospects.
Unemployed
Employed
# Rounds to zero
* Significantly different from “Unemployed”

47. Interest in learning opportunities: A higher percentage of unemployed adults expressed that there were learning activities they would have liked to participate in, but did not in comparison to employed adults.
* Significantly different from “Unemployed”

48. Reasons for not participating in learning activities: The most common reason for not participating in learning activities unemployed adults reported was that it was too expensive, while the most common reason for employed adults is that they were too busy at work.
Unemployed
Employed
* Significantly different from “Unemployed”

49. Highlighted Findings
More adults with recent employment in semi-skilled and elementary occupations are unemployed compared to skilled occupations.
Unemployed adults participated more often in formal education than employed adults, while employed adults were more likely to participate in non-formal education.
Unemployed adults who participated in formal or non-formal education had significantly higher numeracy scores than those who did not.
The most common reason for participating in learning activities for both groups was to improve job performance/career prospects.
The most common reason for not participating in learning activities for the employed was being too busy at work, while for the unemployed it was the financial burden.

50. Discussion What finding stood out the most to you? Why?
How should this information impact policy and practice in your field?
Do the findings about unemployed adults and education status match your experiences and observations?

51. Has PIAAC Data Made a Difference?
Nov 2013, Release of NCES’ First Look and OECD’s Time for the U.S. to Reskill?, a deep dive into the low-skilled population Dec 2013, ED/OCTAE launches Toolkit and national engagement effort around skills in communities, involves federal partners at all appearances Jan 2014, Vice President is tasked in State of the Union to take a close look at federal workforce development programs and efficacy; launches inter-agency skills working group which culminated in an April agency coordinated definition and statement on Career Pathways July 2014, VP’s report, Ready to Work released at WIOA signing July 2014, bipartisan-bicameral passage of the Workforce Innovation and Opportunity Act (WIOA); first reauthorization in a decade; mentions the 36M low-skilled PIAAC number

52. Has PIAAC Data Made a Difference? (cont.)
Feb 2015, Making Skills Everyone’s Business, released by ED/OCTAE as a call to action Feb 2015, America’s Skills Challenge: Millenials and the Future report released with great public and media uptake April 2015, The White House National Economic Council convenes Upskill America Summit, focused on the 24M frontline workers with low skills; holds anniversary summit with employers in April 2016 [Sept 2015], National Science Foundation holds national strategy meeting re STEM education that kicks off with a PIAAC presentation by ETS Jan 2017, ED/OET releases the National Education Technology Plan – Higher Education Supplement, which includes data from PIAAC Feb 2017, National Skills Coalition’s report, Foundational Skills in the Service Sector, is released with analysis of PIAAC data

53. Education and Skills Online (ESO)
An OECD Study

54. Education & Skills Online (ESO) @ http://www. oecd
An assessment tool designed to provide individual level results linked to PIAAC
Measures both cognitive and non-cognitive skills
Available in both English and Spanish in the U.S.
Can be downloaded 24/7 on demand over the internet

55. Education & Skills Online Components
Background Questionnaire
Age
Gender
Highest level of education completed
Born in U.S. or First arrived in U.S.
First language
Employment status
Core Cognitive Assessments
Literacy
Numeracy
Problem Solving in Technology-Rich Environments (optional)
Reading Components (optional)
Non-Cognitive Module (optional)
Behavioral performance competencies
Subjective well-being & health
Career interest and intentionality
Skills use & transfer

56. Education & Skills Online: Uses and Benefits
Appropriate for use in both educational and workplace settings.
Can be used for placement for individuals at any point along a career pathway.
Can be used to evaluate classes, programs or any group of respondents.
Organizations have flexibility to determine which skills to assess.
Database is owned by purchaser (individual/organization/state) and is available for further analyses/research purposes.

57. PIAAC Reports, Releases, and Other Resources

58. Reports and Data Collections
Past
2012 & 2014 U.S. Data Collection
Spring 2016 – U.S. 2012/2014 First Look Report
Summer 2016 – OECD Report with Round 2 Countries
Fall 2016 – Highlights Prison Report 2014
2017 U.S. Data Collection – Small area estimates (estimates for U.S. states and counties based on all three rounds of data)
2017 International Round 3 Data Collection
Future
Winter 2017 – U.S. Young Adults Report
Spring/Summer 2020 – Release of PIAAC Small Area Estimate Study
Cycle Data Collection in U.S. and more than 35 other countries

59. NCES PIAAC Website

60. PIAAC Gateway Website

61. PIAAC Resources Researchers’ Corner: Research Paper Database
Announcements, FAQ
Data Files, Data Analysis Tools, Data Training Tools
Researcher listserv (funding opportunities)
Research Paper Database
Education and Skills Online – an online version of PIAAC
Outreach Toolkits
Research-to-Practice Guides
Event Calendar
Regular updates through the PIAAC Buzz

62. PIAAC Results Portal

63. PIAAC International Data Explorer (IDE)

64. Infographics

67. For more information on PIAAC or suggestions for future PIAAC data collections:
Contact NCES PIAAC Staff
Stephen Provasnik
Holly Xie

68. Thank you! Emily Pawlowski epawlowski@air.org Keren Hendel
1000 Thomas Jefferson Street NW
Washington, DC
General Information:
Thank you!