Using Innovative Assessment Models to Improve Reemployment Outcomes Moderator: Pam Frugoli O*NET/Competency Assessment Team Lead US Department of Labor, ETA 2009

Goals Assist you in making better assessment decisions for your customers – Provide basic assessment information – Discuss two scenarios – Interact with panel of state officials responsible for the use of assessment Provide resources that could assist you as you work with customers Share what you are doing with your customers Discussion

Agenda Introduction (Pam Frugoli) Selecting the Right Test (Janet Wall, Sage Solutions) Scenarios for Discussion (Lauren Fairley-Wright, Workforce Analyst, ETA) Panel Discussion and Audience Interaction – Douglas Reamer, Director, Workforce Innovation, New York State Department of Labor – Glenda Owen, Chief Workforce Service Division, Oklahoma Employment Security Commission 3

Selecting the Right Test Presenter: Dr. Janet Wall President and Founder Sage Solutions 2009

Topics and Organization focus on customer measurement need type and quality of assessment selection factors administration considerations interpretation and use

Know the Customer What might you already know? – Intake Interview: – Intake Interview: background, work experience, communication skills, education level, computer capability, interests, recent training/courses What are their needs? – Stage: – Stage: where are they in the process? career exploration, career transition, want job now

Information Needs What do you want/needs to know about the customer? – Interests, skills, tool/technology capability, math ability, language skills, writing, work readiness, teamwork Is any of this already available, credible, current? – From education institutions (educational tests, grades, employer evaluations, training records)

Decision Considerations What are the stakes? (low or high) Low stakes – decisions not “life changing” – Inspires communication and self knowledge – Can use informal assessment (free or low cost) High stakes – critical decisions – Selection into training and educational programs – Employment selection – Pertinent for dislocated workers

General Considerations Reading level Primary language capability Mode of administration – Paper/pencil, computer Administrator qualifications

Technical Considerations For high stakes decisions – Reliability – Validity (depends on use) – Interpretation – Bias and adverse impact Sources of information – Technical reports – Journal articles – Other users – Your own research and records

Resource Considerations Administration Time Cost – Consider partnerships Administration requirements – Standardization – Equipment needs – Room condition Scoring – Self, staff, automated, in or out house

Interpretation Considerations Self or staff interpretation? – Staff qualifications Clarity of score report Research support of interpretation Danger of over interpretation Measurement error

Other Considerations Respect of copyright Test security Privacy of results Storage Space – Paper files, electronic

General Process Customer Needs Options General and Technical Information Interpretation and Use

Useful Resources Counselors Guide to Career Development Instruments (NCDA) Measuring Up: Assessment Resources for Teachers, Counselors, and Administrators (ProEd, Inc) Assessment Resources:

O*NET Guides O*NET Testing and Assessment Guides: – Testing and Assessment: A Guide to Good Practices for Workforce Investment Professionals ( 65) 65 – Tests and Other Assessments: Helping You Make Better Career Decisions – Testing and Assessment: An Employer’s Guide to Good Practices

Scenarios For Discussion Presenter: Lauren Fairley-Wright Manpower Analyst US Department of Labor, ETA 2009

The Case of Ynez Legal immigrant from Guatemala (8 yrs in the U.S) Occupation = home health aide Education – HS graduate Recent work experience – 4 yrs part-time with most recent employer; no prior work history available Husband recently dislocated; Ynez needs full-time work with benefits Ynez visits the local one-stop for assistance

Challenges to Consider Language limitations – Ynez’s primary language is Spanish Limited to exposure/knowledge of career options No formal training after High School No records available for review

Questions to Consider What additional information do you need to know about this customer in order to assist her? How would you obtain this information – what tools would you use – what are your options considering the challenges that exist? Are formal assessments necessary? Why or why not? Would you recommend self-administered assessment tools? How do you determine this? In selecting the appropriate assessment tools for Ynez, what are some of the “considerations” that are important?

Toolbox/Product Box What’s in your toolbox or product box of assessments? – Examples: Intake Interview Template/Form, Virtual One Stop Assessment Tools, Records Review Template, Collaboration with Community College for Specialized Assessments

The Case of Eugene Civilian recently BRACed from a local military installation Occupation = Budget Analyst Education – Bachelor’s Degree Recent work experience – 4 yrs tenure at most recent employer; 7 yrs at prior employer Eugene visits a local one-stop for assistance Eugene is considering new occupations in a new industry – but is willing to consider remaining in the same occupation depending on opportunities

Challenges to Consider Large number of customers needing assistance – major regional layoffs in financial, retail and hospitality industries. Dwindling one stop center resources (staff, time, financial) State/local government furloughs and hiring freezes Limited federal government opportunities in the region. Eugene does not wish to relocate.

Questions to Consider What additional information do you need to know about this customer in order to assist him? How would you obtain this information – what tools would you use, what records would you review – why? Considering Eugene’s background - are formal assessments necessary? Why or why not? Would you recommend self-administered assessment tools? How do you determine this?

Toolbox/Product Box What’s in your toolbox or product box of assessments? – Examples: Intake Interview Template/Form, Virtual One Stop Assessment Tools, Records Review Template, Collaboration with Community College for Specialized Assessments

New York’s Assessment Strategy Presenter: Douglas Reamer Director of Workforce Innovation NY State Department of Labor 2009

Assessment 27 Profile 1-30Profile 31-69Profile >69 IA Based on Profile Score IA Based on Staff Assessment IA Based on Profile Score Job Search Ready Resume Prep Interview Skills Job Matching Career Development Services Comprehensive Assessment Training

Assessment

Assessment 29 Staff Assessment Criteria – Barriers to Employment and/or Employment Retention – Employment Goal Relevant to the local labor market outlook – Occupational Knowledge, Skills and Abilities Appropriate for Employment Goal

Assessment Comprehensive Assessment – Identify “Barriers to Employment” (beyond skills development) which impede the customer’s ability to search for, accept, and/or retain employment. – Identify an “Employment / Occupation Goal” that is available in the local labor market. – Identify “Proficiency in the Occupational Knowledge, Skills and Abilities” associated with the Employment Goal.

Self Serve Interest Profiler Work Importance Profiler Initial Assessment Job Search Ready ServicesCareer Development Services Job Match Services Ability Profiler JZ Metrix Learning Skills Dev. & Training ETP/TBL (service) Skills Profiler JZ Career Development Services Comprehensive Assessment 1.Identify “Barriers to Employment” (beyond skills development) which impede the customer’s ability to search for, accept, and/or retain employment. 2.Identify an “Employment / Occupation Goal” that is available in the local labor market. 3.Identify “Proficiency in the Occupational Knowledge, Skills and Abilities” associated with the Employment Goal. 1. Address Barriers to Employment 2. Establish Employment Goal 3. Objective Skills Assessment Resume Preparation Interviewing Skills Career Exploration Skills Assessment Technology Based Learning Job Search Ready Services Assessment and Service Requirements Self Services JZ Key Service Categories Services Assessments Assessment Tools JobZone

Oklahoma’s Assessment Strategy Presenter: Glenda Owen Chief, Workforce Service Division OK Employment Security Commission 2009