We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byKorbin Lambeth
Modified over 2 years ago
© 2010 by ACT, Inc. All rights reserved. 1 Creating an Aligned Assessment System to Promote Work Training Readiness and Career Success Steve Robbins Vice President, Research ACT, Inc. Though designed to meet a wide array of needs, all ACT programs and services have one guiding purpose: Helping people achieve education and workplace success.
© 2010 by ACT, Inc. All rights reserved. 2 The Challenges Create a comprehensive assessment system that predicts success in education and work Tailor assessments to critical transition points and context Understand the complexity and interplay of cognitive and noncognitive factors across time and jobs
© 2010 by ACT, Inc. All rights reserved. 3 Do noncognitive factors matter? What we know from the world of work: –Project A from the military –Training and work outcomes
© 2010 by ACT, Inc. All rights reserved. 4 Combining Personality & Cognitive Ability Tests Correlations between general cognitive ability and personality tests and measures of job performance in Project A CogPersBothCriteria.63.26.67Core technical proficiency.65.25.70General Soldiering Proficiency.31.33.44Effort and Leadership.16.32.37Personal Discipline.20.37.42 Physical fitness and military bearing (McHenry, Hough, Toquam, Hanson, & Ashworth, 1990) Creating opportunity for incremental validity especially as criteria vary
© 2010 by ACT, Inc. All rights reserved. 5 Training and Work Outcomes Noncognitive factors are predictive of important job performance criteria (e.g., Rotundo & Sackett, 2002; Casillas et al., 2009) –Task Performance –Organizational Citizenship –Counterproductive Work Behaviors –Safety Noncognitive factors meet a variety of employer needs –Screening –Selection –Training
© 2010 by ACT, Inc. All rights reserved. 6 Validity of Cognitive and Noncognitive Measures for Training and Work Outcomes Training Outcomes Work Performance General Mental Ability.54 a.62 b Math.48 a.52 b Reading.44 a.35 b Conscientiousness.17 c.22 c Emotional Stability.10 c.11 c Notes. a Brown, Le, & Schmidt (2006). b Salgado et al. (2003). c Schmidt et al. (2007) corrected for indirect range restriction.
© 2010 by ACT, Inc. All rights reserved. 7 No Unified Theory of Key Characteristics However, the literature suggests that several factors are important: –Motivation, Social Engagement, and Self-Regulation –Personality characteristics (the Big Five) Role of behavior ratings vs. self-report –Different perspectives –Coaching/training –Monitoring progress
© 2010 by ACT, Inc. All rights reserved. 8 Common and Specific Factors Are the same characteristics (i.e., common factors) associated with educational and work outcomes? Are there unique (i.e., specific) characteristics that contribute to the prediction of educational and work outcomes? In particular, how do age and setting impact the outcomes of interest?
© 2010 by ACT, Inc. All rights reserved. 9 Behavioral Development (Motivation, Social Engagement, & Self-Regulation) Cognitive Development & Acquisition of Foundational Skills (Academic Learning & Achievement) Social Capital School/Work Factors Integrated Pyramid for Success Career Development (Exploration, Crystallization, Choice & Match) Allen & Robbins (2010)
© 2010 by ACT, Inc. All rights reserved. 10 The human side of the pyramid
© 2010 by ACT, Inc. All rights reserved. 11 ACT’s noncognitive programs are part of a comprehensive and integrated set of solutions, which address the three broad areas essential for academic and work success across the lifespan: Academic achievement & foundational skills Behavior (noncognitive) Career planning (noncognitive) Set of Solutions
© 2010 by ACT, Inc. All rights reserved. 12 Achievement & Foundational Skills Grades CollegeWork 6789101112 EXPLORE ® PLAN ® ACT ® COMPASS ® Comprehensive & Integrated Academic & Work Solutions: Grade 6 - Work SRI and BMS SRI Behavior WorkKeys ® Performance, Talent & BMS ACT Interest Inventory Career Planning Fit College Fit
© 2010 by ACT, Inc. All rights reserved. 13 Creating a Comprehensive Set of Behavior (Self & Other Report) Assessments Role of meta-analysis and validity generalization –Robbins et al. (2004). Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin, 130, 261-288. –Robbins et al. (2009). Intervention effects on college performance and retention as mediated by motivational, emotional, and social control factors: Integrated meta- analytic path analyses. Journal of Applied Psychology, 94, 1163-1184.
© 2010 by ACT, Inc. All rights reserved. 14 Behavior: Self-Report DomainMiddle SchoolHigh School & CollegeWork* Motivation (Getting work done) Academic Discipline Commitment to School Optimism Academic Discipline Commitment to College Goal Striving General Determination Study Skills Communication Skills Carefulness Discipline Order Social Engagement (Getting along) Family Attitude toward Education Family Involvement Relationships w/ School Personnel School Safety Climate Social Activity Social Connection Cooperation Goodwill Influence Sociability Striving Self-Regulation (Keeping your cool) Managing Feelings Thinking Before Acting Orderly Conduct Academic Self-Confidence Steadiness Optimism Stability Note. *Creativity and Savvy do not fully map onto these domains.
© 2010 by ACT, Inc. All rights reserved. 15 Behavior: Other Report DomainMiddle School & High SchoolWork Motivation (Getting work done) Initiative Planning & Organizing Sustained Effort Performance Initiative Planning & Organizing Persistence Responsibility Social Engagement (Getting along) Communication Working with Others Communication Working with Others Self-Regulation (Keeping your cool) Managing Feelings Conduct Organizational Citizenship Stress Management Following Rules Adaptability
© 2010 by ACT, Inc. All rights reserved. 16 Career Planning DomainMiddle School & High SchoolWork Interests ACT Interest Inventory DISCOVER Career Information Decision System Fit Inventory Values DISCOVER Career Information Decision System Fit Inventory - Neuman et al. (2009). Job congruence, academic achievement, and earnings - Tracey & Robbins (2006). The interest-major congruence and college success relation: A longitudinal study.
© 2010 by ACT, Inc. All rights reserved. 17 Tailoring Assessments to Critical Transition Points and Context The role of noncognitive measures in promoting work training readiness and career success –Talent supply chain –Meeting various needs
© 2010 by ACT, Inc. All rights reserved. 18 Talent Supply Chain Noncognitive factors are important at all of the key transitions that lead individuals to the world of work. Pre-K Primary (K-8) Secondary (9-12) Post Secondary (2 & 4-yr) Work
© 2010 by ACT, Inc. All rights reserved. 19 Noncognitive Measures Meet Various Workforce Needs PurposeNeedsACT Noncognitive Solutions Pre-Selection-- Screen people in most cost effective way -- Find honest/dependable employees -- Save time in the screening process WorkKeys Performance Recruitment-- Identify people who fit the work environment -- Identify people with skills that match the job WorkKeys Fit Selection-- Select employees with skills that best fit the job -- Save time in selection process -- Select people in most cost-effective way -- Find honest/dependable employees -- Certifying employees WorkKeys Performance WorkKeys Talent WorkKeys Fit
© 2010 by ACT, Inc. All rights reserved. 20 Noncognitive Measures Meet Various Workforce Needs (cont.) PurposeNeedsACT Noncognitive Solutions Coaching/ Development -- Identify other jobs that an employee can fit -- Develop employees for future company needs -- Employees identify areas of improvement WorkKeys Talent WorkKeys Fit Behavioral Monitoring Scales Succession/ Leadership Planning -- Identify candidates for top-level positions -- Develop employees for future needs of company -- Retain top performers WorkKeys Talent WorkKeys Fit Training/ Development -- Identifying work readiness -- Identify basic workplace skill levels -- Educating about career planning -- Job Placement WorkKeys Talent WorkKeys Fit Behavioral Monitoring Scales
© 2010 by ACT, Inc. All rights reserved. 21 Critical Issues Self vs. other report Changing attitudes, behaviors, and skills Complexity of personality & career relationships –Different personality characteristics are important for different jobs (whether across or within jobs) –Moderation effects of personality on ability –Curvilinear effects of personality on performance
© 2010 by ACT, Inc. All rights reserved. 22 Self vs. Other Report Connelly & Ones (2010) conducted meta-analyses of self-other ratings based on 44,000+ individuals across 263 independent samples. –Other ratings are strong predictors of behaviors, particularly for academic achievement and job performance. –In some cases, other ratings yielded predictive validities substantially greater than self-ratings. –When other-ratings are added to self-ratings, results show considerable increases in validity (with gains more pronounced for Conscientiousness, Emotional Stability, and Extraversion).
© 2010 by ACT, Inc. All rights reserved. 23 Changing Attitudes, Behaviors, and Skills Behavior modeling training (BMT), based on Bandura’s social learning theory, has become a widely used psychologically-based approach to training in work environments. A recent meta-analysis by Taylor et al. (2005) summarizing 117 studies found substantial effects of BMT on a variety of training outcomes. Outcome d Attitudes.56 Behaviors.33 Declarative knowledge.69 Procedural knowledge-skills 1.21
© 2010 by ACT, Inc. All rights reserved. 24 “It was about here, wasn’t it, Ed, when you came on board as sales manager?” Harvard Business Review. March 2007. p. 90
© 2010 by ACT, Inc. All rights reserved. 25 Complexity of Personality & Career Different personality characteristics are important for different jobs (whether across or within fields). –Conscientiousness appears to be important for all jobs. –Extraversion is relevant to some jobs (e.g., sales) but not others (e.g., computer programmers).
© 2010 by ACT, Inc. All rights reserved. 26 Complexity of Personality & Career Across Fields Position Computer Programmers O-NET Job Zone Four: Considerable Preparation Needed Education Most occupations in this zone require a four-year bachelor's degree, but some do not. http://online.onetcenter.org/link/summary/15-1021.00 Position Sales Representatives O-NET Job Zone Four: Considerable Preparation Needed Education Most occupations in this zone require a four-year bachelor's degree, but some do not. http://online.onetcenter.org/link/summary/41-4011.00
© 2010 by ACT, Inc. All rights reserved. 27 Complexity of Personality & Career: Comparing Scores Across Fields
© 2010 by ACT, Inc. All rights reserved. 28 Complexity of Personality & Career Moderation Postlethwaite et al. (2009) found that conscientiousness was a stronger predictor of safety behavior for individuals with lower levels of cognitive ability.
© 2010 by ACT, Inc. All rights reserved. 29 Complexity of Personality & Career Curvilinear Relationships Le et al. (in press) found curvilinear relationships between Emotional Stability and job performance (task, OCB, CWB), as well as between Conscientiousness and task performance.
© 2010 by ACT, Inc. All rights reserved. 30 Questions? Correspondence regarding this presentation should be addressed to: Steve Robbins Vice President, Research ACT, Inc. email@example.com
© 2010 by ACT, Inc. All rights reserved. 31 References Allen, J. & Robbins, S. (2010). Effects of interest–major congruence, motivation, and academic performance on timely degree attainment. Journal of Counseling Psychology, 57, 23–35. Brown, K. G., Le, H., & Schmidt, F. L. (2006). Specific aptitude theory revisited: Is there incremental validity for training performance? International Journal of Selection and Assessment, 14, 87-100. Casillas, A., Robbins, S. B., McKinniss, T., Postlethwaite, B., & Oh, I.S. (2009). Using narrow facets of integrity to predict safety: A test validation study. International Journal of Selection and Assessment, 17, 119-125. Connelly, B.S. & Ones, D.S. (2010). An Other Perspective on Personality: Meta-Analytic Integration of Observers’ Accuracy and Predictive Validity. Psychological Bulletin, 136, 1092–1122. Le, H., Oh, I.-S., Robbins, S. B., Ilies, R., Holland, E., & Westrick, P. (in press). Too Much of a Good Thing: Curvilinear Relationships Between Personality Traits and Job Performance. Journal of Applied Psychology. McHenry, J. J., Hough, L. M., Toquam, J. L., Hanson, M. A., & Ashworth, S. (1990). Project A validity results: The relationship between predictor and criterion domains. Personnel Psychology, 43, 335-354. Neumann, G., Olitsky, N., & Robbins, S. (2009). Job congruence, academic achievement, and earnings. Labour Economics, 16, 503-509. Postlethwaite, B., Robbins, S., Rickerson, J., & McKinniss, T. (2009). The moderation of conscientiousness by cognitive ability when predicting workplace safety behavior. Personality and Individual Differences, 47, 711-716.
© 2010 by ACT, Inc. All rights reserved. 32 References (cont.) Robbins, S. B., Lauver, K., Le, H., David, D., Langley, R., & Carlstrom, A. (2004). Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin, 130, 261-288. Robbins, S. B., Oh, I., Le, H., & Button, C. (2009). Intervention effects on college performance and retention as mediated by motivational, emotional, and social control factors: Integrated meta-analytic path analyses. Journal of Applied Psychology, 94, 1163-1184. Rotundo, M. & Sackett, P. R. (2002). The relative importance of task, citizenship, and counterproductive performance to global ratings of job performance: A policy- capturing approach. Journal of Applied Psychology, 87, 66–80. Salgado, J. F., Anderson, N., Moscoso, S., Bertua, C., & de Fruyt, F. (2003). International validity generalization of GMA and cognitive abilities: A European community meta-analysis. Personnel Psychology, 56, 573-605. Schmidt, F. L., Shaffer, J., & Oh. I. (2007). Reassessing the Relative Importance of Cognitive Ability and Personality in Job Performance and Training Performance: Some Surprising New Research Findings. Paper presented at the 2007 ATP conference, Palm Springs, CA. Feb. 6. Taylor, Paul J., Russ-Eft, D.F., & Chan, D.W.L. (2005). A Meta-Analytic Review of Behavior Modeling Training. Journal of Applied Psychology, 90, 692-709. Tracey, T.J.G., & Robbins, S.B. (2006). The interest-major congruence and college success relation: A longitudinal study. Journal of Vocational Behavior, 69, 64-89.
×1= 9 4 1×1= 1 5 8×1= 8 6 7×1= 7 7 8×3= 24.
1 Budapest University of Technology and Economics, BME, 1872 Budapest University of Technology and Economics, BME, 1872 Happy New Year 2012.
Copyright © Action Works 2008 All Rights Reserved - Photos by David D. Kempster 1.
Copyright © 2012, Elsevier Inc. All rights Reserved. 1 Chapter 7 Modeling Structure with Blocks.
1. 2 Choosing and preparing for a career is the most challenging developmental task of all for the late adolescent and young adult. It is essential for.
Create an Application Title 1Y - Youth Chapter 5.
Cracking the Code: ACT Research on College & Career Readiness 2011 GEAR UP Alliance Institute Leadership Strand © 2011 by ACT, Inc. All rights reserved.
CALENDAR NEW CALENDAR
1 Using K-12 Assessment Data from Teacher Work Samples as Credible Evidence of a Teacher Candidate’s Ability to Produce Student Learning Presented by Roger.
C Copyright © 2005, Oracle. All rights reserved. Practice Solutions.
1 California Preschool Learning Foundations. 2 Outcomes Increase your understanding of the California Preschool Learning Foundations, including: why they.
MaK_Full ahead loaded 1 Alarm Page Directory (F11)
Organizational Behavior Individual Differences. © 2003 Prentice Hall Inc. All rights reserved.2–22–2 Organizational Behavior.
Robbins et al., Fundamentals of Management, 4th Canadian Edition ©2005 Pearson Education Canada, Inc. Chapter 1 Managers and Managing in the 21st Century.
1 Before Between After 2 What comes before. _____ 10 _____
The 5S numbers game. 1. This sheet represents our current work place. Our job during a 20 second shift, is to strike out the numbers 1 to 49 in correct.
©2004 by Pearson Education11-1 R.C. Hibbeler Resistência dos Materiais, 5ª ed. 11 – Projetos de Vigas e Eixos.
Management: Arab World Edition Robbins, Coulter, Sidani, Jamali Chapter 1: Introduction to Management and Organizations Lecturer: Amani B AL-Kahtani.
A sample problem. The cash in bank account for J. B. Lindsay Co. at May 31 of the current year indicated a balance of $14, after both the cash receipts.
I 1. can 2 see 3 A 4 to 5 come 6 my 7 the 8.
5 | 2Copyright © Houghton Mifflin Company. All rights reserved. Part Three Markets and Consumer Behavior.
2011 WINNISQUAM COMMUNITY SURVEY YOUTH RISK BEHAVIOR GRADES 9-12 STUDENTS=1021.
Improving individual performance: Effective Performance Management Systems for Small States Regional Caribbean Conference on Improving Public Service Performance.
PP Test Review Sections 6-1 to 6-6 Mrs. Rivas 1. 2.
What We Know about Non- Cognitive Constructs and Workplace Outcomes Paul Sackett.
2011 FRANKLIN COMMUNITY SURVEY YOUTH RISK BEHAVIOR GRADES 9-12 STUDENTS=332.
Older Workers: Employment Expectations CAUCE Conference 2012 Atlanta Sloane-Seale & Bill Kops University of Manitoba.
Chapter 5 Transfer of Training Copyright © 2010 by the McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin.
Orientation and Training Susan A. Abravanel Sydney Taylor June 25 th, 2014.
Time for a BREAK! You have 45 Minutes. Time Left 44.
Presented to: By: Date: Federal Aviation Administration FAA Safety Team FAASafety.gov AMT Awards Program Sun ‘n Fun Bryan Neville, FAASTeam April 21, 2009.
V WFN41 Employee & Manager Self Service Overview EMPLOYEE EMPOWERMENT THROUGH TECHNOLOGY.
PSSA Preparation. Question 1(no calculator) D Question 2 (no calculator)
Symantec 2010 Windows 7 Migration Global Results.
Time and Labor Processing Day 1. Exercise #1a. Enter Time – Positive.
©2004 by Pearson Education13-1 R.C. Hibbeler Resistência dos Materiais, 5ª ed. 13 – Flambagem de Colunas.
David Burdett May 11, 2004 Package Binding for WS CDL.
Subtraction: Adding UP. Category 1 The whole is a multiple of ten.
TCCI Barometer September “Establishing a reliable tool for monitoring the financial, business and social activity in the Prefecture of Thessaloniki”
1.step PMIT start + initial project data input Concept Concept.
Pennsylvania Value-Added Assessment System (PVAAS) High Growth, High Achieving Schools: Is It Possible? Fall, 2011 PVAAS Webinar.
Introduction 1 The Cost of Occupational Fraud 2.
Break Time Remaining 10:00. Break Time Remaining 9:59.
TCCI Barometer March “Establishing a reliable tool for monitoring the financial, business and social activity in the Prefecture of Thessaloniki”
Model and Relationships 6 M 1 M M M M M M M M M M M M M M M M
1 Phase III: Planning Action Developing Improvement Plans.
Copyright © 2003 Pearson Education, Inc. Slide 1 Computer Systems Organization & Architecture Chapters 8-12 John D. Carpinelli.
Unit I Topic 2-7 MAC Protocols for Ad Hoc Wireless Networks Department of Computer Science and Engineering Kalasalingam University 1 CSE 6007 Mobile Ad.
© 2017 SlidePlayer.com Inc. All rights reserved.