Presentation is loading. Please wait.

Presentation is loading. Please wait.

Alabama System Test Coordinator Workshop for EXPLORE Administration

Similar presentations


Presentation on theme: "Alabama System Test Coordinator Workshop for EXPLORE Administration"— Presentation transcript:

1 Alabama System Test Coordinator Workshop for EXPLORE Administration
September 13, 2010

2 ACT’s College and Career Readiness System- EXPLORE Jacqueline Twiggs, MEd Director, P-16 Assessment Services ACT Southeast Region

3 Preparation for College and Career
Emphasize the ALL student goal and acknowledge the national and state movement under way to make college/career ready the standard. (versus proficiency on state minimum competency tests) The term “postsecondary” means preparing students for the transition from high school to any program – certification programs, job training, 2-year colleges, 4-yr colleges, apprenticeship programs, military. When we say college/postsecondary, we mean preparation for life after high school. ACT research went one step further and examined that statement. The result of the study provided empirically based research to support that being Ready for College and Ready for Work require similar skills. Explain the basis of the research for Ready for College and Ready for Work: Same or Different? “To determine how workforce training readiness compares to college readiness, we analyzed data from 476,847 high school juniors in Illinois who took the ACT, the WorkKeys Reading for Information Test, and the WorkKeys Applied Mathematics Test between 2001 and These tests are administered as part of the Illinois Prairie State Achievement Examination program, a statewide assessment administered annually to all eleventh-grade students. We statistically aligned the scores on the two WorkKeys Tests (which represent workforce training readiness) to the scores on the ACT Reading and Mathematics Tests (which represent college readiness).” Although the contexts within which these expectations are taught and assessed may differ, the level of expectation for all students must be the same. Academic preparation lays the foundation for a student to have options and opportunities. By limiting a student’s academic preparation, we close doors and limit opportunities. Students should be choosing what option they want to pursue – not trying to figure out what options they have because of their academic preparation – or lack thereof. BACKGROUND INFORMATION In the next decade, nearly two-thirds of the estimated 15.6 million net new jobs created in the U.S. will be in occupations that require some post-secondary education or considerable on-the-job training. (Remarks Prepared for Delivery by U.S. Secretary of Labor Elaine L. Chao, Hispanic Association of Colleges and Universities 13th Annual Capitol Forum on Hispanic Higher Education, Washington, D.C. Monday, April 14, 2008, cited on the Department of Labor website) The Occupational Information Network (O*NET) system serves as the nation's primary source of occupational information, providing comprehensive information on key attributes and characteristics of workers and occupations. The O*NET database houses this data, and O*NET Online provides easy access to that information ACT focused on Job Zone 3 because the occupations in this zone are likely to offer a wage sufficient to support a small family, provide the potential for career advancement, and are projected to increase in the future (U.S. Department of Labor, 2004). Zone 3 is the highest O*NET level that includes jobs that do not require a bachelor’s degree, but which likely require some combination of vocational training and/or on-the-job experience, or an associate’s or higher degree. Examples include electricians, construction workers, upholsterers, and plumbers. By selecting O*NET Zone 3, we are essentially defining workforce readiness as workforce training readiness, since Zone 3 jobs require high school graduates to have the foundational skills necessary to learn additional job-specific skills throughout their careers. Detailed information on O*NET Prepare all students for success, no matter which path they choose after graduation. In the next decade, nearly two-thirds of new jobs created in the U.S. will require some post-secondary education or considerable on-the-job training. Former U.S. Secretary of Labor, Elaine L. Chao

4 What is EXPLORE? ACT’s College Readiness assessment for 8th and 9th graders A curriculum-based achievement test including: Four Subject Area Tests: English Mathematics Reading Science An Assessment of Career Interest The UNIACT Interest Inventory 72 Items Identification of career areas based on ACT’s World of Work Map

5 Longitudinal Assessments
ACT’s College and Career Readiness System (CCRS) for ALSDE Administration Longitudinal Assessments 8th Grade English, math, reading, science Career and Educational Components Score Scale: 1—25 10th Grade English, math, reading, science Career and Educational Components Score Scale: 1—32 11th Grade English, math, reading, science, optional Writing Test Career and Educational Components Score Scale: 1—36 12th Grade Job skills assessment system Measures real-world skills Measures 10 foundational workplace skills Data must be gathered over a timeline to make informed decisions. We understand that ACT program and solutions are but one component in your many variables that impact education. For the purpose of our workshop we are going to focus on how our products and solutions all fit together and can provide a baseline of data. A data systems is a collection of data and managing of the data – our system is intuitive – empirically derived – you may choose to throw other pieces in to the system of evaluating data but ACT solutions provide the baseline.

6 ACT’s College and Career Readiness Assessments –
What they have in common Tests are curriculum-based achievement tests. They are tests of acquired and/or developed abilities. Test questions correspond to recognized middle and high school learning experiences. Tests consist of complex heterogeneous tasks that require students to apply the skills and knowledge they’ve learned over time to solve them. Tests are developmentally appropriate for the grade level.

7 ACT’s College and Career Readiness Assessments –
What they have in common Test items are written by classroom teachers. Tests are based on ACT’s National Curriculum Survey All skills measured on EXPLORE, PLAN, and the ACT are aligned with over 40 state learning standards All skills measured on EXPLORE, PLAN, and the ACT are included in NAEP Standards

8 College and Career Readiness Assessments
CONTENT How are the test constructed? What do the tests measure? We will begin with the first two questions and hope to cover these by the time we take a break. This will give a basic understanding of the test and the scores so that when you review your data you will have a basis for understanding the numbers you see.

9 ENGLISH TEST Measures students’ understanding of the conventions of standard written English Measures students’ ability to effectively communicate meaning by: Critiquing Revising Editing The English test measures the student’s ability to revise and edit an essay, important elements of the writing process. Students are presented with a variety of writing styles to edit and revise. Colleges and universities have found that the ACT English test is an accurate and reliable measure for placement into freshman credit-bearing English Composition courses. This is why many colleges are opting to not require the additional essay test. Note that even though the four test sound like subject area test, they really are not. They are not appropriate as end-of-course tests. These four tests measure curriculum skills, not course content skills. That is, they measure the kinds of skills students need to study in a variety of subjects. Writing and reading are every teachers responsibility -- including coaches, business ed, technology, vocational education, etc.

10 ENGLISH TEST EXPLORE PLAN ACT Punctuation 6 7 10
Grammar and Usage Sentence Structure 40 53% 30 60% 25 63% Usage/Mechanics Strategy Organization Style 35 47% 20 40% 15 37% Rhetorical Skills Total Items Passages Passage Length 4 300 Words 4 300 Words 5 325 Words

11 MATHEMATICS TEST (Four-Function Calculators are permitted)
Requires Students to Analyze problems – in both real world and purely mathematical settings Plan and carry out strategies Verify appropriateness of solutions Some of the math problems are pure math, many are application

12 MATHEMATICS TEST PLAN: 2 sub-scores; ACT: 3 sub-scores
Content Area Basic Statistical/ Probability Concepts Pre-Algebra Elementary Algebra Pre-Geometry Plane Geometry Coordinate Geometry Intermediate Algebra Trigonometry Total Items EXPLORE 4 10 9 7 30 PLAN 14 8 11 7 40 14 10 9 4 60 ACT

13 READING TEST Measures student proficiencies in understanding and deriving meaning from texts ranging from fiction narratives to informational passages Measures vocabulary by determining the meaning of unfamiliar or multiple-meaning words from context Measures skills used to read and understand published materials Passages are not made up for the test. At the beginning of each passage, we give the title and author of the passage and where it was published.

14 READING TEST Content Area Prose Fiction Humanities Social Sciences
Natural Sciences Total Items EXPLORE 10 30 3 500 Words PLAN 8 9 25 3 500 Words 10 40 ACT 4 750 Words Passages Passage Length

15 SCIENCE TEST (EXPLORE)
Measures student proficiencies in using and reasoning with science information, skills, and knowledge typically acquired up through middle school science courses Asks Students to: Communicate information and use scientific research strategies Make comparisons between, and draw conclusions from scientific findings, studies, and viewpoints. Extrapolate and extend scientific understandings consistent with sound scientific reasoning. The science test assumes students have learned certain content from their science classes. Although this content knowledge is assumed in the science test, they are not asked to recall the specific facts from specific courses. The test measures their ability to “think like a scientist.” The information/facts they need to answer the questions are given in the passages.

16 SCIENCE TEST Stimulus Material Data Representation Research Summaries
Conflicting Viewpoints Total Items EXPLORE 12 10 6 28 PLAN 10 14 6 30 15 18 7 40 ACT

17 SCIENCE TEST- EXPLORE Content Area Format Life Science
1 Life Science Physical Science Earth/Space Science Data Representation Research Summaries Conflicting Viewpoints Content areas are distributed over all formats 1

18 SCIENCE TEST- PLAN & ACT
Content Area Format 1 Biology Earth/Space Science Chemistry Physics Data Representation Research Summaries Conflicting Viewpoints Content areas are distributed over all formats 1

19 ACT’s College and Career Readiness Assessment Scales Relationship
5 10 15 20 25 30 35 40 EXPLORE PLAN ACT 36 32 25 The ACT in its present format was introduced in 1989 on a scale of Plan was introduced in 1987 on a scale of EXPLORE became fully operational in the early 1990s on a scale of We had three tests that measured common skills in a common format, but we had three different score scales. To help educators better track their students academic development, we put PLAN on ACT’s score scale and EXPLORE on PLAN’s score scale. We did not but EXPLORE on ACT’s score scale. We can predict an ACT score from PLAN. We can predict a PLAN score from EXPLORE. But, we will not predict an ACT score from EXPLORE.

20 ACT’s College and Career Readiness System Scales Relationship
5 10 15 20 25 30 35 40 EXPLORE PLAN ACT 36 32 25

21 ACT’s College Readiness Benchmark Scores
*The ACT Benchmark Score indicates a 50% chance of obtaining a “B” or a 75% chance of obtaining a “C” in corresponding credit-bearing college courses. College Readiness Benchmarks are empirically derived. Through research done on a sample of ACT test takers over a period of ten years, we were able to derive benchmark scores that provide a strong probability of success in the first credit-bearing core courses at a postsecondary institution that correspond to the ACT subject tests. Students who meet or exceed a benchmark score have a 50% chance of achieving a B or higher or about a 75% chance of achieving a C or higher in the corresponding credit-bearing college course (college level credit-bearing courses = English Composition, College Algebra, College Biology, and College Social Sciences courses such as U.S. History, Psychology, etc.). Only ACT provides empirically derived benchmarks and only EPAS assessments provide these indicators over time with grade-appropriate assessments. Using these benchmarks assists educators in identifying students early on in their high school career of their current readiness for college, and whether any early intervention is needed to get the student back on track. As EPAS provides longitudinal data on students, the system and the benchmark scores can provide teachers and administrators with the ability to assess how well/effective intervention plans are, and if there is any need for adjustment. Measures vs. Standards The EXPLORE and PLAN College Readiness Benchmark Scores are based on the ACT College Readiness Benchmark Scores. They reflect students’ expected growth from EXPLORE to PLAN to the ACT and assume sustained academic effort throughout high school. The College Readiness Standards enable teachers, students and parents to understand what a test score means—how it translates into usable academic skills. Part of the College Readiness Standards involves information on classroom experiences that teachers can use to move students to a higher level of achievement. This type of integrated systems that can provide the early diagnosis and guidance which will help in achieving their goal. While not to be interpreted as guarantees of college admission or success, the CRBS can provide important indicators of readiness as early as 8th grade. Identify those students not meeting one or more Benchmarks early on while there is still time for academic interventions or coursework planning Using that definition and our extensive empirical evidence consisting of course grades earned by more than 900,000 students in a nationally represented population of both two year and four year postsecondary institutions we know exactly how well high school students must perform on the ACT test. The benchmarks are differentiators, as only ACT can offer them and determine probability of success (based on scores) in initial credit-bearing core courses. This is primarily due to the fact that the ACT is a curriculum based assessment, and therefore can provide such research based information. National Curriculum Survey: Course Placement Service: ‘Crisis at the Core ‘ and ‘On Course for Success’: Trap: Biology as the standard. This decision was based on our National Curriculum Survey of over 200,000 middle school, high school and postsecondary educators and administrators that identified Biology as the ‘entry-level’ Science course. Trap: How stable are the benchmarks The benchmarks are empirically derived and are based on course grades earned by more than 900,000 students in a nationally represented population of both two year and four year postsecondary institutions. They are nationally normed, so they cannot determine probability of success at a particular institution (ACT’s Course Placement Service can assist 2 and 4-year institutions in creating institutional specific benchmarks), but they provide a good overall view of a student’s college readiness. Trap: Why do the Benchmark scores vary? This was research based where ACT made some choice on where to draw the lines- guidelines. Based off of the National Curriculum Survey, ACT also determined the levels of proficiency that initially indicate a higher probability of success in the four core subject areas, and therefore determined the specific benchmark scores for each. Trap: Placement vs. Benchmark The highlighted benchmarks on this slide provide an overall view of a student’s readiness for college. As mentioned before, these are nationally normed benchmarks, and in order to use such scores for placement at a particular college/university an institution would have to work with ACT to complete a Course Placement Service that would provide schools with institutional specific benchmarks for use in placement. Test College Course PLAN The ACT 8th Grade 9th Grade English English Composition 13 14 15 18 Math Algebra 17 19 22 Reading Social Sciences 16 21 Science Biology 20 24 EXPLORE

22 College Readiness Standards (CRS) are the statements that represent widely held learning goals that are important for success in high school, college, and the world of work. The CRS link EXPLORE, PLAN, and the ACT scores to curriculum and instructional decision making.

23 College Readiness Standards by Learning Strands and Score Ranges
Basic Operations and . . . Probability, Statistics, & Data . . . Numbers: Concepts & Properties 16-19 20-23 Scores are seen as Assessment for Achievement, rather than Assessment of Achievement! Standards: ideas for progress Participants can review the row of your choice and identify the grade level by which students should master the skills in that row. I usually choose row ACT does not put a grade level on any College Readiness Standards skill range. You can discuss here about using the Standards for setting goals for the school. Where do they think their students should be scoring when they take EXPLORE? PLAN? When they take the ACT? Explain that the skills in the range are measured on ACT only; the skills in the range are measured on PLAN and the ACT, but not on EXPLORE. Also note that on the Mathematics pages, the last column, Functions, is measured on the ACT only. The College Readiness Standards allows us to answer two of the questions on my earlier slide: What skills do my students already know? What skills do they need to learn?

24 Translating Scores to Skills
Mathematics: Basic Operations & Applications - Score Range: 16 – 19 Solve routine one-step arithmetic problems (using whole numbers, fractions, and decimals) such as single-step percents Solve some routine two-step arithmetic problems Visual representation translating the concept of scores to skills. Do not focus on the words on the target. This is meant to visually represent that the focus is on the skills, not the numbers. The objective is to move the students into different ranges of skills. 8th-Grade On Course Benchmark: 17 Indicates that the student is on course for meeting the ACT benchmark in grade 11-12 EXPLORE 8th-Grade Mathematics Benchmark = 17

25 EXPLORE Reporting Services
Slide Information: Decisions that are made using concrete data often yield the best results. ACT solutions provide you with powerful data tools that can enhance your academic objectives with your students. The beauty of ACT data and reporting is that they apply at all levels – the student, school, district, and state – consistent data to help you frame your objectives and establish/reinforce your priorities.

26 Reporting Services Student Score Report Item Analysis
Career Interest Inventory College Readiness Benchmarks Slide Information With changes to the reporting packages for EXPLORE and PLAN (we will discuss more later) we want to focus on how to best use two specific reports that most everyone will receive - the College Readiness Standards and Item Response Summary Report. Briefly state what the purpose of the other reports are and what they entail. Note: Acknowledge that some of the schools/districts have been using these tools and we hope that we can provide them with a few new ideas to take back and try. College Readiness Standards (w/ Ideas for Progress) Course and Grade Information

27 EXPLORE

28 What do Your Scores Mean?
Composite Score 15 Score Range (1-25) PLAN takes the # of questions you answered correctly and translates this into a score between 1 and 32 The composite score is simply the average of your test scores in English, Math, Reading & Science Your Composite score can range from 1-32 The subscores tell you how well you did in 2 specific areas of each subject( English and Math) and range from These scores are on a different scale than the subject area and composite score and therefore do not add up to equal the subject area score. The percentage listed is the percent of students scoring at or below your score. Or, You scored as high as or higher than __% of other students

29 EXPLORE Score Report Do my scores indicate that I am on track to being ready to succeed in college courses? Compare the students scores on the upper left of the student report to the Benchmark scores printed in the shaded area. Walk the audience through this.

30 USING YOUR REPORT PACKAGE Student Level – EXPLORE and PLAN Item Response
Presenters note: FIRST CLICK: Will bring up red boxes SECOND CLICK: Will flash on the subscore area box and explode the box. THIRD CLICK: Will go back to smaller view of the Item Response report FORTH CLICK: Will flash on the content area box and explode the box Reminder: Important transition from detail to reporting Slide Information The back side of the EXPLORE and PLAN student score reports contain an Item Response. Counselors and teachers need to look at this information closely AND interact with students. This is the back of the PLAN student report. Since there is a new form of the test each year, we can publicize the answers. It is important that you return the students score report and their test booklet to them so they can compare their answers to the questions. On the left side of page 2 of the Student Report, you will find a list of the student responses to each of the EXPLORE and/or PLAN test questions that were on the test. You will find a list on the student’s answer along with the correct answer for each of the four areas (English, math, reading, and science) The area on the right is personalized for the student and comes directly from the “ideas for progress” printed in the College Readiness Standards materials. This information can be used to improve student skills in each of the areas tested by EXPLORE or PLAN (English. math, reading, and science). Notice that each suggestion corresponds to specific content areas. For instance, in math, you will find suggestions for how to improve your skills in “Basic Operations”, “Measurement”, and other important math areas. These suggested actions can be incorporated into a student’s academic plan. The suggestions on the student score report will most likely be different from those for other students in who took EXPLORE or PLAN, because they are based specifically on the individual student EXPLORE and PLAN scores. Remember that the College Readiness Standards show how students’ skills can progress, becoming increasingly sophisticated from score range to score range. The purpose of the Standards is to help counselors, teachers, and administrators, as well as students and their parents, to better understand how the scores relate to the kinds of skills needed for success in high school and beyond. Remember, EXPLORE and PLAN are curriculum-based assessments which means that is measures what students can do and with what they have learned. The knowledge and skills measured by these assessments differ in sophistication and complexity from grade 8 to grade 12. So, the Standards serve as a direct link between what students have learned and what is being taught in the classroom. Background Information: Test Question Analysis Activity Booklets; College Readiness Standards Tables; Sample Test Questions Matched to College Readiness Standards; Curriculum Review Worksheets. These are excellent resources for educators.

31 72 item unisex interest inventory of work relevant activities
Included with EXPLORE, PLAN and the ACT 72 item unisex interest inventory of work relevant activities Identifies personally relevant career options Bridged to World-Of Work Map and Holland Types

32 Career Planning Program: World-of-Work Map
About the Map 26 career areas (groups of similar occupations) arranged into 12 regions Career area locations are empirical. Based on work tasks: DATA: Facts, numbers, files, accounts, business procedures. IDEAS: Insights, theories, new ways of saying or doing something—for example, with words, equations, or music. PEOPLE: People you help, serve, inform, care for, or sell things to. THINGS: Machines, tools, living things, and materials such as food, wood, or metal. Six general types of work (“career clusters”) and related Holland types (RIASEC) are shown on edge of map.

33

34 School Reporting Package
Student Score Reports (2 per student) Student Score Labels (2 per student) Student List Report Profile Summary Report (includes College Readiness Standards Data, Presentation Packet, and Early Intervention Rosters) Item-Response Summary Report Item-Response Interpretive Guide Interpretive Guide for Student & School Reports School Norms (on Student Score Report) Data File w/ School Norms on CD

35 System Reporting Package
Profile Summary Report (aggregated data from all affiliated schools) – (CD) Profile Summary Report by school (CD) Item-Response Summary Report for System and by School (CD) Item-Response Summary report Interpretive Guide (1 copy) Data File for System (CD) EXPLORE Norms for School and System

36 Profile Summary Reports
Profile Summary Reports summarize the performance of your students who took EXPLORE. The report focuses on: performance, access, course selection, and postsecondary and career plans. Reports are for EXPLORE and PLAN

37 What’s in the “Pipeline?”
EXPLORE Page 6

38 Intervention Rosters The EXPLORE Early Intervention Rosters identify students who fit into three categories. This information can help you to design intervention strategies to assist students to reach their academic and career goals. Be sure to translate features into benefits.

39 Benefits: Early Intervention
EXPLORE Intervention Rosters: Identify which students reported that they do not plan to finish high school, or have no post-high school educational plans Identify which students earned an EXPLORE Composite score at or below the national 10th percentile for eighth-grade students Identify which students expressed a need for a lot of help in one or more selected areas.

40 Item-Response Summary Report
EXPLORE Item-Response Summary Reports show the percentage of your students who selected the correct response to each item, the percentage of your students who selected each incorrect response, and the percentage who did not answer the item. In addition, the report gives the average percentage of students who responded correctly to the items in each content area. Be sure to translate features into benefits.

41 USING YOUR REPORT PACKAGE School/District Reports – Item Response Summary Report
EXAMPLE: EXPLORE Item Response Summary Report Presenters note: FIRST CLICK: Will bring up red boxes SECOND CLICK: Will flash on the box and explode the box. The colors and look of this report may change, but the information will stay the same. The Item Response Summary Report will consist of a full page for each subject area of the test. The example here is English. Slide Information The Item-Response Summary Report shows the percentage of your students who selected the correct response to each item, the percentage of your students who selected each incorrect response, and the percentage who did not answer the item. In addition, the report gives the average percentage of students who responded correctly to the items in each content area. Reference group results are provided to assist you in evaluating your students’ performance on each item relative to that of other tested students. The item-level information in this report can help you relate your students’ overall performance on EXPLORE or PLAN to your local curriculum, including identifying consistent patterns of strength or weakness in your tested students’ performance across areas of your curriculum measured by the test. If for example, you find that your students perform consistently lower than the reference group on items within a content area such as Punctuation, you can examine the text of the Punctuation items to obtain specific examples of the skills or knowledge involved. In using this report, you should determine your students’ academic strengths and weaknesses relative to the skills and knowledge measured by the test items, and address apparent weaknesses at the content area level. Reminder: Promote Awareness of the tool – and engaging or getting the right information to the right people. Questions to consider Is this information being shared with department heads and curriculum specialists? Background Information EXPLORE or PLAN Item-Response Summary Report

42 www.explorestudent.org Visual 15: Points of Emphasis
ACT has developed a new website designed to help students and parents understand and use EXPLORE results. At you can: Learn what your EXPLORE scores mean Learn how to improve your skills Learn how EXPLORE can help you see if you are on target for college Use online career exploration tools Find sample EXPLORE test questions

43

44 ACT Southeast Region Staff supporting Alabama
Gennine Brewer Senior Consultant Jacqueline Twiggs Director, P-16 Assessment Services Jim White Assistant Vice President


Download ppt "Alabama System Test Coordinator Workshop for EXPLORE Administration"

Similar presentations


Ads by Google