1. 1 Michigan Merit Examination Edward Roeber Michigan Department of Education

2. 2 Issues Leading Up to the MME High school principals organization felt MEAP testing took too long (days and hours) MEAP assessment program suffered an operational meltdown in 2003 Although students can receive up to $3,000 towards post-secondary education, student didn’t take the MEAP test seriously Participation in the MEAP assessment was too low - especially in suburban districts

3. 3 Proposed MME Legislation Students would take the following tests –College-entrance test and a work skills assessment acceptable to employers, plus and Michigan items needed to secure Federal approval –Testing would occur in the spring of the junior year –Retesting in fall and spring of senior year –One free retake College-entrance readiness tests (PLAN and PSAT) used for dual enrollment eligibility purposes

4. 4 Arguments in Favor of MME Students will receive a “free” college-reportable score on a college entrance test - this would motivate students to participate in and do well on assessments Students will receive regular work skills scores in reading and mathematics - useful in employment Michigan Merit Exam scores could still be used to award post-secondary scholarship in Michigan Lower cost and less testing time

5. 5 Arguments Against MME Little standards-based information from a college entrance and work skills tests - no data for student or school improvement Test administration procedures are very rigid Can all students score “above average?” Since ACT test scores have not changed much over the past decade, how will schools make AYP? MME is far more expensive - $73 versus $19/student

6. 6 MME Legislation PA 592 of 2004 - MME inserted into Career and Technical Preparation Act PA 593 of 2004 - Replaced high school MEAP with MME in School Aid Act PA 594 of 2004 - Replaced high school MEAP with MME in Postsecondary Enrollment Options Act PA 595 of 2004 - Replaced high school MEAP with MME for determining Merit Award Scholarships PA 596 of 2004 - Replaced the high school MEAP with MME in the State School Code

7. 7 Steps Used to Implement MME Visited potential vendors to share ideas Competitively bid the MME program Reviewed the bids Selected new contractor(s) - Pearson/ACT Determined the content of each MME test Conducted pilot test of 37 high schools (statistical alignment)

8. 8 Steps Needed to Implement MME Conducted additional content alignment studies to select MME items Prepared revisions to Michigan’s Accountability Workbook Submitted application to USED in July, 2006 Conducted MME standard setting for NCLB purposes in July Set PLAN/PSAT cuts for dual enrollment

9. 9 Steps Needed to Implement MME November 1, 2006 was the “drop-dead date” for the MME versus MEAP decision for Spring 2007 –Receive approval from USED prior to drop-dead date, OR –continue MEAP High School Assessment (if approval is not received by that date) If approved by USED, implement the MME in grades 11 in the 2006-07 school year; maintain MEAP HST for Grade 12 only

10. 10 Steps Needed to Implement MME Was the USED approved the MME? November 1, 2006 letter from USED: “... It is likely to be fully compliant.. However, additional evidence is needed to show how Michigan’s new assessments meet the standards and assessment requirements under ESEA.”

11. 11 Evidence That Must be Provided Evidence of SBE adoption of achievement standards for the MME Documentation of technical quality indicators, except standard setting and materials previously submitted, as listed in the Peer Review Guidance for the MME as administered in spring 2007.

12. 12 Evidence That Must be Provided Final technical manuals for the 2006-07 MME and MI-Access assessments. Actual MME assessment reports of results at the student, school, and state levels that have been provided.

13. 13 So…it’s a Go! First MME administration - Spring 2007 Students to be assessed: Grade 11 –All students enrolled in Grade 11 during March 2007 take the complete Michigan Merit Exam –Including students who previously took the MEAP in grade 10 to qualify for dual enrollment

14. 14 MME Assessment Design Grade 10: College entrance readiness test (PLAN and PSAT - local school/student decision) Grades 11 and 12: Michigan Merit Examination –English Language Arts - ACT, WorkKeys and a Michigan Social Studies item –Mathematics - ACT mathematics, a few items from the ACT Science section, WorkKeys and some Michigan items –Science - ACT test and Michigan items –Social Studies - all Michigan items

15. 15 MME Assessment Design Assessments Administered Contributes to an English Language Arts Score Contributes to a Mathematics Score Contributes to a Science Score Contributes to a Social Studies Score ACT Plus Writing WorkKeys Applied Mathematics & Reading for Information And Michigan Mathematics Michigan Science and Social Studies Total ELA Score Reading ACT Reading (MC) WorkKeys Reading for Information items (MC) Writing ACT English (MC) ACT Writing (CR) Michigan Social Studies CR item - writing score ACT Mathematics (MC) WorkKeys Applied Mathematics (MC) Selected ACT Science items (MC) Michigan items (15 MC) ACT Science (MC) Michigan Science (55 MC) Michigan Social Studies (57 MC and 2 CR - social studies score) Note: The number of Michigan items may increase in order to field-test new items to replace those released annually.

16. 16 MME Test Organization DayTestSubject SessionParts Total Items or Prompts Testing Time (minutes)Total Admin Time Day 1ACT Plus Writing English 5 7545 Reading4035 Writing130 Mathematics60 Science40 Day 2 WorkKeys Reading for Information 3 3345 Applied Mathematics3345 Michigan Mathematics1420 Days 2,3 or 4 Michigan Science14645 Social Studies 2 3250 2750 TOTAL Minutes465 TOTAL hours7.75

17. 17 Special Populations Students with Disabilities Students with Section 504 Plans English Language Learners

18. 18 Accommodations ACT-approved –Available to students with disabilities and Section 504 plans –Apply only to the ACT State-allowed –Available to students with disabilities, Section 504 plans, and English language learners –Apply to all parts of the MME

19. 19 Accommodation Requests ACT-approved versus State-allowed For the ACT –7,934 ACT-approved –456 denied –1,565 state-allowed materials requested For State-allowed accommodations no approval is required

20. 20 MME Accommodation Summary Table

21. 21 MME Schedule for 2006-2007 MME assessment administration dates: –MME administered statewide March 13 - ACT March 14 - WorkKeys and Michigan Mathematics March 14-16 - Other Michigan tests –Make-up testing scheduled two weeks later March 27 - ACT March 28 - WorkKeys and Michigan Mathematics March 28-30 - Other Michigan tests –Auxiliary test centers offered for non-public students

22. 22 MME Activities for 2006-2007 September-October, 2006 –Conducted high school video test briefing on September 12 –Established 1,111 high schools as test centers - public schools, charter schools, and private schools –Schools identified key testing staff Test Supervisor Backup Test Supervisor, and Test Accommodations Coordinator

23. 23 MME Activities for 2006-2007 November, 2006-February, 2007 –High school Test Supervisor, Backup Test Supervisor, and Test Accommodations Coordinator training at 12 sites (about 3,400 testing staff) –Schools apply for accommodations for students with disabilities and English language learners (both ACT- and state-approved ones) –ACT reviews accommodation requests and determines those eligible for college-reportable or state-reportable scores –Auxiliary test centers set up for non-public school students who attend a non-public school that was not established

24. 24 MME Schedule for 2006-2007 February-March, 2007 –Tests (ACT and Michigan) produced –Accommodations determined and materials ordered –Non-secure and secure materials shipped to high schools –Students complete the demographic and non-cognitive sections of the answer folders for the MME –High school Test Supervisors train room supervisors and proctors –MME assessments administered

25. 25 MME Schedule for 2006-2007 April-July, 2007 –Students and schools receive reports of results Regular ACT reports (college-reportable) WorkKeys reports MME score reports –MME scores used for state accreditation and NCLB AYP accountability purposes

26. 26 MME Schedule for 2006-2007 August-November, 2007 –Retake vouchers produced, sent to schools, and given to students –Students register for MME retake via ACT –Retake on a single fall nationally-scheduled ACT testing Saturday administration - fourth Saturday in October –WorkKeys and Michigan tests administered the following week in schools

27. 27 MME Issues and Challenges Getting USED to approve an ACT-based assessment for NCLB AYP purposes –How much augmentation is needed? –Non-alignment of the WorkKeys tests Alignment of the MME tests to Michigan’s standards - determining needs to be added –Changing nature of the ACT tests Preparing to administer the ACT in Michigan’s schools –Test center establishment –Testing staff training –Accommodations

28. 28 MME Issues and Challenges Preparing the IT work for blending multiple assessments into a coherent assessment –Different psychometric models –Two contractors’ systems with Michigan’s systems Assuring adequate assessment administration preparation - for very secure tests (ACT) –Moving from an ACT-administered test to Michigan educator-administered tests

29. 29 MME Issues and Challenges Accommodations for students with disabilities –What accommodations will be offered and under what conditions? –How much work is it to get accommodations approved? –Will MI-Access need to be modified and/or expanded? Participation of English language learners –What accommodations will be offered and under what conditions? –Will any ELL accommodations result in college-reportable ACT scores? –Why can’t ELLs use the same accommodations as SWDs but not receive college-reportable scores?

30. 30 MME Issues and Challenges Working through the logistics of testing –Multiple (3) answer folders –Inability to pre-ID students –Blended reporting of multiple tests –Alignment of the tests to a state’s standards Working with ACT –Attempting to modify established procedures Electronic registration of students, key testing staff and schools Updating information on staff and schools electronically Considering modern psychometric methods Blending ACT’s procedures with those of states

31. 31 MME Issues and Challenges States using ACT have recently formed an “ACT Users Group” –Identified a number of challenges that states would like to work on together –Identified several issues that states would like to work on with ACT –Next meeting is May 17-18 in Detroit

32. 32 For Questions and Comments Edward Roeber (517) 373-0739 voice Roebere@michigan.gov Email Questions to mme@michigan.govmme@michigan.gov MME Website: www.michigan.gov/mmewww.michigan.gov/mme