1. NOCTI Overview Amie Birdsall and Patricia Kelley February 23, 2012

2. Discussion Topics  NOCTI Overview  Products and Services  Selecting an Assessment  CTEDDI  Q&A

3. NOCTI Overview  Over 40 years serving the CTE community  Non-profit organization  Owned and operated by the 50 states  Each state is a member of NOCTI’s Consortium of States  Works with both secondary and post-secondary populations  Nocti Business Solutions (formerly The Whitener Group) provides services to industry

4. NOCTI Overview - Educational Partners

6.  Over 80 Job Ready tests offered  30 Pathway tests in 10 clusters available  Over 90 private industry employment and/or advancement tests delivered  20 teacher-level assessments  Study guides for all Job Ready and Pathway assessments  Countdown Kits available for all Job Ready assessments  Successful Schools-Successful Programs Guide Supplemental ProductsAssessments NOCTI Products & Services Professional Learning Opportunities Also Offered!

7. Selecting an Assessment  Determine purpose of assessment  Determine audience  Determine results desired  Research available options

8. Selecting an Assessment

9. www.nocti.org Assessment blueprints are available on NOCTI’s website at no cost!

10. Assessment Blueprints Blueprints include: Competency List Content Distribution Sample items/jobs

11. Assessment Blueprints  Content distribution for written and performance  Sample multiple- choice questions  Sample performance job

12. Job Ready Assessments  Assess technical skills at the occupation level (e.g., Accounting, Carpentry, Pre-Engineering)  Measure aspects of occupational competence such as factual and theoretical knowledge  Target secondary and post-secondary program completers

13. NOCTI Job Ready Assessment Blueprint Plumbing

14. Number of Questions: 222 Areas Covered:Accounting

15. NOCTI Pathway Assessment Blueprint Construction Pathway

16. Assessment Results  NOCTI’s Standard Score Reporting Package:  Individual Score Reports  Group Score Reports  Comparative Data  Other Reports Available:  Task-Link Reports  Integrated Academic Reports

17. What data do I receive from NOCTI? Score Reports Online Test Administration  Immediate Score Reports  Aggregate Reports: 2 business days after user codes are released from Client Services Center Paper/Pencil Administration  Aggregate Reports: 10 business days after receipt of correctly completed answer sheets

18. What data do I receive from NOCTI? Immediate Score Reports  Immediate Student Scores  Displayed to student if option is designated  Emailed to Site Coordinator upon completion of testing (can be forwarded to instructor)

19. NOCTI Standard Score Reports Individual Score Reports Provided for each test taker Scores by duty and total Pre-/post-test comparisons Comparative data Individual Score Reports Provided for each test taker Scores by duty and total Pre-/post-test comparisons Comparative data

20. NOCTI Standard Score Reports Group Score Report Provided for the testing group Duty and total scores for each participant Group Score Report Provided for the testing group Duty and total scores for each participant

21. NOCTI Standard Score Reports Analysis of Scores Report Provided for the testing group Comparative data Criterion- referenced cut score Other statistical information Analysis of Scores Report Provided for the testing group Comparative data Criterion- referenced cut score Other statistical information

22. NOCTI Additional Reports: Task-Link Report Includes: Number of items presented for each task Average correct by group Number correct for each test taker Task-Link Report Includes: Number of items presented for each task Average correct by group Number correct for each test taker

24. Project Goals Understand data and assessments Use data effectively Improve learning and skill acquisition for students Plan for continuous improvement of learning and instruction Share ideas and collaborate with other educators

25. Instructional Improvement Cycle

26. A Nursing Assisting Class Classroom Example

27. Areas on a Nursing Assisting Test

28. Nursing Assistant Pretest Areas Pretest % Correct Student 1 Posttest % Correct Student 1 Basic Nursing Assisting 54.6 -- Infection Control 30.0 -- Communication Skills 41.1 -- Personal Care Skills 36.4 -- Mental Health/Social Needs 53.8 -- Client’s Rights 36.4 -- Basic Restorative Devices 50.0 -- Employability Skills 27.3 -- Total 43.4 Individual Nursing Pretest

29. Nursing Assistant Pretest Areas Student 1 % Correct Student 2 % Correct Student 3 % Correct Student 4 % Correct Student 5 % Correct Student 6 % Correct Class Average % Correct Basic Nursing Assisting 54.645.542.727.3 54.541.9 Infection Control 30.020.010.020.010.030.020.0 Communication Skills 41.117.623.5 13.235.325.7 Personal Care Skills 36.454.545.5 36.454.545.5 Mental Health/Social Needs 53.838.542.338.520.853.841.3 Client’s Rights 36.4 38.336.422.545.535.9 Basic Restorative Devices 50.060.050.060.030.080.055.0 Employability Skills 27.318.2 27.39.127.321.2 Total 43.438.435.938.522.848.838.0 Nursing Assisting Pretest

30. Nursing Assisting Posttest Nursing Assistant Pretest Areas Student 1 % Correct Student 3 % Correct Student 5 % Correct GroupSiteStateNation Basic Nursing Assisting 100.092.476.286.479.572.575.5 Infection Control 91.794.572.479.278.270.770.6 Communication Skills 100.0 55.395.083.864.366.4 Personal Care Skills 92.354.552.569.265.851.652.9 Mental Health/Social Needs 97.472.560.371.869.863.164.1 Client’s Rights 84.866.444.764.667.658.157.8 Basic Restorative Devices 87.560.245.253.161.955.656.5 Employability Skills 87.578.537.479.775.666.667.0 Total 93.979.458.476.873.764.165.2

31. Achievement gaps Differences between subgroups Trends Progress What the Data Reveal

32. Collecting Data – More Than NOCTI Includes process of collecting, analyzing, and interpreting information Determines the degree to which students are becoming competent and meeting standards Occurs at many levels (e.g., classroom and program assessments to state and national assessments)

33. School Program Teacher What Data Exist? Class Student Standards National State District/System

34. Icebergs What lies beneath the data scores that influence those scores?

35. What is the data designed to answer? What other data may be needed? What roadblocks are there to interpreting the data? What Questions Can the Data Answer?

36. Plan for Learning Improvement Start with results: Analyze the previous year’s goals, comparing against local unit tests and end-of-course tests, at the student level and classroom level. Triangulate: Find relationships between test data and attendance and discipline referrals.

37. Plan for Learning Improvement Continued: Disaggregate data by subgroups (e.g., socioeconomic status or those receiving free/reduced lunch, or Individualized Education Program/IEP). Use information from advisory groups or parents. Set goals for the year as well as long-term goals.

38. Plan for Learning Improvement Continued: A common denominator is to improve learning through teamwork. Build school department and state data teams. Review annual assessment data with a faculty team to isolate achievement gaps or groups of students with high and low performance.

39. Plan for Learning Improvement Continued: Schedule school and department data team meetings to discuss data and review progress. Discuss and develop strategies, such as curricular alignment with academic and industry standards, curricular sequence, timing, and depth of instruction.

40. Monitor Student Progress Use a progress monitoring system for all students. Graph students’ curricula-based scores. Identify effective instructional components.

41. Thank You Amie Birdsall Amie.Birdsall@nocti.org Pat Kelley Patricia.Kelley@nocti.org