Lessons Learned

Communication, Communication, Communication Collaborative effort Support of all stakeholders Teachers, Principals, Supervisors, Students, Parents, Teacher’s Union, Community Transparent Process Open two-way dialog Multiple methods of distributing and receiving information Teachers need to be able to review and verify the students that count for them and their data multiple times. Helping students achieve by helping teachers excel

What assessments will be included in the model? State testing data mandated. Standardized national assessments are readily available. When district assessments are used… – consider the impact on the teacher Time to administer vs. quality of the data received Is the assessment aligned with the curriculum? – consider the sample size Best practice suggests 10 teachers for a particular course to have appropriate standard error ranges. Consider what will be done with results when the sample size of teachers is less than 10…less than 3…. – consider the complexity of the assessment – consider how the data will be recorded and reported Item bank/Testing platform How will you maintain as much objectivity as possible? Helping students achieve by helping teachers excel

What assessments will be used for unique populations? How will ESE students be measured? – We looked at assessments recommended for use with specific populations. – We are using the Brigance Test of Basic Skills for students in grades KG-2 and 11 & higher and the Battelle and 4 Year Old Standards for PK. – We also have a district created rubric for assessing life skills in low prevalence populations. How will electives like performing arts be measured? – Our first round included performance items. – We moved to written exams based on the content knowledge taught in these courses. – We are testing specials on-line at the elementary level. Group administration Art pictures embedded Music recordings Paper and pencil option What about courses where a teacher has multiple levels in the same section? How will the data be captured? Helping students achieve by helping teachers excel

If district assessments will be used, what measures will ensure their validity and reliability? Content validity Test security Item bank/testing platform – Monitor real time data, rescore, rescan, remove zeros – Conduct an item analysis each time an assessment is administered – Revise exams if the point biserial and p value indicate a change is needed Minimize the use of stand alone pre-tests – Prior year achievement for Reading and Math FCAT, Stanford, PSAT ……. – District semester exam/end of year test from prior year Importance to student – exam exemptions Large item bank so that multiple versions of the exam can be developed District staff dedicated to building exams and better assessments — Collaborate with curriculum supervisors and teachers Helping students achieve by helping teachers excel

Developing and Revising Assessment Test Blueprint created by curriculum supervisors and teacher leaders* to include standards and number of items tested Curriculum supervisor and teacher leaders write and/or select items and prepare assessment for field testing Assessment is transferred to online portal (Achievement Series) Items or whole test is field tested during the year Good items are kept & Poor items are revised or deleted Revised items are stored for current or later use. *Teacher leaders are recommended and approved in conjunction with CTA.

How will common exams be created and the autonomy for teacher decision making be maintained? In most courses a common semester exam would be expected and the norm. In cases where no prior exam existed, we created exams for the courses. – We created ROTC exams. In other cases, the design of the course called for flexibility in pacing and autonomous decision making by the teacher. – We worked with IB teachers in order to find some common ground. In some courses a common exam was easily agreed upon. In some courses where pacing was varied, we agreed upon a common set of questions per topic administered by all in the course of the year. Helping students achieve by helping teachers excel

How can multiple response formats be incorporated? The majority of exams are multiple choice and scanned into our Achievement Series database. Some assessments are open response (Writing). – Common scoring rubric – Scorers trained and calibrated yearly on the rubric – Teachers do not score the papers of their own students Some assessments will be gridded response. – One example is IB Math Analysis Teacher had an excellent open response exam. In order to be as objective as possible, exam utilizes a gridded response that can be evaluated off site. Helping students achieve by helping teachers excel

Test Development Process Curriculum supervisors select teachers to work on each test – the number of teachers varies based on the overall student enrollment. The content supervisors are responsible for content validity. The assessment department is responsible for item quality and overall test quality. Current course teachers develop a test blueprint based on the course objectives and teacher consensus. The weight of objectives or content on the test is based on the time that content is addressed in the curriculum. This serves as a proxy for importance. The blueprint includes content and Webb’s cognitive complexity. We try to focus on the culminating skills for each semester.

Test Development Teacher training in item writing We generally require the teachers to submit items for review as part of the training process. Frequently teachers who are excellent in instructional delivery are not necessarily good at item creation. Challenges Reliable Platform with authoring capabilities Method of scoring – timelines Administering and scanning correctly/ Support staff in place during testing Tech support Maintaining test security Printing costs Writing items that measure higher levels of cognitive skills Endless process Helping students achieve by helping teachers excel

When students succeed, we all win.