Presentation on theme: "An Overview of NAEP: The National Assessment of Educational Progress Philip Jacobs NAEP State Coordinator Tennessee"— Presentation transcript:
An Overview of NAEP: The National Assessment of Educational Progress Philip Jacobs NAEP State Coordinator Tennessee
Introduction Philip Jacobs- Tennessee NAEP State Coordinator Middle school and high school teacher for 10 years High school principal 5 years before taking this position Selected as a NAEP school in
1.What is NAEP? 2.What does NAEP look like? 3.What is the NAEP selection process? 4.How can we Improve? Questions for the Day
What is NAEP?
NAEP is the largest continuing and nationally representative assessment of what our nation’s students know and can do in core subjects Assesses Reading, Math, and Science Results are included in The Nation’s Report Card What is NAEP?
Math, Reading, and Science School letters were sent Sept 3 (hard copies), and Oct ( s) Please appoint your school coordinators and have them to register on MyNAEP Deadline is October 31 Assessment Window: January 26 – March 6, 2015 NAEP 2015 in Tennessee
State assessments are designed to show a student’s achievement and growth over time Measure performance against state standards Vary across the country Report on student, class, school, and district performance NAEP is designed to be a standard of measurement for the nation National standards and frameworks Allows comparison of state performance and policy No individual, school, or district results NAEP Differs from State Assessments
State assessments are yearly Grades K-11 NAEP is biannual Grades 4, 8, and 12 Alternating years for State samples and National samples This year, grade 12 is a national sample 321 Schools 87 Districts NAEP Differs from State Assessments
State assessments test all students in a grade NAEP tests a representative sample, like Gallup polling, QC in factories, or blood tests Race and Ethnicity Socio-economic status Urban/ Rural/ Suburban Tennessee has 321 school participating this year About 40 students per school Over 12,000 students NAEP Differs from State Assessments
Validates our progress Serves as an independent check on TCAP improvement State pride and competitiveness State rankings Economic development Future state assessments will look more like NAEP Credibility with the USDOE We view progress on the 2015 NAEP exam as critically important
12 We literally blew away the other states when it comes to education results. -Governor Bill Haslam We literally blew away the other states when it comes to education results. -Governor Bill Haslam “Recently, the National Assessment of Educational Progress (NAEP) revealed that students in Tennessee made tremendous strides since the most in the nation.” - U.S. Chamber of Commerce Foundation “Recently, the National Assessment of Educational Progress (NAEP) revealed that students in Tennessee made tremendous strides since the most in the nation.” - U.S. Chamber of Commerce Foundation
Tennessee is now within one point of the national average on three of four tests NAEPTennesseeNational Average 4th grade math th grade reading th grade math th grade reading265266
We have set two big goals for the 2015 NAEP assessment To meet these goals would be a tremendous accomplishment for Tennessee educators Be the fastest improving state in the nation, again 2.Meet or exceed the national average in scale score
What does NAEP look like?
Frameworks developed by National Assessment Governing Board Questions go through a rigorous review process involving more than 300 people from across the nation NAEP Test Design
Percentage distribution of Math items by grade and content area Content AreaGrade 4Grade 8Grade 12 Number Properties and Operations Measurement Geometry1520 Data Analysis, Statistics, and Probability Algebra153035
50% Constructed Response 50% Multiple Choice Percent of testing time by item formats: Math
25% Low 50% Moderate 25% High Percent of Time by Complexity Level: Math
GradeLiteraryInformational Percentage distribution of literary and informational passages
GradeRange of Passage Lengths (Number of Words) 4200– –1, –1,500 Passage Length
Grade Multiple Choice Short Constructed Response Extended Constructed Response Percent of testing time by item formats: Reading
g/2013-reading-framework.html g/2013-reading-framework.html matics/2013-mathematics-framework.html matics/2013-mathematics-framework.html 12th Grade College Preparedness Video 12th Grade College Preparedness Video Framework Addresses
Achievement Level Definition AdvancedThis level signifies superior performance. ProficientThis level represents solid academic performance for each grade assessed. Students reaching this level have demonstrated competency over challenging subject matter, including subject-matter knowledge, application of such knowledge to real-world situations, and analytical skills appropriate to the subject matter. BasicThis level denotes partial mastery of prerequisite knowledge and skills that are fundamental for proficient work at each grade. Achievement Level Definitions
Schools pre-select a room large enough for the number being tested NAEP field staff conducts the test administration Everything needed comes from NAEP Students test for about 90 min One subject per student Math, Reading, or Science All tests are in the same room at the same time Results are compiled and released in the Fall What does test day look like?
Pencil and Paper (Operational) Used for results Technology-based Assessment Pilot program Tablets supplied by NAEP NAEP 2017 will be all TBA What kinds of tests are there?
Results are reported for the nation and the states States are ranked by subject and grade There are no district, school, or student- level results This frees you and your students to take risks and do your very best! Remember we were the fastest improving in the nation in 2013! Let’s keep it up! How do we know how we did?
What is the NAEP Selection Process?
NAEP puts together a representative sample of a state’s population using such demographic information as race and poverty. For state assessments, NAEP selects around 100 public schools for each subject at each grade—each school represents about 1% of that state’s public school students in the grade being assessed. The goal of NAEP sampling is for every student in the state to have roughly the same probability of being selected. This means large schools are more likely to be selected because they have a greater percentage of the needed sample for the state. But overall student probability of selection is equal Why was our district/school selected?
Identify all potential schools in each state An initial list of all public schools in the nation is compiled using the U.S. Department of Education’s most current public education system database. The national list is then divided into separate lists of schools within each state to begin the sampling process.
Classify schools into groups. Using the list, schools within each state are classified into groups, first by type of location and then by the racial/ethnic composition of the schools within those locations. This step ensures that the sampling process takes into account the distribution of schools and students across rural, suburban, and urban areas, and the diversity of the student population in each state.
Within each group, order schools by student achievement. Within each group, schools are sorted by a measure related to student achievement to ensure that schools with varying levels of student performance are represented in the NAEP sample. This is done using school-level results on state achievement tests.
Develop an ordered list for sampling The groupings of schools determined in steps 2 and 3 are then placed into a comprehensive ordered list for sampling. The probability of a school being selected for the NAEP sample is calculated based on the size of its enrollment in relation to the size of the state’s student population at the selected grade level and the number of schools needed for the assessment. Schools with large enrollments are more likely to be selected because their students represent a large proportion of the state’s student population.
Select the school sample The sample of schools is drawn for NAEP participation with a systematic sampling procedure. The procedure ensures that each school has the correct selection probability, as calculated in Step 4. By proceeding systematically throughout the entire list, schools of different sizes and varying demographics are selected and a representative sample of students in the state will be chosen for the assessment.
Confirm school eligibility The list of schools selected to participate is sent to each state department of education to verify that the schools are eligible for participation. Some factors that would make a school ineligible include school closure or the school not having students in the grade level being assessed.
Within sampled schools, select students to participate in NAEP In each sampled school, a list is compiled of all students within the grade to be assessed. From this list, a sample of students is randomly selected by NCES for participation in the assessment. Every student in a sampled school has an equal probability of being selected. After the sample is drawn, students are assigned a single subject area in which to answer questions. NAEP staff members work with the school to verify the accuracy of student demographic information.
Why are some schools selected frequently? For state assessments, NAEP selects for the sample around 100 public schools for each subject at each grade—each school represents about 1% of that state’s public school students in the grade being assessed. The goal of NAEP sampling is for every student in the state to have roughly the same probability of being selected. This means large schools are more likely to be selected because they have a greater percentage of the needed sample for the state. Overall student probability of selection is equal
Can states choose which schools are sampled for NAEP? No. NCES selects a sample of schools using the NAEP sampling process to ensure the validity of the results. States verify that the selected schools are eligible to participate. This process ensures that NAEP assesses the most representative sample of students possible.
Example State sample of 1000 Students School SizeSchool probability of selection Student probability within the school Student overall probability of selection /1000=.401/400= *.0025= /1000=.201/200=.005.2*.005= /1000=.201/200=.005.2*.005= /1000=.101/100=.01.1*.01= /1000=.051/50=.02.05*.02= /1000=.051/50=.02.05*.02=.001
NCLB requires that districts and schools that receive federal funds participate in NAEP NAEP is voluntary from a student standpoint. Parents can opt out their students Schools must notify parents of this option Please encourage full participation for accurate results Can we get out of it?
Are SWD and ELL included in the NAEP sample? Yes. Students are randomly selected from a list of all students enrolled in the grade that is to be assessed in each school. NAEP offers a comprehensive array of research-based accommodations for students with disabilities, students with 504 plans, and English language learners to participate in the assessment.
SWD ELL Section 504 Paper & Pencil and Tablet Administration All students are encouraged to take NAEP Remember no student, school, or district results Exempt only severe cognitive disabled students Accommodations
At their own pace Limit is 3 times regular session time NAEP collects data on use student use of extended time Most students who request extended time finish without using any additional time Extended Time
Tape Recording Answers Taking test over multiple days Use of abacus, arithmetic table, non-NAEP ruler, graph paper Rewording test questions Use of dictionary, thesaurus, spell-checker, grammar- checker Oral or written answers translated into English Directions, test materials, or versions in native languages other than Spanish Disallowed Accommodations
How can we Improve?
Encouragement Gain buy-in from students and parents Reward effort Have a faculty member present in the testing room Answer every question Formative Assessment Use NAEP samples as formative assessment items NAEP Questions Tool (NQT) Support Success!
Instruction Teach the Tennessee standards Continue to blend in rigorous tasks Support Success!
Formative Assessment Use NAEP samples as formative assessment items NAEP Questions Tool (NQT) Support Success!
Provides educators, researchers, parents, and students access to released questions from NAEP assessments in several content areas. Provides examples of what NAEP asks students in grades 4, 8, and 12. The NQT includes items that align with TCAP and Common Core. Great resource for ALL grades! Use the NAEP Question Tool for high-quality assessment items tn.gov/education/NAEP tn.gov/education/NAEP NAEP Question Tool (NQT)
Help us meet our statewide inclusion goals NAEP sets goals of 85 percent inclusion of students with disabilities, for each test. In the past, Tennessee did not always meet that goal. However, in 2013, we met our inclusion goals on three of the four tests, and still made significant progress. There are accommodations and accessibility features available. Accommodations on TCAP are now much more closely aligned to NAEP than in the past. 50
Help us meet our statewide inclusion goals NAEP is representative of the state If selected, SWD and ELL students will participate in NAEP with the accommodations posted on the MyNAEP website Exclusions Portfolio assessment students ELL students enrolled in US schools for less than one year 51
In 2013, students who talked about their studies at home two to three times a week scored higher on NAEP mathematics than those who did so less frequently. In 2013, 12th graders who discussed different interpretations of readings in class almost every day scored higher on NAEP reading than those who did so less frequently. In 2011, 12th graders who wrote four to five pages a week for homework scored higher on NAEP writing than those who wrote fewer pages. In 2011, eighth graders who frequently worked together on science projects scored better on NAEP science than those who did not. Actionable Insights to Share with Teachers
ASAP- Put NAEP on School Calendar October- School Coordinators register on MYNAEP November- I will send sampled grade student information to NAEP December- Field Staff will begin to contact schools January- SC sends updated student list to NAEP and receives sampled students, parents notified January to March- NAEP Window Summer- Data crunched Fall- Results released and Tennessee celebrates again! NAEP Timeline