Belinda Chung, College Board Consultant

Belinda Chung, College Board Consultant
PSAT/NMSQT® Connections: Using PSAT/NMSQT Tools to Inform and Improve Teaching and Learning Belinda Chung, College Board Consultant

Today’s agenda January 15, 2011
PSAT/NMSQT Overview PSAT/NMSQT Score Report Plus AP Potential Sample Questions Work SOAS Report Overview SOAS Group Work Group Insights/Recommendations & Action Plan Evaluation

Mission The College Board's mission is to connect students to college success and opportunity. We are a not-for-profit membership organization committed to excellence and equity in education.

PSAT/NMSQT Overview

PSAT/NMSQT Overview Performance compared to the state and nation
Content and Question Types Performance compared to the state and nation Question-by-question analysis Early wake-up call 3.57 million students (2010) 55.5% of test takers are pre-juniors (2010) Utilize tools to help you when Understanding by Design (backwards planning)

What is the PSAT/NMSQT? A comprehensive academic reasoning test, which focuses on skills that all students need to succeed in high school and college. These reasoning skills focus on the three main academic areas, but their attainment affects all areas of learning, as they are transferable process skills that are needed to manage and apply knowledge, regardless of content orientation. These skills are relevant to all students, and are needed by all students, thus the strong focus on inclusion when testing with the PSAT/NMSQT.

PSAT/NMSQT Overview The PSAT/NMSQT is a school-based test
Comprised of five sections: 2 Critical Reading 2 Mathematics 1 Writing Skills Critical Reading, Mathematics and Writing Skills scores are on a scale Unlike the SAT, the PSAT/NMSQT has no official essay, because it would be unfeasible to administer and score affordably. A comprehensive academic reasoning test, which focuses on skills that all students need to succeed in high school and college. These reasoning skills focus on the three main academic areas, but their attainment affects all areas of learning, as they are transferable process skills that are needed to manage and apply knowledge, regardless of content orientation. These skills are relevant to all students, and are needed by all students, thus the strong focus on inclusion when testing with the PSAT/NMSQT. -The PSAT/NMSQT provides schools with materials to conduct a practice essay, which can be administered and scored locally. This program is called ScoreWrite: A Guide to Preparing for the SAT Essay.

PSAT/NMSQT Overview Content and Question Types
Critical Reading Two 25-minute sections 48 questions total : sentence completion passage based reading 50 mins Mathematics Two 25-minute sections 38 questions total: multiple choice student produced response Writing Skills One 30-minute section 39 questions total: improving sentences identifying sentence errors improving paragraphs 30 mins

PSAT/NMSQT Overview Content and Question Types
Performance compared to the state and nation Question-by-question analysis Early wake-up call 3.57 million students (2010) 55.5% of test takers are pre-juniors (2010) Utilize tools to help you when Understanding by Design (backwards planning) Of those 55.5% pre-juniors, over 1.5 million were sophomores; but the pre-juniors also include hundreds of thousands of 9th graders, and tens of thousands of middle school students. Note to presenter: add local data to show how the school or district compares with the national percentage of PSAT/NMSQT takers who were pre-juniors.

Average Test Scores For Juniors
ADD LOCAL DATA **[Local data must be added before using this chart]** Notes to presenter: Include participation statistics when comparing scores, such as the % of test takers below each designation of district/state. For discussion: Does the performance of the local group reflect state and national data? How does the data for juniors compare with data for sophomores?]

Average Test Scores For Sophomores
ADD LOCAL DATA **[Local data must be added before using this chart. Include freshman and change chart title if relevant.]** Note to presenter: For discussion: How does the data for juniors compare with data for sophomores?

Critical Reading Overview

Student’s ability to read critically
PSAT/NMSQT Overview Critical Reading Questions focus on: Student’s ability to read critically Student’s ability to think logically Student’s ability to analyze Student’s ability to evaluate

PSAT/NMSQT Overview 5 skills Determining the Meaning of Words
Critical Reading Critical Reading Two 25-minute sections 48 questions total sentence completion passage based reading 50 mins 5 skills Determining the Meaning of Words Author’s Craft Reasoning and Inference Organization and Ideas Understanding Literary Elements

PSAT/NMSQT Overview Sentence Completion 5. Because their behavior was , Frank and Susan served as models for the children under their excellent care. (A) incorruptible . . pernicious (B) lamentable . . flawed (C) commendable . . exemplary (D) erratic . . unimpeachable (E) reputable . . imperfect Note to presenter: Form W, Question 5

PSAT/NMSQT Overview Sentence Completion 5. Because their behavior was , Frank and Susan served as models for the children under their excellent care. (A) incorruptible . . pernicious (B) lamentable . . flawed (C) commendable . . exemplary (D) erratic . . unimpeachable (E) reputable . . imperfect Note to presenter: Form W, Question 5 Difficulty Level = Easy Skill = Determining the Meaning of Words

PSAT/NMSQT Overview Critical Reading Sample Skill Determining the Meaning of Words Use vocabulary skills, context, roots, prefixes, and suffixes to determine the meaning of words.

PSAT/NMSQT Overview Passage: Passage Based Reading
Line1 Advertisers are interested in finding out what exactly makes pop-up ads on the Internet irritating to viewers. Why are these ads seen as intrusive? One 2002 marketing study determined that when Internet users are focused, Line 5 they perceive interruptions as more severe than when they are not focused. What seems to get the attention of viewers (rather than merely annoy them) is to expose them to pop- up ads only at breaks in content—for example, when they are switching between pages. Another marketing strategy Line 10 is to increase the relevance of the ad. If a consumer is browsing an automobile Web site, a pop-up ad for a car will seem less intrusive than will a travel ad. Note to presenter: Form W, Question 31. Pause to allow time to read the text, or read aloud.

PSAT/NMSQT Overview Passage Based Reading The primary function of the sentence in lines (“One not focused”) is to (A) paraphrase the question posed in line 3 (B) present a relevant research finding (C) hypothesize about a solution to a problem (D) shift the focus to a different medium (E) address a concern of Internet users Note to presenter: Form W, Question 31

PSAT/NMSQT Overview Passage Based Reading The primary function of the sentence in lines (“One not focused”) is to (A) paraphrase the question posed in line 3 (B) present a relevant research finding (C) hypothesize about a solution to a problem (D) shift the focus to a different medium (E) address a concern of Internet users Note to presenter: Form W, Question 31 Difficulty Level = Medium Skill = Organization and Ideas

PSAT/NMSQT Overview Critical Reading Sample Skill Organization & Ideas Understand the organization of a reading passage, and identify the main and supporting ideas.

Mathematics Overview

PSAT/NMSQT Overview Mathematics The primary aim of the math section is to assess how well students understand and reason within a math framework: Can they apply what they already know to new situations, i.e. transfer their learning and knowledge? Can they reason and problem-solve within a high school math context? It is recommended that students use a scientific or graphing calculator in the math section. Page 17 of the Official Student Guide gives students full information on calculator use.

PSAT/NMSQT Overview 4 Content Skills 5 Process Skills
Mathematics Mathematics Two 25-minute sections 38 questions total multiple choice student produced response 50 mins 4 Content Skills Number and Operations Algebra and Functions Geometry & Measurement Data, Statistics, & Probability 5 Process Skills Problem Solving Representation Reasoning Connections Communication

Question 30 (Student–Produced Response)
PSAT/NMSQT Overview Mathematics Question 30 (Student–Produced Response) Each year from January 1 through January 8, the number of daily customers c at a music store is estimated by c = x, where x is the day of the month. How many more customers are estimated to be at the music store on January 5 than on January 1? Note to presenter: Form W, Question 30. The PSAT/NMSQT also gives students practice entering their answers on the grid and that’s good preparation so they’ll feel familiar with these types of questions on the SAT. Students should be encouraged to review the instructions for gridding in their answers for Student Produced Response questions ahead of time. These instructions can be found in the Official Student Guide. Note that gridding answers incorrectly is one of the most common careless errors on the test, so students should double check answers on these questions. (One common problem is gridding two ovals in the same column. And whether the correct answer is written in the spaces or not, it is what is gridded that determines if it is correct or wrong.)

Question 30 (Student–Produced Response)
PSAT/NMSQT Overview Mathematics Question 30 (Student–Produced Response) Each year from January 1 through January 8, the number of daily customers c at a music store is estimated by c = x, where x is the day of the month. How many more customers are estimated to be at the music store on January 5 than on January 1? The correct answer is 30. For January 1, the day of the month, x, is equal to 1, and so the estimated number of customers on January 1 is c = = For January 5, the day of the month, x, is equal to 5, and so the estimated number of customers on January 5 is c = = Therefore, there are estimated to be 1232–1202 = 30 more customers at the music store on January 5 than on January 1. Note to presenter: Form W, Question 30. The PSAT/NMSQT also gives students practice entering their answers on the grid and that’s good preparation so they’ll feel familiar with these types of questions on the SAT. Students should be encouraged to review the instructions for gridding in their answers for Student Produced Response questions ahead of time. These instructions can be found in the Official Student Guide. Note that gridding answers incorrectly is one of the most common careless errors on the test, so students should double check answers on these questions. (One common problem is gridding two ovals in the same column. And whether the correct answer is written in the spaces or not, it is what is gridded that determines if it is correct or wrong.) Level of Difficulty = (M) Skill(s) = Algebra and Functions Problem Solving

PSAT/NMSQT Overview Student-Produced Response
Algebra and Functions Solve problems using algebraic expressions and symbols to represent relationships, patterns and functions of different types. Problem Solving Solve abstract and practical problems, applying and adapting a variety of strategies. Monitor progress and evaluate answers in terms of questions asked.

Writing Skills Overview

PSAT/NMSQT Overview Writing Skills Writing Skills questions test students’ ability to edit written work and find the most effective means to communicate a thought.

PSAT/NMSQT Overview 5 Skills Writing Skills
Writing Skills One 30-minute section 39 questions total: improving sentences identifying sentence errors improving paragraphs 30 mins 5 Skills Manage Word Choice and Grammatical Relationships between Words Manage Grammatical Structures Used to Modify or Compare Manage Phrases and Clauses in a Sentence Recognize Correctly Formed Sentences Manage Order and Relationships of Sentences and Paragraphs

PSAT/NMSQT Overview Identifying Sentence Errors Understanding the difference between criticism and insult enhances one’s ability to engage in constructive argument. No error A B C D Note to presenter: Form W, Section 5, Question 22 E

PSAT/NMSQT Overview Identifying Sentence Errors Understanding the difference between criticism and insult enhances one’s ability to engage in constructive argument. No error A B C D Note to presenter: Form W, Section 5, Question 22 E Level of Difficulty = (E) Skill = Correctly formed sentences

Correctly Formed Sentences Recognize correct sentence structure.
PSAT/NMSQT Overview Identifying Sentence Errors Correctly Formed Sentences Recognize correct sentence structure.

Paper Score Report Plus
Student Feedback: Paper Score Report Plus & My College QuickStart

Score Report Plus Scores and percentiles
Personalized feedback on skills Student answers Next Steps Note to presenter: Remind participants that the content on the student score report is also presented in other reports; e.g. My College QuickStart, Student Data File, and SOAS reports.

My College QuickStart™
My College QuickStart is your students’ online post-test feedback. It includes five sections. A student enters a section by clicking on the section box. The five sections are: My Online Score Report My SAT Study Plan My Personality Major and Career Matches … and My College Matches The sections are numbered to provide guidance to students. The numbers suggest a sequence that presents the information in a meaningful flow – however, following the numbers is not critical.

Signing in Students use the access code on their PSAT/NMSQT® paper score report to sign in at To view My College QuickStart, students sign in at Each student will need a College Board account and an access code to sign in to My College QuickStart. It only takes a student a couple of minutes to create a College Board account if he or she does not already have one. A student’s access code is provided in two locations on the inside of a student’s paper score report.

My Online Score Report Look Beyond Scores
Performance on each skill measured State and national percentiles Projected SAT® score ranges Once the student is signed in, a click on Section 1 opens My Online Score Report, which lets students see his/her own test results. Students can see how they performed on each skill tested by the PSAT. These are the same skills tested on the SAT. In addition, students can compare their performance to state and national percentiles, and view their projected SAT score ranges.

My Online Score Report Your Answers Questions and answer explanations
Students can filter questions In the Your Answers section of the Online Score Report, students can see how they performed on each question and learn from their mistakes by reviewing the questions along with explanations for the available answer choices. The passage-based reading questions also include the relevant passages. Students can sort the PSAT questions by those questions they omitted or answered incorrectly, by the question’s skill or level of difficulty.

My SAT Study Plan Prepare for the SAT® Personalized skills to improve
SAT practice questions An official SAT practice test Section 2, My SAT Study Plan, creates a customized SAT study plan for each student based on the student’s PSAT results.

My SAT Study Plan Review and Practice
Review PSAT/NMSQT® questions missed Answer SAT ® practice questions of the same type Students can begin their individual study plans by reviewing the PSAT questions they missed or omitted. Then they can try SAT practice questions that measure the same skills.

My SAT Study Plan My Skills Feedback Based on PSAT/NMSQT results
Hundreds of practice questions, organized by skill “Skills not scores” has been a major focus of the PSAT. In My SAT Study Plan, students will find hundreds of practice questions, organized by skill so students can focus on the skills that need the most work. They can also view questions they answered incorrectly or omitted.

My SAT Study Plan Warm Up for the SAT® Essay Strategies for success
Official scoring guide Practice essay questions Sample essays for the each score 1-6 Students also get information about managing the SAT Essay. Students learn strategies for success, view the official scoring guide, read practice essay questions, and see scored sample essays at each score level, 1 – 6.

Express SAT® Sign-Up A final advantage to the My SAT Study Plan is that students who took the PSAT are pre-registered for the SAT. Using Express SAT Sign-Up, students can OK the pre-filled forms, choose their preferred test dates, and submit their registrations.

My Personality My Personality is a section that helps students better understand their own strengths, interests, and preferences.

My Personality My Personality uses the O-R-A Personality Profiler which is a research-validated assessment tool based on the works of Carl Jung, Gordon Allport, and David Saunders. Here are some sample questions.

My Personality The O-R-A Profiler provides feedback to students about their strengths, motivations, optimal learning environments, and social interactions. Students will be able to determine their personality type and then explore majors and careers that match their type and their interests.

My Major & Career Matches
Connect to the Possibilities Major a student chose Related majors Related careers The 4th section of My College Quick Start covers major and career matches and connects students to MyRoad –an interactive website that lets them explore major and career profiles, read firsthand accounts from students and professionals, and learn about required skills and academic preparations.

My Major & Career Matches
Research Majors Description of majors Courses students should take in high school to prepare Courses they will take in college Jobs students can expect achieving different degree levels For each major, students can find a description, courses to take in high school to prepare, courses to take in college, and jobs students can expect achieving different degree levels in that major.

My College Matches Start the College Search Starter list of colleges
Criteria to customize search Ability to save searches The last section of My College QuickStart is My College Matches. This section provides students with a starter list of colleges that match each student’s preferred major in his/her home state. Students may then take that list and tailor it by refining the search criteria.

My College Matches Get the Facts Admissions Cost and financial aid
Majors offered Sports, housing and more College profiles provide information about college admissions criteria, costs and financial aid, dates and deadlines, athletics and more.

My College QuickStart Tips
Arrange for a computer lab session to introduce students to My College QuickStart TIP Use free lesson plans to guide students through structured activities My College QuickStart is a valuable part of the PSAT and makes the PSAT more than just a one-day experience in October. Through My College QuickStart, students can better understand their test results, prepare for the SAT, learn about themselves, their preferences, and their interests, and connect course-taking with their future career plans. Students will continue to have access to My College QuickStart throughout high school. Introducing the My College QuickStart tools and encouraging students to use them do become items on a Counselor’s To-Do list. In support of your work, The College Board has prepared both tips and resources for you. It is recommended, when possible, that students be introduced to MCQS in a computer lab session. It helps students get started and ensures that all have an opportunity to get online. Students will need to have their paper score reports on hand because they will need their access codes. Counselors can also facilitate their students’ use of My College QuickStart by taking advantage of the prepared lesson plans that guide students through structured practice and planning activities. Other free resources include letters to parents and an implementation guide. Counselors have access to a My CollegeQuickStart interactive demo account which is available on the My College QuickStart resources page at TIP Check out the demo for educators

AP Potential

What is AP Potential? A web-based tool
Free for schools that administer the PSAT/NMSQT ® Provided by the maker of the AP ® Exams Designed to ensure that no student is overlooked Based on proven research What is AP Potential? It is an easy-to-use web-based tool, available exclusively from The College Board, maker of the AP exams. AP Potential generates rosters of students likely to score a 3 or better on a given AP exam – in addition, it generates rosters for 25 different AP courses. AP Potential is free for all schools that administer the PSAT. AP Potential promotes equity because it is designed to ensure that no student with the potential to succeed is overlooked. Finally, AP Potential was created from research studies that found strong correlations between PSAT scores and AP exam results.

AP Potential Identifies “diamond-in-the-rough” students
Promotes equity Helps identify students who initially might have been overlooked for AP courses Is a useful tool for principals, teachers, and counselors to Expand AP programs Increase enrollment in current AP courses

The PSAT/NMSQT and AP Connection
Research shows a strong relationship between PSAT/NMSQT and AP Studies conducted in 1998 and 2006 Analyzed performance of more than a million students Showed strong correlations between PSAT/NMSQT® scores and AP® Exam results Download the research reports at Note to presenter: Refer participants to the document included on the workshop CD, The Relationship Between PSAT/NMSQT Scores and AP Examination Grades: A Follow-up Study, College Board Report No , Maureen Ewing, Wayne J. Camara and Roger Millsap. This is the newest research that follows up on the original 1998 study.

Traditional Methods of Identifying Students for AP Courses
Teacher recommendations Self-nomination History of courses completed Student grades Student discussion Note to presenter: Ask attendees about their current methods of identifying/recruiting students for AP. What are the shortcomings of these approaches?

Access to AP is Important
AP access is linked to success in college. Rigorous high school curriculum is key Participation in AP is linked to college completion. It is a stronger indicator than socioeconomic status and GPA. Note to presenter: You may wish to refer to the following resources; -Answers in the Toolbox, 1999 U.S. Department of Education Study -College Persistence, graduation, and Remediation. College Board Research Note RN-19: Camara, Wayne, 2003.

Signing in to AP Potential™ : www.collegeboard.com/appotential
To use AP Potential, first go to and sign in using your College Board user name and password. If you don’t have a College Board account, you must create one.

Signing in to AP Potential www.collegeboard.com/appotential
The next step is to enter your school’s AP Potential access code. Access codes are on the top right corner of the Roster of Student Scores and Plans. Your roster is delivered in December with your score reports.

AP Potential Expectancy Tables
See data from the correlation study showing the percentage of students at any given PSAT/NMSQT score range. Here is a little bit of background on the AP Potential research study - To view data from the 2006 correlation study, open Expectancy Tables from the AP Potential menu on the left. There you can see, for each of the 25 different AP courses, the number of students in the sample size and the individual PSAT skill or skill combination used in the correlation study. In Biology, the preferred skill combination is academic preparation in Critical Reading and Math. Thus, a student who scored a 50 in Critical Reading and a 66 in Math has a 77.4% probability (based on the correlation tables) of earning a 3or better on the AP Biology Exam.

AP Potential Select PSAT/NMSQT® Administration Year
Step 1 of 4 Select the PSAT/NMSQT ® administration year for which you’d like to generate a roster. The Expectancy Tables can provide some understanding about how AP Potential works. Now, let’s follow each step. Step 1: Select the administration year for which you’d like to generate rosters. You may select data from 2008, 2009, or Besides generating rosters, this data storeroom allows you to see academic growth and skill development for an individual student or for groups of students.

AP Potential Select Subjects
Step 2 of 4 Select the AP subject areas your school offers. Next, select AP subject areas. Some educators start by looking for potential students course by course for the AP courses their schools offer. You can also look at AP courses their schools do not offer but may consider adding to the curriculum if there’s a critical mass of students who possess the academic skills for those courses.

AP Potential Select Pool
Step 3 of 4 Choose a grade level and define your pool. Step 3 is to choose a grade level and define the pool. If you choose “All Grades,” you generate a roster of every student who tested, including students in grades other than 9th, 10th, or 11th. If you are missing a student in a given grade, check in “All Grades.” It is possible that the student failed to enter a grade on his or her answer sheet. A pool reflects the probability of students receiving an AP Exam grade of 3, 4, or 5. For example, if you set your pool at 61, your roster will include all students whose probability of receiving an AP Exam grade of 3, 4, or 5 is anywhere between 61% and 100%. You may adjust the percentages to increase or decrease the number of students included on your roster of potential AP students. You may select a different percentage for each AP subject. Two important considerations: First, there is a default percentage for each course. Because the pool is a sliding scale, the default is simply a place to begin – a percentage which you can increase or decrease. The default percentage is not a percentage recommended by the College Board. It is nothing more than a mid-point, a place to begin. Second, AP Potential was created to facilitate opening access to AP courses – particularly to those students who for whatever reason have been traditionally underrepresented. Setting a pool at a lower percentage casts a wider net in looking for students who might be ready for more academic rigor and helps to begin conversations with students who never saw themselves in an AP class.

AP Potential Student Roster
Step 4 of 4 Read the instructions for using the roster and see your list of students at the bottom of the screen. Step 4 which is the Student Roster step reveals a page with many options for viewing and sorting your data. The initial roster will have all students who tested. The “Y” beside the name indicated the student is in the pool you defined. A click on a student name will allow you to view student details. A click on the subject or course name will allow you to view the student list for that subject.

AP Potential Subject Detail
Student names Correlations and Student scores Ethnicity Gender The subject detail page can be sorted 4 different ways: by student name; by correlations and student scores; by ethnicity; and/or by gender. You may again, click on any student name on the roster to reveal individual student detail. Please be mindful of this caution: “Results represent probabilities, not absolutes. User should not use this product as a means to exclude students from courses.” The College Board also recommends that AP Potential never be used as the sole consideration for AP course placement.

AP Potential Student Detail
All courses selected noted Status of meeting the criteria Probability for scoring a 3 noted The student detail includes date of birth, ethnicity, current grade, and gender. His or her PSAT scores for a specific administration year are shown. Also included is his or her status of meeting the pool criteria as well as the individual probability for scoring a 3 or better. This AP Potential page can be printed and shared with a student.

AP Potential Tips TIP Export data into an Excel spreadsheet to make it easy to read and use. TIP Create letters to parents. AP Potential™ provides sample letters in English and Spanish. There are additional resources on the Step 4 page. First, you can export the roster data onto an EXCEL spreadsheet to make the information easy to read and to use. You can delete columns that you think are unnecessary or ad ones to personalize the data to your school. Second, there is a sample letter which you can use to inform parents about their students potential for success in AP classes. The sample letter is available in English or Spanish. You may, of course, personalize the tone and the details of your letter. AP Potential is a valuable resource in opening options for students and helping them better connect the choices they are making today with their plans and hopes for the future.

AP Potential Tips After considering potential AP students using your school’s standard process, check AP Potential for additional students that might have been overlooked. Use AP Potential to make a case for professional development - more teachers will need to be trained to handle an increased number of AP courses. Review AP Potential for 9th and 10th graders in preparation for 11th or 12th grade AP courses and ensure that they are taking the correct preparatory courses

Activity A: Getting to know the Test
Attendees should refer to the activities section of their participant’s manual. This activity will allow workshop attendees to better understand: The content assessed by the PSAT/NMSQT See the academic reasoning skills necessary to answer test questions Assess how the content of the test aligns with the skills of their students

SOAS Introduction Audience Administrators Department heads
Curriculum specialists Teachers Other staff interested in students’ college preparatory skills Note: the more inclusive the testing within each grade, the more meaningful and applicable the results. The SOAS information is useful to many educational professionals in your building. Certainly, administrators, department chairpersons, curriculum specialists, and teachers will be interested in the skill analyses provided in the reports. As integral contributors to the teaching and learning community, we counselors are often the professionals who introduce the SOAS to our colleagues. In addition, we can link conversations about school-wide learning goals to student performance gains as demonstrated by the PSAT. A Tip: Inclusive testing gives the best results. The more student data you provide, the more informative your feedback with be. The SOAS reports are grade specific; therefore the reports are listed as Class of 2011, Class of 2012, etc. You may also view reports by multiple school terms. An important footnote for these reports – they are generated only if you had 25 or more students test within a grade level.

Instructions: Part I, Solo Activity (20 minutes)
Activity A Review the PSAT/NMSQT skills list in your binder in order to become familiar with the skills for your academic area. You will have 20 minutes to take sample PSAT/NMSQT Mark your answers in the exam booklet Immediately after answering a question, refer to the List of Academic Skills list and, in the test book next to the question number, identify the skill group(s) that you feel is needed to answer the question. Participants should review the Skills list (in binder) prior to taking the test to familiarize themselves with the academic skills contained on the PSAT/NMSQT.

Instructions: Group Activity (20 minutes)
Activity A Gather in groups by subject matter (no more than 7 per group). Each group needs a Leader and a Recorder. You will have 20 minutes to review at least 5 questions from the test-taking activity, to compare notes on the skills needed to the answer questions: English teachers, do at least 3 Critical Reading questions and 2 Writing Skills questions Math teachers, do at least 3 multiple choice questions and 2 SPR questions Take minutes to answer the following questions: The group portion of Activity A will allow participants to discuss the items they reviewed in the individual portion of the activity. This will allow participants to see how accurately they were able to identify the skills needed to answer the questions and to discuss how prepared they feel their students are for the PSAT/NMSQT.

How did the “group’s” coding match the test developer’s coding?
Discussion Topics: Activity A How would the content covered in these questions be familiar to your 10th grade students? 11th grade students? How are these test questions similar to question types you would present in class? Was there agreement in the group on the skills coding? If no, what kinds of issues caused disagreement? How do these skills compare to the skills you would expect your students to have? (10th? 11th?) How did the “group’s” coding match the test developer’s coding? After the 20 minute discussion period, the entire workshop group should reconvene and discuss each group’s answers to the questions.

The Summary of Answers and Skills (SOAS)

Logging In Repeat Users www.collegeboard.com/results
If you’ve accessed SOAS reports before, just login with your username and password to see this year’s reports at

Logging In New Users If it’s your first time, you’ll need your school’s SOAS access code If it’s your first time accessing SOAS reports, you’ll need your school’s access code- Your school access code can be found on your PSAT Roster of Student Scores and Plans which is included in your school’s score report shipment. Access codes will also be ed to your school’s PSAT Coordinator and mailed to your school principal.

Supporting Materials SOAS Resources
Before we dive into the report details, I’d like to mention several resources that are available to help you maximize use of the SOAS reports. First- from the SOAS website, educators can sign up to attend a free webinar to get a walk-through of the SOAS report. There are also several online resources available in the sidebar on the right side of the screen. The resources include a Skills Reporting Tools video, ‘how-to’ Tutorials, Test Questions and Answer Explanations in viewable and printable format, the PSAT Skills List, alignment reports linking PSAT Skills to state standards, and information on College Readiness Benchmarks. We’ll go into some of these in more detail towards the end of this section.

Your Reports Links to your school’s reports and will be available in the box once you are successfully logged in. Click on the plus sign to show all the reports available to you. Each link is a separate report for individual grade levels. Reports are provided if your school tested 25 or more students in a grade.

Report Orientation Table of Contents Page(s) Title Page
Performance Overview 1 Skills Analysis 2, 6-7, 11 Question Analysis 3-4, 8-9, 12-13 Comparable Group Analysis 5, 10, 14 Let’s go through an SOAS report. Each report is 14 pages long. It is helpful to think of one report as consisting of an introductory Performance Overview, and 3 different presentations of data: Skills Analysis, Question Analysis, and Comparable Group Analysis. The three presentations of data – Skills Analysis, Question Analysis, and Comparable Group Analysis – will be repeated for each section of the PSAT: Critical Reading, Math, and Writing.

Performance Overview Page 1 Number of students in report
Mean scores and score distribution College Readiness Benchmarks The Performance Overview shows the number of students included in the report, mean scores and score distribution, and the number of students who have met or exceeded the new College Readiness Benchmark- an indicator of whether students are on track for success in college.

Skills Analysis Pages: 2, 6-7, 11 Skills
Triangles show a comparison to the state and nation Each test question is linked to a skill The Skills Analysis tables (repeated on pages 2, 6-7, and 11) show how your school performed on each of the skills measured by the PSAT. These same skills are measured by the SAT and ReadiStep. You can see how your students’ performed compared to other students in the state and the nation. And, for each test question, you can see the percent of students who answered the question correctly, and whether the question was easy, medium, or hard.

Question Analysis Pages: 3-4, 8-9, 12-13
Student responses and answer patterns Comparisons to the state and nation The Question Analysis tables (repeated on pages 3-4, 8-9, and 12-13) show student responses and answer patterns for each test question. You can also see how your students’ performed compared to other students in the state and the nation. Correct answers are displayed in gray, and the responses for the incorrect and omitted answers and displayed to the right.

Question Analysis Tips: Look for questions your Students answered
incorrectly that the students in the state and nation answered correctly Look for commonly Wrong answers When first looking at the report, there are 2 information pieces to pay attention to: First, look for questions your students answered incorrectly that the students in the state and nation answered correctly. Second, look for commonly wrong answers. The “eyeball” technique works well because concerns about your students’ performance can readily be seen by the length of the gray bar labeled “School.” When a large percentage of your students answered the question correctly, the gray bar will be quite long. When your students had trouble with a question, it is interesting to note which “incorrect response or responses” attracted them. Notice questions 7 and 8 where only about 10-20% of the students selected the correct answer. For question 23, many students chose answer C, though B is correct. You can look at the question 23 and the skills associated with it to identify areas for improvement. The Test Questions and Answer Explanations and Skills and Alignments will be helpful supportive materials. Another way to identify problem areas is to look for questions your students got wrong but students in the state and national answered correctly. You can identify the skills measured in these questions and target these skills for improvement. There is a lot of information to be curious about on this page. How did my students do? When missing a question, did they gravitate toward the same wrong answer? Or did they not? If a large percentage of students chose the same wrong answer, what was so attractive about that answer? What questions did students omit? This opportunity to observe how students reacted to individual questions offers a unique perspective on your students’ demonstration of college readiness skills.

Comparable Group Analysis
Pages: 5, 10, 14 The third section is the Comparable Group Analysis (repeated on pages 5, 10, 14). It allows you to easily ‘eyeball’ the test questions where your students performance diverged from what was expected. The data is produced based on a statistical model called Comparable Group.

SOAS Comparable Group What is a comparable group?
A useful statistical model A statistically created group (virtual group) Mirrors your group’s performance profile This creates an expected performance indicator for your group on each question. Provides more “actionable” feedback than state or national averages on questions/skills. I’d like to take a moment to clarify what the comparable group is. It is a useful model, and is a statistically created group (a virtual group if you will) which mirrors your group’s performance profile. The comparable group is not a real school; rather it is a sample group of students across the entire test-taking population that was chosen because they performed similarly to students at your school. No demographic data is factored in - the comparable group is created solely based on student performance. The comparable group is a statistical model used to create an expected performance indicator on each question providing more actionable feedback than state or national averages on questions and or skills.

Comparable Group Analysis
Pages: 5, 10, 14 Darker blue = significantly below the comparable group. Lighter blue = significantly above the comparable group Back to the report. The table allows you to easily find test questions for further investigation. Look for questions where the dot falls in the darker blue and lighter blue shaded area. These are questions where your students’ performance diverges from what would be expected.

Resources Several resources are available to help make your report actionable so you can maximize its benefits – available online on the right side of the web page. Here you’ll find: A walk-thru video and a tutorial An interactive tool that allows you to view and print test questions and answer explanations Information on skills and alignment to state standards And test score information

Resources Test Questions and Answer Explanations
I’d like to call your attention to the Test Question and Answer Explanation tool. Reference to test question details such as the question itself, the correct answer, and the full answer explanation is especially helpful to interpreting your SOAS reports fully. To use the tool, simply click on the subject tab and check the test questions you would like to see, then click Build PDF.

How does it all add up? 3. View the question details
2. Find the skill measured by this question Identify a question for review So how can you use this information to make improvements in your classrooms? Here’s just one example of how all of the resources we’ve reviewed can be used to help you make curriculum adjustments and improve student learning. The SOAS report will help you identify a test question that your students struggled with Also look at the skill measured by this question Use the Test Question and Answer Explanations tool to review question details Look up of your state’s standards it is aligned to. Make adjustments to your classroom curriculum to improve student learning and better meet your state standards.

Other Relevant Resources: Before, During, and After SOAS review

Understanding PSAT/NMSQT Scores
PSAT score related FAQ’s Information on percentiles and mean scores Score conversions This publication is mailed with score reports in December and also available online at It is included on the participants’ CD’s.

PSAT/NMSQT Summary Reports
A school summary report will look similar to this state-level report and is a good source of information.

PSAT/NMSQT Summary Reports
Summary Reports are provided to all schools that tested at least 50 juniors or sophomores. State level summary reports are also available (College Bound Juniors/Sophomores reports) Reports contain: number of test-takers ethnic background participation in Student Search Service self-reported GPA performance by score range college major Schools that test at least 50 juniors and/or 50 sophomores receive Summary Reports in December, for juniors and sophomores showing the number of test-takers and the mean test scores, with standard deviations, for the local group, the state and the nation. The reports also provide group distributions in math, critical reading, and writing skills scores; ethnic background; college major, and other information. (**A Special Summary Report is an option available for a fee for schools that tested fewer than 50 students, or for schools with juniors and sophomores who tested at another school and need one aggregate report of their students’ data.) Review summary data compared to national and state before beginning so that there is some sense of overall comparison based purely on scores. It will give the question by question review some perspective. Note to presenter: use the next few slides to illustrate how this report can be used to compare local group to the state and the nation.

Activities

Activity B Review Skills Analysis
Instructions Work within your own school group/academic area, using your school’s SOAS report. Identify the skill that your students had the lowest performance on. Using the Test Question and Answer Explanations, look at each question aligned to that skill. Things to look for: Skills furthest below the state/national average Skills that are furthest to the left (i.e. those that are the weakest for the group). Skills that came to teachers’ attention after review of Question Analysis. Attendees should refer to the activities section of their participant’s manual. They will need a copy of their school/districts SOAS report. This activity will allow workshop attendees to: Develop strategies for working with the Skills Analysis sections of their SOAS data Identify PSAT/NMSQT skills that posed particular problems for their students Participants should be given minutes to complete this activity. On a worksheet they should record the skills they identify as problematic during their analysis and any thoughts they have on the questions for consideration (as listed on next slide).

Activity B Review Skills Analysis
Questions for Consideration Within your group, answer the following questions: Is there a pattern in the skills you identified as problematic? Are there actions or strategies that might help address the issues? Participants should be given minutes to complete Activity D. On a worksheet they should record the skills they identify as problematic during their analysis and any thoughts they have on the questions for consideration.

Activity C Review Question Analysis
Instructions Work within your own school group/academic area, using your school’s SOAS report. Refer to the Test Questions and Answer Explanations to make note of problem questions and how they might be addressed. Things to look for: Easier questions (difficulty level 1-3) where less than a 2/3 of the students answered correctly. A larger proportion of your students answered incorrectly or omitted an answer in comparison to the state/nation Commonly wrong answers Review any questions that generated teacher interest when you took the test in Activity A. Attendees should refer to the activities section of their participant’s manual. They will need a copy of their school/districts SOAS report. This activity will allow workshop attendees to: Develop strategies for working with their SOAS data Identify PSAT/NMSQT test questions that posed particular problems for their student Participants should be given 25 minutes to complete this activity. On the a worksheet they should record the items they identify as problematic during their analysis and any thoughts they have on the questions for consideration (as listed on next slide).

Activity C Review Question Analysis
Questions for Consideration Within your group, answer the following questions: Are there any patterns in the questions you identified as problematic? Are there actions or strategies that might help address the issues? Summarize your findings for sharing. Participants should be given 25 minutes to complete this activity. On the a worksheet they should record the items they identify as problematic during their analysis and any thoughts they have on the questions for consideration (as listed on next slide).

Activity D Comparable Group Analysis
Instructions Work within your own school group/academic area, using your school’s SOAS report. Using the Test Question and Answer Explanations, look at each question. Things to look for: Questions that fall into the dark blue area, where your students performed below expectation. Questions that fall into the light blue area, where your students performed higher than expectation. Attendees should refer to the activities section of their participant’s manual. They will need a copy of their school/districts SOAS report. This activity will allow workshop attendees to: Develop strategies for working with the Skills Analysis sections of their SOAS data Identify PSAT/NMSQT skills that posed particular problems for their students Participants should be given minutes to complete this activity. On the a worksheet they should record the skills they identify as problematic during their analysis and any thoughts they have on the questions for consideration (as listed on next slide).

Activity D Comparable Group Analysis
Questions for Consideration Within your group, answer the following questions: Is there a pattern in the skills aligned with the test questions your students performed below expectation on? Is there a pattern in the skills aligned with the test questions your students performed expectation on? Are there actions or strategies that might help address the issues? Participants should be given minutes to complete Activity D. On the a worksheet they should record the skills they identify as problematic during their analysis and any thoughts they have on the questions for consideration.

For Group Discussion To Help Students and Parents 1. How does the school educate parents about the wealth of information on Score Report Plus? 2. What plan does the school have in place to make sure students receive and understand their scores?

For Group Discussion Using Your SOAS Data Identify the skills associated with the questions students answered incorrectly. Make inferences about what led to errors. Identify where in the curriculum the skills are (or are not) taught. Work with departments and classrooms to review common concerns. What recommendations can you make to your school for increased student success?

Closing Thoughts

Using Assessments for School Improvement
Identify skills to help guide instruction and curriculum planning Connect more students with challenging high school courses and college attendance Increase Advanced Placement Program courses in your school system; more professional development for teachers Close the achievement gap in your community

The Excellence and Equity Challenge
“Without preparation, opportunity is an empty promise.” Alan Page, former NFL star, current Minnesota State Supreme Court Justice Austin American Statesman,

Looking Ahead: 2011 PSAT/NMSQT Wednesday, October 12th Saturday, October 15th
The formula for scheduling the PSAT/NMSQT: the third Saturday of October and the preceding Wednesday. Note to presenter: Poll the attendees about their October fall break. How many schools have one and is it typically during the Columbus Day week? Provide feedback to the College Board.

Belinda Chung, College Board Consultant

Similar presentations

Presentation on theme: "Belinda Chung, College Board Consultant"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Belinda Chung, College Board Consultant

Similar presentations

Presentation on theme: "Belinda Chung, College Board Consultant"— Presentation transcript:

Similar presentations

About project

Feedback