1. FRAPPYs Utilizing Free Response AP Problems (Yay!)
as Formative Assessments
Welcome! Introductions...
Jason, Paul
AP Stats experience
Reading experience
AP Annual Conference • Washington, DC • June 16, 2010
Jason M. Molesky
Assessment & Accountability Coordinator
Lakeville Area Public Schools
Lakeville, MN
Paul Buckley
Department of Mathematics
Gonzaga College High School
Washington, DC

2. Ben’s Chili Bowl 1213 U. Street, NW
Jason...no matter what conference I am speaking at, I always try to leave the audience with ONE key piece of information. It’s easiest to give this up front rather than make people sit and listen for an hour...so here it is...THE MOST IMPORTANT THIN YOU”LL LEARN TODAY...

3. Paradoxical Intent
Jason: Most of us use FRQs in our class...they are an excellent source of questions...nothing we share today will be “new”. We’ll just be giving you a different approach. To start, though, suppose we didn’t want to get the biggest bang for our buck out of FRQs...how could we use them in the most ineffective manner possible? Turn to a partner and discuss. Share.

4. Learning Targets What constitutes a quality statistical response?
Session Goal: Introduce the FRAPPY process as formative assessment to strengthen student understanding and develop statistical communication skills.
Paul: Goal is to introduce a FR process based on our reading experiences that takes ordinary FRQs and incorporates them as formative assessments in the classroom. We’ll explain some background research on assessment, walk you through the process, and work through one problem together.
What constitutes a quality statistical response?
How do we utilize AP Free Response Problems as formative assessments to develop statistical understanding and effective communication skills?

5. FRAPPYs
Part 1:
Background Information
Formative Assessment

6. AP Statistics Free Response Questions
Paul: How does CB define the discipline of “statistics”? Key difference between stats and other mathematics courses is the communication component.
Jason: What are students expected to do with their knowledge of statistics? Boils down to conceptual understanding and clear communication.
Clear Communication
Statistical Knowledge

7. The Purpose of The FRAPPY
Important Note #1: We over-TEST our kids.
We under-ASSESS them.
Important Note #2: AP Statistics is NOT a test- preparation course.
Our goal is to teach students statistical concepts, how to apply them, and how to communicate statistically.
The primary intent of the FRAPPY process is NOT test preparation...it is a tool to help develop statistical understanding and communication skills.
Jason: Based on this, there are two important notes we must address. 1) We over test our kids (relate back to ineffective use of FRQs) and 2) AP Stats is NOT a test prep course.
Paul: The primary intent of the FRAPPY process is to develop the conceptual understanding and communication skills noted in the CB definition.

8. Attributes of Statistical Proficiency
Procedural Fluency
Adaptive Reasoning
Jason: What does Statistical Proficiency look like? Conceptual Understanding - Comprehension of mathematical concepts, operations, and relations.
Procedural Fluency - Skill in carrying out procedures flexibly, accurately, efficiently, and appropriately.
Strategic Competence - Ability to formulate, represent, and solve mathematical problems.
Adaptive Reasoning - Capacity for logical thought, reflection, explanation, and justification.
Productive Disposition - Habitual inclination to see mathematics as sensible, useful, and worthwhile.
Our goal is to help students pull this all together...
Strategic Competence
Conceptual Understanding
Productive Disposition
National Research Council. (2001). Adding it up: Helping children learn mathematics. J Kilpatrick, J. Swafford, and B. Findell (Eds.). Mathematics Learning Study Committee, Center for Education. Washington, DC: National Academy Press.

9. Reasoning and Sense Making
“Reasoning and Sense Making are the foundations for the processes of mathematics...”
“...teachers must judiciously select tasks that require [students] to figure things out for themselves and ask probing questions.”
Paul: NCTM also notes that Reasoning and Sense Making are key components of mathematical proficiency. If our use of assessments focuses solely on computation and basic interpretation, we are falling short. We must provide opportunities for students to reflect on and explain their thinking. This is best achieved through effective formative assessment practices.

10. What is Formative Assessment?
How do we increase Conceptual Understanding and Communication Skills {ie. Statistical Proficiency}?
Jason: So how do we go about developing Statistical Proficiency? Quality instruction and student-involved Formative assessment. Note, formative assessment is not a “THING”...it’s a “PROCESS”...goal is to improve learning, not assign a grade, etc.
Stiggins, R., (2006) Classroom Assessment for Learning.
Assessment Training Institute, Educational Testing Service, Portland, Oregon.

11. Strategies of Assessment For Learning
Where Am I Going?
Provide Clear, Understandable Learning Targets
Use Examples and Models of Strong and Weak Work
Where Am I Now?
Offer Regular Descriptive Feedback
Teach Students to Self Assess and Set Goals
Jason: Brief overview of the 7 strategies. Note, there is nothing “new” here. The research on formative assessment suggests to maximize the power of assessments as learning tools, we must become more intentional about the student-involved application of these strategies.
How Can I Close the Gap?
Focus on ONE Learning Target
Teach Students Focused Revision
Engage Students in Self-Reflection
Chappuis, J., (2009) Seven Strategies of Assessment for Learning. Assessment Training Institute, Educational Testing Service, Portland, Oregon.

12. ...not Rocket Science
Paul: Jason tends to get all technical when he starts talking assessment...here’s what it boils down to...
The POWER is in the Intentional, Student- Involved use of FRAPPYs as Learning Tools!
Sadler, R.- Presented by Rick Stiggins Assessment Training Institute

13. FRAPPYs Part 2: The Purpose of the FRAPPY The FRAPPY Process
Turn to your partner and note ONE thing you picked up from Part 1.

14. What Have We Learned as Readers?
AP FRQ’s are QUALITY assessment items.
COLLABORATION is critical for success.
There are 4 things that will maximize a student’s score on a Free Response Question.
Proper application of a Statistical Concept(s).
CLEAR COMMUNICATION
Paul and Jason: Experience as Readers...what have we learned? Encourage folks to sign up to Read!
Process for approving a FRQ for operational exam.
Collaboration model in grading and how it can be incorporated in FRAPPY Process.
Communication items (No bald answers, must be in context, limit calculator speak, etc....)

15. The FRAPPY Process The FRAPPY Process “Intent of Question”
“Free Response AP Problem, Yay!”
1 Class Period
“Intent of Question”
Sample Student Responses
Rubric Discussion
Score and Reflect
Jason: Based on my experience as a Reader and on the research on formative assessment, it became clear I needed to think differently about how I use FRQs in my classroom. Came to the conclusion I could train my students as Readers...if I learned so much about the concepts through the Reading process, why wouldn’t I want to provide my students with the same opportunity? Overview of the Process...

16. Where Am I Going? FRAPPY Step 1 15 Minutes to work on Problem
READ the ENTIRE problem.
Determine “Intent of Question”
What are they looking for?
Formulate Response
Proper application of Statistical Concept(s)
CLEAR COMMUNICATION
Context, Context, Context!
Paul: Step 1...goal is to get students thinking about the intent of the problem and the key components that are necessary for a full response. This is equivalent to setting the “target” referenced by Stiggins, et al.

17. What Does “Good” Look Like?
FRAPPY Step 2
Discuss Intent of Question
View sample student responses
Available at AP Central
How would YOU score these?
What constitutes...
Minimal
Developing
Substantial
Complete
Jason: Step 2...intent is to familiarize students with “good” and “poor” work. What does “good” look like?

18. Where Am I Now? FRAPPY Step 3 Share Rubric/Scoring Guideline
Review Intent of Question
Identify “model solution” for each part
Score your papers with a partner
Each part receives
Essentially Correct,
Partially Correct, or
Incorrect
Assign 1-4 score using a holistic approach
Paul: Step 3...teaching students to self-assess. Why should we grade the responses when we can have the students do it? More power if students can assess, score, and reflect on their responses than simply getting the feedback from us.

19. How Can I Close the Gap? FRAPPY Step 4 Effective Feedback is critical
What made each part an E, P, or I?
What could you have done differently to increase your score?
How would you approach a similar problem in the future?
Note key items on your FRAPPY.
File for AP Review and Prep.
Jason: Feedback...this is critical. Allow time to discuss responses. At the Reading, we refer to these as “what if” questions. Frowned upon at the Reading, but they are a valuable discussion starter and reflection tool in the formative assessment process. Encourage students to mark them up and File the FRAPPYs away for review/prep in April.

20. FRQs as Formative Assessment FRQs as “Test Preparation”
When Should You FRAPPY?
FRQs as Formative Assessment
FRQs as “Test Preparation”
Jason: Note, I implied the FRAPPY would be done BEFORE April. Test Prep vs Formative Assessment. By definition, formative assessment must happen DURING the learning. So, FRQs can not be saved for the final exam preparation in April. They should be incorporated throughout the course.
Review
Review
Review
Review
Review
Teach
Teach
Teach
Teach
Teach
Review
Review
Test
Test
Test
Test
Test
Descriptive Stats
Producing Data
Probability
Inference
Exam Prep
Sep
Oct
Nov
Dec
Jan
Feb
Mar
Apr
May
FRAPPY early. FRAPPY often.

21. FRAPPYs Part 3: It’s FRAPPY Time!
Jason: Turn to partner and note ONE thing you learned from this section.
Part 3:
It’s FRAPPY Time!

22. It’s FRAPPY Time!
Paul (and Michael?): Hand out yellow sheet. Guage time to determine how long to allow for Intent and work. 5 min?
Jason: What was the intent of the question? Key elements? Correct approach, Check conditions, Correct mechanics, Correct interpretation in context

23. Complete What did other students say?
Paul: Before we grade your response...take a look at how others responded...what do you think of this one? Perfect! Must have been one of Jason’s students :)
Complete

24. Developing What did other students say?
Jason: How about this one? No indication of method used. Correct mechanics and interpretation of interval. Incorrect final conclusion. Developing.

25. What does “Good” look like?
Rubrics, student samples, etc. can be found at AP Central. I have them all organized by year and can provide after the presentation if you have a flash drive. Can not post on StatsMonkey. Note, students DO have acccess to all of these through a simple google search, so using them as summative assessments is not always the safest practice.

26. How did YOU do? FRAPPY Step 3 Score your papers with a partner
Each part receives
Essentially Correct,
Partially Correct, or
Incorrect
Assign 1-4 score using a holistic approach
What did you do well? What could use some work? How could you improve your response?
Paul: Final step...score own paper and note suggestions based on the rubric. Could include a reflection piece on the back. “When I encounter an inference problem, I must remember to...”

27. FRAPPYs
Jason: Wrapping it up...
Part 4:
Wrapping it up

28. Concluding Thoughts
Jason: Final thoughts...as noted earlier, nothing new here. If our goal is to increase statistical proficiency, intentional use of formative strategies can produce substantial learning gains. Our hope is that you will be able to incorporate the FRAPPY process throughout your year as a means to better prepare your students for success on the exam and, more importantly, to help increase their learning in your course!

29. Resources http://web.mac.com/statsmonkey/
- Past Free Response Question and Student Samples
- StatsMonkey Resource Clearinghouse, FRAPPYs
Jason: Suggested resources...
- Pearson Assessment Training Institute
- NCTM Focus in High School Mathematics, Assessment Handbook

30. Questions - Discussion