1. Can student attitudes be used as predictors of success in problem solving? Karen Cummings Southern Connecticut State University New Haven, CT

2. Outline Existing work on measuring student attitudes/beliefs/expectations in physics My recent work in measuring student attitudes/beliefs/expectations in regard to problem solving Teaching problem solving in student centered, conceptually focused courses.

3. Problem Solving Attitudes Survey Maryland Physics Expectations Survey (MPEX) like scoring philosophy and administration. Some questions were taken from the MPEX (and modified or not).

4. Maryland Physics Expectation Survey Work done by Richard Steinberg, Joe Redish and Jeff Saul (U Maryland) Likert scale survey. Statements are made. Respondents “strong agree” to “strongly disagree” with the statement. Several “clusters” of questions that probe student attitudes, beliefs and expectations in regard to learning physics.

5. Beliefs about learning physics: " Understanding" physics basically means being able to recall something you've read or been shown. Beliefs about the content of physics knowledge: The most crucial thing in solving a physics problem is finding the right equation to use. Beliefs about the structure of physics knowledge: A significant problem in this course will be being able to memorize all the information I need to know. Beliefs about the connection of physics to reality: To understand physics, I expect to think about my personal experiences and relate them to the topic being analyzed Beliefs about the role of mathematics in physics: All I learn from a derivation is that the formula obtained is valid and that it is OK to use it in problems.

6. “Testing” procedure “Right” or “Favorable” answers are determined by giving a survey to a group of “experts” Students are given the survey pre and/or post-instruction Motion of student responses toward those given by the experts is seen as progress.

7. Beliefs about learning physics: " Understanding" physics basically means being able to recall something you've read or been shown. Strongly agree Agree Neutral Disagree Strongly Disagree

8. General Trend:

9. The Problem Solving Attitudes Survey Part of a project to develop a measure of student quantitative problem solving ability Based on the assumption that “expert” introductory physics problem solvers share characteristic attitudes.

10. Markers of Expertise in Quantitative Problem Solving Expertise is characterized by more than just the ability to generate a correct answer. Confidence-even in the face of uncertainty Ability/tendency to evaluate progress. Well established and acknowledged processes for solving standard types of problems. Conceptual basis overrides distraction of various surface features or mathematical tasks.

11. Problem Solving Attitudes Survey MPEX like scoring philosophy and administration. Some questions were taken from the MPEX (and modified or not). 23 total questions. Given to PER experts, RPI students (pre- post), Southern CT and McDaniel students (post).

12. Example Expert Response Distributions (n=25) Question #4849505152535658 Agree233 2 12114 Neutral01011442 Disagree12122217199 # of comments531449611

13. The three discarded questions: 53.In solving textbook problems in introductory physics, one should NOT start out by ignoring the details presented in the problem statement. 56. In solving textbook problems in introductory physics, identifying the correct equation or equations to use is the most important part of the process. 58. In solving textbook problems in introductory physics, I often find it helpful if I can find a similar problem that is worked out in the textbook. I can then use that problem solution as a pattern for use in the solving the new problem.

14. Scoring the Survey Agree and Strongly agree answers are given a value of +1 Disagree and Strongly disagree answers are given a value of -1 All expert answers are taken to be -1 or +1.

15. The Remaining 20 questions are Very Good Discriminators Between Top 1/3 and Bottom 1/3 of Students at RPI Breaking by FMCE score, the top third was more “expert like” than the bottom third on every question. Breaking by numerical course grade, the top third was more “expert like” on 18/20 questions. The progression toward expert was bottom, middle, top in 13/20 questions with grouping by FMCE score and 10/20 questions with grouping by course grade. The 10 are a subset of the 13, so this information may be used help to weed out questions.

16. Question #67645170 Expert Top-0.28-0.550.00-0.27 middle0.02-0.280.00-0.09 bottom0.25-0.100.450.11 Difference top/bottom-0.53-0.45 -0.38 # 67. If I came up with two different approaches to a problem and they gave different answers, I would not worry about it; I would just choose the answer that seemed most reasonable. (Assuming the answer is not in the back of the book.) (MPEX) #64. Equations are not things that one understands in an intuitive sense; they are just givens that one can use to calculate numerical answers. (Modified MPEX)

17. Question #67645170 Expert Top-0.28-0.550.00-0.27 Middle0.02-0.280.00-0.09 Bottom0.25-0.100.450.11 Difference top/bottom-0.53-0.45 -0.38 # 51. In solving problems in physics, being able to handle the mathematics is the most important part of the process. #70. If I’m not sure about the right way to start a problem, I’m stuck. There is nothing I can do with that problem except to go see the teacher or a friend for help.

18. Question #61695548 Expert1.00 Top0.620.600.530.88 middle0.570.700.470.79 bottom0.430.400.210.53 Difference top/bottom0.20 0.320.35 #61. I have a general approach that I apply in solving all problems that are solvable using conservation of linear momentum. #69. To be able to use an equation in a problem (particularly in a problem that I haven't seen before), I need to know more than what each term in the equation represents. (MPEX)

19. Question #61695548 Expert1.00 Top0.620.600.530.88 middle0.570.700.470.79 bottom0.430.400.210.53 Difference top/bottom0.20 0.320.35 55. In solving textbook problems in introductory physics, I can often tell when my work and/or answer is wrong, even without looking at the answer in the back of the book or talking to someone else about it.

20. # 48. Suppose you are given two problems. One is about a box sliding down an inclined plane. There is friction between the incline and the box. The other is about a person swinging on a rope. There is air resistance between the person and the air molecules. You are told that both problems can be solved using the concept of conservation of energy. Which of the following statement do you MOST agree with? Chose only one answer. A) The two problems can be solved using very similar methods B) The two problems can be solved using somewhat similar methods C) The two problems must be solved using quite different methods D) The two problems must be solved using very different methods E) There is not enough information given to know how the problems will be solved

21. Question #61695548 Expert1.00 Top0.620.600.530.88 middle0.570.700.470.79 bottom0.430.400.210.53 Difference top/bottom0.20 0.320.35 55. In solving textbook problems in introductory physics, I can often tell when my work and/or answer is wrong, even without looking at the answer in the back of the book or talking to someone else about it.

22. Markers of Expertise in Quantitative Problem Solving Confidence-even in the face of uncertainty Ability/tendency to evaluate progress. Well established and acknowledged processes for solving standard types of problems. Conceptual basis overrides distraction of various surface features or mathematical tasks.

23. Scoring the Survey Agree and Strongly agree answers are given a value of +1 Disagree and Strongly disagree answers are given a value of -1 All expert answers are taken to be -1 or +1. Then student answers which agree with experts are awarded one point.

24. Correlation Between Survey Score andNR Exam Average (RPI)2720.28 FMCE Post-test (RPI)2720.35 Instructor Assessment of Ability (McDaniel)170.34 Instructor Assessment of Ability (SCSU)200.48 Correlation Between Final GradeNR and FMCE Post-test (RPI) for these students2720.44

26. Conclusions and Future Tasks: These questions show promise as a diagnostic tool for evaluation of intro. courses. Expert pool should be broadened to improve face validity. Number of questions can be further reduced. Survey will be combined with other types of questions already developed to produce a preliminary problem solving assessment. Then re-evaluate correlations and perform validation studies. Work funded in part by an NSF-POWRE grant.

27. Teaching Problem Solving in Student Centered Classrooms. As a man asked last night: Where does one find the time to teach problem solving when instructional time is shifted to building conceptual understanding through hands-on activities?

28. Spent time wisely- Do What Helps Teach and Model an “expert-like” approach to solving problems. Heller’s-University of Minnesota Reif’s- Carnegie Mellon GOAL (Beichner)

29. Do What Helps Teach and Model an “expert-like” approach to solving problems. In general A) Draw a Picture B) Identify what is given/ unknown, assign variable names and coordinate system C) Identify guiding concept D) Draw Diagrams/Graphs/Pictures E) Chose equations F) Solve algebraically-Use dimensional analysis to check answer G) Plug in numbers, check to reasonableness.

30. Do What Helps Teach and Model an “expert-like” approach to solving problems. Model the problem solving approach in doing 1- 2 “touch stone” problems before the homework is assigned. Give the students 1-2 similar problems to do in class as a group, before homework is assigned. Teach is there as resource for individual interventions. Collect the homework, or use a computer based homework system. Force students to do the work. “You are what you grade.”

31. Don’t Waste Time Don’t assign problems that require “tricks”. Challenge students with a few, but not all of the assigned problems. Don’t go over all the homework after it is due. If less then 25% of the class want to see a problem done, don’t do it. I don’t post solutions and the students have never complained.

32. Thank You For Coming