UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing An Exploratory Study of a Novel Formative Assessment Tool to Promote Students’ Circuit Problem Solving Greg Chung American Educational Research Association Annual Meeting San Francisco, CA - April 7-11, 2006

2/∞ Learning Environment Large-class instruction Instructor marches along with Little direct feedback from students Little or no information about what students know Instructional format ill-suited to allow students to exercise their problem solving skills and test their knowledge

3/∞ Course Selection EE 10 (Circuit Analysis I) Gateway into EE discipline Challenging for students First exposure to the problems of interpretation, mathematical analysis, and design of circuit First exposure to problem solving relying on graphical, abstract description of a system

4/∞ Current Study Focus on individualized understanding and problem solving Student solve problems in real-time and present results to instructor Interaction is private and anonymous, which encourages student to interact Real-time feedback to instructor enables discovery and focus on areas of deficiency Focus on feasibility and utility Technical issues, self-reports of perceived impact

5/∞

6/∞ Student’s View

7/∞ Instructor’s View

8/∞ Results Instructor perceptions All students in session participated, drastically improved interaction Clear and immediate feedback Rate of receiving questions and observing responses to problems is much higher than conventional sessions Method exceeds interactivity of one-on-one from instructor perspective

9/∞ Overall Impression Very PositivePositiveModerateNegative Comments The section was very helpful. Because students were able to ask and answer questions anonymously, it made it easier for me to ask questions that I wouldn't normally ask (for fear of sounding stupid). I also found some students in the section very helpful. With its smaller size, the section enabled students to use each other as a resource. The process of learning seems to be slowed down a little too much. Though the discussion can be changed to fit our learning needs, every example takes too long. This make the class a little boring.

10/∞ Perceived Learning Much more effective More effectiveSimilar Less effective Much less effective Comments The sessions helped to reinforce what I had learned from lectures and the book. It was a good way to solidify any potential questions I may have had regarding specific circuits. Using the computer based tools was a nice alternative to pencil and paper or white-boarding. I think that the answer to this question is based on the type of individual. From my perspective, it is easier for me to take notes on problems and go over it at a later time, individually. I felt some pressure when solving the problems in a group setting.

11/∞ Perceived Interaction Comments I think that maintaining anonymity is very crucial in the interaction aspect of the discussion. Many, including myself, may feel a little embarrassed asking a "dumb" question but w/ this method, I don't feel that people will hesitate to ask those questions. The whole "instant messaging" system was cool, but seemed impersonal. Also, it felt intimidating to message the professor. It seemed to make more sense if we just asked the questions in person rather than messaging. Much more effective More effectiveSimilar Less effective Much less effective 49710

12/∞ Perceived Engagement Comments I was definitely more prone to sit and give my full attention in this section than I am normally in any discussion. I did not fall asleep, where normally I will doze off during normal discussion. I think it's a lot easier to pay attention because I feel like I actually have to do the problem myself, rather than sit back and let some brainy kid figure it out for me, like I will tend to do when I feel lazy normally. Whenever we were assigned a problem to do, I always ended up taking out a piece of paper and pencil to write out the problem. Having the problem on the computer made it harder to see the whole problem because the screen was too small to fit the problem into the screen. Much more effective More effectiveSimilar Less effective Much less effective

13/∞ Perceived Impact Learning of specific analysis techniques Development of circuit analysis problem solving skills Completing homework assignments22574 Performance on exams42761 Performance on the course34931 Your improvement on attitudes toward electrical engineering Your attitudes toward EE10/EE115A in particular Your attitudes toward engineering in general Your making new friends/networking = computer-based section did not have much of an effect, 5 = had a very big effect

14/∞ Concluding Remarks Focus of study was on utility and feasibility Did not design study to test for instructional effects Scaling up an issue Processing 30 students’ responses was tractable … 60 students became hard Future work Experimental study to test effects on learning Automated scoring to support scaling

15/∞ next presentation ©2006 Regents of the University of California