Challenges in Computing Education Research: What We Need to Know to Reach Computational Thinking for All Mark Guzdial School of Interactive Computing

The Two Cultures

Story Research Questions of Computing Education Story 1: Teaching Computing to Computing-related Majors Measuring knowledge, measuring naïve understanding, changing the task Bottom line: It’s much harder than we think, and we don’t really know why. Story 2: Teaching Computing to Everyone Computing for All at Georgia Tech – : One course for all – : Contextualized Computing Education Studying context and computing Evaluation Results

Computing Education Research Key Question: How do people understanding computational processes and devices; and how can we improve that understanding? As a discipline, like other domain-specific education research fields. Mathematics education, physics education, chemistry education, etc. Applying the general findings of education research, with the insights of a domain, to explain learning about the domain.

The “Geek Gene” Question What influences people’s understanding of computation and their ability to learn computation? Why do so few people find programming easy to learn, but those that do find it very easy? Hypothesis #1: Some people have the “geek gene” Hypothesis #2: Some people accidentally get the right background. Math education and “unitization”

Potential Causes for the “Geek Gene” Experiences with time, process, representation, and agency Spatial Intelligence (Shea & Lubinsky, 2001) 6

“Why Johnny Can’t Compute” How should we be teaching computing? What is a developmentally appropriate curriculum? What are the factors that influence that developmental path, e.g., culture, motivation? Multiple studies have sought to measure learning about programming in undergraduate years. Results are always less than we anticipate Yet, people seem to have significant naïve understanding of computing 7

Rainfall Problem: Elliot Soloway (1989) Students were given some form of this problem: Write a program that repeatedly reads in positive integers, until it reads the integer After seeing 99999, it should print out the average.

Results Results at Yale: CS1 Students: Only 14% correctly solved the problem (correct other than syntactic errors). CS2: Only 36% got it correct. Junior/Senior Systems Programming Course: 69% This problem has been replicated at schools like University of Michigan and Georgia Tech Always with the same results Why is this hard?

Large MIMN Studies McCracken ITiCSE Working Group 2001 “Build a calculator.” Five institutions, four countries Average score: out of 110 Lister ITiCSE Working Group 2004 Maybe the problem is design. Using Multiple Choice Questions in a multi-institutional, multi-national study.

Lister Study

Naïve Understanding of Computing Studies by Miller (1981) and Pane (2001) asked people to tell a person/computer how to solve some task. Results tell us how people think about specifying process before learning a language. 12 “Make a list of those employees who make more than 8 dollars/hr. and also are over 50 years old. List should be organized by employee name.”

Naïve Computing No one uses loops Some IF’s, never ELSE’s Preference for qualification instead of iteration “For all the last names that start with ‘G’”… No object-oriented programming There is a sense of instances with properties No notion of classes or methods. All programs from perspective of programmer, never of agent/object Heavy use of production rules, event-driven programming. 13

Replacing ELSE’s Thomas R.G. Green (1977) OPTION 1: if (value < 10) then value = value + 10; else sum = sum + value; end if. OPTION 2: if (value < 10): value = value + 10; not (value < 10): sum = sum + value; end (value < 10).

Parallelism in Commonsense Computing Suppose we sell concert tickets over the telephone in the following way – when a customer calls in and asks for a number (n) of seats, the seller 1) finds the n best seats that are available, 2) marks those n seats as unavailable, and 3) deals with payment options for the customer (e.g. getting credit or debit card number, or sending the tickets to the Will Call window for pickup). Suppose we have more than one seller working at the same time. What problems might we see, and how might we avoid those problems? (Lewandowski et al. 2007)

Results:

Summary: CompEd Research Questions for Computing Majors What is hard about programming? How do we measure student understanding about computing? Where is the challenge about our tools versus about computing? Can changes to our tools make programming more obvious? Can we build on students’ naïve understanding? 17

1961 MIT Sloan School Symposium

Learn Programming to Re-Think Process Everywhere Alan Perlis argued that computer science should be part of a liberal education. –Explicitly, he argued that all students should learn to program. Why? –Because Computer Science is the study of process. –Automated execution of process changes everything Including how we think about things we already know

“A handful of people, having no relation to the will of society, having no communication with the rest of society, will be taking decisions in secret which are going to affect our lives in the deepest sense.” The Power and Fear of Algorithms The Economist (Sept., 2007) spoke to the algorithms that control us, yet we don’t understand. Synthetic Biology, Credit Ratings, Adjustable Rate Mortgages, Search Rankings C.P. Snow foresaw this in Those who don’t understand algorithms, can’t understand how the decisions are made.

21 The challenge of teaching computing is not just about computing majors (Endy et al., Nature 2005)

Alan Kay’s Dynabook (~1970) Alan Kay (2004 ACM Turing Awardee) sees the Computer as humanity’s first metamedium A medium that can represent all other media. Programming as an important new medium The computer-as- Dynabook is for creative metamedia exploration and reading

23 Why should everyone learn computing? Perlis: To have a powerful new tool to think with. Snow: Because it’s necessary to participate and understand in the modern world. Kay: To have full expressive power with the most powerful and creative medium humans have ever invented.

24 Context and Computation Context: What students want to do with computation. An application domain. Biology: Invent new medicines. Engineering: See and test models before realizing them. Media: Manipulate sounds and images to express new thoughts in new ways.

25 Relationship between context and computing 1. A subject that intersects a context 2. Tools that can be seen from context as a lens 3. A lens that offers a new way of seeing and doing in other contexts. Computing as literacy Segregationist Integrationist Synergist Lewis & Smith, inroads, June 2005

Examples of Context and Computing Tool Model Bio2010 (US NRC) calls for programming and computational models for all Biology students. Physics teaching VPython for labs where they solve three-body problems. Lens/Literacy Model: Jeanette Wing’s “Computational Thinking” Computer science provides the tools and metaphors for understanding our world

27 Teaching Computing as Literacy Logo Boxer Idit Harel and Instructional Software Design Project (ISDP)

28 Example: Teaching Computing as Lens on Physics Bruce Sherin Emile

Implementing the Subject Model Fall 1999: All students at Georgia Tech must take a course in computer science. Considered part of General Education, like mathematics, social science, humanities… Why did Georgia Tech make that decision? Computing is important Computing was a College. Solved a problem for Engineering Made a competitive distinction for Liberal Arts

One-class CS1: Pass (A, B, or C) vs. WDF (Withdrawal, D or F) Success Rates in CS1 from Fall 1999 to Spring 2002 (Overall: 78%) Architecture46.7% Biology64.4% Economics53.5% History46.5% Management48.5% Public Policy47.9%

Contextualized Computing Education What’s going on? –Research results: Computing is “tedious, boring, irrelevant” Since Spring 2003, Georgia Tech teaches three introductory CS courses. –Based on Margolis and Fisher’s “alternative paths” Each course introduces computing using a context (examples, homework assignments, lecture discussion) relevant to majors. –Make computing relevant by teaching it in terms of what computers are good for (from the students’ perspective)

Contexts, Results, and Themes Media Computation Computing for Engineers (MATLAB) Robotics for CS1 – Institute for Personal Robotics in Education Pre CS1 –Contexts with Girl Scouts Beyond CS1 –Impact on CS2 –Gameboy programming for Computer Organization

Media Computation: Teaching in a Relevant Context Presenting CS topics with media projects and examples –Iteration as creating negative and grayscale images –Indexing in a range as removing redeye –Algorithms for blending both images and sounds –Linked lists as song fragments woven to make music –Information encodings as sound visualizations 33

Results:CS1“Media Computation” Change in Success rates in CS1 “Media Computation” from Spring 2003 to Fall 2005 (Overall 85%) Architecture46.7%85.7% Biology64.4%90.4% Economics54.5%92.0% History46.5%67.6% Management48.5%87.8% Public Policy47.9%85.4%

Voices from Media Computation Students Intl Affairs student (female): “I just wish I had more time to play around with that and make neat effects. But JES [IDE for class] will be on my computer forever, so… that’s the nice thing about this class is that you could go as deep into the homework as you wanted. So, I’d turn it in and then me and my roommate would do more after to see what we could do with it.” “I dreaded CS, but ALL of the topics thus far have been applicable to my future career (& personal) plans—there isn't anything I don't like about this class!!!" "Media Computation is a CS class but with less severity. The media part of the class is extremely visually interesting. I would only take another CS class if it were Media Computation."

36 Survey One Year Later 19% of respondents had programmed since class ended "Did the class change how you interact with computers?” 80% say “Yes” “Definitely makes me think of what is going on behind the scenes of such programs like Photoshop and Illustrator.” 'I understand technological concepts more easily now; I am more willing and able to experience new things with computers now’ 'I have learned more about the big picture behind computer science and programming. This has helped me to figure out how to use programs that I've never used before.’

Results at Gainesville College (Tew, Fowler, Guzdial, SIGCSE 2005)

Results at U. Illinois-Chicago (Sloan and Troy, SIGCSE 2008)

Introducing Computing in an Engineering Context Developed in collaboration with Civil, Mechanical, and Aerospace Engineering. Uses Engineering problems and MATLAB Covers traditional CS1 topics Among our 3 CS1’s, these are the first students to program outside of class.

Results:CS1 for Engineering

Side Trip: How about CS?

Comparing Spring 2004

A Context for CS1 for CS majors: Robotics Microsoft Research has funded the Institute for Personal Robotics in Education Tucker Balch, Deepak Kumar, Doug Blank Joint between Bryn Mawr and Georgia Tech Goal is to develop a CS1 with robotics as the context. Includes a camera and media computation functions

Some Attitudes Results All students enjoyed the robot, were comfortable with it, and found it easy to get working. Personalizing the robot improved the course, in students’ opinion. Reported that the class was about computer science Found homework challenging BMC students did more on homework “because it was cool.”

W’s vs. F’s, statistically significant

Pre-CS1: “Georgia Computes!” Use in Girl Scouts Workshops

What about beyond CS1? If students have a contextualized CS1, how do they perform in a context-free CS2? Are students as well-prepared in CS2 after a contextualized CS1? Is context useful in CS2 and beyond?

The Impact of Contextualized CS1 on CS2 enrollment CS2 Data Structures Enrollment Bryn Mawr College Data for 12 years of CS2

Including last two years (since robots) CS2 Data Structures Enrollment

50 A Contextualized CS2: MediaComp Data Structures How did the Wildebeests charge over the ridge in Disney's "The Lion King"?

Research Question: Is context still useful in a second course? Mixed model research design Interviews informing whole-class survey 11% agreed with “Working with media is a waste of time that could be used to learn the material in greater depth.” “I didn’t take this class to learn how to make pretty pictures.” A majority of the class (70%) agreed or strongly agreed that working with media makes the class more interesting. 67% of the students agreed or strongly agreed that they were really excited by at least one class project 66% reported doing extra work on projects to make the outcome look “cool.” (Yarosh and Guzdial, JERIC, Jan 2008)

Context: Programming a Gameboy Traditional Computer Organization using an imaginary processor, vs. Computer Organization using a Gameboy Not all the same topics (e.g., Gameboy graphics modes) Common focus on C programming, bit manipulations, model of the computer (including memory and I/O) Post-class survey comparison: No significant performance difference. Big motivation difference. 100% of students in Gameboy class did extra on at least one homework, because it was fun. 30% of traditional students never put any extra effort in ( p =0.046). (Tew, Dorn, Leahy, Guzdial, JERIC, 2008)

Summary: What do we know about contexts for Computing Ed? Including context has led to evaluation reports of: Increased retention (in CS1, across multiple schools, with both majors and non-CS majors, across genders and ethnicities), Some evidence of increased enrollment in CS2, and Student reports of increased time-on-task, even beyond CS1. Not all contexts work for all students. Context is about personal engagement Some students do just fine with computing Fibonacci numbers Therefore, we need to offer multiple contexts, for all Snow’s “cultures”

54 Summary: Does Contextualized Computing Lead to Literacy Models? The Computing-as-subject model is not viable for all students from a motivation perspective. Programming is detail-oriented, unforgiving. It’s hard. The Computing-as-tool model is motivating. Maybe it enhances learning of the context, too. Maybe it can be a step towards a literacy model. Computing-as-literacy model requires an investment in becoming literate. What motivates that investment?

Summary: What don't we know about context? What is a context? What do students perceive as a relevant context? When are contexts successful? For how long? For whom? When do contexts lead to valuing literacy?

56 Some PhD Student Work on Contexts Do contexts help with learning ? Over time? Allison Elliot Tew is developing a reliable and valid language- independent measure of CS1 knowledge. When the context is fixed, how do you teach computing within that context? Brian Dorn is teaching computing to graphic designers who script Adobe Photoshop What is needed for teachers to succeed with contextualized computing education? Lijun Ni is studying teachers attending our workshops

Conclusions There is a lot we don’t know about teaching computing to computing majors. Why it’s hard How to assess knowledge How to build tools that take advantage of naïve understanding Computing is important for everyone, We should aim to be able to teach computing to everyone. We don’t yet know how Contextualized Computing Education has great promise for achieving the goal of teaching everyone about computing.

58 Thank you! For more on MediaComp approach: Supported by: US National Science Foundation, Microsoft Research, Georgia Tech's College of Computing, Georgia’s Department of Education, GVU Center, Al West Fund, President's Undergraduate Research Award,Toyota Foundation

Media Computation Impact on CS2 Trial at Columbus State University Fall 2007, 3 “sections” of CS1 for Majors: Media Computation Java vs. traditional –We also taught two other CS1 sections without the Media Computation focus (using the Lewis/Loftus book) and are assessing the difference. The Media Computation class had a significantly higher completion and pass rate and appears to have a higher student satisfaction rate. We will be able to better compare the two methods after this semester. Students from both groups are taking the same CS2 class. (Wayne Summers, Dept Chair) Spring 2008: Same CS2 for all 3 sections. Results: –No difference on retention rates, midterm exam grades, final exam grades. (ACM Southeast)