Media computation as a context for learning computing Mark Guzdial College of Computing/GVU Georgia Institute of Technology

Story Perlis’ challenge: Computer science is more important than calculus Evidence that we’re not there yet  And what the solution might look like Our attempt: Introduction to Media Computation How this might be used/useful elsewhere

Computer science is more important than Calculus In 1961, Alan Perlis argued that computer science is more important in a liberal education than calculus  Explicitly, he argued that all students should learn to program. Calculus is about rates, and that’s important to many. Computer science is about process, which is important to everyone

How close are we to being able to teach everyone CS? Not very  CS1 is one of the most despised courses for non-majors At many departments, CS retention rates are lower than the rest of campus  At Georgia Tech: 65% for 1995 cohort, vs. 73% for Engineeering Drop-out rates near 50% at many institutions Female enrollment in CS has been dropping nationally

Why? Several recent studies and books claim that CS instruction tends to dissuade anyone but white males  “Tedious,” “taught without application relevance,” “boring,” “lacking creativity,” “asocial”

The potential of computing How can we realize the potential impact of computing if future professionals in other disciplines despise computer science? To realize that potential, we need practitioners in other disciplines to understand computing.  At least some of those practitioners need to understand it as a creative, exciting venture—as we do!

The best uses for computing technologies will come from others Thomas Edison vs. D.W. Griffith If we want computing technologies to become useful, they have to get out of our hands. It can’t be just through applications  That presumes that the current technologists know how everyone else wants to do things and should do things  Suggestion: D.W. Griffith knew things that Edison didn’t.

The Challenges We need to motivate CS, potential CS, and non-CS students to care about computing We need to make it social, creative, relevant, exciting, and not tedious  Which is how many of us already see Computing, but that’s not getting communicated

Intro Computing at Georgia Tech CS1321 “Introduction to Computing”  Required of every student at Georgia Tech  Uses Scheme and How to Design Programs  Followed up by CS1322 “Introduction to Object- Oriented Programming” in Java COE 1361 “Computing for Engineers”  Engineering problem-solving in MATLAB  Still in pilot stage: students a term  800 new Engineering Freshmen a year enter Tech

Our Attempt: Introduction to Media Computation A course for non-CS and non-Engineering majors  International Affairs, Literature, Public Policy, Architecture, Management, Biology, etc. 120 students this semester, planning in the Fall  2/3 female in this semester’s CS1315 Focus: Learning programming within the context of media manipulation and creation

Motivating the Computing As professionals, these students will often the use the computer as a communications medium. All media are going digital, and digital media are manipulated with software. Knowing how to program, then, is a communications skill.

Programming as a Communications Skill Knowing how to program means to understand one’s tools.  Maybe means can transfer tool skills more easily  Students tell us that they’re excited to learn how PhotoShop works. And it means that, if you have to, you may be able to grow your own

Programming as Communicating Process A program is a succinct, executable process description That makes valuable for explaining process  We use examples from Biology (e.g., explaining DNA transcription) and Management (e.g., simulations) to make this point

Python as the programming language Huge and contentious issue Use in commercial contexts legitimatizes the choice  Industrial Light & Magic, Google, Nextel, etc. Minimal syntax Looks like other programming languages  Potential for knowledge transfer Actually using Jython ( for Java class librarieshttp://

How the class was developed Created in response to “recent unpleasantness”  On-line surveys, meetings with students Developed with an advisory board with representatives from across campus  Architecture, Math, Electrical and Computer Engineering, Literature, Center for Enhancement of Teaching and Learning Trialed in faculty workshop in mid-December

Course Objectives Students will be able to read, understand, and modify programs that achieve useful communication tasks  Not programming from a blank piece of paper Students will learn what computer science is about, especially data representations, algorithms, encodings, forms of programming. Students will learn useful computing skills, including graphing and database concepts

Data structures Some data structure discussions come naturally  Sounds are in arrays  Pictures are matrices of pixels, each of which has red, green, and blue components  Files are stored in trees Much of our discussion of data structures comes in the form of encodings comparisons  RGB vs. CMYK for encoding color  22.1 Khz vs Khz for audio sampling rates

def negative(picture): for px in getPixels(picture): red=getRed(px) green=getGreen(px) blue=getBlue(px) negColor=makeColor(255-red,255-green,255-blue) setColor(px,negColor) def clearRed(picture): for pixel in getPixels(picture): setRed(pixel,0) def greyscale(picture): for p in getPixels(picture): redness=getRed(p) greenness=getGreen(p) blueness=getBlue(p) luminance=(redness+blueness+greenness)/3 setColor(p, makeColor(luminance,luminance,luminance))

def chromakey(source,bg): for x in range(1,getWidth(source)): for y in range(1,getHeight(source)): p = getPixel(source,x,y) # My definition of blue: If the redness + greenness < blueness if (getRed(p) + getGreen(p) < getBlue(p)): #Then, grab the color at the same spot from the new background setColor(p,getColor(getPixel(bg,x,y))) return source

Use a loop! Our first picture recipe def decreaseRed(picture): for p in getPixels(picture): value=getRed(p) setRed(p,value*0.5) Used like this: >>> file="/Users/guzdial/mediasources/barbara.jpg" >>> picture=makePicture(file) >>> show(picture) >>> decreaseRed(picture) >>> repaint(picture) original

Recipe to Increase the Volume def increaseVolume(sound): for sample in getSamples(sound): value = getSample(sample) setSample(sample,value * 2) Using it: >>> f="/Users/guzdial/mediasources/gettysburg10.wav" >>> s=makeSound(f) >>> increaseVolume(s) >>> play(s) >>> writeSoundTo(s,"/Users/guzdial/mediasources/louder-g10.wav")

A Sunset-generating function How do we turn this beach scene into a sunset? What happens at sunset?  Tried increasing the red, but that failed. New Theory: As the sun sets, less blue and green is visible, which makes things look more red.

A Sunset-generation Function def makeSunset(picture): for p in getPixels(picture): value=getBlue(p) setBlue(p,value*0.7) value=getGreen(p) setGreen(p,value*0.7)

Introducing IF: Making Barb a redhead def turnRed(): brown = makeColor(57,16,8) file = r"C:\Documents and Settings\Mark Guzdial\My Documents\mediasources\barbara.jpg" picture=makePicture(file) for px in getPixels(picture): color = getColor(px) if distance(color,brown)<50.0: redness=getRed(px)*1.5 setRed(px,redness) show(picture) return(picture) Original:

Generalizing Algorithms We talk about algorithm complexity later in the course, after the media is done. We talk about different approaches to the same problem, where the criteria might be aesthetics or correctness, instead of speed or size  For example, generating greyscale During the media, we point out similar themes in different functions.  We refer to them as “sub-recipes”

Scaling the picture down def copyBarbsFaceSmaller(): # Set up the source and target pictures barbf=getMediaPath("barbara.jpg") barb = makePicture(barbf) canvasf = getMediaPath("7inX95in.jpg") canvas = makePicture(canvasf) # Now, do the actual copying sourceX = 45 for targetX in range(100,100+((200-45)/2)): sourceY = 25 for targetY in range(100,100+((200-25)/2)): color = getColor(getPixel(barb,sourceX,sourceY)) setColor(getPixel(canvas,targetX,targetY), color) sourceY = sourceY + 2 sourceX = sourceX + 2 show(barb) show(canvas) return canvas

Scaling the picture up def copyBarbsFaceLarger(): # Set up the source and target pictures barbf=getMediaPath("barbara.jpg") barb = makePicture(barbf) canvasf = getMediaPath("7inX95in.jpg") canvas = makePicture(canvasf) # Now, do the actual copying sourceX = 45 for targetX in range(100,100+((200-45)*2)): sourceY = 25 for targetY in range(100,100+((200-25)*2)): color = getColor(getPixel(barb,int(sourceX),int(sourceY))) setColor(getPixel(canvas,targetX,targetY), color) sourceY = sourceY sourceX = sourceX show(barb) show(canvas) return canvas

Recipe for halving the frequency of a sound def half(filename): source = makeSound(filename) target = makeSound(filename) sourceIndex = 1 for targetIndex in range(1, getLength( target)+1): setSampleValueAt( target, targetIndex, getSampleValueAt( source, int(sourceIndex))) sourceIndex = sourceIndex play(target) return target This is how a sampling synthesizer works! Here’s the piece that does it

Compare these two def half(filename): source = makeSound(filename) target = makeSound(filename) sourceIndex = 1 for targetIndex in range(1, getLength( target)+1): setSampleValueAt( target, targetIndex, getSampleValueAt( source, int(sourceIndex))) sourceIndex = sourceIndex play(target) return target def copyBarbsFaceLarger(): # Set up the source and target pictures barbf=getMediaPath("barbara.jpg") barb = makePicture(barbf) canvasf = getMediaPath("7inX95in.jpg") canvas = makePicture(canvasf) # Now, do the actual copying sourceX = 45 for targetX in range(100,100+((200-45)*2)): sourceY = 25 for targetY in range(100,100+((200-25)*2)): color = getColor( getPixel(barb,int(sourceX),int(sourceY))) setColor(getPixel(canvas,targetX,targetY), color) sourceY = sourceY sourceX = sourceX show(barb) show(canvas) return canvas

Both of them are sampling Both of them have three parts:  A start where objects are set up  A loop where samples or pixels are copied from one place to another To decrease the frequency or the size, we take each sample/pixel twice In both cases, we do that by incrementing the index by 0.5 and taking the integer of the index  Finishing up and returning the result

Using your personal pictures

And messin’ with them

Data-first Computing Real users come to a user with data that they care about, then they (unwillingly) learn the computer to manipulate their data as they need. CS1315 works the same.  We use pictures of students in class demonstrations.  Students do use their own pictures as starting points for manipulations.  They started doing this in the second week How often do students use their second week of CS1 on their own data? How does that change the students’ relationship to the material?

Rough overview of Syllabus Defining and executing functions Pictures  Psychophysics, data structures, defining functions, for loops, if conditionals Sounds  Psychophysics, data structures, defining functions, for loops, if conditionals Text  Converting between media, generating HTML, “flattening” media and saving to a database Movies Then, Computer Science

Computer science as a solution to their problems “Writing programs is hard! Are there ways to make it easier? Or at least shorter?”  Object-oriented programming  Functional programming and recursion “Movie-manipulating programs take a long time to execute. Why?”  Algorithmic complexity “Why is PhotoShop so much faster?”  Compiling vs. interpreting  Machine language and how the computer works

Assignments encourage collaboration Homework are all collaborative Quizzes are preceded by nearly-identical, collaborative pre-quizzes Two “take-home exams” (programming assignments) are non-collaborative “Lablets” on application software are collaborative

Assignments encourage creativity For several homeworks, the task is to manipulate media in some way, but we don’t care what media  For example, creating a collage or building an animation Encouraging homework results to be posted to CoWeb in galleries We’ve purchased webcams to loan to students to create their own media

First Homework assignment (Due last Friday) Homework 1: Write a program named hw1 to accept a picture as input, and change its pixels as follows: Set the green component to 125% of its current value Decrease the blue by 25% Decrease the red by 75%

Solutions shared in the CoWeb

Grade distribution Much better than anticipated.

Tools to support media computation JES: Jython environment for students MediaTools for exploring media

Assessing the effort We don’t know if this will work  Will students learn? Will they learn similarly to the other CS1 classes?  Will the class change their attitude toward CS? Hopefully, something better than what we’ve been seeing Will the differences appear among the women, specifically?  If there are problems in student performance, where are they?

Assessment measures Comparing all of our CS1 classes in terms of learning and motivation, broken out by gender and major  Surveys: Initial, midterm, and final Observational study of student performance to understand problems and strategies  Watched two groups working on HW1 Interview study of impact on women

How can this be useful to you? We’d like to find partners in developing and trialing the course as a non-majors programming course.  We have software, draft of book, and slides in development.  We’d like to develop something that works more than just at Tech. We have Java classes to offer to majors courses that implement most of the media interface.  Media focus may be motivating to our majors, and may help with retention issues.

public greyScale(String filename) throws java.io.IOException // parms: string defining the file path { JavaPicture src = new JavaPicture(); src.setFilename(filename); src.loadImage(); // System.out.println("Height = " + src.getHeight() ); // System.out.println("Width = " + src.getWidth() ); for (int x=1; x < src.getHeight();x++) { for (int y=1; y < src.getWidth();y++) { JavaPixel p = src.getPixel(y,x); int nC = (p.getRed()+p.getGreen()+p.getBlue() )/ 3; p.setBlue(nC); p.setRed(nC); p.setGreen(nC); } src.saveImage("C:\\grey.jpg"); }

Summary Perlis’ challenge suggests that CS is more important than Calculus  But need to update our pedagogy to make it happen Media Computation may be a useful context to motivate student performance Our class is aimed at addressing the challenges we’ve identified, and we’re trying it this semester The approach right now is aimed at non-majors, but certainly could be used with majors, too.

Acknowledgements Course materials development: Jason Ergle, Claire Bailey, David Raines, Joshua Sklare, Adam Wilson, Andrea Forte, Mark Richman, Matt Wallace, Alisa Bandlow. Assessment: Andrea Forte, Rachel Fithian, Lauren Rich Thanks to Bob McMath and the Al West Fund, to GVU and CoC, and the National Science Foundation

For further information Course CoWeb: Where we planned the course: