SLIDE 1IS246 - SPRING 2003 Lecture 12: Theory and History of Computation IS246 Multimedia Information (FILM 240, Section 4) Prof. Marc Davis UC Berkeley SIMS Monday and Wednesday 2:00 pm – 3:30 pm Spring

SLIDE 2IS246 - SPRING 2003 Today’s Agenda Review of Last Time –Movie Ideas Feedback Theory and History of Computation –Turing and von Neumann –Ideas of Computation –Discussion Questions Action Items for Next Time

SLIDE 3IS246 - SPRING 2003 Today’s Agenda Review of Last Time –Movie Ideas Feedback Theory and History of Computation –Turing and von Neumann –Ideas of Computation –Discussion Questions Action Items for Next Time

SLIDE 4IS246 - SPRING 2003 Movie Ideas Feedback Time constraints –Make sure your movie will be between 2:30 and 3:00 minutes long –Add shot durations to your annotated storyboards POV –Who is making your message movie? –To whom is it intended? –What do you want your audience to do with it? –How do your cinematographic choices (especially shot POV) reflect these decisions?

SLIDE 5IS246 - SPRING 2003 Today’s Agenda Review of Last Time –Movie Ideas Feedback Theory and History of Computation –Turing and von Neumann –Ideas of Computation –Discussion Questions Action Items for Next Time

SLIDE 6IS246 - SPRING 2003 Computation in Intellectual History Computation as instrumentality –PCs, PDAs, embedded processors, etc. Computation as ideas –Modeling process –Languages for modeling process –Primitives, combination, abstraction –Parameterization –Black boxing functionality –Optimization

SLIDE 7IS246 - SPRING 2003 Universal Turing Machine An abstract representation of a computing device –It has a read/write/erase head that scans a (possibly infinite) one-dimensional (bi-directional) tape divided into squares, each of which can be inscribed with a symbol (e.g., 0 or 1) –Computation begins with the machine in a given "state” scanning a given square –It can erase what it finds on the square and print a symbol (e.g., 0 or 1), move to an adjacent square, and go into a new state –A table of instructions specifies, for each state, what the machine should write, which direction it should move in, and which state it should go into –The machine’s behavior is determined by three parameters The state the machine is in The square it is scanning A table of instructions –The table can list only finitely many states, each of which becomes implicitly defined by the role it plays in the table of instructions

SLIDE 8IS246 - SPRING 2003 von Neumann Machine Stored program –Program is no longer external to machine itself –A “conditional control transfer” permitted the program sequence to be interrupted and reinitiated at any point –Computer can modify both its data and programs von Neumann machines allowed the construction of practical digital computers

SLIDE 9IS246 - SPRING 2003 Today’s Agenda Review of Last Time –Movie Ideas Feedback Theory and History of Computation –Turing and von Neumann –Ideas of Computation –Discussion Questions Action Items for Next Time

SLIDE 10IS246 - SPRING 2003 Programming As Representation There is a structure of formal symbols that can be manipulated according to a precisely defined and well-understood system of rules There is a mapping through which the relevant properties of the domain can be represented by symbol structures This mapping is systematic in that a community of programmers can agree as to what a given structure represents

SLIDE 11IS246 - SPRING 2003 Programming As Representation There are operations that manipulate the symbols in such a way as to produce veridical results—to derive new structures that represent the domain in such a way that the programmers would find them accurate representations Programs can be written that combine these operations to produce desired results

SLIDE 12IS246 - SPRING 2003 Levels of Representation Physical machine –Wires, chips, disks, or pipes, valves, sinks, or tinker toys Logical machine –And-gates, or Or-gates, inverters, etc. Abstract machine –Machine instructions for manipulating stored symbols High-level language –Java, C++, Lisp, etc. (operator, data structures, etc.) Representation language –Represents primitives and operations about a domain

SLIDE 13IS246 - SPRING 2003 Algorithms and Programming Algorithm –A step-by-step description of a procedure to achieve a desired result Programming –Primitives –Means of combination –Means of abstraction

SLIDE 14IS246 - SPRING 2003 From Algorithms to Programs Algorithm –A step-by-step description of a procedure to achieve a desired result –How can we walk a square? Walk forward Turn Walk forward Turn Walk forward Turn Walk forward

SLIDE 15IS246 - SPRING 2003 LOGO Square Example TO SQUARE FORWARD 5 RIGHT 90 FORWARD 5 RIGHT 90 FORWARD 5 RIGHT 90 FORWARD 5 END

SLIDE 16IS246 - SPRING 2003 LOGO Square Example TO SQUARE :SIZE FORWARD :SIZE RIGHT 90 FORWARD :SIZE RIGHT 90 FORWARD :SIZE RIGHT 90 FORWARD :SIZE END

SLIDE 17IS246 - SPRING 2003 LOGO Square Example TO WINDOW :SIZE SQUARE :SIZE END

SLIDE 18IS246 - SPRING 2003 Computation for Designing Artifacts Four computational ideas/techniques from Carlo Sequin –Procedural generation –Parameterization –Optimization –Evolutionary power

SLIDE 19IS246 - SPRING 2003 Procedural Generation Rather than creating artifacts directly, the user may design a generating program that will then generate the desired artifact The empowering aspect of this approach is that the generating procedure will not just create the one artifact originally desired, but, with minor variations to the program, it can produce many different artifacts that may all fit a specified set of constraints or usage

SLIDE 20IS246 - SPRING 2003 Parameterization For classes of frequently needed artifacts, the procedural generation mentioned above can be captured in a robust and more general program that contains a modest number of parameters that can be easily adjusted by non-programming users A judicious selection and coupling of such parameters can enhance the likelihood that any arbitrary combinations of parameters still produce a meaningful output, although it may be far from desirable or optimal with respect to some specific application However, the ease of modifying the parameter values and previewing the expected outcome, would allow even novice users to achieve better than average results obtained by un-aided users

SLIDE 21IS246 - SPRING 2003 Optimization Given that the tedium of creating individual artifacts can be greatly reduced by procedural generation, users can explore a far larger space of possibilities than they could if they had to craft each artifact individually This allows them to home in on a more optimal solution than they could by building a few prototypes If the constraints and goal functions are well understood, then the generating program may contain its own evaluation loop that allows it to explore many options on its own and gradually converge towards a local optimum

SLIDE 22IS246 - SPRING 2003 Evolutionary Power The ease of exploration afforded by the use of procedural generation permits an informed user to more clearly see and locate the boundaries of the paradigm captured in a generating program By making these boundaries more visible, it also becomes more obvious to ask what lies beyond Often such questions can be answered with a modest re-programming effort that enlarges the scope of the generator

SLIDE 23IS246 - SPRING 2003 Today’s Agenda Review of Last Time –Movie Ideas Feedback Theory and History of Computation –Turing and von Neumann –Ideas of Computation –Discussion Questions Action Items for Next Time

SLIDE 24IS246 - SPRING 2003 Discussion Questions On Winograd and Flores (Ana Ramirez) –Winograd and Flores discuss the importance of abstraction and separation of concerns in computing. Increased levels of abstraction and separation of concerns result in an increase in programmer productivity and sometimes a decrease in performance. Moore's law has placed an importance on the programmer’s productivity, since computers double in speed every 18 months, most performance problems disappear once the newest processor comes out. We are quickly approaching a point where Moore's law will no longer hold. Will we find a new technology that continues this rapid increase in speed in processor technology or will the emphasis be moved from programmers' productivity back to program performance? How will the role of abstraction and separation of concerns change?

SLIDE 25IS246 - SPRING 2003 Discussion Questions On Winograd and Flores (Ana Ramirez) –The abstraction boundaries must be crossed occasionally in order to increase performance. One such case is in systems that must operate in real-time. How does this show up in multimedia systems?

SLIDE 26IS246 - SPRING 2003 Discussion Questions On History of Computation (Catherine Lai) –Look at the pervasive presence of computing in modern science and technology, has the history of computing established a significant presence in the history of science and technology? –Is technology the creator of demand or a response to it? –What role do governments play in fostering and directing technological innovation and development? –What does the term “computer” mean to you? Have we prematurely united its multiple historical resources into one thing? What about for computing? Is there a dual nature or tripartite structure to it?

SLIDE 27IS246 - SPRING 2003 Discussion Questions How can we think about motion pictures and computation in terms of each other as “representational” systems? How could we describe an editing process computationally? How would we need to describe video to be able to operate on it computationally?

SLIDE 28IS246 - SPRING 2003 Today’s Agenda Review of Last Time –Movie Ideas Feedback Theory and History of Computation –Turing and von Neumann –Ideas of Computation –Discussion Questions Action Items for Next Time

SLIDE 29IS246 - SPRING 2003 Readings for Next Week Monday 03/10 “Computational Media Theory” –Required Lev Manovich. The Language of New Media. The MIT Press, Cambridge, Massachusetts, 2001; pp Gilles Bloch, G.R. From Concepts to Film Sequences, Yale University Department of Computer Science, 1987; pp Chitra Dorai and Svetha Venkatesh. Computational Media Aesthetics: Finding Meaning Beautiful. IEEE Multimedia, 8 (4); pp –Optional Marc Davis and David Levitt. Time-Based Media Processing System (US Patent 6,243,087), Interval Research Corporation, USA, 2001; pp

SLIDE 30IS246 - SPRING 2003 Readings for Next Week Wednesday 03/12 “Automated Media Analysis: Audio” –Required Jonathan Foote. An Overview of Audio Information Retrieval. Multimedia Systems, vol. 7, 1999; pp Kenichi Minami, Akihito Akutsu, Hiroshi Hamada, and Yoshinobu Tomomura. Video Handling with Music and Speech Detection. IEEE MultiMedia, vol. 5, 1998; pp –Optional (On IS246 Course Web Site) Christian Spevak and Emmanuel Favreau. Sound Spotter - A Prototype System for Content-Based Audio Retrieval. In: Proceedings of the 5th International Conference on Digital Audio Effects (DAFx-02). Hamburg, George Tzanetakis, Perry Cook. Automatic Musical Genre Classification of Audio Signals. In Proc. International Symposium for Audio Information Retrieval (ISMIR 2001) Bloomington, USA, October 2001.