SM1205 Interactivity Topic 01: Introduction Spring 2011SCM-CityU1

Self-Introduction L1 Instructor – Oscar Au PhD (HKUST, 2007) Teaching: interactivity, programming, physical computing, hardware hacking – SM1204, SM1205, SM2240, SM2608 Research: computer graphics, user interface Spring 2011SCM-CityU2

Interactivity Human-human interaction Human-object interaction Human-computer interaction Spring 2011SCM-CityU3

Human-Computer Interaction Also known as HCI User interface – A place where interaction between humans & machines occurs Types – Command-line interface – Graphical user interface – Natural user interface Spring 2011SCM-CityU4 User Interface

Command-Line Interface Keyboard Spring 2011SCM-CityU5 DosUnix

Graphical User Interface (GUI)GUI Mouse & keyboard Spring 2011SCM-CityU6

Natural User Interface (NUI)NUI Keyboard? No. Mouse? No. Spring 2011SCM-CityU7

NUI Example: Multi-Touch Effortless interaction with digital content through natural hand gestures and touch Spring 2011SCM-CityU8

NUI Example: Multi-Touch Microsoft Surface Spring 2011SCM-CityU9

NUI Example: Kinect (wiki)wiki Controller-free gaming & entertainment experience for Xbox 360 by Microsoft Xbox 360 by Microsoft – Released Nov Spring 2011SCM-CityU10

NUI Example: Kinect (wiki)wiki Spring 2011SCM-CityU11

NUI Example: Kinect (wiki)wiki Spring 2011SCM-CityU12

NUI Example: Kinect (wiki)wiki Natal sensor – RGB camera – Depth camera – Multi-array microphone Capabilities – 3D motion capture, facial recognition, voice recognition etc. Spring 2011SCM-CityU13

How Microsoft Kinect Works Depth camera – Shooting out beams of infrared light – Measuring how long it takes them to reflect off of objects in the scene and return to an infrared camera Video: Look at Kinect using IR goggles Video: Kinect – The Way Games Work Spring 2011SCM-CityU14

How Microsoft Kinect Works Spring 2011SCM-CityU15

Microsoft Office Labs Vision 2019 Spring 2011SCM-CityU16

Class Schedule Development platform: Flash CS5 + AS 3.0 Tentative topics – Flash fundamentals – AS 3.0 fundamentals – Keyboard & mouse – Interactivity with sound, image, video – Webcam interactivity (color, motion, pattern) Spring 2011SCM-CityU17

Assessment Attendance: 10% 3 Assignments: 90% – Tentative: A1 (20%), A2 (30%), A3 (40%) Curved grading: A+, A, A-, B+, …. Spring 2011SCM-CityU18

Late Policy Attendance – Late policy: 5 late minutes per class > 5 minutes late  no attendance score for that class Assignment & project submission – Late policy: 3 late days in total Use them wisely! Spring 2011SCM-CityU19

SM1205 Interactivity Flash Fundamentals Spring 2011SCM-CityU20

What is Flash (wiki)?wiki Originally by Macromedia, later by Adobe – Latest version: Adobe Flash CS5 Rich media content platform – Animation, interactivity, image/video, online games Youtube, Google Videos, Yahoo Maps, …. Support both bitmaps & vector graphics Programming language: ActionScript 3.0 Spring 2011SCM-CityU21

ActionScript 3.0 Scripting language Object-oriented language Popular for – Web design, – Animation – Multimedia applications – … Online language reference Spring 2011SCM-CityU22

Spring 2011SCM-CityU23 Stage Tools Timeline Layer Panels: Library + Properties

Workflow Spring 2011SCM-CityU24 Files PublishView Flash Player

Let’s start a simple example! Spring 2011SCM-CityU25

Document Properties Change stage properties & save as Ex01.fla Spring 2011SCM-CityU26

Use Commands Test movie – Control > Test Movie (Ctrl + Enter) Play (for previewing animation) – Control > Play (Enter) Publish – File > Publish (Shift + F12) Spring 2011SCM-CityU27

Shape & Drawing Object Shape Spring 2011SCM-CityU28

Shape & Drawing Object Drawing object – Make sure object drawing is enabled Spring 2011SCM-CityU29

Layers Spring 2011SCM-CityU30 Layer3 Layer2 Layer1

Symbols and Library Symbol: reusable content – Stored in the library – Required for many operations E.g., motion tween, access in ActionScript Spring 2011SCM-CityU31 Convert to Symbol Instance

Symbols and Library Each instance can have its own properties, effects, and filters Spring 2011SCM-CityU32

Symbols and Library But if you change the symbol, every of its instances will be changed correspondingly Spring 2011SCM-CityU33

Tween Tween (between) can be assigned to keyframe intervals – Automatically interpolate frames between keyframes Motion Tween – Mainly for keyframes with symbol instances Shape Tween – Mainly for keyframes with shapes/drawing objects Spring 2011SCM-CityU34

Motion Tween Example Step 1: right click a symbol instance at Frame 1 & select Create Motion Tween Spring 2011SCM-CityU35

Motion Tween Example Step 2: At Frame 25, change properties of the symbol – E.g., move it to a new place, change its alpha value Spring 2011SCM-CityU36

Motion Editor Very powerful but not that intuitive to control Spring 2011SCM-CityU37

Predefined motion – Window > Motion Presents Spring 2011SCM-CityU38

Shape Tween Example Step 1: Right click a shape or drawing object at Frame 1 & select Create Shape Motion – E.g., a red square Step 2: Insert a blank keyframe at Frame 20 & draw some shape, e.g., a circle Spring 2011SCM-CityU39

Recommend Flash Books Spring 2011SCM-CityU40

SM1205 Interactivity Programming Fundamentals Spring 2011SCM-CityU41

Talk to the Computer Computers & humans speak in different languages Spring 2011SCM-CityU42 English, Mandarin, Cantonese, …

Talk to the Computer Is there any interpreter for us? – Yes: through voice recognition – No: recognition is successful only at the level of predefined sets of voice commands E.g., “Xbox, pause” Computers do NOT have fuzzy human brains – Need new languages with stricter grammars/syntaxes, called programming languages Spring 2011SCM-CityU43

Programming Language Now humans “speak” in programming languages – E.g., Java, C, C++, Python, Flash ActionScript, … But computers still speak in binary Still need an interpreter – A special computer program which converts the code written in a programming language to the binary the computer understands Different programming languages need different interpreters Spring 2011SCM-CityU44

Spring 2011SCM-CityU … Write Code Code in Programming Language

Programming Language Computers are NOT good at tolerating errors You must exactly follow the grammars of programming language – E.g., many programming languages are case-sensitive Spring 2011SCM-CityU46