1. Locative Media Lalya Gaye Ubiquitous Computing course
IT-University in Göteborg
31 November 2007

2. Introduction Locative Media Lecture Aims and scope
Overview of the field
Technology overview
Discussion of design and prototyping approaches
Design issues: focus on sustainability in locative media

3. Introduction Lecture Content
Ubiquitous computing: recap
Ubicomp technologies
Locative Media: definition and origins
Themes, projects and related design issues
Characteristics, challenges and design opportunities
Technologies available to the general public
Sustainable Design?

4. Ubiquitous Computing Recap

5. Ubiquitous Computing Recap Mark Weiser’s vision (1991)
disappearing computer
everyday world literally used as interface
“The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.”

6. Ubiquitous Computing Recap
The computer: calculator -> information system -> interactive -> pc -> mobile, integrated, networked
Levels of interaction: electrical -> symbolic -> textual -> visual -> social, tangible
Evolution of the user interface: from immersing the user in the computer’s world to computing increasingly adapting to the user’s world and skills.
Ubicomp = opposite of virtual reality: embedded reality.

7. Ubiquitous Computing Recap Evolution of computer-human interaction:
more of the human’s everyday world and everyday skills in computing
computers an increased part of our everyday life
requiring less specialised knowledge to operate them
relying increasingly on user’s everyday skills
smaller computers
from one computer for many user, to many computers

8. Ubiquitous Computing Recap Designing ubicomp systems:
Focus on the interaction between user & technology (as opposed to form and function), on what experience the user gets from it, on what added-value ubicomp brings to his/her life.
Follow needs and requirements but also entice new behaviours?

9. Ubiquitous Computing Recap
Enhance people’s activities by making computing available at hand, when and where needed (including when the users are mobile)
Computing naturally blending into everyday settings, vanishes into the background
The physical and social world around us as digitally augmented and distributed interface
Manipulating digital data = manipulating entities in the physical world
Literally build on people’s everyday use of the physical and social world, in situation and in real time.
Peripheral awareness
Greenfield: “information processing dissolving into behaviour”
IT + everyday life as design material (f. ex. I/O Brush)

10. Ubiquitous Computing Recap Implementing the ubicomp vision:
Many interconnected computers per person
Mobile devices combined with computers embedded in the environment (e.g. post-hoc augmentation of everyday objects with sensors and networked communication)
With awareness of physical & social context + each other
-> Mapping the digital world to the physical one
-> User interface: tangible and embedded in the real world

11. Ubiquitous Computing Recap Implementing the ubicomp vision:
Distributed interface: networking mobile devices and embedded computers (sensors, processors, etc) -> flexible and seamless integrated whole -> e.g. any display or input device can become one’s own (user mobility)
Interaction in context and in real time (f.ex. tracking things and people -> relevant information and interaction opportunity to the right person at the right time)

12. Ubiquitous Computing Recap Types of systems: “walk-up-pop-up”
wearables
ambient displays
intelligent work environments
augmented, interconnected everyday objects
etc
Media cup, TecO

13. Ubiquitous Computing Technologies

14. Ubiquitous Computing Technologies
Ubiquitous Computing (Weiser): computing interweaved in everyday life, “where the action is” (Dourish)
context awareness
embedded sensor networks
global positioning
wearable computing
augmented & mixed-reality
ad hoc and p2p user networks

15. Ubiquitous Computing Technologies * Embedded sensor networks Sensors:
- in everyday environments
- on people
- on artefacts
Sensor fusion: combining different data and placements to gather context

16. Ubiquitous Computing Technologies * Context-aware computing
“computer-based devices [that] reach out into the real world through sensors” [Gellerson].
“A system is context-aware if it uses context to provide relevant information and/or services to the user, where relevancy depends on the user’s task.” [Dey & Abowd, 1999].

17. Ubiquitous Computing Technologies * Context-aware computing
Enables computing to run into the background and adapt to changes of context in order to present appropriate behaviour to specific situations.
“presentation of information and services to a user”
“automatic execution of a service” depending on context appropriateness
or “tagging of context to information for later retrieval” [Dey].

18. Ubiquitous Computing Technologies * Context-aware computing
Gellersen et al.

19. Ubiquitous Computing Technologies * Context-aware computing
Gellersen et al.

20. Ubiquitous Computing Technologies * Tangible computing
Input, data, output and networking contained and accessed within the same tangible artefact
Paper, cups, pens, umbrellas or specially designed artefacts
Tangible objects as active entities that respond to the environment, to user manipulation and people’s activities in general
Building on the users’ cognitive abilities

21. Ubiquitous Computing Technologies * Social computing
Incorporating understandings of the social world into interactive systems
Social traces left by people on objects or places
Mobile social networks between co-located acquaintances
enhancing user awareness by providing them information about others and their activity

22. Ubiquitous Computing Technologies * Augmented reality
Superimposing a digital world upon the real one
User experiences both as co-existing parts of the same reality
User is able to interact with their combination in real time
Interfaces:
3D computer graphics seen through transparent head-mounted displays or augmented glasses
Spatialised audio cues heard through headphones

23. Ubiquitous Computing Technologies * Augmented reality
Mixed-reality: digital world not directly overlaid on the physical one but still presented as part of the same reality, f.ex.
with both realities displayed on the screen of hand-held device)

24. Ubiquitous Computing Technologies * Wearable computing
Computing incorporated into clothing
Make use of body-related information or interaction forms to control processes : - body movements - biometrics
Embedded displays (e.g. glasses)

25. Ubiquitous Computing Technologies * Platforms: Smart-Its Smart Dust
Pin & Play
Tiny OS
etc

26. Ubiquitous Computing Technologies Smart-Its:
sensors: sound, light, acceleration (2d), pressure
core board: context-recognition, communication interface (RF)

27. Locative Media: Background

28. Locative Media Background
Typical contexts of use for ubicomp: home, office work, cafeterias, grad-students research labs, etc
Locative media = media with sense of place
New media + urban aesthetic practices + community uses of public space + contextual art + mobile, ubiquitous and geographical technologies
City, public spaces
Ubiquitous computing in public space:
Minority Report dystopia (video: 44:20) vs. current creative uses and appropriations of public space?

29. Locatived Media Background Urban aesthetic practices
Mobility as creative act
Creative use of public space
Walking:
aboriginal walkabouts
situationist dérive, psycho-geography

30. Locative Media Background Urban aesthetic practices
Mobility as creative act
Creative use of public space
Graffiti
Reclaim the Streets
Urban sports:
skateboarding
parkour (video)
-> urban space as resource for aesthetic movements

31. Themes and Projects

32. Locative Media Projects
Themes
Pervasive Gaming: the world as a game-board
Space annotation: media with a specific position in space
Location awareness & GPS-enabled locative media
Mobile music & locative audio
Radio pirates
Social spaces
etc

33. Locative Media Projects Locative Media Projects
Pervasive Gaming
Locative Media Projects
Pervasive Gaming
The world as game-board
Botfighters and Pirates!
Backseat Gaming (video)
Can You See Me Now? (video)
iPerG
...
Can You See Me Know? Blast Theory + Equator

34. Locative Media Projects
Space Annotation
Media with a specific position in space
User-authored social cues
Virtual: Geonotes (video) Urban Tapestries (animations)
Physical: Yellow Arrow (video) Grafedia
Grafedia, grafedia.net
Yellow Arrow, Count Media

35. Locative Media Projects
GPS & Positioning
Hundekopf, knifeandfork
GPS-drawing
Non-linear narratives:
Hundekopf (video)

36. Locative Media Projects
GPS & Positioning
Tracking and mapping paths
Biomapping (video), Drift, Net_Derive (video)...
Biomapping, Christian Nold
Drift, Teri Rueb

37. Locative Media Projects
Mobile Music and Locative Audio
Audio space annotation
Mobile music sharing/listening:
distributed
ad hoc
sound walks
Mobile music making:
situated
collaborative
Wearable audio
User content!!!

38. Locative Media Projects
Mobile Music and Locative Audio
Audio space annotation
Hear&There
(Rozier, MIT Medialab, 1999)
Tacticle Sound Garden [TSG] (video)
(Mark Shepard, Buffalo Univ )
Tejp / Audio tags
(PLAY & FAL, )
Sound as public display, Peripheral awareness, Community re-appropriation of public space

39. Locative Media Projects
Mobile Music and Locative Audio
Audio space annotation
Audio Bombing (video)
(Fleming et al., 2007)
Sonic Graffiti (video)
(C-Y Lee, 2007)
Sound as public display, Peripheral awareness, Community re-appropriation of public space

40. Locative Media Projects
Mobile Music and Locative Audio
Audio space annotation
[Murmur] (murmur.ca)
Sound as public display, Peripheral awareness, Community re-appropriation of public space

41. Locative Media Projects
Mobile Music and Locative Audio
Sound walks
Electric walks (Christina Kubisch)
Drift (Rueb)
34n118w (Knowlton, Spellman, 2005)
Craving (Garnicnig, Haider, 2007)
Seven Mile Boots (Beloff et al., )
The Case at Kulturhuset
(Knifeandfork, 2004)
Riot! (Mobile Bristol, Hewlett Packard)
mapping audio world to physical paths

42. Locative Media Projects
Mobile Music and Locative Audio
Distributed and located music
Location 33 (Carter & Liu, USC, 2005)

43. Locative Media Projects
Mobile Music and Locative Audio
Mobile music sharing
SoundPryer (Mattias Östergren, Interactive Institute, 2001)
TunA
(Arianna Bassoli et al.,
Medialab Europe, 2002)
Social aspect of mobile computing: ad hoc networks, distributed social networks, etc
-> spontaneous and situated music sharing with people in public space

44. Locative Media Projects
Mobile Music and Locative Audio
Mobile music sharing
Bass Station
(Mark Argo & Ahmi Wolf, 2003)
Push!Music (Håkansson et al., 2005)

45. Locative Media Projects
Mobile Music and Locative Audio
Situated music making
Sonic City (video)
(Gaye et al., FAL & PLAY, )
Sound Lens (Toshio Iwai, Tokyo Univ.)
Solarcoustics: CONNECT (Barnard, ITP/NYU, 2005)
Sensor technology + GPS -> situated music making Interacting with the environment, mobile soundscapes

46. Locative Media Projects
Mobile Music and Locative Audio
Situated music making
Sound Mapping (video)
(Mott et al., Reverberant, 1997)
Sonic Interface
(Akitsugu Maebayashi, 1999)
Warbike
(McCallum, )
Skatesonic (video) (van Toder, 2006)

47. Locative Media Projects
Mobile Music and Locative Audio
Collaborative mobile music making
ImprovE (video)
(Wideberg & Hasan, 2006)
CosTune
(Nishimoto et al., ATR, 2001)
Malleable Mobile Music
(Atau Tanaka, Sony CSL, 2004)
Ad hoc & distributed networks throughout the city -> collaborative music making

48. Locative Media Projects
Mobile Music and Locative Audio
Collaborative mobile music making
China Gates (Clay, Majoe, 2006)
Sequencer404 (Hatcher, Jimison et al., 2006)
Cellphonia (Bull et al, 2006)
Ad hoc & distributed networks throughout the city -> collaborative music making

49. Locative Media Projects
Mobile Music and Locative Audio
Wearable audio
Nomadic Radio (Shawney, MIT Medialab, 1998)
Sonic Fabric (Alice Santaro, 2002)

50. Locative Media Projects
Mobile Music and Locative Audio
Wearable audio
”Personal instruments”
(Krzysztof Wodiczko, 1969)
(Chelle Hugues, RCA/CRD, 2000)

51. Locative Media Projects
Mobile Music and Locative Audio
Wearable audio
Robotcowboy (Wilcox, 2007)
Hearing Sirens (Cathy van Eck, 2007)

52. Locative Media Projects
Mobile Music and Locative Audio
Output: Headphones vs boombox vs using everyday objects
SoundbugTM speakers & piezos
Flower Speakers (LET’S corporation, Japan, 2004)

53. Locative Media Projects
Radio Pirates
Bit Radio
(Bureau of Inverse Technology)
7/11 (video) (New Beginnings, Göteborg)
Key Chain Radio Station
(Rikako Sakai, Ivrea, 2004)

54. Locative Media Projects
Social Spaces
Hummingbirds
Jabberwocky (video)
MobiTip

55. Charateristics, Challenges and Design Opportunities

56. Characteristics of Locative Media
Interaction Properties
Interactions happening anywhere, on the move :
taking advantage of the mobile setting: playing with social and geographic dynamics implied by mobility
-> outdoors everyday space, location and social context becoming resources for interaction as you move through space
-> spontaneous & situated collaborations with people around or distributed across the city

57. Characteristics of Locative Media
Interaction Properties
Interactions happening anywhere, on the move
becoming embedded in the physical and social context of everyday life
-> people managing interaction in heterogeneous context
-> and in simultaneity with other activities (crossing a street... waiting for the bus...)
tunA, Bassoli et al, Medialab Europe, 2002

58. Characteristics of Locative Media
Technical Opportunities & Challenges
Usage extended over time and space
Ergonomics
Same application, many devices
Same application, many places
Access variability
Ad-hoc meetings, windows of opportunity
Shifting social roles and contexts
Shifting physical context
Heterogeneous environment
Scales of interaction
Merging digital and physical realms

59. Characteristics of Locative Media
Design Issues
User-authored content spread across public space: raises questions about
property of information
privacy & surveillance (loca)
spamming?
Augmenting environments and supporting activities with embedded computation: what if it changes what makes things what they are?
If ubicomp spreads into public space, according to whose will?
Top-down corporations, government vs bottom-up citizens, communities? Conflicts of interests?

60. Characteristics of Locative Media
Design Issues
User control (Greenfield): How do you know you are interacting with a computer if invisible? How do you protect your privacy? avoid false commands? How do you know where to look for interaction?
How to query/notify presence, access, place, manipulate media?
How is the place? Who is there? What activities are going on there? How mobile is/are the user(s)? What meaning do the place, activities, and things around have and for whom?

61. Characteristics of Locative Media
Design Issues
Pro-active and calm computing vs engaging
Ubicomp vs pervasive computing: at hand when needed vs always on everywhere
Connect physical and virtual world: technical and HCI issue but also sociological, aesthetic, even political and environmental. F.ex. Yellow Arrow vs Geonotes:
physical vs virtual markers
Graffiti style interaction vs screen-based

62. Enabling technologies Available to General Public

63. Enabling Technologies
Available to the General Public
Mobile peer-to-peer
Tracking, positioning and placement
Sensing and data-processing
Content creation and manipulation

64. Enabling Technologies
Available to the General Public
* Server-Client
* Mobile peer-to-peer:
Bluetooth
WiFi
Infrared

65. Enabling Technologies
Mobile Peer-to-Peer
* Bluetooth
Standard communication protocol for wireless personal area network (PANs)
Connect and exchange information (commands, files) between devices
Microwave radio frequency -> non-directional
Short range (power-class-dependent: m)
Use: BluetunA, bluejacking, Nokia’s Digidress

66. Enabling Technologies
Mobile Peer-to-Peer
* WiFi
Wireless local area network
Radio, non-directional
Internet and VoIP phone access, network connectivity for for consumer electronics, etc
Connect to local access points
Server-client vs ad hoc networks

67. Enabling Technologies
Mobile Peer-to-Peer
* Phones vs Wifi-enabled PDAs
Connectivity: closed/open network vs operators
Cost
Range
Distributed vs ad hoc vs server-client
Compatibility
Programmability: SDK, OS
Memory, speed

68. Enabling Technologies
Mobile Peer-to-Peer
* Platform: Opentrek
Peer-to-peer networking platform specifically designed for Wireless Ad Hoc Networks
Cross-platform!
Ad hoc networking -> collaborate

69. Enabling Technologies
Available to the General Public
* Tracking, positioning and placement
Phone cells
WiFi hotspots
GPS
Virtual media
Physical markers: 2D barcodes, RFID, user ID to phone

70. Enabling Technologies
Tracking, Positioning and Placement
* Global Positioning System (GPS)
30 geo-stationary satellites -> location, speed, direction, path
Shadows, accuracy
Use: CYSMN?, GPS drawing, Drift
GPS-enabled phones, PDAs
Platform: Geotracing

71. Enabling Technologies
Tracking, Positioning and Placement
* Geotracing

72. Enabling Technologies
Tracking, Positioning and Placement
* Placing media: socialight.net
In-place and remote annotation with smart-phone /PDA
social network community
sound, text, images, video
google maps + GPS

73. Enabling Technologies
Tracking, Positioning and Placement
* RFID
Radio-frequency identification
Storing and remotely retrieving data
Storage & processing + antenna
Physical markers
Tagging objects
Range: 5-20cm
Passive (powered by inductivity when used) vs active RFID

74. Enabling Technologies
Tracking, Positioning and Placement
* RFID
Uses:
Passports
ransport payments
Product tracking
Automotive
Animal identification
RFID in inventory systems
Human implants
RFID in libraries
Controversy: privacy issues. Shielding?

75. Enabling Technologies
Tracking, Positioning and Placement
* 2D barcodes
QR (Quick Response) code, Datamatrix code, etc
Physical markers
Can store between one and 500 characters
Tag objects, places
Scan with cameraphones -> hyperlink (physical mobile interaction)
How to: Kaywa reader + generator:

76. Enabling Technologies
Tracking, Positioning and Placement
* Unique ID to phone
Physical markers with unique IDs
Tag objects, places
Send number to server -> store & retrieve media
Arrows available, but not ID generator

77. Enabling Technologies
Available to the General Public
* Sensing:
sensors
data processing: microcontrollers

78. Enabling Technologies
Sensor Data Processing
* Micro-controllers
Basic Stamp II, Basic X – 24
Tutorial:
Arduino
open source hardware physical computing I/O platform
cheap (20 Euro)
easy (Processing)
assemble yourself
stand-alone or connect to computer (MAX/MSP, etc)

79. Enabling Technologies
Available to the General Public
* Creating and manipulating content:
Mobile Processing
Python
J2ME
miniMIXA
PdA (Pd on PDAs, linux)
Keyworx

80. Enabling Technologies
Creating and Manipulating Content
* Mobile Processing
Open source programming environment for design and prototyping software for mobile phones.
Similar to Processing environment.
Runs on Java powered mobile devices.
Bluetooth -> communication
Control example: attach light sensor on screen so sending info from phone to laptop

81. Enabling Technologies
Creating and Manipulating Content
* MiniMIXA
Commercial DJ software for mobile phones, PDAs
* Keyworx
Multimedia platform (base for GeoTracing f.ex.)
* PDa (Puredata anywhere): Pd for Linux on PDAs

82. Enabling Technologies
Creating and Manipulating Content
* Python PyS60
Interactive object-oriented language
Nokia S60 phones and more
Record, playback, play MIDI notes, control MAX/MSP patches...
PyS60: and
Tutorial (Jürgen Scheible - Mobilenin)

83. Enabling Technologies
Hacking mobile phones
3rd party software (Java, etc)
Hacking hardware: use camera, microphone, speakers, audio out...

84. Sustainable Design?

85. Sustainable Locative Media?
Issues
Problem in particular with Ubicomp: technology spread everywhere
Production, use, reuse, disposal
Use of energy + where to get it from?
Computers get smaller but not batteries
Issues with spreading technology into the wild: not as controlled environment as homes or offices
Littering: what happens to the embedded technology after use or break-down? who is responsible/accountable ?
Physical & virtual littering?
Peak oil!

86. Sustainable Locative Media?
Possible Approaches
Recycling?
Use of existing material and sources of energy?
Biodegradable material, f. ex. paper markers?
The simpler the better?
Wearability?
When should power be on? How should the system know when it should be on/off?

87. Sustainable Locative Media?
Design Inspirations
* Hacking
Repurposing existing technology

88. Sustainable Locative Media?
Design Inspirations
* Parasating?
Re-using existing features and properties of space and sources of energy in the environment: power, airflow, conductivity, etc.
paraSITE
Glitch (Tejp)

89. Sustainable Locative Media?
Design Inspirations
* Body-generated energy?
steps, body-heat, etc
Humand-Powered Objects Workshop: Bike4Tea, DynamoMouse...

90. Sustainable Locative Media?
Design Inspirations
* Ephemeral computing (Jernström)?
Deploying and packing up temporary and re-usable ubicomp infrastructures
SiSSy (video)

91. Resources: http://www.cs.chalmers.se/idc/ituniv/kurser/07/uc/locmedia/