Real-Time Cities: an Introduction to Urban Cybernetics Harvard Design School: SCI Spring 2014 Jason Lipshin Exercise #3: Case Studies in Informatically Driven Actuation

Actuation + Scale For this assignment, I focused on various ubiquitous computing projects which provided dynamic feedback to the user through novel forms of mechanical/physical actuation. I attempted to organize my case studies according to the heuristic of scale – from small everyday objects to interventions taking place at the scale of a building. As mentioned previously in this class, scale is often used as an organizing or taxonomic device in ubiquitous computing research. Particularly within popular HCI literature, like Mike Kuniavsky’s Smart Things and Adam Greenfield’s Everyware, the relationship between ubicomp and scale plays an important role. This emphasis can even be seen within the earliest days of the field – Mike Weiser’s enumeration of “pads, tabs, and boards” can be read as a taxonomy of scale. 1 | Move-It Sticky Notes 2 | Botanicus Interacticus 3 | Stock Market Skirt 4 | CoMotion Bench 5 | Co2morrow

URL: 1 | Move-It Sticky Notes (Kathrin Probst et al, 2014) A lot of people still rely on pen and paper for taking short notes. Post-its are still the most popular paper media for informal note taking. With Move-it Sticky Notes, the designers present the prototype for an active paper interface that combines the affordances of note taking on paper with the capabilities of digital systems and extends it with active feedback functionality. Move-it Sticky Notes combine traditional Post-it notes with a technologically enhanced paperclip, which can be moved and thus give active feedback to the user. Three different demo applications demonstrate applicability of the system for active reminding, status monitoring, and interactive bookmarking.

1 | Move-It Sticky Notes

Which of the 10 possible categories of scenarios does the Project belongs to? Answer: VIII. Environments, architectural spaces, and smart artifacts that solicit people’s needs and desires that they sense via direct input from the inhabitants, or sensed from contextual information. Which sensing technology/ types of sensors are deployed to impliment spatial gaze? Answer: The project uses a smart pen and handwriting recognition software to sync the user’s handwritten notes with personal information management (PIM) tools. However, the most interesting part of the project is the actuation of the post-it paper folding using a smart paper clip. How the performance of the piece/instalation is conditioned? Answer: The smart clip attached to the piece of paper bends slightly when the user has a notification or reminder. This bending of the smart paper clip (made of shape memory alloy) provides the user with a visual cue or reminder apart from the clutter and information overload of the desktop.

1 | Move-It Sticky Notes Which actuation technology is used to provide for dynamism of the piece? Answer: Upon occurrence of a specified event, the Move-It smart paper clip gives active motion feedback by actuating the shape motion alloys which compose the paper clip. What connectivity technology is used to connect different parts of the informatically driven system at question? Answer: Each smart sticky note has a greycode on the backside of the note. Once identified, the Move-It desktop application is able to establish a distinct association between the digital notes and the physical paper note with attached greycode. Is the system operates based on a memory of the past and/or anticipation of the future through recognition of past patterns? Answer: No, there is no memory or adaptive quality to the system.

1 | Move-It Sticky Notes What is the extreme vision of the scenario? Answer: This scenario is about the subtlety of the interaction. I believe an “extreme vision” of the scenario would be anathema to the designer’s original intentions. Why is the project significant? Answer: The project is significant because it attempts to rethink the mundane nature of a digital notification or reminder in a more physical and playful way. What do you identify as a poetic aspect to the logic of operation of the system at question? Answer: The physicality of the interaction and the sense of animacy bestowed on an inanimate object like a paper clip provides this project with a poetry and charm.

1) Move-It Sticky Notes What challenges are inherent in deployment of the project or maintaining the integrity or level of operation of the system in the long run? Answer: This project is very high tech for a simple interaction, so it is probably not scalable or sustainable from a cost perspective. However, I believe that it still represents an important way to bring a more physical aspect to the notion of a reminder. What is your evaluation of the design of the physical manifestation of the system in terms of its esthetics? Answer: Again, the primary charm of the system is in the sense of animacy and physicality bestowed to the paper clip. It’s aesthetics plays on the creation of a sense of wonder and cuteness, perhaps related to the notion of the techological sublime that was discussed early in this course. How can the system scale up or down to fit a different context of operation? How can its logic of operation or technology be re- appropriated for the different context? Answer: This project was built for the context of desktop computing where post-its are already used in abundance. This project could have other potential applications in other places where post-its are already in use (perhaps refrigerators, public bulletin boards, etc.)

1 | Move-It Sticky Notes How the system will deal with nodal or total failure in operation? Answer: Since the project is subtle, failure will likely just result in the paper clip not working. Nothing too apocalyptic would come of this total failure because of the localized and subtle nature of the interaction. What is the system’s logic of operation and information flow/architecture? Answer: The flow of the project moves from written notes to personal information management systems, whereby the handwritten notes on captured via the smart pen, analyzed via handwriting recognition software and imported to the calendar. Once the reminder had been synced with the personal information management system and the system recognizes a notification event, it sends that notification to the greycode attached to the back of the post-it note, which then bends the shape memory allow of the smart paper clip. What is the nature of actuation explored by the system? Does it explore generation of change that is registred through other senses beyond vision? Answer: The nature of the actuation is physically embodied within the paper clip, but from the user’s perspective it primarily relies on vision. However, rather than frontal visual interaction, this project engages peripheral vision – a topic of great interest in recent HCI literature.

Project Video: 2 | Botanicus Interacticus (Disney Research Pittsburgh, 2012) Botanicus Interacticus is a technology that transforms real, everyday plants into gesture-based, interactive control mechanisms. Simply by inserting an electrode connected to the Botanicus Interacticus system into soil, the system can sense when human beings touch the plant, working off the inherent electrical properties of the plant itself. Plants attached to Botanicus Interacticus respond to physical touch by humans. A digital “avatar” of the plant also grows in response to the frequency and quality of touch.

2 | Botanicus Interacticus Which of the 10 possible categories of scenarios does the Project belongs to? Answer: V. Where the space is created around a piece that one of the audience members interacts with the piece while others maintain the persona of spectators of the act. Which sensing technology/ types of sensors are deployed to impliment spatial gaze? Answer: The project uses a small electrode connected to the Botanicus Interacticus system. When inserted in soil, the electrode can sense when a plant has been touched, building off the inherent electrical properties of plants themselves (in other words, their internal circuitry). How the performance of the piece/instalation is conditioned? Answer: In the installation of the piece at the SIGGRAPH conference, the user was able to control a digital avatar of the plant on a flat screen, using the physical plant as a controller.

2) Botanicus Interacticus Which actuation technology is used to provide for dynamism of the piece? Answer: The piece is more about novel input techniques (gesture recognition and the plant as controller), rather than novel forms of actuation (the digital avatar of the plant on screen). What connectivity technology is used to connect different parts of the informatically driven system at question? Answer: Again, the plant’s natural circuit is immersed in soil, which is connected to the copper electrode, which is connected to the Botanicus Interacticus gesture recognition digital system. Is the system operates based on a memory of the past and/or anticipation of the future through recognition of past patterns? Answer: No, there is no memory or adaptive quality to the system.

2) Botanicus Interacticus What is the extreme vision of the scenario? Answer: The extreme version of the scenario could perhaps measure the effect of human gestural touch on plant growth over long periods of time. Thus, the scaling up could take place in a temporal sense. Why is the project significant? Answer: The project is significant because it rethinks gestural input beyond the typical models of the mouse or swipe on an iPhone. In a way, it seems very similar to the Makey Makey project completed by the Life Long Kindergarden group at the MIT Media Lab. What do you identify as a poetic aspect to the logic of operation of the system at question? Answer: The novelty of using a living thing like a plant as an input controller for a digital system is extremely provocative and poetic. There is also something interested about using the inherent electrical properties of plants, as opposed to simply building a regular circuit.

2) Botanicus Interacticus What challenges are inherent in deployment of the project or maintaining the integrity or level of operation of the system in the long run? Answer: Although the designers say that it implements machine learning techniques in order to be precise with the recognition of gestural input, I’m sure that there are issues with the system’s accuracy. Also, watering the plants would be difficult if they are attached to Botanicus Interacticus. What is your evaluation of the design of the physical manifestation of the system in terms of its esthetics? Answer: I think the aesthetics of the system are attractive and minimalist and although there is complicated technology going on within the digital part of the system, the simplicity of the circuit created provides a liveness in feedback that is clear and pleasurable for the user. How can the system scale up or down to fit a different context of operation? How can its logic of operation or technology be re- appropriated for the different context? Answer: As I mentioned in a previous answer, it could be interesting to see how touch affects plant growth over long- periods of time. This study could be akin to research done on the effect of classical music on plant growth.

2) Botanicus Interacticus How the system will deal with nodal or total failure in operation? Answer: Since the project is subtle, failure will likely just result in the system simply not working. What is the system’s logic of operation and information flow/architecture? Answer: The flow of the project moves from the gestures of the human user, whose vibrations and interruptions of the natural electrical circuit of the plant are sensed by an electrode in the soil. The electrode then sends that information to the Botanicus Interacticus digital system, which then reads the input and maps it to a digital representation of the plant on the screen. What is the nature of actuation explored by the system? Does it explore generation of change that is registred through other senses beyond vision? Answer: The nature of the actuation is simply visual output on a screen, but the input is gesture-based and wholly novel, engaging the sense of touch.

Stock Market Skirt is a feminist art project that extracts and analyzes stock prices from online stock market quote pages on the internet. These values are then used to determine whether to raise or lower a skirt via a stepper motor. According to the artist’s website: “when the stock prices rises, the hemline is raised; when the stock market prices fall; the hemline is lowered.” This project provides a novel feedback mechanism through the actuation of the dress’ dynamic hemline. URL: 3 | Stock Market Skirt (Nancy Patterson, 1996)

3) Stock Market Skirt Which of the 10 possible categories of scenarios does the Project belongs to? Answer: IV. Where the space is created around a piece that communicates the designer’s concept and the audience is engaged in a voyeuristic act. Which sensing technology/ types of sensors are deployed to impliment spatial gaze? Answer: The project is simply scraping real-time financial data from the Internet, so there are not really sensors in this project. How the performance of the piece/instalation is conditioned? Answer: In the installation of the piece, the user is confronted with screens which display stock price information. The smart dress sits at the center of these TV screens and the dress’ hemline is raised or lowered depending on the levels culled from the financial data.

3) Stock Market Skirt Which actuation technology is used to provide for dynamism of the piece? Answer: A simple stepper motor is used to raise or lower the dress’ hemline. What connectivity technology is used to connect different parts of the informatically driven system at question? Answer: The artist’s website does not disclose this information, but I assume that the stepper motor is connected to some sort of micro-controller which is then connected to a computer with an Internet connection. Is the system operates based on a memory of the past and/or anticipation of the future through recognition of past patterns? Answer: No, there is no memory or adaptive quality to the system.

3) Stock Market Skirt What is the extreme vision of the scenario? Answer: The art project is already making a pretty extreme statement! Perhaps if someone was actually wearing the dress? Why is the project significant? Answer: The project is significant because it uses wearable technology to make a political statement. Many projects in this space rely simply on novelty and a DIY aesthetic, whereas this project humorously engages feminist issues in a novel medium. What do you identify as a poetic aspect to the logic of operation of the system at question? Answer: The poetic aspect of the project is the humorous comment on the connection between female sexuality and wealth within advanced Western capitalist societies.

3) Stock Market Skirt What challenges are inherent in deployment of the project or maintaining the integrity or level of operation of the system in the long run? Answer: The project was created in 1996 and still has staying power. I believe that it would be much easier to create and maintain this project with the advent of micro-controllers like Arduino and much easier ways to scrape data in real-time from the Internet. What is your evaluation of the design of the physical manifestation of the system in terms of its esthetics? Answer: The aesthetics of both the dress and the televisions screen are very 1990s and cyberpunk-inspired, looking very outdated by today’s standards. However, I still think that the conceptual framing of the project is interesting and relevant today. How can the system scale up or down to fit a different context of operation? How can its logic of operation or technology be re- appropriated for the different context? Answer: One could imagine different contexts in which real-time data manifests itself in a shape-changing wearable garment. I believe Neri Oxman in the Mediated Matter group and Leah Buchley of the High-Low Tech group at the MIT Media Lab have both explored this possibility.

3) Stock Market Skirt How the system will deal with nodal or total failure in operation? Answer: Since the project is subtle, failure will likely just result in the system simply not working. What is the system’s logic of operation and information flow/architecture? Answer: A digital program scrapes the Internet in real-time for financial data about the NYSE. If the stock prices rise, that information is sent to a a micro-controller which raises the hemline of the Stock Market Skirt. If the stock prices lower, than the same process makes the hemline of the skirt fall a bit. What is the nature of actuation explored by the system? Does it explore generation of change that is registred through other senses beyond vision? Answer: The nature of the actuation is a change in length to the hemline of a dress. It engages a sense of tactility as well as vision, in which actuation is executed directly on the body.

The CoMotion Bench is a shape changing piece of furniture. It explores how physical changes to the shape of the bench can change social situations around it. The bench was tested in three different contexts: a concert hall, an airport, and a shopping mall. In the user testing for this project, the researchers emphasized how contextual atmospheres may affect the users’ experience of the shape changing bench and its surroundings. URL: 4 | CoMotion Bench (Majken Kirkegaard Rasmussen et al, 2014)

4) CoMotion Bench Which of the 10 possible categories of scenarios does the Project belongs to? Answer: IX. Environments, architectural spaces, and smart artifacts that are digitally enhanced so that they act as mediators of what we define as NSA (No Strings Attached) or fleeting/fearless social encounters amongst their inhabitants. Which sensing technology/ types of sensors are deployed to impliment spatial gaze? Answer: Sensors are placed at various points along the length of the bench to detect the pressure of whether people are sitting on them. Depending on which sensors detect a person, linear actuators within the bench will change the bench’s shape to change the social relationships between the sitters. How the performance of the piece/instalation is conditioned? Answer: The users of the bench were not made aware that they were sitting on the smart bench – its actuatable nature was meant to be kept a surprise. Once the bench started to move, the sitters were initially unsure whether they were hallucinating, but quickly came to realize that the bench was trying to push them closer to their fellow sitter.

4) CoMotion Bench Which actuation technology is used to provide for dynamism of the piece? Answer: A series of linear actuators which change the degree of incline of the bench, the bench’s length, etc. What connectivity technology is used to connect different parts of the informatically driven system at question? Answer: The pressure sensors and linear actuating system are all contained within the bench object itself. Is the system operates based on a memory of the past and/or anticipation of the future through recognition of past patterns? Answer: No, there is no memory or adaptive quality to the system.

4) CoMotion Bench What is the extreme vision of the scenario? Answer: One could imagine a more anthropomorphized bench which more actively tried to push its sitters together. At present, the bench’s interactions were more subtle such that the user is even unsure if it is changing shape. Why is the project significant? Answer: The project is significant because it tries to reconfigure social relations within a public space. It does so through a novel, smart object which seems to be endowed with a cheerful personality, which tries to bring sitters on a bench closer together. What do you identify as a poetic aspect to the logic of operation of the system at question? Answer: The poetic aspect of the project is the humor and subtlety of the interaction. When presenting this project at the Tangible, Embedded, and Embodied Interaction Conference in Munich this February, the designer said that many users of the system believed that the bench had a “personality.” Achieving this effect through a series of sensors and actuators speaks to the sense of poetry created by this public art project and intervention.

4) CoMotion Bench What challenges are inherent in deployment of the project or maintaining the integrity or level of operation of the system in the long run? Answer: The project is merely a proof of concept and its highly complex mechanical nature will surely show wear-and- tear as more people sit on the bench. Also, it is unclear if the project will still be interesting after its initial novelty wears off. What is your evaluation of the design of the physical manifestation of the system in terms of its esthetics? Answer: The aesthetics of the system are highly attractive. Maintaining the subtlety of interaction with the bench such that people tend to anthropomorphize the object speaks to the skill in execution of the mechanical system. How can the system scale up or down to fit a different context of operation? How can its logic of operation or technology be re- appropriated for the different context? Answer: I think that user testing with the bench already demonstrated the idea that the object would perform differently depending on the context (i.e. shopping mall vs. airport, etc.) However, one could also imagine other kinds of furniture which also created new forms of interaction between users.

4) CoMotion Bench How the system will deal with nodal or total failure in operation? Answer: Failure of the system could be dangerous to the sitter or totally mundane. With regard to the latter, the CoMotion bench would simply turn into a regular bench. What is the system’s logic of operation and information flow/architecture? Answer: Pressure sensors embedded along the length of the bench detect the presence of a human sitter. Depending on the location of the sitter, a system of linear actuators will change the shape of the bench, such that the two or more sitters become closer to one another. What is the nature of actuation explored by the system? Does it explore generation of change that is registred through other senses beyond vision? Answer: The nature of the actuation is the shape changing nature of the bench. The nature of the interaction is both tactile and visual.

C02morrow is an art installation exhibited on the façade of the Royal Academy GSK Contemporary. It is a living visualization of greenhouse gas fluxes from a network of sensing technologies in the UK. The physical form of the sculpture is based on a scrubber molecule which helps to sequester CO2 out of the air. The sensing technology was provided by Siemens Smart Cities technology initiative. URL: 5| Co2morrow (Alessandro Marianantoni and Marcos Lutyens, 2010)

5) Co2morrow Which of the 10 possible categories of scenarios does the Project belongs to? Answer: IV. Where the space is created around a piece that communicates the designer’s concept and the audience is engaged in a voyeuristic act. OR I. Collective amazement and spectacular demonstration of the technological sublime Which sensing technology/ types of sensors are deployed to impliment spatial gaze? Answer: Co2 sensors are deployed throughout the UK. The scale of this project is amongst its key novelties. How the performance of the piece/instalation is conditioned? Answer: The sculpture is attached to façade of a building. Passerbys stand in awe of the beauty of the sculpture but the purple haze also gives them a real-time visualization of the air quality collected at various places all over the UK.

5) Co2morrow Which actuation technology is used to provide for dynamism of the piece? Answer: The actuation is simple the purple glow of a light whose brightness and intensity correlates to the volume of CO2 sensed. What connectivity technology is used to connect different parts of the informatically driven system at question? Answer: The artist’s website does not provide this information. Is the system operates based on a memory of the past and/or anticipation of the future through recognition of past patterns? Answer: No, there is no memory or adaptive quality to the system.

5) Co2morrow What is the extreme vision of the scenario? Answer: This scenario, because it is gathering data from sensors distributed across the UK, and because the display mechanism is already displayed on the façade of a building, is pretty extreme. The scale of intervention is already quite large both in terms of data collection and actuation. Why is the project significant? Answer: The project is significant because of its physical beauty and the scale of data collection. The Air Quality Egg project and the Warning Signs wearable computing project mentioned in my earlier case studies presentations have a very similar concept, but here the data collection id decoupled from the visualization. That is what is interesting to me. What do you identify as a poetic aspect to the logic of operation of the system at question? Answer: The poetic aspect of the project is the visualization, which provides a “fuzzier” interpretation of air quality data than your typical bar graph. Provided the information in this way is interesting, though its capabilities to actually inform someone about environmental issues is suspect.

5) Co2morrow What challenges are inherent in deployment of the project or maintaining the integrity or level of operation of the system in the long run? Answer: The project is no longer in operation, but if it were, it would have to deal with maintenance of the AQ sensors distributed throughout space, as well as wear-and-tear on the physical sculpture. What is your evaluation of the design of the physical manifestation of the system in terms of its esthetics? Answer: The aesthetics of the system are highly pleasing – I like the idea of providing dwellers in urban space with a “fuzzier” impression of the air quality in their nation. However, again, I am unsure of the sculpture’s educational capabilities. How can the system scale up or down to fit a different context of operation? How can its logic of operation or technology be re- appropriated for the different context? Answer: Again, the system is already scaled to a very large level, both in terms of data collection and presentation. A “smaller version” of the project might look like the Air Quality Egg – sensing would take place locally within the object and then the object would glow certain colors depending on the quantity of the particulate matter sensed.

5) Co2morrow How the system will deal with nodal or total failure in operation? Answer: Failure of the system would likely result in a misleading visualization on the sculpture that no longer communicated meaningful data. What is the system’s logic of operation and information flow/architecture? Answer: Co2 sensors are distributed throughout the UK. Those sensors send real-time data on air quality to the sculpture in London. The digital system within the sculpture translates the amount of particulate matter sensed into the brightness of purple light emitted by the scuplture. What is the nature of actuation explored by the system? Does it explore generation of change that is registred through other senses beyond vision? Answer: The nature of the actuation is purely visual. Conceptually it is not very novel, but the execution aesthetically deserves merit.