TouchDevelop Chapter 5-7 Chapter 5 Audio Chapter 6 Camera, Graphics and Video Chapter 7 Sensors Mengfei Ren

5 Audio 5-1 Music 5-2 Radio 5-3 Sounds 5-4 Microphone 5-5 Media Player on a Home Network

5 Audio A smartphone could play music such as MP3 files, audio streamed over the internet or media center All the media files could be synchronized to the phone using Zune, or purchased and downloaded via the Marketplace and the Internet

5-1 Music The smartphone supports three music formats, AAC, MP3, and WMA. The TouchDevelop API provides the media resource for retrieving collections of all the songs, albums, and playlists. However, these collections couldn’t changed by TouchDevelop. [1]

5-1 Methods for Songs, Albums and Playlists [1]

5-1 Methods for Songs, Albums and Playlists(cont.) [1]

5-1 Play the Songs When there is new album or playlist sent to the player, it creates a queue of songs to be played. The songs will be played in order that they appear in the album or playlist. Requesting to play a new song when there is a song playing, the current one will be terminated and the queue be cleared before the new one starts. Playing songs is a background event.[1] Demo 5-1

5-1 Methods for Playing Songs

The song’s volume ranges from 0.0 to 1.0. The value of 0.0 means mute, however the value of 1.0 does not mean the maximum which the phone is capable. The value of 1.0 is related to the current volume of the cell phone. If the phone is under mute mode, the speaker is set to off and there is no any audible sounds.

5-2 Radio The radio plays as a background activity. The radio will only stop if the method radio  stop is called or if the user shuts off the radio externally to the script. [1] Demo 5-2

5-3 Sounds The Sound datatype is used for audio recordings in the WAV format. The files in WAV format is commonly used used for short sound clips (30 seconds or less).[1] – This format is commonly used for uncompressed audio and therefore the files tend to be large. – No method is provided for stopping playback of the clip before it is finished.

5-3 Methods for Sound Datatype

5-4 Microphone API for making a recoding var snd := senses  record microphone The word “Recording…” and a stop button will be displayed on the screen.

5-5 Media Player on a Home Network A media center can transmit videos and music onto the phone over a Wi-Fi network. The phone could be a remote control for the media player. TouchDevelop provides the Media Server datatype for a media center. The script could search and retrieve the songs and videos. To play a song from media server: player  play home media (mlink) Demo 5-3

5-5 Methods for Media Server[1]

6 Camera, Graphics, Video 6-1 Camera 6-2 Static Graphics Drawing and Display 6-3 Playing Videos from the Internet 6-4 Media Player

6-1 Camera To create the picture, the cell phone should at least have a camera on the back, which is the primary camera. To access the camera, TouchDevelop provides the senses service. Also the Camera datatype to retrieve information about the camera and take a quick low- quality picture. To show the picture on the wall, there are two methods.

6-1 Ways to Take Pictures High-quality picture: senses  take camera picture Not immediately take the picture. It calls the phone’s built-in software for using the camera and displays a preview image on the screen. Take the picture until the button pressed. Low-quality picture: senses  front camera  preview Demo 6-1

[2] Demo 6-2

6-1 The at, pixel and set pixel Methods The methods could be used only on pictures containing a modest number of pixels. The picture taken by the camera contain as many pixels as the camera’s resolution. When showed on the screen, the picture is scaled to fit the screen’s size, however, its original number of pixels is stored in the memory. The at and pixel methods both retrieve the color of a particular pixel.

6-1 The at, pixel and set pixel Methods (cont.) However, the at method does not care where the pixel is located within the picture. It provides more efficient access all the pixel than the pixel method. The equivalence between the two ways to access a particular pixel:

6-1 Picture Effects [2]

Example: Picture Overlaying Demo 6-6

6-2 Static Graphics Drawing TouchDevelop provides many methods for drawing lines and shapes. The create picture method is to create a new board to draw lines and shapes. var pic := media  create picture(400,200) Demo 6-3

6-2 Drawing Methods Picture [2]

6-3 Playing Videos from the Internet TouchDevelop doesn’t provide access to video files held on the phone. Nor can a script download a video to the phone. A script can access and play videos which are streamed from the internet or a media server.[2] Demo 6-4

6-4 Media Player A media center could send pictures to the phone and stream videos to the phone. [2]

6-4 Media Player (cont.) Demo 6-5 TouchDevelop does not provide methods to stop the video playing from within the script. To stop the playing video, – Tap the screen and see the pause/continue button – Pause the video and tap the back button to stop the execution in the script.

7 Sensors 7-1 The Sensors 7-2 Sensor-Driven Events 7-3 Accelerometer 7-4 Compass 7-5 Gyroscope 7-6 Motion

7-1 Sensors GPS(Global Positioning System): obtains the phone’s current location on the map Accelerometer: measures gravitational and acceleration Compass: returns the direction of magnetic north Gyroscope: measures the phone’s orientation in 3D space [3]

7-1 Sensing Methods [3]

7-2 Sensor-Driven Events A script can use the event of effect an action, such as pausing the playback of an audio recoding. Demo 7-1 [3]

7-3 Accelerometer The phone measures the force by the accelerometer when it is shaken or moved, which combines acceleration with gravity. The acceleration is in any direction in three dimensional space. The TouchDevelop API use the Vector3 datatype to present the current force in three dimensions. Demo 7-2, 7-3

7-3 Accelerometer (cont.) (0,0,-1) – the phone is lying flat on the table (0,-1,0) – the phone is held vertically with its bottom edge on the table The TouchDevelop reports the force in g (gravitational) units. Any stationary object is subjected to a force of 1g in the downwards direction [3]

7-4 Compass Report the phone’s orientation with respect to magnetic north. Demo 7-4

7-5 Gyroscope

7-6 Motion The TouchDevelop provides the Motion datatype to present the current location which is measured by the combination of the various sensor components. [3]

7-6 Motion (cont.) The force caused by gravity and the additional force caused by acceleration is separated. The method just returns a true acceleration without the gravitational force. The accurate value of the orientation can be obtained for the time when it was captured. The true orientation is known as its attitude that measured via three methods, pitch, roll and yaw.

7-6 Motion (cont.) These methods is measured in degrees, respectively relative to three orthogonal axes. The axes are diagrammed in the plane.

References [1] bookchapter5.pdfhttps://az31353.vo.msecnd.net/cpd/kyei- bookchapter5.pdf [2] bookchapter6.pdfhttps://az31353.vo.msecnd.net/cpd/axwq- bookchapter6.pdf [3] bookchapter7.pdfhttps://az31353.vo.msecnd.net/cpd/mxnl- bookchapter7.pdf