1. Palm Multimedia and VoIP Design CS525 Semester Research/Design Marc Pevoteaux Ron Erickson

2. Palm History  The Original Pilot 1000 Model Pilot 1000 Model  Manufactured by Palm Computing in 1996 division of U.S. Robotics  Jeff Hawkins – Palm Pilot Inventor

3. Palm History  The Latest Version of Palm Hardware Palm Treo  Computabilty with Windows Mobile Platform

4. Palm OS Cobalt  Palm OS Cobalt Architecture

5. Multimedia Sound Simple Sound  Single voice, monophonic, square-wave sound synthesis, system alerts.  A single-tone (individual amplitude, pitch, duration) can be played by calling SndDoCmd.  Pre-defined system sounds (alerts) can be played through the SndPlaySystemSound method.  Standard MIDI can be played through the SndPlaySmf function..

6. Multimedia  Sampled Sound  Stereo, multi-format, sampled data recording and playback  SndStreamCreate opens a new sampled sound "stream" from/into which you record/playback buffers of "raw" data.  SndPlayResource is used to play sound data that's read from a (formatted) sound file. Creates a simple stream based off the data header.

7. Multimedia  MIDI  Musical Instrument Digital Interface  Records stored in a MIDI database file (sysFileTMidi).  Can be passed to SndPlayResource method.  Can be created programmatically using DmNewRecord and DmWrite.

8. Multimedia  Sound Streams  A sound-stream reads or sends sampled data from the hardware.  Built in support for:  sndFormatPCM  sndFormatIMA_ADPCM  sndFormatDVI_ADPCM  sndFormatMP3  sndFormatAAC  sndFormatOGG

9. Multimedia  Sound Streams  Several streams (15) can run concurrently.  Quantization, sampling rate, channel count must be sent via SndStreamCreate().  Buffer contents processed by SndStreamStart() and SndStreamBufferCallback().  All callbacks run on the same process. If a callback for a stream takes to long other callbacks will be delayed and the stream will seem “glitchy”.  Double buffering scheme allows read and write simultaneously.

10. Multimedia  Multimedia Sessions  Sources  Specifies a device such as a microphone or camera  Created by calling MMSessionCreate().  Destinations  Specifies a device such as a screen or the speakers.  Setup by calling MMSessionAddDest().

11. Multimedia  Multimedia Sessions – Continued  Streams  Setup by finalization of Source or Destination.  Sources and Destinations have at least one stream associated with them, each stream represents a single kind of data it processes. Usually more than one stream is assigned (audio and video).  Tracks  A track defines a route from a source device to a destination device.  Tracks responsible for encoding or decoding (using codecs) and routing from Source to Destination.  Sources and Destinations can have multiple tracks so that each stream can be routed to a separate device.

12. Multimedia  WiFi Connections  API provides support for WiFi wireless networking through a set of IOCTL commands that can be used to find, connect, and communicate over wireless networks.  A WiFi interface can be located by calling IOSGetDriverNameByIndex().  IOSIoctl Function  Returns information about the devices capabilities such as encryption support, transmission rates, signal strength, or SSID can be queried by using  Returns information about available networks.

13. Multimedia  Applications  Applications start, listen, and end.  Applications poll for events.  System event queue and Application event queue. System passes events to the Application if they are not handled.  Application only have to listen for their events.  Events can communicate with separate threads.

14. Palm VoIP Design  Concept create a simple VoIP application that utilized existing Palm hardware.  Hardware  Microphone  Speakers  WiFi Capability  Software  Client Application  Two separate streams (TX and RX)  Server Application

15. Palm VoIP Design

16.  Client Application  Establish a Wireless connection to a server  Receive data – bring in and digitize the voice through the hardware (microphone)  Encode data – encode data into a compressed format  Transmit Data – transmit data over internet connection (WiFi)  Receive Data – receive data from internet connection (WiFi)  Decode Data – decode compressed data  Present Data – send data to hardware (speakers) for audio presentation  Manage events

17. Palm VoIP Design  Connecting  WiFi through IOSGetDriverByNameByIndex() and managed through ioctl functions.  Transmitt Data (TX)  Transmit Session  Source Microphone (ADC)  Destination NIC/WiFi card (packeted stream)

18. Palm VoIP Design  Encoding Data (TX)  Track routes data through codec.  Palm hardware limited in processing power.  Special ITU-T voice codec designed for low packet construction delay  G.711u, 64Kbps data rate, 1ms construction delay.  Track routes packeted data to NIC.  Transmitting Data (TX)  No dedicated circuits = network delay. Subnet would need to be used for proof of design to guarantee QoS.

19. Palm VoIP Design  Receiving Data (RX)  Receive Session  Destination NIC/WiFi card (packeted stream)  Destination Speakers or Headset  Decoding Data (RX)  Track routes data through codec for decompression.  Presenting Data  Track routes decompressed data to destination (speakers).  Managing Events  Application event loop montiors both transmit and receive events.

20. Palm VoIP Design  Server Application  Simple router  Complex – Dedicated service lines, database to manage connections.

21. Palm VoIP Design  Feasibility  Exploration of the Palm API and hardware capabilities reveal that a VOIP application could potentially be created.  Implementation of a VoIP application using the standard API and Palm hardware would most likely only be able to be demonstrated in concept.  Default hardware acquisition rates would most likely delay conversion to packets and a transmission delay would almost always be present due to non-dedicated network circuits.  In reality success of such application would require incorporating a custom DSP piece of equipment that could convert voice directly into packets and possibly working with a ISP to put together a dedicated network that could guarantee QoS.

22. Summary  Palm OS Cobalt has not become widely accepted Garnett.  Palm OS Cobalt is a complete rewrite of the Palm OS architecture.  Cobalt ensured that the older 68K-based applications will continue to run.  Cobalt has built-in sound manager for mixing sound, background and a video playback.  The Palm hardware and software could work together using the existing API’s and WiFi hardware to use the Internet to transmit telephone calls …. VoIP.