1 Towards Pervasive Connectivity in Mobile Computing Frank Siegemund European Microsoft Innovation Center November 2006

2 Outline  Motivation  Related Work  System Architecture  Implementation  Application  Conclusions

3 Motivation  Most middleware assumes basic connectivity –Web Services –Jini –Corba  Problematic in mobile scenarios –Mobile P2P systems –Mobile CSCW  Heterogeneity –Bluetooth, WLAN, IrDA, GPRS –IPv4 vs. IPv6 –NATs and firewalls

4 Mobile P2P  Bootstrapping problem  Establish connectivity between mobile users  Mobile devices as first- class citizens in P2P applications  Exploit knowledge about connectivity in the group  More reliable/robust group establishment

5 Scenario  Content sharing between mobile users  Conference scenario  Skiing trip  Existing group collects connectivity parameters  Sharing by means of visual tags/glyphs  Handheld-embedded cameras for capturing tags

6 Outline  Motivation  Related Work  System Architecture  Implementation  Application  Conclusions

7 Related Work  Capturing visual tags with handheld devices and low-cost cameras –Rekimoto et al.  Visual tags in Pervasive Computing –Rohs and Zweifel –Many interesting scenarios  Visual tags to bypass Bluetooth device discovery –Scott et al.

8 Related Approaches  Out-of-band discovery for Bluetooth –IrDA –RFID  Collaborative approaches for device discovery –Siegemund and Rohs  Manual configuration  Send SMS with information about group –Complicated from user perspective

9 Outline  Motivation  Related Work  System Architecture  Implementation  Application  Conclusions

10 System Architecture

11 Collecting Connectivity Data  Overlay between group members  Exploit heterogeneity –Various communication technologies in group –Different security constraints  Construct group connectivity profile  Create tags to transmit connectivity profile  Candidate group member captures tags  Simplified user interaction

12 Group Overlay  Overlay can contain both mobile and stationary nodes  Overlay provides multicast group  Nodes in group have synchronized state  Connectivity properties of nodes shared between group members  NATs can be dealt with via different group peers  Direct connections not always possible  Group used to circumvent connectivity constraints

13 Visual Tags  Tags captured from video stream  Captured by handheld- embedded digital cameras  Direct user feedback about detected tags possible  Small video resolution requires displaying multiple tags  Concrete tag layout unimportant

14 Outline  Motivation  Related Work  System Architecture  Implementation  Application  Conclusions

15 Implemenation  Heterogeneous environment –Windows Mobile 5.0 smartphones –Window XP/Vista desktops  Implementation on desktops using Windows P2P Infrastructure  Mobile devices join/create P2P overlay –Socket interface –Lightweight database for data synchronization  Native implementation

16 Visual Tags  Tag detection in DirectShow  Analyze single video frames  Basic algorithm –Black and white conversion –Region detection –Identify guide bars –Decode tags –Determine tag index –Finish if all tags have been decoded –Join group Similar to Rekimoto et al.

17 Improvements  Determine threshhold for black/white conversion in feedback loop –Different lightning conditions  Mark tag in video frame to provide feedback to users –Tag detected or not  Synchronization –Tag contains time a single tag is displayed –Capturing device can calculate schedule for reading tags

18 Evaluation  240 x 320 video frame resolution  Implementation on 195 Mhz smartphone –State of the art (no high-end model)  400 ms per frame –Can be improved significantly !!! –Copy filter  90 ms for thresholding  250 ms for region detection  Algorithm deals with small rotations of the tag  More sophisticated tag detection systems available

19 Outline  Motivation  Related Work  System Architecture  Implementation  Application  Conclusions

20 Example Application  Mobile P2P chat application  Messaging for user groups on mobile devices  Overlay to distribute connectivity parameters  Tags to exchange connectivity profiles  Deal with NATs and connectivity constraints

21 Outline  Motivation  Related Work  System Architecture  Implementation  Application  Conclusions

22 Conclusions  Basic connectivity problematic in mobile scenarios  Current middleware solutions insufficiently address this problem  Facilitate groups to establish connectivity in mobile scenarios  Groups create finger print of network environment  NATs and security constraints can be dealt with  Visual tags user friendly way to communicate connection properties  Useful for mobile P2P and CSCW scenarios  Real-life applications

23 Contact Frank Siegemund European Microsoft Innovation Center Ritterstrasse 23, Aachen