1. FLoD: A Framework for Peer-to-Peer 3D Streaming IEEE INFOCOM 2008 Shun-Yun Hu*, Ting-Hao Huang, Shao-Chen Chang*, Wei-Lun Sung*, Jehn-Ruey Jiang*, and Bing-Yu Chen National Central University*, National Taiwan University Taiwan, R.O.C.

2. Outline Networked Virtual Environments (NVEs) Motivation P2P NVE Neighbor Discovery P2P 3D Streaming Conclusion

3. Outline Networked Virtual Environments (NVEs) Motivation P2P NVE Neighbor Discovery P2P 3D Streaming Conclusion

4. 4

5. 5 Adaptive Computing and Networking Lab, CSIE, NCU

6. 6

7. NCU ACNLab

8. Massively Multiplayer Online Games MMOGs are growing quickly  Multi-billion dollar industry  10 million subscribers for World of Warcraft  600,000 concurrent users

9. NASA World Wind

10. Google Earth

11. Zoom in…

12. NCU ACNLab To HsinChu..

13. and NTHU..

14. It is going to be 3D

15. NTHU 3D Student Center…

16. To the ground

17. 3-Dimensional Virtual Tourism Google Earth Virtual Earth NASA World Wind X3D Earth NCU ACNLab

18. DARPA SIMNET: screenshot From Bruce Sterling's "War is Virtual Hell,“ (1993).

19. CALVIN: a distributed collaborative virtual environments for architectural layout designs (1996)

20. 20 NVE A networked virtual environment (NVE) is a computer-generated virtual world where multiple geographically dispersed users can assume virtual representatives (or avatars) to concurrently interact with each other in real time through networked devices. Also called a distributed virtual environment (DVE)

21. 21 NVE Examples of NVEs include  early DARPA SIMNET and DIS systems  early distributed collaborative virtual environments  currently booming Massively Multiplayer Online Games (MMOGs)  future multi-user 3DVTs  future multi-user Web 3D browser

22. Outline Networked Virtual Environments (NVEs) Motivation P2P NVE Neighbor Discovery P2P 3D Streaming Conclusion NCU ACNLab

23. Motivation Two future trends for NVEs  More and more users  More worlds with larger and more dynamic contents

24. NCU ACNLab Motivation Q: How to support millions of concurrent users? A: Utilizing peer-to-peer schemes to relief the load of servers

25. NCU ACNLab Motivation Q: How to support large and dynamic worlds? A: 3D streaming is needed

26. Outline Networked Virtual Environments (NVEs) Motivation P2P NVE Neighbor Discovery P2P 3D Streaming Conclusion NCU ACNLab

27. 27 Model for NVEs Many nodes on a 2D plane An avatar needs to know only those within Area of Interest (AOI)‏ ★ : self ▲ : neighbors Area of Interest (AOI)

28. How does each node receive the relevant messages? Completely connected point-to-point Client/Server Client/Server cluster Partially connected point-to-point (peer-to-peer) NCU ACNLab

29. 29 A simple solution (point-to-point)‏ But…too many irrelevant messages N * (N-1) connections ≈ O(N 2 )  Not scalable! Source: [Funkhouser95]

30. 30 A better solution (client-server)‏ Message filtering at server to reduce traffic N connections = O(N)  server is bottleneck Source: [Funkhouser95]

31. 31 Current solution (server-cluster)‏ Still limited by servers. Expensive to deploy & maintain. Source: [Funkhouser95]

32. The Problem Client-server: resources limited by provisioning Resource limit [Funkhouser95]

33. The Solution Peer-to-Peer: resources grow with demand Resource limit [Keller & Simon 2003]

34. 34 P2P NVE Neighbor Discovery We need to solve the neighbor discovery problem in a fully-distributed, message- efficient manner with specific goals:  Consistent  Good neighborship consistency  Scalable  Limit & minimize message traffics  Responsive  Direct connection with AOI neighbors

35. 35 Neighborship Consistency (1) Definition # of current AOI neighbors observed # of current AOI neighbors

36. 36 :is observed neighbor Neighborship Consistency (2) An example Neighborship Consistency = 4 / 5 = 80% :is actual neighbor

37. Voronoi-based Overlay Network : VON Use Voronoi diagram to solve the neighbor discovery problem  Each node constructs a Voronoi diagram of its neighbors  Identify enclosing and boundary neighbors  Mutual collaboration in neighbor discovery

38. 38 Voronoi Diagram 2D Plane partitioned into regions by sites, each region contains all the points closest to its site Site Region

39. Voronoi-based Overlay Network : VON ● node i and the big circle is its AOI ■ enclosing neighbors ▲ boundary neighbors ★ both enclosing and boundary neighbors ▼ normal AOI neighbors ◆ irrelevant nodes

40. 40 Procedure (JOIN) 1)Joining node sends coordinates to any existing node Join request is forwarded to acceptor 2)Acceptor sends back its own neighbor list Joining node connects with other nodes on the list Acceptor’s region Joining node

41. 41 Procedure (MOVE) 1)Positions sent to all neighbors, mark messages to B.N. B.N. checks for overlaps between mover’s AOI and its E.N. 2)Connect to new nodes upon notification by B.N. Boundary neighbors New neighbors

42. 42 Procedure (LEAVE) 1)Simply disconnect 2)Others then update their Voronoi diagram new B.N. is discovered via existing B.N. Leaving node (also a B.N.) New boundary neighbor

43. Outline Networked Virtual Environments (NVEs) Motivation P2P NVE Neighbor Discovery P2P 3D Streaming Conclusion

44. What is 3D streaming? Continuous and real-time delivery of 3D contents over network connections to allow user interactions without a full download. Contents are fragmented, transmitted, reconstructed, and then displayed.

45. 3D streaming vs. media streaming Video / audio media streaming is very matured User access patterns are different for 3D content  Highly interactive  Latency-sensitive  Behaviour-dependent  Non-sequential

46. 4 types of 3D streaming Object streaming Scene streaming Visualization streaming Image-based streaming

47. 47/ Object streaming Hoppe 1996 Progressive Meshes

48. 48/ Scene streaming Many objects Object selections & transmissions Teler &Lischinski 2001

49. P2P-based 3D Streaming Models & assumptions  Many 3D objects(position, orientation)  User navigations withAOI visibility  Objects are fragmented (base & refinement pieces)  Data are initially stored at server

50. 50/ NCU ACNLab Observation Limited & predictable area of interest (AOI)‏ Overlapped visibility = shared content

51. FLoD: Flowing Level-of-Details Assume P2P-NVE overlay, such as VON Download 3D content from AOI neighbors Basic design  Each object has a unique ID and associated progressive mesh data  Scene description records object ID, orientation, and scale  World is partitioned into cells

52. IEEE INFOCOM 2008

53. Steps of FLoD 1. Peers exchange incremental formation of cached 3D data periodically 2. A peer sends requests for downloading scene descriptions 3. A peer sends requests for downloading object data 4. A peer asks the server if none of the peers responds

54. Object Prioritization Visual importance

55. Peer selection Multi-level request

56. Prototype experiment Progressive modeling and rendering of the scene (by NTU) P2P neighbor discovery and 3D streaming (by NCU)

57. Partition Cell-based construction Use an actual game scene 100x game scene (514KB -> 51.8MB)

58. Fragmentation

59. Simulation setup Environment  1000x1000 world, 100ms / step, 3000 steps  client: 1 Mbps / 256 Kbps, server: 10 Mbps (both)‏ Objects  Random object placement (500 objects)‏ User behavior  Random way-point  clustering movement (1.5 * ln(n) hotspots)‏

60. Metrics Server's perspective  Requests can be redirected (bandwidth usage) User's perspective  Visual quality (fill ratio)  Interactivity (base latency)

61. Server bandwidth usage

62. Client bandwidth usage

63. Fill ratio

64. Base latency

65. Outline Networked Virtual Environments (NVEs) Motivation P2P NVE Neighbor Discovery P2P 3D Streaming Conclusion

66. NVE neighbor discovery can be solved in a P2P manner with  Good neighborship consistency  Low message overhead NVE 3D streaming can be solved in a P2P manner with  Minimal server resource usage  Low control message overhead  Short Latency

67. Conclusion Future work  Multi-user X3D browser  Multi-user X3D browser with P2P 3D streaming  TOPOS (Greek; common place): Authenticable P2P personal 3D space publication and navigation system  H E AVEN (a Hybrid Architecture for massively multi-user Virtual ENvironments): client assisted MMOGs

68. Adaptive Computing and Networking Lab. Q & A Thank you! http://acnlab.csie.ncu.edu.tw

69. 69 Related Work (1): DHT-based: SimMUD [Knutsson et al. 2004] (UPenn) Pastry (DHT mapping) + Scribe (Multicast) Fixed-Sized Regions Coordinators

70. 70 PASTRY (P2P overlay) SCRIBE (Multicast support) MMOG GAME Related Work (1): DHT-based: SimMUD

71. 71 Related Work (1): DHT-based: SimMUD

72. 72 Related Work (2): Neighbor-list Exchange [Kawahara et al. 2004] (Univ. of Tokyo) Fully-distributed Nearest-neighbors List exchange High transmission Overlay partition

73. 73 Related Work (3): Mutual Notification: Solipsis [Keller & Simon 2003] (France Telecomm R&D) Links with AOI neighbor Mutual cooperation Inside convex hull Potentially slow discovery Inconsistent topology

74. 74/ Visualization streaming Large volume Time-varying Dedicated servers Olbrich & Pralle 1999

75. 75/ Image-based streaming Server- rendered Thin clients Less responsive Cohen-Or et. al. 2002