1. RESTful APIs and Delivery Formats for 3D Resources Kristian Sons (DFKI) Rémi Arnauld (AMD) Johannes Behr (Fraunhofer IGD) The multimedia point of view
A.G. Malamos
Multimedia Lab
Dept. of Informatics Engineering
TEI of Crete, Heraklion, Crete, Greece
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY
AUGUST 8-10 – VANCOUVER, CANADA

2. The multimedia dimension of graphics
Graphics and especially virtual worlds have a clear multimedia dimension in the sense of coexistence of multiple medias presented together in real time to provide an integrated audiovisual result.
Performance in that case is a convolutional result of several factors applied to graphics, video and image textures and sound. Such performance factors might be...LOD (resolution), Delay, Jitter and of course.... user expectations.
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY
AUGUST 8-10 – VANCOUVER, CANADA

3. Our experiments in real-time 3D delivery
HTTP vs Websockets (The two most popular approaches on application networking)
The MPEG DASH (Dynamic Adaptive Streaming over HTTP)
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY
AUGUST 8-10 – VANCOUVER, CANADA

4. HTTP vs Websockets The 1st experiment
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY
AUGUST 8-10 – VANCOUVER, CANADA

5. HTTP vs Websockets (1/5) What is websockets?
WebSocket is a protocol providing full-duplex communications over a single TCP connection. The WebSocket protocol was standardized by the IETF as RFC 6455 in 2011, and the WebSocket API is being standardized by the W3C.
Client
Server
Handshake (upgrade HTTP)
Full-duplex Data Channel
Persistent Connection
Time
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY
AUGUST 8-10 – VANCOUVER, CANADA

6. HTTP vs Websockets (2/5) TCP/IP WEBSOCKETS STOMP HTTP
The websockets testbed
TCP/IP
WEBSOCKETS
STOMP
Simple (or Streaming) Text Orientated Messaging Protocol
HTTP
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY
AUGUST 8-10 – VANCOUVER, CANADA

7. HTTP vs Websockets (3/5) The data-set
We have currently uploaded several models from tens of thousands to millions of point coordinates, including colour, faces and normals information
The dome of the church of St. Ioannis in the TEI area, in Crete
Traditional Cretan musical instrument
Ancient Theater Demetriados
An olive tree
Egyptian Cat
from the The Classic Archaeology Collection
in the Museum of Archeology and History of Art
of the University of Athens
an amphora
a wheel-made plate
Corinthian oinochoe
a wheel-made lekythos)
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY
AUGUST 8-10 – VANCOUVER, CANADA

8. HTTP vs Websockets (4/5) Models are in X3D-X3DOM format
Three Web-based distribution platforms available to compare:
Direct display
progressive streaming over HTTP requests
Websocket-STOMP progressive transmission
Optional use of LZW compression into streams
Two progression modes:
Serial (neighborhood) progression
Area sort (biggest triangles first)
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY
AUGUST 8-10 – VANCOUVER, CANADA

9. HTTP vs Websockets (5/5) a. Serial progression
b. Area sort progression
Webservices have better performance than HTTP requests
Entropy Compression (LZW) increase performance in Webservices and it degrades performance of http requests
Streaming has better performance than model downloading
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY
AUGUST 8-10 – VANCOUVER, CANADA

10. MPEG DASH in real-time graphics delivery?
The 2nd experiment
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY
AUGUST 8-10 – VANCOUVER, CANADA

11. MPEG DASH (1/4) Source: MPEG DASH IF
The MPEG-DASH streaming adaptation approach is conceptually simple.
A client (player) application accesses the Media Presentation structure of a media, normally stored in an XML file with the extension .mpd, and selects the appropriate Adaptation Sets corresponding to the characteristics of the end-device and the end-user (e.g. audio tracks in specific languages, video tracks in specific encodings).
Then, the client begins downloading and playing Representations from these adaptation sets, while at the same time probing for QoE metrics (e.g. achieved FPS rate, remaining power, computational resources available).
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY
AUGUST 8-10 – VANCOUVER, CANADA

12. MPEG DASH 2/4 The media (3D) file preparation MPD Structure
Mesh parameters
Texture Image/Video
3D Model
MPD Structure …
Representation #1
Representation #2
Representation #3
Multimedia Presentation Description
Stream preparation 
X3D VR World
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY
AUGUST 8-10 – VANCOUVER, CANADA

13. MPEG DASH 3/4 The MPEG DASH player Client Web Server Network Memory
Host Server1
Host Server2
Host Server(n)
Presentation Quality (LOD) Decision
Client
Network
Memory
Battery
FPS
Timer
Metrics Probing
X3DOMDisplay
Web Server
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY
AUGUST 8-10 – VANCOUVER, CANADA

14. MPEG DASH 4/4 The test-bed and results
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY
AUGUST 8-10 – VANCOUVER, CANADA