1. Sub-Project 2: Multimedia Distribution Platforms for Cyber-Physical Systems
NUS PI: Roger Zimmermann
ZJU PI: TBD
NUS Co-PIs: Ben Leong
ZJU Co-PIs: TBD
11/27/2018 1

2. Motivation (1)
Challenge Current large-scale multimedia systems, such as networked games, are mostly built based on an integrated digital media distribution architecture that is designed from scratch for each application.
Goal Aims to design a general-purpose, extensible common platform that addresses the requirements of all networked CPS apps.For example:
Media transmission protocols
Sensor- and meta-data annotations …
11/27/2018 2

3. Motivation (2) Trends User-generated video content is growing rapidly.
Mobile devices make it easy to capture video.
Example Mobile Video Scenario (GeoVid)
Viewable scene modeling (i.e., geo-properties)
Video and (sensor) meta-data acquisition
Indexing, querying, and presentation of results 1) 2) 3) d 
11/27/2018 3

4. Ex.: Spato-Temporal Range Query on Video Streams
area
11/27/2018 4

5. Search and Result Presentation (2D)
11/27/2018 5

6. GeoVid Demonstration

7. Architecture of Mobile Sensor-Rich Video Management System
User Interface
Meta-data Representation
Meta-data
Meta-data Processor
Viewable scene descriptions
Mobile Device
User
Missing Video frame request
Indexing and Storage
Spatio-temporal range query
Query Interpreter
Query
Video content
Video Ranking
Video
results
Spatio-temporal Relevance
User
Spatio-temporal overlap properties
Video Summarization
11/27/2018 7

8. Dynamic Adaptive Streaming over HTTP (DASH)
Traditional media streaming (e.g., based on RTP/RTSP/RTCP) streaming faces several challenges in large-scale systems:
Special-purpose servers/proxies/peers for media (complex)
Difficulties in firewall traversal: protocols use both TCP and UDP transmissions
Difficulties in caching data (no “web caching”)
Advantage of RTP/RTSP/RTCP
Short end-to-end latency
11/27/2018 8

9. DASH Standard (Dec. 2011) ISO/IEC Standard:
“Information technology — MPEG systems technologies — Part 6: Dynamic adaptive streaming over HTTP (DASH)”
JTC 1/SC 29; FCD
Uses HTTP protocol to “stream” media
Divides media into small chunks, i.e., streamlets
Makes playback adaptive
Encode media into multiple different streamlet files, e.g., a low, medium, and high quality version (different bandwidth).
Supported by: Move Networks, Apple, Microsoft, Netflix, …
11/27/2018 9

10. DASH Limitations
DASH standard only describes server-to-client dissemination:
Upload mechanism is undefined!
We are working on a low latency DASH upload protocol for mobile devices.
MPD: Media Presentation Description
11/27/2018 10

11. DASH Upload Design Features:
Real-time segmentation on the mobile device
Chunked, resumable upload via HTTP
Segment-level, server-side transcoding for quality adaptation
[ACM MMSys 2012]
End-to-end startup latency on iPhone 4:
11/27/2018 11

12. Smartphone Apps Android App
Available for download
iPhone App
Available in the iTunes App Store
11/27/2018 12

13. Search and Result Presentation (3D) e.g., with Google Earth
11/27/2018 13

14. Other Research Peer-to-peer video and audio streaming
Texture streaming in virtual worlds
Geo-graphic routing and sensor networks
Mesh networking
VoIP and HD communications
Etc.

15. Multimedia Distribution Platforms for Cyber-Physical Systems
Core deliverables
The cyber-physical node server that would effectively be the web-server equivalent for the cyber-physical web.
The hyper-renderer client for a cyber-physical environment server - equivalent of the normal web-browser for the cyber-physical web server.
Co-space, which is the whole system, comprising of the servers, middleware, and the clients, will then be used to develop the various cyber-physical applications.
11/27/2018 15

16. Thank You http://geovid.org
Roger Zimmermann, Seon Ho Kim, Ben Leong Sakire Arslan Ay, Beomjoo Seo, Min Min Htoon, Liang Ke, Hao Jia, Wang Guanfeng, Ma He, Fang Shunkai, Zhang Lingyan, Shen Zhijie, Zhang Ying, Cui Weiwei, Yin Yifang
National University
of Singapore
University of
Southern California
Further information at:
11/27/2018 16