1. Report from MesosCon North America 2016 1-2 June 2016, Denver, U.S.
J Blomer
20 June 2016
EP-SFT weekly meeting

2. J Blomer, MesosCon 2016 report
Conference on the Mesos eco system
Involved parties
Apache Foundation: hosting Mesos open source software
Mesosphere: Company commercializing Mesos
Linux Foundation: hosting MesosCon conference
2 days conference + 1 day hackathon
600 – 700 participants
Most participants from industry: Microsoft, IBM, JP Morgan Chase, … but also many web companies: Twitter, Netflix, yelp
48 presentations from 154 submissions
Presentation on CernVM-FS together with Mesosphere
Recall the main components of the CernVM ecosystem
20/6/16
J Blomer, MesosCon 2016 report

3. J Blomer, MesosCon 2016 report
Recall the main components of the CernVM ecosystem
20/6/16
J Blomer, MesosCon 2016 report

4. The ``Apache Big Data Stack’’
By and large: Java World
Zookeeper: consistent distributed database to store the cluster state
Mesos: cluster scheduler
Requires frameworks to be useful, e.g. Spark, Ceph, Marathon for web services, …
Hadoop (HDFS): PB storage, often text files
Spark: orchestration of distributed processing pipelines
Cassandra: distributed NoSQL database
Kafka: distributed message queue for data streams (e.g. Twitter feeds)
Demo: Life map of New York taxis
Recall the main components of the CernVM ecosystem
20/6/16
J Blomer, MesosCon 2016 report

5. J Blomer, MesosCon 2016 report
Take-Away Notes
Size of computing, example Twitter: 30k nodes, 250k containers, 500PB data  Similar to our large data centers
The Apache Stack provides building blocks, many home-grown systems at larger scale
A lot of interest in machine learning and stream processing (Spark instead of Hadoop)
Little focus on individual node performance; focus on high-availability and scalability
CernVM-FS presentation very well received; seems that container distribution is a menace for everyone.
Recall the main components of the CernVM ecosystem
20/6/16
J Blomer, MesosCon 2016 report