1. Presented by Daniel Ortiz Esquivel R&D Software Engineer YAPC::Europe September 2015 Site: https://developer.williamhill.comhttps://developer.williamhill.com Twitter: https://twitter.com/dortizesquivelhttps://twitter.com/dortizesquivel OMNIA Distributed & Reactive platform for data management

2. Motivations 2 Omnia: Distributed & Reactive platform for data management Users Feeds System 3 Party In order to be in a position to innovate we need to control and understand our data Social Network s IoT Omnia

3. DMP based on the Lambda architecture and the Reactive principles What is Omnia? 3 Chronos Data Source NeoCortex Speed Layer Fates Batch Layer Hermes Serving Layer Data Flow Input Output Omnia: Distributed & Reactive platform for data management

4. Resources: Reactive Extension for Perl: https://github.com/eilara/Rx.pl https://github.com/eilara/Rx.pl Carl Mäsak YAPC:2013: https://www.youtube.com/watch?v=EnlYURS Kgk0 https://www.youtube.com/watch?v=EnlYURS Kgk0 Ran Eliam : https://www.youtube.com/watch?v=ymEGZE i4hO4 https://www.youtube.com/watch?v=ymEGZE i4hO4 Reactive principles 4 Responsive Resilient Message Driven Elastic The Reactive Manifesto http://www.reactivemanifesto.org/ Omnia: Distributed & Reactive platform for data management

5. Chronos is a reliable and scalable component which collect data from different source and organize them into Streams of observable events. Chronos: Data acquisition 5 Incident: { type: “bet”, version: “1.0”, time: “2015-09-03 06:00:10”, acquisitionTime: “2015-09-03 06:00:06”, source: “BetSystem”, payload: {…. Any valid JSON} } Omnia: Distributed & Reactive platform for data management Chronos Data Source TCP HTTP WS … JMS HTTP Poll SSE Adapter Stream s ConverterPersistence BetsDeposits Prices Stream = Adapter + Converter + Persistence

6. Chronos: Data acquisition 6 Omnia: Distributed & Reactive platform for data management Chronos 1 (SSE, Bets placed) Chronos 2 (JMS, Deposits) Chronos 3 (HTTP, Events) Chronos N (SSE, Twitter) ….… Chronos 2 (JMS, Deposits) (SSE, Bet Placed)

7. Fates represents the long term memory of Omnia. It organizes the incidents that Chronos collected into timelines and also elaborates new information as views by using machine learning, logical reasoning and time series analysis. Fates: Batch layer 7 Omnia: Distributed & Reactive platform for data management Customer: 123 Login Deposit Bet placed … Logout Event: 78 Started Fault Penalty … Goal Timelines & Views BetsDeposits EventsSession Fates Batch Layer

8. Fates: Batch layer 8 Omnia: Distributed & Reactive platform for data management Timelines Views Jobs Fates

9. NeoCortex represents the short term memory of Omnia. It offers a framework to develop micro services on top of Apache Spark. It performs fast and real time data processing with the data acquired by Chronos. NeoCortex: Speed layer 9 Omnia: Distributed & Reactive platform for data management NeoCortex BetsDeposits EventsSession Micro Services Output

10. Hermes is a scalable and full duplex communication for B2C and B2B. Hermes: Serving Layer 10 Omnia: Distributed & Reactive platform for data management B2C Browser B2B Load balancer Push Server Distribute Cache Push Server Push Server … TCP WS HTTP JS API WH Apps Cache

11. Custom advert, bonus, data load prediction, bot detection... Omnia Inception 11 Chronos Data Source NeoCortex Speed Layer Fates Batch Layer Hermes Serving Layer Input Output Omnia: Distributed & Reactive platform for data management Users become a new data producer

12. Real time monitoring and elasticity Docker and Mesos: S cale In&Out based on demand, Omnia inception +1 12 Chronos Data Source NeoCortex Speed Layer Fates Batch Layer Hermes Serving Layer Input Output Omnia: Distributed & Reactive platform for data management JMX

13. Omnia infrastructure 13 Omnia: Distributed & Reactive platform for data management Omnia Docker Marathon Mesos Node

14. https://careers.williamhillplc.com/ Questions 14 Omnia: Distributed & Reactive platform for data management https://omnia.williamhill.com/