1. Building an enterprise Data Fabric at NatWest Markets
Royal Bank of Scotland
jPrime Sofia
31st May 2017

2. Who am I? NatWest Markets (NWM) Technology
Director, Royal Bank of Scotland
Developer & Architect
Development Manager for Data Fabric
I’m a developer and technical lead, with a passion for solving business challenges simply but properly
Mike Fulke

3. What’s this talk about?
2014
2015
2016
2017
We started Data Fabric Q4 2014
Now NatWest Markets strategic data service
Just 2 of us!
We went live Q (4 of us)
Developed through to date (12 of us)
I’ll explain our journey building Data Fabric, with a live demonstration, then some time for Q&A….

4. What’s the background? NWM Architecture (2016)
Thousands of applications
Hundreds of critical applications
Dozens of primary systems
Range of data transfer patterns
Tens of thousands of data flows
Multiple producers of similar data
Applications manage own data
Lineage and explains challenging

5. What’s the problem? Systems in silos aligned to different departments
Our systems have evolved in a business-aligned way for decades
Systems in silos aligned to different departments
So there’s lots of functional duplication across the system stacks
Copying data led to duplicated reconciliations
We must become leaner and more sustainable… so we’re simplifying
In parallel, improving data quality is a key outcome in the control space
Regulation in the banking industry is now very strict
Firms need to demonstrate effective management of technology

6. What are we doing about it?
Data Fabric vision (2015)
Rationalise the data layer…behind a common API, implemented by a multi-tenant PaaS, tackling common data problems…
…reducing people and systems costs, improving hardware utilisation, lowering data centre footprint, offering modern features and capabilities for our developers…

7. Data Fabric – Product Definition
“Data Fabric provides data storage, query and distribution as a service, enabling application developers to concentrate on business functionality.”

8. Data Fabric – Technology Stack
RBS
Agile
Technology “preferences”…
Open Source
Wide adoption
Modern
Cheaper

9. Data Fabric – Development Approach
Integration Architecture
“Internal Open Source”
Contributions Welcome
Self-service Approach
2-week Iterations
Fully Automated Testing
100% Uptime
Fortnightly Releases, Intraday

10. Data Fabric – Software Architecture

11. Data Fabric – Topology

12. Data Fabric – Query Observable Enumerable
“pass the query through the data”
“pass the data through the query”
…naturally BSON allows the same data representation both at rest and “in transit”…

13. Demonstration – storage, query, streaming and state management
Internet
Data Fabric
Console
>
Data Fabric
Console
>
Write some data ….. Query for the data …. Setup a watch

14. Next Steps for Data Fabric
Continue onboarding systems
Better ACID characteristics
Streamed materialisation for join and reporting
JDBC and Reporting tools
Open source – gradually?
Contact us if you are interested!

15. Questions Please!