1. An overview of InfiniteGraph, the distributed graph database. Darren Wood Chief Architect, InfiniteGraph

2. For today’s discussion… InfiniteGraph – Background / History – Architecture – What we are working on Consistency Models Challenges/Requirements of Distibution Data model – Schema vs Schema-less Copyright © InfiniteGraph

3. Building on Objectivity/DB Derived from Objectivity/DB core – Distributed Object Database – Native core (broad platform support) with C++, Java, C# and Python Bindings – Thousands of deployments, many are 24x7x365 – Markets : VLDB, Data Fusion / Metadata, Complex Object Models, Relationship Analytics Copyright © InfiniteGraph

4. Relationship Analytics Experience Relationship Analytics – Large part of Objy/DB Government business – Significant growth in the commercial space – Many OEM opportunities – Mostly based on graph theory – Hand coded algorithms Copyright © InfiniteGraph

5. Needed something more… Objectivity Provides – Powerful distributed object storage – Flexible class based persistence – Enterprise DB features But…. – Lots of custom coding – Complex deployment – No generic algorithm framework Copyright © InfiniteGraph

6. Enter InfiniteGraph Simple Graph focused API Automated distribution and deployment Mostly configuration driven Java class based persistence Property model support Asynchronous navigation Indexing framework Copyright © InfiniteGraph

7. Some code… Copyright © InfiniteGraph Vertex alice = myGraph.addVertex(new Person(“Alice”)); Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); bob.addEdge(new Call(timestamp), charlie); AliceCarlosCharlieBob Meets CallsPays Calls

8. Basic Architecture Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Blueprints User Apps Objectivity/DB Distributed Database Session / TX Management Placement

9. What Are We Working On ? Copyright © InfiniteGraph

10. Targeting Large Graphs Graphs grow quickly – Billions of phone calls / day in US – Emails, social media events, IP Traffic – Financial transactions Some analytics require navigation of large sections of the graph Must distribute data and go parallel Copyright © InfiniteGraph

11. Consistency Models Trading off full consistency for performance Relax locking, allow stale reads Gain 100x edge ingest rates Great for “social graph” applications These tend to have relaxed consistency requirements API allows choice per operation Copyright © InfiniteGraph

12. Pipelining Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Session / TX Management Placement (MDP) Placement (Pipelining) V1V1 V1V1 V2V2 V2V2 V3V3 V3V3 E 12 E 23 Pipeline Manager Staging Containers Pipeline Containers E(1->2) E(3->1) E(2->3) E(2->1) E(2->3) E(3->1) E(1->2) E(3->2) E(1->2) E(2->3) E(3->1) E(2->1) E(2->3) E(3->1) E(3->2) E(1->2)

13. Distributing Navigation Graph algorithms naturally branch Breaking up the process is relatively simple Orchestrating it is more challenging Copyright © InfiniteGraph Alice CarlosCharlieBob Meets CallsPays DaveEveChuck Calls Lives With Meets

14. Distributed API Application(s) Partition 1Partition 3Partition 2Partition...n Processor Partitioned Graphs are Ugly Copyright © InfiniteGraph

15. Making it scale… A distributed data tier supports parallel IO A distributed cache/processor tier for processing “navlets” Ability to deal with remote data reads (fast) Navlets hit processor/cache instances based on consistent hashing of their start vertex Copyright © InfiniteGraph

16. Flexible Data Models Schema isn’t for everyone – Great for relatively static models – Support for inheritance helps – So does auto-evolution Looking to support document style elements – Support for ad-hoc attributes – Ability to mix both approaches Copyright © InfiniteGraph

17. Other Projects Blueprints / Gremlin support Native Visualizer and Management Console Built in Qualifiers More Indexing Enhancements Algorithms Large scale samples Copyright © InfiniteGraph

18. Thank you ! Copyright © InfiniteGraph darren.wood@infinitegraph.com Twitter - @infinitegraph