1. Software Engineering for Business Information Systems (sebis) Department of Informatics Technische Universität München, Germany wwwmatthes.in.tum.de Factors influencing the database selection for B2C web applications. Master Thesis Kick-Off Presentation Al-Saeedi, Bilal | 30.11.2015

2. Agenda Introduction Motivation ? Research questions ? 1 © sebis 140122 Matthes Slides sebis 2014 2 Road Plan Progress ? Future plan ? 3 Research Approach Research plan ? Present what has been done ? 2

3. Introduction – Motivation © sebis 140122 Matthes Slides sebis 2014 3 NOSQL DatabasesRelational DBMS

4. Introduction – Research Questions © sebis 140122 Matthes Slides sebis 2014 5 What are the influencing factors for the selection of a database for a B2C application ? What are the challenges in terms of query expressiveness and complexity of the database ? What are the benefits and shortcomings of the database in terms of B2C application scenarios ?

5. Research Approach © sebis 140122 Matthes Slides sebis 2014 6 DB selectionConstructs identificationUse cases identificationLiterature reviewUse cases implementationResults documentation

6. Databases Selection © sebis 140122 Matthes Slides sebis 2014 7 Community Strength MaturityPopularity Availability Scalability Open Source Support Reliability

7. Databases Selection © sebis 140122 Matthes Slides sebis 2014 9 Database Categories Key-Value Document Based Column Based Graph Full Text Search Engines RDBMS MySQL Postgres Oracle …..... Redis Riak Memcached …..... MongoDB Couchbase Cloudant …..... Cassandra HBase Accumulo …..... Neo4j Titan OrientDB …..... Elasticsearch Apache Solr BaseX ….....

8. Constructs © sebis 140122 Matthes Slides sebis 2014 10 Introduction Installability Basic Concepts Possible Use cases Basic Features Query Language Transaction Support Pipeline Support Data Modeling Data Layout Relational Data Support Referential Integrity NormalizationData Evolution Nested Data Support Searching Data Full Text Search Support Regular Expressions Support Query OptionsIndexingQueries Filtering and Grouping data Sorting Special Features Any Database Specific Features Administration and Maintenance ConfigurationScalability Persistence, and ACID Support Handling failure Backup and Upgrade SecurityAvailabilityData migration

9. B2C Use Cases © sebis 140122 Matthes Slides sebis 2014 11 Redis Session Management. Cart Management. Cache Service. Login, Cart, and Products Analytics Service. MongoDB Users Management. Shipping Management Product Review & Wish list Management History Data. Product & Category Management. Cassandra Analytics Service Logging Service Neo4j Recommendations System. ElasticSearch Product Search. MySQL Finance (orders & payments) Inventory Management.

10. Constructs – Redis © sebis 140122 Matthes Slides sebis 2014 13 Introduction Installability Basic Concepts Possible Use cases Basic Features Query Language Transaction Support Pipeline Support Data Modeling Data Layout Relational Data Support Referential Integrity NormalizationData Evolution Nested Data Support Searching Data Full Text Search Support Regular Expressions Support Query OptionsIndexingQueries Filtering and Grouping data Sorting Special Features Pub/Sub Support Expire Configuring as a cache Lua Scripting Administration and Maintenance ConfigurationScalability Persistence, and ACID Support Handling failure Backup and Upgrade SecurityAvailabilityData migration

11. Redis – Basic Concepts © sebis 140122 Matthes Slides sebis 2014 14 http://www.slideshare.net/dvirsky/kicking-ass-with-redis

12. Redis – Transaction Support © sebis 140122 Matthes Slides sebis 2014 16 Watch/UnwatchMulti/Exec Discard

13. Redis – Relational Data Support © sebis 140122 Matthes Slides sebis 2014 18 (Hash) Customer (Hash) Order (Hash) Shipping (Hash) Payment 1* * * 1* Set (orderIDs) Orders:customerID Set (orderIDs) Orders:paymentID Set (orderIDs) Orders:shippingID Set (shippingIDs) Shippings:orderID

14. Redis – Relational Data Support © sebis 140122 Matthes Slides sebis 2014 19 Submit an order Get all customer Shippings

15. Redis – Indexing © sebis 140122 Matthes Slides sebis 2014 22 Primary IndexKey Hash { “SKU”: “sku”, “Name”:”ProductName”, “Color”: “ProductColor”, “Price”: “ProductPrice” } Products:ProductID  Primary Index Set {ProductIDs} Product.Color.Index: ProductColor Sorted Set {ProductIDs, Price} Product.Price.Index Secondary Index Composite Index Sorted Set {CategoryID:Price:ProductID, 0} Product.Category.Price.Index Retrieval

16. Redis – Filtering, Grouping and Sorting © sebis 140122 Matthes Slides sebis 2014 23 Group Data Set {ProductIDs} Product. Manufacture.Index: CompanyName Set {ProductIDs} Product.Color.Index: ProductColor Filter Data Sorting Data SORT command  sorts lists, sets, and sorted sets

17. Redis – Use Cases Implementations © sebis 140122 Matthes Slides sebis 2014 28 Cart Management Service Session Management Service Cache Service Analytics Service Other Redis Examples: Inspect Data Structures Job Queue. Handling Data Relation Pipeline Support Transaction Support https://github.com/alronz/B2C-Database-Selection-ImplementationsCode: https://github.com/alronz/b2c-database-selectionDocumentation:

18. Road Plan © sebis 140122 Matthes Slides sebis 2014 29 Today CompleteCompleteOngoingOngoing Not Started EndStart RevisingRevising

19. Technische Universität München Department of Informatics Chair of Software Engineering for Business Information Systems Boltzmannstraße 3 85748 Garching bei München wwwmatthes.in.tum.de Bilal Al-Saeedi rose@in.tum.de Thank you for your attention! MSc. Informatics Student

20. Backup slides © sebis 140122 Matthes Slides sebis 2014 3

21. Constructs – Redis © sebis 140122 Matthes Slides sebis 2014 13 Introduction Installability Basic Concepts Possible Use cases Basic Features Query Language Transaction Support Pipeline Support Data Modeling Data Layout Relational Data Support Referential Integrity NormalizationData Evolution Nested Data Support Searching Data Full Text Search Support Regular Expressions Support Query OptionsIndexingQueries Filtering and Grouping data Sorting Special Features Pub/Sub Support Expire Configuring as a cache Lua Scripting Administration and Maintenance ConfigurationScalability Persistence, and ACID Support Handling failure Backup and Upgrade SecurityAvailabilityData migration

22. Databases Selection – Key-Value © sebis 140122 Matthes Slides sebis 2014 3 Built-in persistence support. Vey fast in-memory database Variety of storage data types. Open source

23. Databases Selection – Document-Based © sebis 140122 Matthes Slides sebis 2014 3 JSON-like storage format named BSON Great features such as ad hoc queries, Indexing, sharding, etc. Easy to use and has great community Open source

24. Databases Selection – Column-Based © sebis 140122 Matthes Slides sebis 2014 3 Open source SQL-Like Query language called CQL Decentralized, no single point of failure Great community

25. Databases Selection - Graph © sebis 140122 Matthes Slides sebis 2014 3 Supports a powerful and easy to use graph query language called Cypher. Support complex data modeling. Open source Great community

26. Databases Selection – Full Search Engine © sebis 140122 Matthes Slides sebis 2014 3 Scalable Map/Reduce model Memory efficient and fast. Open source Great community

27. Databases Selection - RDBMS © sebis 140122 Matthes Slides sebis 2014 3 Relational Database System Client/Server Architecture SQL compatibility Open source RDBMS Great community

28. Introduction – Research Questions © sebis 140122 Matthes Slides sebis 2014 4 System product quality model—ISO/IEC9126.

29. Databases Selection © sebis 140122 Matthes Slides sebis 2014 8 http://db-engines.com/Magic Quadrant for Operational DBMS

30. Constructs – Redis © sebis 140122 Matthes Slides sebis 2014 13 Introduction Installability Basic Concepts Possible Use cases Basic Features Query Language Transaction Support Pipeline Support Data Modeling Data Layout Relational Data Support Referential Integrity NormalizationData Evolution Nested Data Support Searching Data Full Text Search Support Regular Expressions Support Query OptionsIndexingQueries Filtering and Grouping data Sorting Special Features Pub/Sub Support Expire Configuring as a cache Lua Scripting Administration and Maintenance ConfigurationScalability Persistence, and ACID Support Handling failure Backup and Upgrade SecurityAvailabilityData migration

31. Constructs – Redis © sebis 140122 Matthes Slides sebis 2014 15 Introduction Installability Basic Concepts Possible Use cases Basic Features Query language Transaction Support Pipeline Support Data Modeling Data Layout Relational Data Support Referential Integrity NormalizationData Evolution Nested Data Support Searching Data Full Text Search Support Regular Expressions Support Query OptionsIndexingQueries Filtering and Grouping data Sorting Special Features Pub/Sub Support Expire Configuring as a cache Lua Scripting Administration and Maintenance ConfigurationScalability Persistence, and ACID Support Handling failure Backup and Upgrade SecurityAvailabilityData migration

32. Constructs – Redis © sebis 140122 Matthes Slides sebis 2014 21 Introduction Installability Basic Concepts Possible Use cases Basic Features Query language Transaction Support Pipeline Support Data Modeling Data Layout Relational Data Support Referential Integrity NormalizationData Evolution Nested Data Support Searching Data Full Text Search Support Regular Expressions Support Query OptionsIndexingQueries Filtering and Grouping data Sorting Special Features Pub/Sub Support Expire Configuring as a cache Lua Scripting Administration and Maintenance ConfigurationScalability Persistence, and ACID Support Handling failure Backup and Upgrade SecurityAvailabilityData migration

33. Constructs – Redis © sebis 140122 Matthes Slides sebis 2014 17 Introduction Installability Basic Concepts Possible Use cases Basic Features Query language Transaction Support Pipeline Support Data Modeling Data Layout Relational Data Support Referential Integrity NormalizationData Evolution Nested Data Support Searching Data Full Text Search Support Regular Expressions Support Query OptionsIndexingQueries Filtering and Grouping data Sorting Special Features Pub/Sub Support Expire Configuring as a cache Lua Scripting Administration and Maintenance ConfigurationScalability Persistence, and ACID Support Handling failure Backup and Upgrade SecurityAvailabilityData migration

34. Constructs – Redis © sebis 140122 Matthes Slides sebis 2014 24 Introduction Installability Basic Concepts Possible Use cases Basic Features Query language Transaction Support Pipeline Support Data Modeling Data Layout Relational Data Support Referential Integrity NormalizationData Evolution Nested Data Support Searching Data Full Text Search Support Regular Expressions Support Query OptionsIndexingQueries Filtering and Grouping data Sorting Special Features Pub/Sub Support Expire Configuring as a cache Lua Scripting Administration and Maintenance ConfigurationScalability Persistence, and ACID Support Handling failure Backup and Upgrade SecurityAvailabilityData migration

35. Constructs – Redis © sebis 140122 Matthes Slides sebis 2014 26 Introduction Installability Basic Concepts Possible Use cases Basic Features Query language Transaction Support Pipeline Support Data Modeling Data Layout Relational Data Support Referential Integrity NormalizationData Evolution Nested Data Support Searching Data Full Text Search Support Regular Expressions Support Query OptionsIndexingQueries Filtering and Grouping data Sorting Special Features Pub/Sub Support Expire Configuring as a cache Lua Scripting Administration and Maintenance ConfigurationScalability Persistence, and ACID Support Handling failure Backup and Upgrade SecurityAvailabilityData migration

36. Redis – Relational Data Support © sebis 140122 Matthes Slides sebis 2014 20 Difference Union Join

37. Redis – Configuration & Persistence © sebis 140122 Matthes Slides sebis 2014 27 Can be started without configuration file. Redis configuration is in redis.confi, format confugirationKeyWord argument1 argument2... argumentN Configured also using command line./redis-server --port 6380 --slaveof 127.0.0.1 6379 Configuration on the fly without restarts CONFIG SET SAVE "900 1 300 10" Configuration Persistence Snapshotting or RDB Persistence Periodically and with condition save 100 50 / BGSAVE Append-Only File or AOF Persistence appendonly yes / BGREWRITEAOF fsync every, fsync every second, Never fsync  fsync every for full durability

38. Redis – Expire © sebis 140122 Matthes Slides sebis 2014 25 Expire SetEx

39. Redis – Configuration © sebis 140122 Matthes Slides sebis 2014 27 Can be started without configuration file. Redis configuration is in redis.confi, format confugirationKeyWord argument1 argument2... argumentN Configured also using command line./redis-server --port 6380 --slaveof 127.0.0.1 6379 Configuration on the fly without restarts CONFIG SET SAVE "900 1 300 10"

40. Redis – Indexing © sebis 140122 Matthes Slides sebis 2014 22 Primary IndexKey Hash { “SKU”: “sku”, “Name”:”ProductName”, “Color”: “ProductColor”, “Price”: “ProductPrice” } Products:ProductID  Primary Index Set {ProductIDs} Product.Color.Index: ProductColor Sorted Set {ProductIDs, Price} Product.Price.Index Secondary Index Composite Index Sorted Set {CategoryID:Price:ProductID, 0} Product.Category.Price.Index Retrieval