1. Realtime insight in your application usage with NodeJs, ElasticSearch and Kibana Onno de Haan

2. Hier uw naam Realtime insight in your application usage with NodeJs, ElasticSearch and Kibana Onno de Haan

3. Agenda About VanMeijel Why we need insight in Metacom (our ERP-system) Requirements for monitoring tool Developing the tool Final solution Demo ! Please feel free to ask questions at any time

4. About VanMeijel Founded in 1987 Based in The Netherlands, Emmeloord Satellite office in Belgium, Sint-Niklaas Around 110 employees Software is targeted at construction companies 300+ customers, 8000 - 9000 users

5. Metacom – Our ERP system Contains modules for - Estimation - Reporting - Planning - Invoicing - Cost Control - Procurement - Relation Management - Resource Planning - and many more

6. Metacom – Our ERP system Build in OpenEdge, currently version 11.5 - Millions of code lines - Around 20 – 25 years of age - Lots of different techniques (DB-access on client, client-server, procedural, OO) - Deployed via Round Table, recently moved to Subversion / Env / Jenkins

7. About VanMeijel Why we need insight in Metacom Requirements for monitoring tool Developing the tool Final solution Demo Please feel free to ask questions at any time

8. Application usage is changing - People are working mobile - People want / expect Apps - Customers don’t want big server parks: web interface Changing world Integrations Maintenance Different license models Move to webinterface Why we need insight in Metacom

9. Customers are integrating 3th party applications One webservice often replaces many “real” users Integrations weaken our license model Changing world Integrations Maintenance Different license models Move to webinterface Why we need insight in Metacom

10. What methods are called How often are they called How many business objects are being created / edited per method How many systems are connecting to Metacom Changing world Integrations Maintenance Different license models Move to webinterface Why we need insight in Metacom

11. System is complex Around 3000 – 4000 windows/dialogs Contains lots and lots of settings What is the performance of our webservice methods Changing world Integrations Maintenance Different license models Move to webinterface Why we need insight in Metacom

12. In the future we need to move to “pay for usage” How many business documents are created - For example how many invoices Are business documents updated frequently or not? How many reports are being generated Changing world Maintenance Integrations Different license models Move to webinterface Why we need insight in Metacom

13. How many users are active? On what time of the day / week? How about evenings, weekends? Changing world Maintenance Integrations Different license models Move to webinterface Why we need insight in Metacom

14. What functions / modules are used What is not used anymore Top 100 of most used functions Changing world Maintenance Integrations Different license models Move to webinterface Why we need insight in Metacom

15. About VanMeijel Why we need insight in Metacom Requirements for monitoring tool Developing the tool Final solution Demo Please feel free to ask questions at any time

16. Requirements monitoring tool Web based Real time Easy to analyze Flexible, schemaless database Light-weight; JSON

17. About VanMeijel Why we need insight in Metacom Requirements for monitoring tool Developing the tool Final solution Demo Please feel free to ask questions at any time

18. Events are send to our AppServer API (written in OpenEdge) - Client - Triggers - Business Logic - Webservices AppServer API sends events to webserver Webserver stores events and hosts a viewer Architecture First try, january 2014 Different license models Developing the tool

19. NodeJS as webserver MongoDB as database Visualization was a problem Architecture First try, january 2014 Second try, march 2014 Developing the tool

20. We replaced MongoDB with ElasticSearch Visualization is super easy with Kibana Migration from MongoDB to ElasticSearch was difficult Architecture First try, january 2014 Second try, march 2014 Developing the tool

21. About VanMeijel Why we need insight in Metacom Requirements for monitoring tool Developing the tool Final solution Demo Please feel free to ask questions at any time

22. Clients push events to AppServer AppServer start HTTP-session to REST-service in cloud Fires JSON event, simplified example: { “customerid”:”vma”, “datetime”:”2015-11-05T14:36:54”, “appid”: “mtc”, “appversion”: “9.2.2.1” “type”:”ui”, “function”: “wcustomers” } OpenEdge events NodeJS Webserver ElasticSearch database Kibana for visualisation Final solution

23. OpenEdge events NodeJS Webserver ElasticSearch database Kibana for visualisation Final solution var restify = require('restify'); var elasticsearch = require('elasticsearch'); var server = restify.createServer( { name : "stats" }); server.listen(‘1234’,’127.0.0.1’, function() { console.log('%s listening at %s ', server.name, server.url);}); server.use(restify.queryParser()); server.use(restify.bodyParser()); server.use(restify.CORS()); var client = new elasticsearch.Client({ host: 'localhost:9200' }); server.post({path : '/stats', version: '0.0.1'}, postNewMtcStat); function postNewMtcStat(req, res, next){ var body = req.body; res.setHeader('Access-Control-Allow-Origin','*'); client.create({index: 'usage', type: 'usage', body: req.body }, function(err, success){ if(success){ res.send(201, 'saved'); return next(); }else{ return next(err); } }); }

24. OpenSource, Apache License Based on Lucene Distributed database, very scalable Near real-time search Document oriented, schema changes not a problem! OpenEdge events NodeJS Webserver ElasticSearch database Kibana for visualisation Final solution

25. OpenEdge events NodeJS Webserver ElasticSearch database Kibana for visualisation Final solution Cool Flexible Fast (but not on this laptop ) Very easy to use

26. OpenEdge events NodeJS Webserver ElasticSearch database Kibana for visualisation Final solution

27. DEMO

28. QUESTIONS

29. Thank you for your