1. Deep Learning For Application Performance Optimization
Dr Zoran Sevarac, Deep Netts
Stavi deep netts logo
Who Am I
Who is my audience
How many of you has been doing application performance tuning? To many things to take into account, too much noise, some things not predictable It takes a while to see if assumptions are right
Itcan be very frustrating High pressure if performanceproblems have impact on business, and it almost always does
This is work in progress – na kraju ih pozovi da nam daju intereannti use case
This session is about how do you do it, so I’m not saying that it works , but if you try it and it works please let me know 
The idea: how do you apply deep learning to app performanceoptimization Why deep learning is a good choice?

2. Vision It would be great to have applications that are able to:
learn optimal configuration,
self-tune
provide best performance in different scenarios
What is this all about
The reason why I’m doing this session is that I hope that one day …

3. Motivation Performance is always critical
Performance tuning must be tailored for
Application
Load
Infrastructure
Performance tuning is time consuming, error prone, guesswork
When something changes it has to be repeated
Ideally, that should be turned into systematic, repeatable, even continuous process
Why is this important
End user experience, business
guesswork – too many parameters
How Many Users Can Your Web Site Handle?
Does My Site Crash Under Load?
How Many Hits/Sec Can My Web Site Serve?
15 30

4. Model a dependency between configuration, load, and app performance
How
Performance = F (Load, Resources, Configuration)
INPUTS
OUTPUTS
Deep
Learning
Model
Load
Resources
(CPU, Memory)
Configuration
Estimated
Performance Metrics
Modeling application behaviour
Why using dl for perf tuning makes sense? How do you use this model
Some settings are numeric and some nominal
Mixed optimization problem
Continual or discrtee
Model a dependency between configuration, load, and app performance

5. Why
Modeling a complex, unknown function, with many unknown parameters – typical use case for machine learning (regression)
Model should predict app performnce for specific load and available resources
It can be used to optimize resource usage or ensure quality of service
Task: find configuration settings that will provide performance with optimal usage of resources
Why this makes sense
Too many moving parts that influence each other
Also explain how to use this model
Optimizae resource consumption, user experience, lower cost
Predict performance for the given load and configurtaion
Probably not ideal, but some insight is better then none

6. Inputs: Load and Configuration
Number of concurrent users
Request rate (per second)
Number of available CPUs
Amount of available Memory
Additional configuration settings and runtime parameters (numeric and nominal)
Anything else that’s relevant and has impact on application performance
Average in 1, 5, 15 minutes
Can take numeric and nominal inputs
Very flexible,

7. Outputs – Performance metrics
Average response time (can be skewed with outliers)
Apdex score:
(satisfied req+ tolerating requests/2) / totalRequests
Min, Max response time (better to discretize)
Total execution time (batch processing)
Garbage Collection (frequency, time, total time)
Depends of the optimization goals
Frequency , ratio for min, max
Discretization, group of response times, frequency
Min –

8. General Pipeline 1 2 4 3 5 6 Load Test, App Monitoring
Data Preprocessing
new loads, usage scanarios 4 3
Evaluation with Test Data
Model Building & Tuning
tune existing model
The architecture of the automated app performance tuning based on deep learning diagram – nacrtaj entire pipeline
At the moment working with buch of scripts and figuring out best practice, what works and what doesent
In future the goes is to support it using jenkins 5 6
Deployment to Production
Model Monitoring

9. Data Collection Prepare configurations and loads to test
Run loads using different configurations
Monitor and record performance metrics
Automate/script the process
How much data?
Generate enough variability in order to provide data for model evaluation
Sta tacno rade MC FR?

10. Apache JMeter
Users = Thread Group, Requests = Samplers, Results = Listeners (summary, graphs)

11. Data Preparation Clean data Visualize data Remove correlated inputs
Filter outliers?
Errors , missing values
Visualize data
Remove correlated inputs
Remove inputs that are not correlated with output
Prepare data for model building (calculate outputs avg responses (time window size?), Apdex score, etc.)
Normalize (scale to range [0,1] or [0.1, 0.9])
Transform nominal inputs to one-hot binary vectors

12. Feed Forward Neural Network for Regression
I N P U T S
O U T P U T S
One of the common questions is how to configure since there are many options
So for the regression type of problems this is the recommended configuration
Network inputs correspond to input parameters (load and configuration)
Outputs correspond to performance metrics that we want to guess
A sequence of Layers
Each layer performs operations on its inputs and sends outputs to next layer
Hidden Activation: TanH or ReLu
Output Activation: Linear
Loss Function:
Mean Squared Error

13. Network Training – Backpropagation
Each layer has a number of tunable parameters called ‘weights’ and training consists of tuning these parameters
Training algorithm seeks for a set of layer weights that produce min error – a value of Loss Function
In every training iteration, algorithm changes all the weights for a small amount that's proportional to the network error - difference between actual and target output.
Not going into details, but basically its an optimization problem, making small modifications of the parameters while looking for the minimum of the error function
Sad objasni sta zapravo trening radi –

14. DEMO: Network Training using Deep Netts
How you actualy train the network
Kada bih ovde sad ubacio jedan demo za tuning JVM-a kako se
Kreiranje modela JVM-a
Modeling performance
uopsteno

15. Architecture Tuning How many hidden layers and layer sizes
More and bigger layers gives bigger capacity, but possible over-fitting
Goal is to find the minimum network size that gives good prediction
Automated procedures for tuning:
Grid and Random search
Advanced stuff: custom loss and regularization for perf tuning

16. Using Trained Network
FeedForwardNetwork nnet = FileIO. FileIO.createFromFile(“traindedNet.dnet", FeedForwardNetwork .class); float[] inputs = monitoredParams; nnet.setInput(inputs); float[] predictedPerf = nnet.getOutput()

17. Case Studies Ecommerce Web Application, Tomcat Server
30% Faster Average Response Time with same resources
Billing application, batch processing, Web Logic 15% less time to generate 6 million bills, after tuning 8.5h
Elastic Search, same performance with 20% less memory
Web app on Tomcat server
If you have an interesting use case please get in touch with us, we are interested to collaborate
The architecture was bad, adding more servers didn’t help
After tuning 8.5h, 15 % faster (GC and Web Logic)

18. Why Deep Learning For APT
Less feature selection and engineering – features are learned
Good with high dimensions
(many input parameters)
Good with large amounts
of data - more data == better
Flexible model (easily supports custom settings)

19. Deep Learning Tools
Neuroph, open source, educational, entry level
Deep Netts, more advanced options, but easy to use, with specialized wizrads and workflows
Deep Learning 4J, many advanced options, not so easy to use (requires deeper understanding of deep learning in order to use available options)
Tensor Flow, Keras, Python based (with Java API), very powerful but not easy to use
Napravi uporednu tabelu
Koji je za koga bolji
Zavisti od profila korisnika I njegovog znanja I potreba, sta mu je cilj
Poenta poruke: ako si softver developer/java programer I hoces da primenjujes deep learning bez duboko ulazenje u teoriju i matematiku deep netts je najbolja opcija

20. Ongoing Development
Build a model that will recommend the optimal configuration profile
Jenkins pipelines to automate the entire procedure - continiuos performance tuning
If conditions are met, trigger the reconfiguration, model rebuilding etc.

21. Conclusion
Machine Learning is already used by many app performance solution providers
Deep Learning achieved promising results in our tests
It has potential to revolutionize application performance management
Got an interesting use case? We’d like to hear it!
DL technology us Perfect match for application performance problems
If you have an interesting use case let us know
work in progress, experimenting and figurng out what works and what doesent, but we got some promissing results, and looking at the trends thats the next big thing
so dont fire your performance engineers, use this as their assistnat, that will help them to be give faster reponse, and be more efficent
which will automaticly make your system faster, and make your users happy, and help grow business in a sustainable way

22. Oracle Data Science Cloud
COMING SOON
Support for all open source ML libraries
Support for Team Collaboration
Workflow support for entire lifecycle from data analysis and model building to monitoring
Visit them at the booth!

23. Experts in Modern Development
Cloud
Microservices and Containers
Java, JavaScript/Node.js, PHP, Python
DevOps
Continuous Delivery
Open Source Technologies
SQL/NoSQL Databases
Machine Learning, AI, Chatbots
developer.oracle.com/ambassador
@groundbreakers

24. Learn More