1. Modeling of Parametric Dependencies for Performance Prediction of Component-based Software Systems at Run-time
Simon Eismann, Jürgen Walter, Joakim Kistowski, Samuel Kounev
International Conference on Software Architecture
Seattle,

2. Motivation: VideoProvider
Resource demand describes how long a request occupies a physical resource

3. Required Features Input parameters Parametric dependencies
ROBOCOP [1]
Input parameters
Parametric dependencies
Output parameters
Instance-level dependencies
Multiple descriptions
Non-causal dependencies
PCM [2]
OUR APPROACH
[1] Bondarev, Egor, et al. "Modelling of input-parameter dependency for performance predictions of component-based embedded systems." Software Engineering and Advanced Applications, st EUROMICRO Conference on. IEEE, 2005.
[2] Reussner, Ralf H., et al. Modeling and simulating software architectures: the Palladio approach. MIT Press, 2016

4. Approach
Design of a formalism independent dependency resolution approach
Resolution of dependencies prior to model solution
Implementation for the Descartes Modeling Language (DML)
Due to time constraints
[1] Huber, Nikolaus, et al. "Model-based self-aware performance and resource management using the Descartes modeling language." IEEE Transactions on Software Engineering 43.5 (2017):

5. Dependency Graph Node: Edge: Model variable and callpath to it
Example: Resource demand of component ‘ImageProvider‘ for requests of class ‘browsing’ if called by the component ‘MobileWebUI’
Optionally contains characterization, e.g., exp(7)
Edge:
Represents dependency
Can have multiple sources
Contains an equation that allows to calculate characterization for target if all sources are known
Dependency
Known Variable
+ Callpath
Unknown Variable
+ Callpath

6. Dependency Graph Resolution
Known Variable
+ Callpath
Unknown Variable
+ Callpath

7. Approach
[1] Huber, Nikolaus, et al. "Model-based self-aware performance and resource management using the Descartes modeling language." IEEE Transactions on Software Engineering 43.5 (2017):

8. RQ1 RQ2 RQ3 Research Questions
Can model variables be accurately described by correlations?
Can we provide multiple, independent characterizations of a single resource demand ?
Does modeling parametric dependencies on component instance level improve the prediction accuracy?
RQ1
RQ2
RQ3

9. Case Study 1 – Video Transcription
Implementation of the video store example
Video transcription using CMUSphinx1
10 to 30 seconds of english spoken content from YouTube
Evaluation:
Characterization on training data set
Evaluation of characterization fit
Evaluation of prediction accuracy
1https://cmusphinx.github.io/

10. Evaluation Resource Demand Estimation
Evaluation of characterization fit on the training data set
File size better than number of lines
Both estimations have reasonable errors
Evaluation of prediction accuracy of performance model using each dependency on evaluation data set
Performance analysis using the presented approach followed by a QPN simulation
Response time prediction accuracy:
File size: %
Number of lines: %
Both estimations provide sufficient prediction accuracy for capacity planning

11. RQ1 RQ2 RQ3 Research Questions
Can model variables be accurately described by correlations?
Can we provide multiple, independent characterizations of a single resource demand ?
Does modeling parametric dependencies on component instance level improve the prediction accuracy?
RQ1
RQ2
RQ3

12. Case Study 2 – Subtitle Caching
Subtitles can either be cached or have to be retrieved from a database
Cache probability depends on language and popularity distribution
Two workloads, American traffic and European traffic

13. Prediction without dependencies
Performance model using mean over both traffic types
Four load levels (Utilization from 8% to 79.6%)
Comparison of measured values to predictions
Dependency needs to be modeled
However, this dependency does not apply to all subtitle provider instances
Only if access frequency is tied to language and popularity
Example: Backup service iterates over all available subtitles

14. Prediction with dependencies
Modeling of the dependency as:
Same Experiments with the new model:

15. Conclusion
No existing formalism allows to substitute unmeasurable parameters using correlation
We designed a formalism independent dependency resolution approach
We implemented the proposed approach for the Descartes Modeling Language (DML)
Evaluation across two case studies shows a mean error below 5% for utilization and below 10% for response time

16. Thank you for your attention!
Slides are available at
Simon Eismann, Jürgen Walter, Joakim Kistowski, Samuel Kounev
International Conference on Software Architecture
Seattle,