1. Distributed Load Testing of CMIS Alfresco Benchmark Framework 2.0 Derek Hulley Senior Developer Michael Suzuki Senior Developer

2. Contents Essential Features Framework Architecture Building and Deploying a Test Alfresco-specific Tests Creating Test Users CMIS Test Deployment Execution Results and Analysis

3. Key Features of the Framework Load Tests are First Class projects Full Maven project lifecycle Java Automated testing GitHub: derekhulley/alfresco-benchmark Central Control Scale up the load drivers Preset test defaults and override only as required Scalability NoSQL storage of results Independent load drivers compete for work Deep Analysis Errors, messages and results are all retained Every event execution is recorded and kept

4. Framework Architecture Benchmark Server Tomcat 7 Rest API MongoDB Config Data Services MongoDB Test Data UI Benchmark Driver (xN) Benchmark Driver Tomcat 7 Extras (Selenium) Servers / APIs Load Balancer Servers / APIs Test Services Rest API

5. Writing a New Test Maven archetype https://wiki.alfresco.com/wiki/Benchmark_Framework_2.0#Developing_Tests 01-Using-the-archetype.wmv 02-running-new-project-locally.wmvhttps://wiki.alfresco.com/wiki/Benchmark_Framework_2.0#Developing_Tests 01-Using-the-archetype.wmv 02-running-new-project-locally.wmv Import into Eclipse Build and run Deploy and Execute Then Phase out sample code Phase in real test code

6. CMIS Load Test: Infrastructure BM Driver Virginia 1 54.167.71.95:9080 (CMIS) BM Driver Sydney 1 54.206.178.67:9080 (CMIS) BM Driver Ireland 1 54.74.38.161:9080 (Sign Up, CMIS, Sample) BM Server Ireland http://54.247.12.28:9080/alfresco- benchmark-server http://54.247.12.28:9080/alfresco- benchmark-server BM Server Ireland http://54.247.12.28:9080/alfresco- benchmark-server http://54.247.12.28:9080/alfresco- benchmark-server Config Data Test Data Alfresco 5.0.a CE

7. Preparing for the Tests Deploy and run Signup Test alfresco-benchmark-tests-ent-signup-2.0.war 05-Creating-users-in-Alfresco.wmvalfresco-benchmark-tests-ent-signup-2.0.war 05-Creating-users-in-Alfresco.wmv Permissions and test directories 06-Prepare-the-Alfresco-server.wmv

8. GitHub: Search ‘benchmark-cmis’GitHub: Search ‘benchmark-cmis Pull requests for new scenarios welcome Repository-agnostic Scenario 01 Find a random folder List folder contents Scenario 02 Get test folder Create folder Upload file Download file Delete folder Scenario 03 Get test folder Create folder Upload file Search in folder Delete folder Scenario weightings adjustable CMIS Load Test: Scenarios

9. CMIS Load Test: Sweet Spot Load driver “sweet spot”: Ireland IRELAND_03: 240 sessions at 0.25s intervals 07-Finding-the-sweet-spot.wmv Driver Alfresco

10. Data Comparison Raw results CMIS-IRELAND_05.xlsx CMIS-VIRGINIA_03.xlsx Compared CMIS Scenario (Ireland-Ireland vs Virginia-Ireland) CMIS.IRELAND_05CMIS.VIRGINIA_03xN cmis.scenario.01.findFolder1559626.21 cmis.scenario.01.listFolderContents816437.94 cmis.scenario.02.createTestFolder312096.74 cmis.scenario.02.deleteTestFolder1302792.15 cmis.scenario.02.downloadFile2639115.04 cmis.scenario.02.retrieveTestFolder2334515.00 cmis.scenario.02.uploadFile1597724.86 cmis.scenario.03.createTestFolder311946.26 cmis.scenario.03.deleteTestFolder532304.34 cmis.scenario.03.retrieveTestFolder2436715.29 cmis.scenario.03.searchInFolder251877.48 cmis.startSession1441229.43