Distributed Load Testing of CMIS Alfresco Benchmark Framework 2.0 Derek Hulley Senior Developer Michael Suzuki Senior Developer

Contents Essential Features Framework Architecture Building and Deploying a Test Alfresco-specific Tests Creating Test Users CMIS Test Deployment Execution Results and Analysis

Key Features of the Framework Load Tests are First Class projects Full Maven project lifecycle Java Automated testing GitHub: derekhulley/alfresco-benchmark Central Control Scale up the load drivers Preset test defaults and override only as required Scalability NoSQL storage of results Independent load drivers compete for work Deep Analysis Errors, messages and results are all retained Every event execution is recorded and kept

Framework Architecture Benchmark Server Tomcat 7 Rest API MongoDB Config Data Services MongoDB Test Data UI Benchmark Driver (xN) Benchmark Driver Tomcat 7 Extras (Selenium) Servers / APIs Load Balancer Servers / APIs Test Services Rest API

Writing a New Test Maven archetype 01-Using-the-archetype.wmv 02-running-new-project-locally.wmvhttps://wiki.alfresco.com/wiki/Benchmark_Framework_2.0#Developing_Tests 01-Using-the-archetype.wmv 02-running-new-project-locally.wmv Import into Eclipse Build and run Deploy and Execute Then Phase out sample code Phase in real test code

CMIS Load Test: Infrastructure BM Driver Virginia :9080 (CMIS) BM Driver Sydney :9080 (CMIS) BM Driver Ireland :9080 (Sign Up, CMIS, Sample) BM Server Ireland benchmark-server benchmark-server BM Server Ireland benchmark-server benchmark-server Config Data Test Data Alfresco 5.0.a CE

Preparing for the Tests Deploy and run Signup Test alfresco-benchmark-tests-ent-signup-2.0.war 05-Creating-users-in-Alfresco.wmvalfresco-benchmark-tests-ent-signup-2.0.war 05-Creating-users-in-Alfresco.wmv Permissions and test directories 06-Prepare-the-Alfresco-server.wmv

GitHub: Search ‘benchmark-cmis’GitHub: Search ‘benchmark-cmis Pull requests for new scenarios welcome Repository-agnostic Scenario 01 Find a random folder List folder contents Scenario 02 Get test folder Create folder Upload file Download file Delete folder Scenario 03 Get test folder Create folder Upload file Search in folder Delete folder Scenario weightings adjustable CMIS Load Test: Scenarios

CMIS Load Test: Sweet Spot Load driver “sweet spot”: Ireland IRELAND_03: 240 sessions at 0.25s intervals 07-Finding-the-sweet-spot.wmv Driver Alfresco

Data Comparison Raw results CMIS-IRELAND_05.xlsx CMIS-VIRGINIA_03.xlsx Compared CMIS Scenario (Ireland-Ireland vs Virginia-Ireland) CMIS.IRELAND_05CMIS.VIRGINIA_03xN cmis.scenario.01.findFolder cmis.scenario.01.listFolderContents cmis.scenario.02.createTestFolder cmis.scenario.02.deleteTestFolder cmis.scenario.02.downloadFile cmis.scenario.02.retrieveTestFolder cmis.scenario.02.uploadFile cmis.scenario.03.createTestFolder cmis.scenario.03.deleteTestFolder cmis.scenario.03.retrieveTestFolder cmis.scenario.03.searchInFolder cmis.startSession