Distributed Load Testing of CMIS Alfresco Benchmark Framework 2.0 Derek Hulley Senior Developer Michael Suzuki Senior Developer Good day. The subject I am … I am joined by …

Contents Essential Features Framework Architecture Building and Deploying a Test Alfresco-specific Tests Creating Test Users CMIS Test Deployment Execution Results and Analysis End:

Key Features of the Framework Load Tests are First Class projects Full Maven project lifecycle Java Automated testing GitHub: derekhulley/alfresco-benchmark Central Control Scale up the load drivers Preset test defaults and override only as required Scalability NoSQL storage of results Independent load drivers compete for work Deep Analysis Errors, messages and results are all retained Every event execution is recorded and kept

Framework Architecture Benchmark Server Benchmark Driver Benchmark Driver (xN) Tomcat 7 Benchmark Driver (xN) Tomcat 7 Extras (Selenium) UI Test Services Rest API Rest API Services MongoDB Config Data MongoDB Test Data Servers / APIs Load Balancer

Writing a New Test Maven archetype Then Import into Eclipse https://wiki.alfresco.com/wiki/Benchmark_Framework_2.0#Developing_Tests 01-Using-the-archetype.wmv 02-running-new-project-locally.wmv Import into Eclipse Build and run Deploy and Execute Then Phase out sample code Phase in real test code Execute demo: ~3:30 (01-Using-the-archetype.wmv) mvn archetype:generate -DarchetypeRepository=http://artifacts.alfresco.com/nexus/content/groups/public/archetype-catalog.xml org.alfresco:alfresco-benchmark-sample-archetype

CMIS Load Test: Infrastructure BM Driver Ireland 1 54.73.146.12:9080 (Sign Up, CMIS, Sample) BM Server Ireland http://54.217.124.68:9080/alfresco-benchmark-server Config Data Test Data Alfresco 5.0.a CE BM Driver Sydney 1 54.206.178.67:9080 (CMIS) BM Driver Virginia 1 54.167.28.179:9080 (CMIS)

Preparing for the Tests Deploy and run Signup Test alfresco-benchmark-tests-ent-signup-2.0.war 05-Creating-users-in-Alfresco.wmv Permissions and test directories 06-Prepare-the-Alfresco-server.wmv Live demo failure You are receiving this email because your Amazon CloudWatch Alarm "CPU Thrashing" in the EU - Ireland region has entered the ALARM state, because "Threshold Crossed: 1 datapoint (99.92) was greater than or equal to the threshold (50.0).“ Switched to cmis.alfresco.com ~2 min Show results of DEMO4 Collection, login and user search

CMIS Load Test: Scenarios GitHub: Search ‘benchmark-cmis’ Pull requests for new scenarios welcome Repository-agnostic Scenario 01 Find a random folder List folder contents Scenario 02 Get test folder Create folder Upload file Download file Delete folder Scenario 03 Search in folder Scenario weightings adjustable Execute demo: ~3:30 (Using-the-archetype.wmv)

CMIS Load Test: Sweet Spot Load driver “sweet spot”: Ireland IRELAND_03: 240 sessions at 0.25s intervals 07-Finding-the-sweet-spot.wmv Alfresco Ensure only Ireland is connected. http://54.247.12.28:9080/alfresco-benchmark-server/api/v1/tests/CMIS/drivers Look for Driver

Data Comparison Raw results Compared CMIS.IRELAND_05 CMIS.VIRGINIA_03 CMIS Scenario (Ireland-Ireland vs Virginia-Ireland) CMIS.IRELAND_05 CMIS.VIRGINIA_03 xN cmis.scenario.01.findFolder 155 962 6.21 cmis.scenario.01.listFolderContents 81 643 7.94 cmis.scenario.02.createTestFolder 31 209 6.74 cmis.scenario.02.deleteTestFolder 130 279 2.15 cmis.scenario.02.downloadFile 26 391 15.04 cmis.scenario.02.retrieveTestFolder 23 345 15.00 cmis.scenario.02.uploadFile 159 772 4.86 cmis.scenario.03.createTestFolder 194 6.26 cmis.scenario.03.deleteTestFolder 53 230 4.34 cmis.scenario.03.retrieveTestFolder 24 367 15.29 cmis.scenario.03.searchInFolder 25 187 7.48 cmis.startSession 14 412 29.43 Ensure only Ireland is connected. http://54.247.12.28:9080/alfresco-benchmark-server/api/v1/tests/CMIS/drivers Look for Raw results CMIS.IRELAND_05 CMIS.VIRGINIA_03 Compared

References and Questions Presentation Links Presentation Demo Videos GitHub: derekhulley/alfresco-benchmark derekhulley/benchmark-cmis SVN: Alfresco Signup Test V2.0 Wiki: Benchmark Framework 2.0 Maven Artifacts: org/alfresco/alfresco-benchmark-server/2.0.0 org/alfresco/alfresco-benchmark-sample-archetype/2.0.0 org/alfresco/alfresco-benchmark-tests-ent-signup/2.0 org/alfresco/alfresco-benchmark-tests-cmis/1.0-SNAPSHOT Twitter: @derekhulley @suzukimichael Ensure only Ireland is connected. http://54.247.12.28:9080/alfresco-benchmark-server/api/v1/tests/CMIS/drivers Look for