1. Distributed Load Testing of CMIS Alfresco Benchmark Framework 2.0
Derek Hulley Senior Developer
Michael Suzuki
Senior Developer
Good day.
The subject
I am … I am joined by …

2. Contents Essential Features Framework Architecture
Building and Deploying a Test
Alfresco-specific Tests
Creating Test Users
CMIS Test
Deployment
Execution
Results and Analysis
End:

3. Key Features of the Framework
Load Tests are First Class projects
Full Maven project lifecycle
Java
Automated testing
GitHub: derekhulley/alfresco-benchmark
Central Control
Scale up the load drivers
Preset test defaults and override only as required
Scalability
NoSQL storage of results
Independent load drivers compete for work
Deep Analysis
Errors, messages and results are all retained
Every event execution is recorded and kept

4. Framework Architecture
Benchmark Server
Benchmark Driver
Benchmark Driver (xN)
Tomcat 7
Benchmark Driver (xN)
Tomcat 7
Extras (Selenium) UI
Test
Services
Rest API
Rest API
Services
MongoDB Config Data
MongoDB Test Data
Servers / APIs
Load Balancer

5. Writing a New Test Maven archetype Then Import into Eclipse
01-Using-the-archetype.wmv 02-running-new-project-locally.wmv
Import into Eclipse
Build and run
Deploy and Execute
Then
Phase out sample code
Phase in real test code
Execute demo: ~3:30 (01-Using-the-archetype.wmv) mvn archetype:generate -DarchetypeRepository=
org.alfresco:alfresco-benchmark-sample-archetype

6. CMIS Load Test: Infrastructure
BM Driver Ireland :9080 (Sign Up, CMIS, Sample)
BM Server Ireland
Config Data
Test Data
Alfresco 5.0.a CE
BM Driver Sydney :9080 (CMIS)
BM Driver Virginia :9080 (CMIS)

7. Preparing for the Tests
Deploy and run Signup Test
alfresco-benchmark-tests-ent-signup-2.0.war 05-Creating-users-in-Alfresco.wmv
Permissions and test directories
06-Prepare-the-Alfresco-server.wmv
Live demo failure
You are receiving this because your Amazon CloudWatch Alarm "CPU Thrashing" in the EU - Ireland region has entered the ALARM state, because "Threshold Crossed: 1 datapoint (99.92) was greater than or equal to the threshold (50.0).“
Switched to cmis.alfresco.com
~2 min
Show results of DEMO4
Collection, login and user search

8. CMIS Load Test: Scenarios
GitHub: Search ‘benchmark-cmis’
Pull requests for new scenarios welcome
Repository-agnostic
Scenario 01
Find a random folder
List folder contents
Scenario 02
Get test folder
Create folder
Upload file
Download file
Delete folder
Scenario 03
Search in folder
Scenario weightings adjustable
Execute demo: ~3:30 (Using-the-archetype.wmv)

9. CMIS Load Test: Sweet Spot
Load driver “sweet spot”: Ireland
IRELAND_03: 240 sessions at 0.25s intervals
07-Finding-the-sweet-spot.wmv
Alfresco
Ensure only Ireland is connected.
Look for
Driver

10. Data Comparison Raw results Compared CMIS.IRELAND_05 CMIS.VIRGINIA_03
CMIS Scenario
(Ireland-Ireland vs Virginia-Ireland)
CMIS.IRELAND_05
CMIS.VIRGINIA_03 xN
cmis.scenario.01.findFolder
155
962
6.21
cmis.scenario.01.listFolderContents 81
643
7.94
cmis.scenario.02.createTestFolder 31
209
6.74
cmis.scenario.02.deleteTestFolder
130
279
2.15
cmis.scenario.02.downloadFile 26
391
15.04
cmis.scenario.02.retrieveTestFolder 23
345
15.00
cmis.scenario.02.uploadFile
159
772
4.86
cmis.scenario.03.createTestFolder
194
6.26
cmis.scenario.03.deleteTestFolder 53
230
4.34
cmis.scenario.03.retrieveTestFolder 24
367
15.29
cmis.scenario.03.searchInFolder 25
187
7.48
cmis.startSession 14
412
29.43
Ensure only Ireland is connected.
Look for
Raw results
CMIS.IRELAND_05
CMIS.VIRGINIA_03
Compared

11. References and Questions
Presentation Links
Presentation Demo Videos
GitHub:
derekhulley/alfresco-benchmark
derekhulley/benchmark-cmis
SVN:
Alfresco Signup Test V2.0
Wiki:
Benchmark Framework 2.0
Maven Artifacts:
org/alfresco/alfresco-benchmark-server/2.0.0
org/alfresco/alfresco-benchmark-sample-archetype/2.0.0
org/alfresco/alfresco-benchmark-tests-ent-signup/2.0
org/alfresco/alfresco-benchmark-tests-cmis/1.0-SNAPSHOT
Twitter:
@derekhulley
@suzukimichael
Ensure only Ireland is connected.
Look for