Cultural Heritage in REGional NETworks REGNET Project Meeting Content Group

June 2002REGNET Project Team Meeting Content Group 2 Why load tests and stress tests Complex systems make increasing demands on web servers Multiple objects can interfere High volumes can overwhelm systems Fixes need to be identified To identify bottlenecks in the system configuration To identify hardware requirements at the CSCs

June 2002REGNET Project Team Meeting Content Group 3 Methodology Performance testing objectives Pass / fail criteria Profiles of real users Test requirements Creation of test scripts and test execution Data gathering and correlation from the test tool and web / database server

June 2002REGNET Project Team Meeting Content Group 4 Performance testing objectives Find existing bottlenecks Verify current system capacity Verify scalability of the system Verify performance requirements Determine optimal hardware / application configuration

June 2002REGNET Project Team Meeting Content Group 5 Pass / fail criteria I (Proposal) Single transaction or single user: Successful completion of the test scripts without any failures and within the expected time allocation per transaction: Response time per page request < 1 seconds Multiple users carrying out multiple transactions in different roles: Successful completion of the test without any failures and within acceptable time per transaction / page delivery: Response time per page request < 5 seconds with up to 50 concurrent users A load test is successfully performed to validate performance to a minimum of 3600 different catalogue item requests hour A stress test is successfully performed to validate that the system and its single components do not reach their bottlenecks whilst serving pages to up to 100 concurrent users in different roles Tests are performed via LAN. Bandwith is not tested at this project stage. Bandwith monitoring can be used to determine necessary bandwith for production system. The database has its expected size These are very bold statements !!

June 2002REGNET Project Team Meeting Content Group 6 Pass / fail criteria II Web-server performance, the usability perspective Result page appearing time, Users View: < 0.1 s:User feels that the system is reacting instantaneously < 1.0 s:User experiences a slight delay but is still focused on the current website < 10 s:This is the maximum time a user keeps focused on a website, but the attention might already be distracted > 10 s:User is most likely distracted from the current website and looses interest Nielsen

June 2002REGNET Project Team Meeting Content Group 7 Profiles of real users (Proposal, to be discussed in Sofia) End user sends different requests to the system and views different items from the collection Special users performs different database functions (add, modify, delete) on single objects System administrator performs different database functions (add, modify, delete) on volume data Ratio of End User : Special User : System Administrator 22 : 2 : 1

June 2002REGNET Project Team Meeting Content Group 8 To Do´s in Sofia Estimation on test criteria Based on experience of current web- server usage rates Agreement on task assignement Agreement on test schedule

June 2002REGNET Project Team Meeting Content Group 9 To Do´s after Sofia Finalisation of test criteria Based on experience of current web-server usage rates Detailed user modelling (test scenarios) Test data gathering Software acquisition Set-up of test environement Test case implementation Carrying out of tests Results gathering Results evaluation + Recommendations Performance testing QA

June 2002REGNET Project Team Meeting Content Group 10 Technical test requirements Datasets are loaded onto the system / available for loading to perform different activities Different users exist in the system and have appropriate permissions Test hard- and software configuration PC running on Windows/NT, LINUX, etc., JDK1.2 or higher and jmeter installed, connected via LAN to the webserver A well equipped PC should be able to handle up to 100 concurrent threads (virtual users) Test scripts implemented Exclusive access to the system (1 night ?!) Access to server logs (processor usage, memory usage, disk I/O rates, network traffic) and jmeter logs

June 2002REGNET Project Team Meeting Content Group 11 Timeline (proposal) TBD