1. 1 Specification and Implementation of Dynamic Web Site Benchmarks Sameh Elnikety Department of Computer Science Rice University

2. 2 Dynamic Content Is Common 1 3 2

3. 3 Generating Dynamic Content http Database Server Web Server Dynamic Content Generator

4. 4 DynaServer Project Scale Performance of Dynamic Content Sites –Caching –Clustering –Replication –Dynamic content delivery networks

5. 5 Motivation We Need Workloads to Evaluate Different Architectures No Workloads for Dynamic Content

6. 6 Benchmarks Online Bookstore –Models amazon.com Auction Site –Models ebay.com Bulletin Board –Models slashdot.org Common Applications Stress Different Parts of the System

7. 7 Contributions Online Bookstore –Followed TPC-W specification –Wrote implementation Auction Site –Developed specification & implementation Bulletin Board –Developed specification & implementation

8. 8 Outline Benchmarks – Online Bookstore – Auction Site – Bulletin Board Bottleneck Characterization – Experimental Setup – Measurement – Performance Results

9. 9 Online Bookstore Benchmark Models amazon.com Follows TPC-W (TPC Feb 2000) Activities –Browsing, buying, administrative updates Persistent Data –Images in Web server file system –All others in database

10. 10 Online Bookstore Benchmark Promotion Shopping Cart Next Interaction Image

11. 11 Online Bookstore Benchmark 14 Interactions (as specified in TPC-W), e.g. –Home –Best sellers –Secure payment –Shopping cart Workload Mixes (as specified in TPC-W) –Browsing (95% read-only) –Shopping (80% read-only) –Ordering (50% read-only)

12. 12 Auction Site Benchmark Models Ebay.com Activities –Browsing, selling, bidding User Sessions –Visitor, seller, buyer Persistent Data –Images in Web server file system –All others in database

13. 13 Auction Site Benchmark 26 interactions –Browsing by category –Bidding –Buying –Selling –Leaving comments –User page Workload Mixes –Browsing (100% read-only) –Bidding (85% read-only)

14. 14 Bulletin Board Benchmark Models Slashdot.org Activities –Discussion thread (story & comments) –Ratings User sessions –Visitor, moderator Persistent Data –Images in Web server file system –All others in database

15. 15 Bulletin Board Benchmark 24 interactions –Browsing by category –Submit new story –Search story titles –Display story & filter comments No Full Text Search Workload Mixes –Browsing (100% read-only) –Submission (85% read-only)

16. 16 Bottleneck Characterization Experimental Setup Measurement Performance Results

17. 17 Software http Apache PHP MySQL sql http Database Server Web Server Dynamic Content Generator

18. 18 Hardware CPU AMD Athlon 1.33GHz Memory 768MB Disk 60GB, 9msec, 5400rpm Network 100Mbps Ethernet http Apache PHP MySQL sql

19. 19 Performance Metric Throughput –Number of interactions per minute

20. 20 Resources Monitored Web Server Database CPU Memory Disk Network

21. 21 http Apache PHP MySQL sql Emulated Clients Emulated Clients Java Emulator –Session duration –Think time –Markov model

22. 22 Measurement Driving the System –Several clients –Ramp-up time –Steady-state measurement

23. 23 Online Bookstore Benchmark Throughput

24. 24 Bookstore CPU Utilization (Shopping Mix)

25. 25 Bookstore Resources (Shopping Mix) ResourceWeb ServerDatabase CPU30%71% Memory200 MB390 MB Disk<20 block/s Network3.2 + 0.6 Mb/s0.6 Mb/s Conclusion Database CPU is the bottleneck

26. 26 Auction Site Benchmark Throughput

27. 27 Auction CPU Utilization (Bidding Mix)

28. 28 Auction Resources (Bidding Mix) ResourceWeb ServerDatabase CPU100%58% Memory70 MB250 MB Disk<20 block/s Network55 + 2 Mb/s2 Mb/s Conclusion Web Server CPU is the bottleneck

29. 29 Bottlenecks Online Bookstore –Models amazon.com Bidding –Models ebay.com Bulletin Boards –Models slashdot.org –CPU of Database –CPU of Web Server

30. 30 Conclusions 3 Benchmarks – Online bookstore, auction site, bulletin board – Open-source software – Available implementations (PHP, Servlets, EJB) Capacity Planning Research on Dynamic Content

31. 31 Collaborators Cristiana Amza, Anupam Chanda, Alan Cox, Romer Gil, Willy Zwaenepoel Department of Computer Science - Rice University Emmanuel Cecchet, Julie Marguerite INRIA Rhône - Alpes Karthick Rajamani IBM Austin Research Lab

32. 32 Download the Benchmarks Benchmark Implementations in – PHP – Servlets – EJB http://www.cs.rice.edu/CS/Systems/DynaServer/