1. Paruj Ratanaworabhan, Ben Livshits, David Simmons, and Ben Zorn JSMeter: Characterizing the Behavior of JavaScript Web Applications 1 In collaboration with Corneliu Barsan, and Allen Wirfs-Brock

2. Why Measure JavaScript? Standardized, de facto language for the web –Support in every browser, much existing code Major Microsoft investment –Web Application Companion based on Script# Benchmarks are driving browser marketing: “browser wars” –Are benchmarks representative? –Limited understanding of its behavior in real sites 2

3. 3 ZDNet 29 May 2008 ghacks.net Dec. 2008

4. Artificial Benchmarks versus Real World Sites 4 Maps 7 V8 programs: richards deltablue crypto raytrace earley- boyer regexp splay 8 SunSpider programs: 3-draytrace access-nbody bitops-nsieve controlflow crypto-aes date-xparb math-cordic string-tagcloud JSMeter Goals of this Talk Show behavior of JavaScript benchmarks versus real sites Consider how benchmarks can mislead design decisions Goals of this Talk Show behavior of JavaScript benchmarks versus real sites Consider how benchmarks can mislead design decisions 11 real sites

5. 5 Instrumentation Framework \ie\jscript\*.cpp Source-level instrumentation custom jscript.dll custom trace files website visits Offline analyzers custom trace files

6. Understanding JavaScript Behavior CodeEvents JavaScript Objects 6

7. Code Behavior CodeEvents JavaScript Objects 7 Function size Instructions/call Code locality Instruction mix Function size Instructions/call Code locality Instruction mix

8. 8

10. Number of Functions Real sites V8SunSpider Loaded = 2,717 Executed = 41% Hot = 3% Loaded = 97Loaded = 13 Executed = 88% Hot = 11% Executed = 80% Hot = 20% 10 Code|Objects|Events

11. 8749,7431,7922,747,484 591 Real sites V8 SunSpider Instructions Executed per Call 11 Code|Objects|Events function(a,b) { var i=0,elem,pos=a.length; if(D.browser.msie) { while(elem=b[i++]) if(elem.nodeType!=8) a[pos++]=elem; } else while(elem=b[i++]) a[pos++]=elem; return a }

12. Object Allocation Behavior CodeEvents JavaScript Objects 12 Allocation by types Lifetime distribution Live heap composition Allocation by types Lifetime distribution Live heap composition

13. V8 Total Memory Allocation by Type 13 Code|Objects|Events Real Applications Types in real applications are: - Different than in the benchmarks - Functions play a larger role - Objects play a smaller role

14. Live Heap Over Time (gmail) 14 Functions, objects grow steadily Strings are transient Code|Objects|Events

15. Live Heap over Time (ebay) 15 Heaps repeatedly created, then discarded Heap contains mostly functions Code|Objects|Events

16. Event Handlers in JavaScript CodeEvents JavaScript Objects 16 Number of events Sizes of handlers Number of events Sizes of handlers

17. Event-driven Programming Model Single-threaded, non-preemptive event handlers Example of some standard event-handler attributes: –onabort, onclick, and onmouseover Very different from batch processing of benchmarks Handler responsiveness critical to user experience 17 Code|Objects|Events

18. 6 Number of Events Handled Real sites V8 669 18 Code|Objects|Events

19. Impact of Benchmarks What gets emphasis –Making tight loops fast –Optimizing small amounts of code Important issues ignored –Garbage collection (especially of strings) –Managing large amounts of code –Optimizing event handling –Considering JavaScript context between page loads 19

20. Conclusions JSmeter is an instrumentation framework –Used to measure and compare JavaScript applications Behavior in benchmarks significantly differs from real sites –Misleads designers, skews implementations Next steps –Develop and promote better benchmarks –Use insights from our measurements to guide designers –Make measurement infrastructure broadly available to JavaScript developers 20

21. 21 google (Web 1.0)bing (Web 2.0) Object Lifetimes Heap Content

22. Backup Slides 22

23. Total Bytes Allocated 23 V8Real Applications Many benchmarks allocate little data

24. Sizes of Handlers Median executed handler instructions Maximum executed handler instructions 24 Code|Objects|Events bingmapgmailfacebook Max = 1,107,532 Median = 314 Max = 594,482 Median = 506 Max = 89,785 Median = 380 crypto Max = 86,000,000