Presentation is loading. Please wait.

Presentation is loading. Please wait.

Performance Pitfalls  Paul Lockwood. About the Presenter  Über-geek First computer in 1981 First fulltime IT job in 1989 Clipper/Dbase IV, PowerBuilder,

Similar presentations


Presentation on theme: "Performance Pitfalls  Paul Lockwood. About the Presenter  Über-geek First computer in 1981 First fulltime IT job in 1989 Clipper/Dbase IV, PowerBuilder,"— Presentation transcript:

1 Performance Pitfalls  Paul Lockwood

2 About the Presenter  Über-geek First computer in 1981 First fulltime IT job in 1989 Clipper/Dbase IV, PowerBuilder, Java, C#  Contracted most of career Lived/ worked in four different countries Vast exposure to different approaches  Stand on Shoulders of Giants

3 Causes are Well Known  Memory Issues Mainly leaks, sometimes churn/ GC issues  I/O Disk, network  100% CPU Runaway threads  Bad SQL, very poor DB Design  Blocking lock around cache reads Knock on effect of spin-waits, waiting on I/O

4 Performance Basics  Orders of Magnitude Services are thousands, even millions of times slower Architecture can guarantee poor performance  Typical Tuners think in milliseconds We Look for Big Mistakes Generally Ignore:  Value/reference boxing  Static method vs v-table lookup  Most marshaling  Hash lookups  Concatenating strings

5 Removing Guesswork SummaryWarm AverageMagnitude InProcess680ns1 Named Pipes Pooled (10K requests)35us10 2 Named Pipes (Connect, Disconnect)237us10 3 Web Service Local [REST result]1ms [1.6ms]10 4 Web Service LAN [REST result]5.5ms [4ms]10 4 Web Service Internet [REST result]80ms [35ms]10 5 MySQL Local Pooled0.5ms10 3 MySQL Local Non Pooled6.2ms10 4 MySQL LAN Pooled6ms10 4 MqSQL LAN Non Pooled2,700ms10 7 1ns is one CPU Cycle on a 1Ghz Processor

6 Architectural Mistakes  Overuse of SOA, DI/IoC, Reflection etc ~100ms per SOA call is typical All are difficult to debug  Layer and compile-in often the best choice  Homegrown Frameworks Since 2000s these mostly fail Architect’s continued control prevents rescue  Buzzword/Resume Driven Architectures

7 Demos #1  JMeter to load test  IDE to attach, pause and view all threads

8 N-Tier: Popular but slow

9 Demos #2  WinDbg – Analyzing a Production Dump  Screen Captures from Real World Tuning

10 Personal Quick Wins  Find/ Remove Memory Leaks Static HashTable is classic example  Remove.Net Exceptions used in Logic  Remove unnecessary Locking Even saw a Logging Sink with contentions! ReaderWriterLocks are easy fixes  Short circuit SOA/ Web Service calls  SQL Tuning, even just adding Indexes!  Very chatty SQL calls -> pure Server or pure client  ‘Real OOP’ Code -> Trivial SQL Code  Stop Log.Debug writing to Database  Cache Lookup data (lazy load Singleton with IDispose) Consider Implementing ‘Clear Cache Page’

11 More Difficult Fixes  Complex sproc vs 10klocs of C#/Java code Automated Developer Tests essential with complexity  SQL Optimization Hints, really understand data, rewrite sprocs..  3-Tier Library total re-write Fought Architecture Team in a multinational  Native Memory Leak Very hard to reproduce caching issue Advice from others totally misleading IISDiag confirmed the leak, then JMeter isolated  Code Simplifications Hard coded logic -> Table driven methods XMLDocument parsing -> xsd.exe/ XmlBeans

12 Locating the Problem  Existing Excuses Only happens in live Environments  We have special hardware  Cannot reproduce traffic Problem Domain is insanely complex  Hardware is not good enough, need 64Gb etc  Test machine Replicate live data if possible CPU Horsepower is not really important IDE useful to compile/ debug Exclusive access to database, test hardware!!  Taking Dumps (Not Recommended) WinDbg, sos.dll, adplus, IISDiag (auto scripts) etc Like CSI, it’s a snapshot in time Total experts only – sole job function perhaps?

13 Final Slide Feedback is much appreciated  DotNetWorkaholic.com

14 N-Tier/ Distributed Objects “Objects have been around for a while, and sometimes it seems that ever since they were created, folks have wanted to distribute them. However, distribution of objects, or indeed of anything else, has a lot more pitfalls than many people realize, especially when they’re under the influence of vendors’ cozy brochures.” Martin Fowler (2002)  Distributing Objects: Increases Expense Increases Complexity Reduces Performance - The Mysterious Allure of distributing objects

15 Scalability  Look at Performance first Scalability seems easier/ sexier  Product X, Y or Z won’t Scale! Really? Use common sense; ask why not Any ORM should scale: tune/cache the hot paths  DataBase writes are an ideal Bottleneck Most apps are mostly reads, few writes. i.e. Cache for reads Async audit/ analytics tracking to other servers  Horizontal vs Vertical scaling How many people have ten to fifteen web/app servers in production?  Major Enemies Session State Pessimization (Pessimistic Optimization) Insane Architects :)


Download ppt "Performance Pitfalls  Paul Lockwood. About the Presenter  Über-geek First computer in 1981 First fulltime IT job in 1989 Clipper/Dbase IV, PowerBuilder,"

Similar presentations


Ads by Google