We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byBilly Cressy
Modified about 1 year ago
reimagining the business of apps ©2013 NativeX Holdings, LLC
Measure Everything Understanding your systems.
Why should we measure? Understanding the current state of our code and systems.
Experiences Why StatsD and Graphite may help you!
©2013 NativeX Holdings, LLC Experiences System performance degradation
©2013 NativeX Holdings, LLC Experiences Understanding when releases have an effect on our systems
©2013 NativeX Holdings, LLC Experiences Load balancing – When is my server active
©2013 NativeX Holdings, LLC Experiences Transitioning traffic to a new system
Convinced Yet? Hopefully so.
Other reasons you should use StatsD and Graphite?
©2013 NativeX Holdings, LLC StatsD and Graphite Blending data tracking and data representation Its Open Source High Performance Fire and Forget (UDP) Data Representation through Graphs Render API Automatic Measure Creation (Low Configuration) Data roll-off Complex Data Aggregation through Render API
StatsD High Performance Data Logging
©2013 NativeX Holdings, LLC StatsD How does it work? Node JS UDP Pluggable Backend (Graphite, Console, Repeater, Create Your Own)
©2013 NativeX Holdings, LLC StatsD Why NodeJS? High throughput Simple code Simple calls Async Everything How does it work? It Listens for metrics on UDP socket It Assigns a metric to its own bucket It Aggregates values for each bucket until they need to flush Each bucket flushes every X seconds (configurable) and resets the value in the bucket
©2013 NativeX Holdings, LLC StatsD Why UDP? Fire and Forget Unless you majorly overload your network with UDP traffic it will not bring down your systems if StatsD fails Little network overhead, small messages Ask me how many times we crashed StatsD
©2013 NativeX Holdings, LLC StatsD – Metric Types Timers – Track Times by Millisecond How long does it take to make calls to other people’s APIs Do I know how long specific units of code take to run? How long does it take for my API to respond? Counters – Track an Integer Value How many times does my page get hit per flush interval How many times does my API get hit per flush interval How many users hit my API per flush interval Gauges – Track a Value What was this value at the last point before flush interval? What is our CPU on our serer running at? How much memory do we have available? How much disk space do we have available?
©2013 NativeX Holdings, LLC StatsD Lets talk about creating metrics If the metric name did not exist previously, a bucket will be created to store the data. “.” represents a step down in hierarchy. Think of it as a file system but replace “\” with “.”. MDC.Twitter.HashTags becomes MDC Twitter HashTags
©2013 NativeX Holdings, LLC StatsD Show me some code already, OK NStatsD.Client.Current.Increment("testing.increment"); NStatsD.Client.Current.Decrement("testing.decrement"); NStatsD.Client.Current.Timing("testing.timing", 2345); NStatsD.Client.Current.Gauge("testing.gauge", 45); Also has ability to do sample rate, which allows you to only send metrics every x requests. Value is expressed as a double from 0.0 to 1.0 Can it send tracking from SQL Server? It can Check the Questions page
Graphite Data Aggregation and Graphing
©2013 NativeX Holdings, LLC Graphite How does it work? Graphite Web – for assembling and rendering graphs Render API – An API which takes querystring parameters to generate graphs or json Web App – Allows you to generate graphs by assembling simple urls which call the render api
©2013 NativeX Holdings, LLC Graphite Render API How easy is it? Let me show you. Example Graph - et=stats.gauges.MDC.Twitter.Mentions et=stats.gauges.MDC.Twitter.Mentions Need JSON? et=stats.gauges.MDC.Twitter.Mentionshttp://dev.statsd.teamfreeze.com/render/?width=800&height=500&targ et=stats.gauges.MDC.Twitter.Mentions&format=json
©2013 NativeX Holdings, LLC Making Measurements Meaningful Understanding data is all about knowing the context and presentation Grids (Nice filtering, but little impact) Charts (Depends on the Type and what You’re Measuring) Line (Great for comparing things over time) Gauge (Great for showing general system health as an average) Focus (Selectable Timeframe for in depth analysis over time) Alerting - Graphs are nice, but do we really want to stare at graphs all day
Lets Build a Graph Graphite - How simple can it be?
©2013 NativeX Holdings, LLC Graphite Dashboards What if I want to create a collection of graphs that I want to repeatedly look at? /dashboard – Created named dashboards with graphs which you generate
Setting Up StatsD & Graphite
©2013 NativeX Holdings, LLC Components StatsD nodeJS module [Listener; Aggregation] Graphite Carbon carbon-relay carbon-cache carbon-aggregator Whisper Django Webapp
©2013 NativeX Holdings, LLC Graphite – Python modules Store and render time-series data carbon - a daemon that listens for time-series data whisper - a simple database library for storing time-series data graphite webapp - A Django webapp that renders graphs on- demand Query = look in carbon cache + whisper files
©2013 NativeX Holdings, LLC Carbon daemons Receive, cache, store carbon-relay.py Basically a Load Balancer carbon-cache.py – daemon receives incoming data into RAM flush to whisper on an interval carbon-aggregator.py – optional Works similar to StatsD TCP
©2013 NativeX Holdings, LLC Whisper The datastore Each metric in a.wsp file.wsp are pre-allocated; fixed size Retention and resolution are predefined & static Each metric type creates different number of.wsp files Key TypeBalloon Factor Sets1 Counter2 Timer9 Gauge1 Resolution Retention ShortLong Aggregated Precise Storage Configuration
©2013 NativeX Holdings, LLC Storage-Schemas.conf /graphite/conf/storage-schemas.conf [title_of_schema] Priority = 110 Pattern =.* Retention = 20s:7d,1m:21d,15m:3y Fixed size database requires parameters [title] – can be anything Priority - Carbon checks priority from high to low Pattern - regex match or default is used ^servers\. Or ^user_metrics\. Retention – 20 seconds for 7 days, 1 minute for 21 days, 15 minutes for 3 years
©2013 NativeX Holdings, LLC StatsD Configuration File: /dConfig.js Ex: /opt/graphite/statsd/dConfig.js JSON Object Graphite backend config (could be a carbon-relay or –cache) graphitePort graphiteHost legacyNamespace: Should ‘counts’ for counters and timers have ‘stats_counts’ prefix? [for backwards compat] port – for the StatsD UDP socket flushInterval: In ms, how often to flush values to backend and reset to 0.
Lessons Learned What did we learn the hard way?
©2013 NativeX Holdings, LLC Lessons learned Keep Flush interval’s the same StatsD & Carbon config 10 seconds typically Limit bucket architecture Bad idea – reporting per server in a cluster Buckets total size is static Filled up whisper storage? corrupt data! Countries example
©2013 NativeX Holdings, LLC Lessons learned Data points per pixel More data points per pixel = rounded data Some crazy numbers Show example
©2013 NativeX Holdings, LLC Lessons learned NStatsD C# library Retention ratios don’t exactly work Need to handle % sent to NStatsD Example Start in the cloud Scalability Data Precision & Accuracy Company adoption brings over use Limit use to monitoring general trends
©2013 NativeX Holdings, LLC Rolling your own system What can you do with StatsD and Graphite
©2013 NativeX Holdings, LLC Questions? Graphite - https://graphite.readthedocs.org/en/0.9.10/https://graphite.readthedocs.org/en/0.9.10/ StatsD - measure-everything/http://codeascraft.com/2011/02/15/measure-anything- measure-everything/ StatsD GitHub - https://github.com/etsy/statsd/https://github.com/etsy/statsd/ StatsD Metrics - https://github.com/etsy/statsd/blob/master/docs/metric_types.md https://github.com/etsy/statsd/blob/master/docs/metric_types.md NStatsD - https://github.com/robbihun/NStatsD.Clienthttps://github.com/robbihun/NStatsD.Client Graphite ** - Github on the stored procedure - https://github.com/peschuster/graphite- client/blob/master/source/Graphite.TSql/sp_graphitesend.sql https://github.com/peschuster/graphite- client/blob/master/source/Graphite.TSql/sp_graphitesend.sql DLL, should be placed in the shared location - https://github.com/peschuster/graphite- client/tree/master/source/Graphite.TSql https://github.com/peschuster/graphite- client/tree/master/source/Graphite.TSql ** super useful for setting graphite up the right way
Monitoring with InfluxDB & Grafana Andrew Lahiff HEP SYSMAN, Manchester 15 th January 2016.
Module 10: Monitoring ISA Server Overview Monitoring Overview Configuring Alerts Configuring Session Monitoring Configuring Logging Configuring.
Ideas to Improve SharePoint Usage 4. What are these 4 Ideas? 1. 7 Steps to check SharePoint Health 2. Avoid common Deployment Mistakes 3. Analyze SharePoint.
PRESENTED BY Alan Eldridge Sales Consultant Tableau Software Australia.
Clusterpoint Margarita Sudņika ms RDBMS & NoSQL Databases & tables → Document stores Columns, rows → Schemaless documents Scales UP → Scales UP.
Hands-On Microsoft Windows Server 2008 Chapter 11 Server and Network Monitoring.
Marcelo R.N. Mendes. What is FINCoS? A set of tools for data generation, load submission, and performance measurement of CEP systems; Main Characteristics:
© 2008 Quest Software, Inc. ALL RIGHTS RESERVED. Perfmon and Profiler 101.
Google App Engine Danail Alexiev Technical Trainer SoftAcad.bg.
©Company confidential 1 Performance Testing for TM & D – An Overview.
IMDGs An essential part of your architecture. About me
Windows Server 2008 Chapter 11 Last Update
Cloud Computing for the Enterprise November 18th, This work is licensed under a Creative Commons.
Meet with the AppEngine Márk Gergely eu.edge. What is AppEngine? It’s a tool, that lets you run your web applications on Google's infrastructure. –Google's.
Microsoft Virtual Academy Module 7 Installing and Configuring System Center 2012 R2 Virtual Machine Manager.
What Can You do With BTM? Business Transaction Management touches the following disciplines: Performance Management Application Management Capacity.
SSRS 2008 Architecture Improvements Scale-out SSRS 2008 Report Engine Scalability Improvements.
CS526 - Advanced Internet & Web Systems May 11, 2009 Nathan Archer.
Copyright 2007, Information Builders. Slide 1 Machine Sizing and Scalability Mark Nesson, Vashti Ragoonath June 2008.
Module 8: Concepts of a Network Load Balancing Cluster.
Amazon Web Services. Amazon Web Services (AWS) - robust, scalable and affordable infrastructure for cloud computing. This session is about:
Enterprise PI - How do I manage all of this? Robert Raesemann J Jacksonville, FL.
Capacity Planning for LAMP Architectures John Allspaw Manager, Operations Flickr.com Web Builder 2.0 Las Vegas.
Distributed Time Series Database InfluxDB/openTSDB.
Chapter 14 Chapter 14: Server Monitoring and Optimization.
1 © Prentice Hall, 2002 The Client/Server Database Environment.
Distributed Data Stores – Facebook Presented by Ben Gooding University of Arkansas – April 21, 2015.
Microsoft Confidential © 2012 Microsoft Corporation. All rights reserved.
Cloud Computing Computer Science Innovations, LLC.
COOKIES AND SESSIONS.
Platform as a Service (PaaS) Presenter: Arun Kumar Singh.
GENERAL SCALABILITY CONSIDERATIONS
CERN IT Department CH-1211 Genève 23 Switzerland t Integrating Lemon Monitoring and Alarming System with the new CERN Agile Infrastructure.
Ch. 31 Q and A IS 333 Spring 2016 Victor Norman. SNMP, MIBs, and ASN.1 SNMP defines the protocol used to send requests and get responses. MIBs are like.
If you have a transaction processing system, John Meisenbacher
+ Logentries Is a Real-Time Log Analytics Service for Aggregating, Analyzing, and Alerting on Log Data from Microsoft Azure Apps and Systems MICROSOFT.
Distributed Logging Facility Castor External Operation Workshop, CERN, November 14th 2006 Dennis Waldron CERN / IT.
How WebMD Maintains Operational Flexibility with NoSQL Rajeev Borborah, Sr. Director, Engineering Matt Wilson – Director, Production Engineering – Consumer.
The Replica Location Service The Globus Project™ And The DataGrid Project Copyright (c) 2002 University of Chicago and The University of Southern California.
Ceilometer + Gnocchi + Aodh Architecture 1 Ceilometer (data collection): to efficiently collect, normalise and transform data produced by OpenStack services.
Internet Information Server 6.0. IIS 6.0 Enhancements Fundamental changes, aimed at: Reliability & Availability Reliability & Availability Performance.
Test Loads Andy Wang CIS Computer Systems Performance Analysis.
Austin code camp 2010 asp.net apps with azure table storage PRESENTED BY CHANDER SHEKHAR DHALL
Nagios Performance Tuning Nick Scott
Module 5: Managing Public Folders. Overview Managing Public Folder Data Managing Network Access to Public Folders Publishing an Outlook 2003 Form Discussion:
Designing High Performance BIRT Reports Mica J. Block Director Actuate Corporate Engineers Actuate Corporation.
Reset and Recycle IIS Reset Application Pool Management Error Codes New HTTP Sub-status codes Custom/Detailed Errors Tracing in IIS7 and.
Institute of Computer and Communication Network Engineering OFC/NFOEC, 6-10 March 2011, Los Angeles, CA Lessons Learned From Implementing a Path Computation.
CLOUDWATCH VERY SIRIUS MONITORING. SENDING METRICS Metrics can be sent in simply through additional handlers to common tools No handler yet for collectd.
© 2017 SlidePlayer.com Inc. All rights reserved.