Cleveland SQL Saturday Catch-All or Sometimes Queries

Slides:



Advertisements
Similar presentations
JQuery MessageBoard. Lets use jQuery and AJAX in combination with a database to update and retrieve information without refreshing the page. Here we will.
Advertisements

Tuning: overview Rewrite SQL (Leccotech)Leccotech Create Index Redefine Main memory structures (SGA in Oracle) Change the Block Size Materialized Views,
Application Generator Merrill Networking Services.
HOW TO IMPORT AND EXPORT DATA. Why do I need to use Export/Import? Here are some examples: If you have a laptop that you take home (or have a home version.
SQL SERVER 2012 XVELOCITY COLUMNSTORE INDEX Conor Cunningham Principal Architect SQL Server Engine.
Tempdb Parasites Jason Hall-Dir. of Client SQL Sentry Blog-jasonhall.blogs.sqlsentry.net.
No, Thanks, I’ll Use a Spreadsheet
Introduction Optimizing Application Performance with Pinpoint Accuracy What every IT Executive, Administrator & Developer Needs to Know.
Physical Database Design & Performance. Optimizing for Query Performance For DBs with high retrieval traffic as compared to maintenance traffic, optimizing.
1099 Why Use InterBase? Bill Todd The Database Group, Inc.
Moving Around in Scratch The Basics… -You do want to have Scratch open as you will be creating a program. -Follow the instructions and if you have questions.
ITGS Databases.
Sofia, Bulgaria | 9-10 October The Query Governor Richard Campbell Stephen Forte Richard Campbell Stephen Forte.
Operating Systems Lecture 14 Segments Adapted from Operating Systems Lecture Notes, Copyright 1997 Martin C. Rinard. Zhiqing Liu School of Software Engineering.
Visual Basic for Application - Microsoft Access 2003 Finishing the application.
Continuous Improvement. Start Simple and Continually Improve E.g., Gmail Labels 1.
© 2015 albert-learning.com How to talk to your boss How to talk to your boss!!
SQL Saturday Pittsburgh So you want to be a DBA What is a DBA? Michael John Ed Wagner SQL Saturday Pittsburgh October 3rd 2015.
How to kill SQL Server Performance Håkan Winther.
Copyright Office Material Copyright Request System.
SQL Saturday Cleveland Top development and Design Mistakes Michael John SQL Saturday Cleveland February 6th 2016.
The purpose of a CPU is to process data Custom written software is created for a user to meet exact purpose Off the shelf software is developed by a software.
VAdata Tools VAdata: Virginia’s Sexual and Domestic Violence Data Collection System.
Parameter Sniffing in SQL Server Stored Procedures
Memory COMPUTER ARCHITECTURE
Dynamic SQL Writing Efficient Queries on the Fly
A Real Problem What if you wanted to run a program that needs more memory than you have? September 11, 2018.
Virtual Memory - Part II
Attendance Tracking Module
SQL Saturday Pittsburgh
Database Performance Tuning &
SQL Injection Attacks Many web servers have backing databases
Parameter Sniffing in SQL Server Stored Procedures
loooooooooooooong Introduction!
Reminder: Please choose a “challenging” topic for your project. If it cannot be done by the end of this semester, please show us what you have done toward.
Dynamic SQL Writing Efficient Queries on the Fly
Database Performance Tuning and Query Optimization
CSE 120 Principles of Operating
DevOps Database Administration
SQL Server May Let You Do It, But it Doesn’t Mean You Should
Marcos Freccia Stop everything! Top T-SQL tricks to a developer
Agenda Database Development – Best Practices Why Performance Matters ?
DevOps Database Administration
Lecture 12 Lecture 12: Indexing.
Number and String Operations
Exceptions & Error Handling
Statistics: What are they and How do I use them
Steve Hood SimpleSQLServer.com
Advanced SQL: Views & Triggers
TEMPDB – INTERNALS AND USAGE
Selected Topics: External Sorting, Join Algorithms, …
Please thank our sponsors!
There’s not always room for one more. Brian Bershad
Chartboost Help Site Competitive Analysis and Proposal
Four Rules For Columnstore Query Performance
Please use speaker notes for additional information!
“All About Me” Staff Development Day
Summit Nashville /3/2019 1:48 AM
Chapter 11 Database Performance Tuning and Query Optimization
Tonga Institute of Higher Education IT 141: Information Systems
Scheduling.
Data Structures & Algorithms
Tonga Institute of Higher Education IT 141: Information Systems
Running a Java Program using Blue Jay.
Exceptions 10-May-19.
CSE 153 Design of Operating Systems Winter 19
COMP755 Advanced Operating Systems
Using wait stats to determine why my server is slow
CMPE 152: Compiler Design May 2 Class Meeting
Best Practices in Higher Education Student Data Warehousing Forum
Presentation transcript:

Cleveland SQL Saturday Catch-All or Sometimes Queries Michael John mljohn11@Comcast.net 1

Cleveland SQL Saturday About Me! 8th SQL Saturday 20 years as a developer, DBA, and general IT pain. Started with SQL 6.0 Became the DBA because I was the only person who knew “SELECT * FROM TABLE” First job in IT was an entry level developer at 17k per year. Three years later I was Director of IT with a staff of 45. Making a little more than 17k! I wish I had more time to write again. And more money for longer vacations! Been there, done that! 2

Cleveland SQL Saturday What I spend all my time on! 3

Cleveland SQL Saturday My Rants Triggers Cursors Cryptcnmes Doing work on a production machine A server is NOT a desktop!!! Begging for disk space Untested code / procedures being run in production. “Google programming” Copy and paste coding practices Just because you can, doesn’t mean you should. Incompetence being touted as authority 4

Cleveland SQL Saturday SQL Saturday 2015, October 3rd. Is promoting a Pittsburgh function something that’s allowed in Cleveland? 5

Cleveland SQL Saturday Agenda What are catch-all queries? Examples Issues How to handle them Bad Practices Questions, and maybe correct answers 6

Cleveland SQL Saturday What are Catch-All Queries? The term was coined by Gail Shaw in the article on her web site. http://sqlinthewild.co.za/index.php/2009/03/19/catch-all-queries/ This presentation acknowledges this article as a source. 7

Cleveland SQL Saturday What are Catch-All Queries? Queries that contain many optional parameters Queries that run in many different ways, depending upon the parameters Plan cache is likely less than optimal Typically the back end to a search screen Reporting Reporting Reporting 8

Cleveland SQL Saturday What Are Catch-All Queries?   They are also hidden by developers If you see a proc with… IF @parameter = something BEGIN --Do some select IF @parameter = something else BEGIN --Do some other select Converting a procedural query to a set based query may create a catch-all 9

Cleveland SQL Saturday What are Catch-All Queries?   Reports that have many parameters All, one, or many values Customers Date Ranges Status list Any number of other values Look for reports with drop downs that contain “ALL”, “None”, and so forth as a value that can be selected 10

Cleveland SQL Saturday What are Catch-All Queries? 11

Cleveland SQL Saturday Example of a catch-all query   This is an actual example from my current system Example #2 12

Cleveland SQL Saturday What issues do they cause???   Many possible execution plans, the best one may not be used. Table scans, table scans, table scans, depending how its written Larger tables, large result sets, or a high number of executions may be very slow. User complain. It’s too (fill in the blank) Difficult to maintain Code may be hard to follow 13

Cleveland SQL Saturday How do we handle them???   You don’t. You can only lessen the pain! Seriously re-consider the architecture Was this ever planned for in the first place? Did this evolve over time? User requests? Do the business rules still apply? “It’s the way we always did it” Are there other methods that may work better? Full-Text search? Lucene? Create big old denormalized tables Create big “word lists” Data warehouse? Do this in code, not T-SQL? Do these need to be real time?? 14

Cleveland SQL Saturday How do we handle them???   DO NOT WRITE CODE, CHANGE INDEXES, OR FIX SETTINGS AS YOUR FIRST STEP Find the pattern. Understand what is happening. Capture values of the parameters over a period of time Example #3 Capture the performance over time. Correlate this to the parameters being used. Mornings? Afternoons? End of the month? Example #3a.xls Determine the net effect of a change!!!!! Execution time is all that really matters 15

Cleveland SQL Saturday How do we handle them???   From the analysis… Only certain people or locations use specific features. User training! Majority of the searches are on a single field, which comes from one location This was never in the original design, it evolved What the users really want is a Google-like search or an “incremental” search Almost HALF of the executions that occur on this screen can be replaced by a report. They pick a wide open search, and export it into a spreadsheet. 16

Cleveland SQL Saturday How do we handle them???   Actions This is being broken into 2 separate screens, 2 reports were also developed The Google-like search is being implemented with full text search A complete result set will be pulled into a “cache” and the heavy lifting will be done in code (Maybe!) Lots of user training! 17

Cleveland SQL Saturday How to handle them. Method #1   WHERE clause WHERE ISNULL(@parameter, field) = field Really a bad way to do this. NON-SARGABLE Likely a table scan Example #4 18

Cleveland SQL Saturday How to handle them. Method #2   Select it all into a temp table, delete what you don’t need. Works pretty well for a small result set Performs poorly with larger results or frequent executions IO, IO, it’s off to work we go! It just looks funny!!!! Example #5 19

Cleveland SQL Saturday How to handle them. Method #3   Create different procedures for the different combinations Good performance Bad for maintenance Example #6 20

Cleveland SQL Saturday How to handle them. Method #4   Use the data you collected, create different procedure(s) for the most used combinations Something else for the rest If you need to do this, you are going to wait Good performance Bad maintenance Example #6X 21

Cleveland SQL Saturday How to handle them. Method #5   Table Variables / AND EXISTS Useful in reporting scenarios Example #7 22

Cleveland SQL Saturday How to handle them. Method #6   For numeric types, set a number value that works for the different possibilities NULLS will not work Handle this in a subquery, depending upon the situation Can be a bit hard to understand Example #8 23

Cleveland SQL Saturday How to handle them. Method #7   MIN/MAX/BETWEEN NULLS will not work Handle this in a subquery, depending upon the situation Can be a bit hard to understand Example #8A 24

Cleveland SQL Saturday How to handle them. Method #8   MIN/MAX/BETWEEN/CHECKSUM NULLS will not work Handle this in a subquery, depending upon the situation Can be a bit hard to understand Example #8B 25

Cleveland SQL Saturday How to handle them  Method #8 Think outside the box and change the rules! Example #XXX 26

Cleveland SQL Saturday How to handle them. Method #9  Dynamic SQL Example #9 and 9a 27

Cleveland SQL Saturday How to handle them. Method #10  Whatever works best Example #11 28

Cleveland SQL Saturday Take Aways  Determine the pattern Gather the FACTS Make a decision Be prepared to change Be prepared to be challenged 29

Cleveland SQL Saturday Questions? Suggestions?  30