1. Andrey Nikolov
CSI: Database

2. Sponsors
Gold sponsors:

3. About me Software developer at Kodar Ltd.
Almost 20 years of experience in developing database applications
Working with SQL Server since version 2000
MCSA: SQL Server 2012/2014
KODAR Ltd.
Systems for efficient software
solutions and optimizations
3 |

4. Case

5. Case
Incident Description: Stupid Jerks LTD is a small ISV startup, which not long ago developed their first commercial application – Unseen University Dashboard. It uses a small database to hold values of sensors and show these values in a dashboard (WPF application). On January 21st, 2016, their client (Innocent Guys LLC) installed in on a dedicated computer in their office, connected to a 65” flat screen monitor, where they can monitor in real time the data coming from the sensors. Recently few complaints were received, that the system drive of this computer becomes almost entirely full. Actions Taken: Innocent Guys LLC hired an IT consultant. She concluded that the Unseen University Dashboard is responsible for filling up the system drive. Her recommendation was to contact Stupid Jerks LTD for further investigation and clean up disk space on the system drive.
We need to find out:
What caused the system drive to be filled;
Why it happened;
Who is responsible for this incident;
What should be done to avoid future incidents like this one.

6. Database Recovery Models
It is not “Mode”, but “Model”  Possible values are:
Model
Require Log backups
Point-in-time recovery
Simple No
To the end of backup
Full
Yes
Bulk-logged

7. Write-ahead logging 1. Database is in consistent state Transaction Log
2. Update query comes in and a new transaction is started
Buffer Pool
(data pages in RAM)
3. Pages are loaded from disk
SQL query
4. Pages are modified in RAM
5. Changes are written to the log
6. Transaction commit is confirmed
Database file
Lazy Writer
7. Pages are written to disk

8. Backup and Restore Backup types Restore sequence to a point-in-time
Full
Differential
Transaction log
Restore sequence to a point-in-time
Latest full backup prior PiT
Latest (not all!) differential backup prior PiT
All transaction log backups to the PiT

9. Bulk-logged Recovery Model
Switching from Full to Bulk-logged does not break the log chain
Minimally logged operations reduces transaction log size, but increases log backup size
After minimally logged operations, switch back to Full model and take a log backup

10. Case

11. Case
Incident Description: After last incident, CTO of Innocent Guys LLC is aware that Unseen University Dashboard’s database is in Full recovery model. He studied thoroughly about the different recovery models and database backups. Using the newly acquired knowledge, he scheduled a weekly full database backups on 19:00 every Sunday, daily differential backups every evening at 22:00 and hourly transaction log backups. He even tried to make the application highly available, but CEO’s nephew brought his computer back home. Suddenly, On January 21st, 2017, the computer crashed and refused to boot. A quick look from CTO revealed that the system drive is completely full. Actions Taken: Innocent Guys LLC hired an IT consultant. She concluded that the Unseen University Dashboard is responsible for filling up the system drive. Her recommendation was to contact Stupid Jerks LTD for further investigation and clean up disk space on the system drive.

12. Case

13. Case
Incident Description: After freeing disk space on the system drive (again!), CTO is finally able to invest his time in improvements of the monitoring system. They really need these improvements, because the number of sensor updates is expected to grow exponentially in the next couple of months. So, he rolled up his sleeves and started the next phase of the project, which was quickly implemented by the talented developers from Complete Newbies LTD. Another lesson learned was that they need a reliable monitoring system to monitor the monitoring system. And soon enough it alerted him that the database grew too much. Actions Taken: Thanks to the monitoring system they noticed the problem before the system drive runs out of free space. He immediately contacted Complete Newbies LTD for support.

14. Service Broker
Messaging feature that receives, processes, and sends messages
Tightly integrated with the database engine (by using DML)
Messages can be sent to:
a queue in the same database as the sender
another database in the same SQL Server instance
to another SQL Server instance either on the same server or on a remote server
Messages are:
Asynchronous
Ordered
Reliable
Durable

15. Service Broker structure
Service Broker enabled Database
Service
Queue
Service program
Service Broker enabled Database
Service
Queue
Service program
Dialog
Message
Message
send
send
receive
receive

16. Advantages of Service Broker
Database integration enhances application performance and simplifies administration
Message ordering and coordination for simplified application development
Loose application coupling provides workload flexibility
Related message locking allows more than one instance of an application to process messages from the same queue without explicit synchronization
Automatic activation allows applications to scale with the message volume

17. Why Asynchronous, Queued Applications?
We have to size our servers to handle peak load. If the systems are connected with a queue, the peaks can be leveled by shifting some of the work to the slack times.

18. Case

19. Case
Incident Description: With the expected data size growth, the partnership between Innocent Guys LLC and Complete Newbies LTD reached a point, when the developer had full control over the production server. This allowed them to deliver updates as fast as possible and to help with the database maintenance. One of the first things done by the project manager was to link the database to a source control system. Soon he noticed database schema changes, which are not made by their developers. Actions Taken: The project manager arrived at the office of Innocent Guys LLC to investigate on site. He finds out that every couple of hours the foreign keys in the SensorValues table are changed and all developers swears they didn’t did this.

20. Important XE Terms Packages – top level containers of metadata
Events – points of interest inside the engine
Actions – perform a specified task when the event fires
Targets – the event destinations
Predicates – a filters that defines whether or not the event will fire

21. XE architecture Event Predicate Actions Targets Execution POI
If event is enabled, collect default payload data
Event
Execution
Evaluate the predicate
Collect additional data (if necessary)
Predicate
POI
Collect data for the actions
Actions
Dispatch the event data
Directly to synchronous targets
To intermediate memory buffers for asynchronous targets
Targets
Execution

22. Case

23. Case
Incident Description: After last incident, IT departments of Innocent Guys LLC and Complete Newbies LTD agreed, that it is not a good practice to use the production environment for development. The development was moved to a dedicated SQL Server instance. Soon after that, users started to complain of sporadic slowness. It appears at different times during the day – sometimes in the mornings, sometimes in the late afternoon, and occasionally multiple times in one day. Actions Taken: The project manager arrived at the office to investigate this issue on site. He checked the plan cache for expensive queries and checked the wait stats, but he didn’t manage to understand why the system is sometimes slow.

24. Wait Stats
SQL Server is permanently tracking why execution threads have to wait.
Can be obtained by querying sys.dm_os_wait_stats
Top waits could point you to the issue
To interpret them, look at SqlSkills SQL Server Wait Types Library:

25. Case

26. Case
Incident Description: Soon after the reason for the sporadic slowness was found, the project manager implemented a baseline. He keeps not only wait stats and general server state, but also stats for most important queries. When reviewing the baseline data for the last month, he noticed that queries for inserting sensor values in one of the tables executes significantly longer than similar queries, which inserts the same data in a different table. Actions Taken: The project manager executed these queries in SQL Server Management Studio and captured their actual execution plans. They were the same, but first query performed much more writes than the second one and he can’t figure out why.

27. Can you spot the bug?
bool CheckUniqueExistsOnTable(string tableName) { return * from sys.indexes i inner join sys.index_columns ic on i.index_id = ic.index_id and i.object_id = ic.object_id inner join sys.columns c on c.object_id = i.object_id and c.column_id = ic.column_id where i.is_unique_constraint = 1 and i.object_id = and c.name = N'Name'"); } void CreateUniqueOnTable(string tableName) var name = "uq_" + Guid.NewGuid().ToString(); ExecuteSql($"alter table {tableName} add constraint [{name}] unique(Name)"); void UpdateDatabase() if (!CheckUniqueExistsOnTable("dboTable_1")) CreateUniqueOnTable("dbo.Table_1"); if (!CheckUniqueExistsOnTable("dbo.Table_2")) CreateUniqueOnTable("dbo.Table_1"); // Copy-pasting is dangerous

28. Case
Instance does not start

29. When Instance Does Not Start
Common reasons:
Service account password incorrect or account locked or disabled
Corrupt or missing master database files
Corrupt or missing model database files
Unable to create tempDB
Unable to open the error log
Minimal configuration mode (-f option)

30. Q&A

31. Sponsors
Gold sponsors:

32. Thank you for not leaving the room till the end of my session!
Here is a smiley face for you: