1. Optimizing database access and diving into .Net SqlClient
Mladen Prajdić
SQL Server MVP
@MladenPrajdic

2. About me
Welcome to Slovenia
The sunny side of Alps

3. Disclaimer I use SELECT * in demos only for brevity 

4. Agenda Building foundation... Client Data Access
New exciting stuff! Async SqlDataReader
Consistency is the key! TransactionScope
Reduce, Reuse, Recycle! Connection Pooling
Make it go faster! Batching Commands
What’s it doing? Statistics
Multiple interleaves? MARS

5. Correct Connection Use
using (SqlConnection conn =
new SqlConnection(_connString)) {
conn.Open();
// db access code here
// return data reader or objects }

6. Client Data Access Building the foundation...

7. TDS Data Flow Result set Result set Result set “Done” token “Done”
using (SqlCommand cmd =
new SqlCommand("SELECT * FROM Table1; SELECT * FROM Table2", conn)) {
SqlDataReader rdr =
cmd.ExecuteReader(CommandBehavior.CloseConnection);
// return the list of data objects
return CreateListOfBusinessObjects(rdr);
// or return the SqlDataReader itself
return rdr; }

8. TDS Data Flow Result set Result set Result set Row Data Row Data
“Done”
token
Result set
“Done”
token
Result set
“Done”
token
“Row”
token
Row Data
“Row”
token
Row Data
“Row”
token
Row Data

9. TDS Data Flow Result set Result set Result set Row Data Row Data
“Done”
token
Result set
“Done”
token
Result set
“Done”
token
“Row”
token
Row Data
“Row”
token
Row Data
“Row”
token
Row Data
Column
Header
Column Data
Column
Header
Column Data
Column
Header
Column Data
Small datatype INT
Large datatype NVARCHAR(MAX)
Medium datatype CHAR(50)

10. SqlDataReader Data Access
Default access (Non-Sequential)
Sequential access
// default = non-sequential access
var rdr = cmd.ExecuteReader();
// sequential access
var rdr = cmd.ExecuteReader(CommandBehavior.SequentialAccess);

11. Non-Sequential (Default) Access1 2
Column
Header
Column Data
Column
Header
Column Data
Column
Header
Column Data 1 2
Column
Header
Column Data
Column
Header
Column Data
Column
Header
Column Data
We can go back and read all previous columns’ data

12. Non-Sequential (Default) Access
The full row is stored in client memory
Move back and forward in the row
Possible scalability issues
Possible Out of Memory exceptions

13. Forward reading only. Trying to go back throws an error.
Sequential Access 1 2
Column
Header
Column Data
Column
Header
Column Data
Column
Header
Column Data 1 2
Column
Header
Column Data
Column
Header
Column Data
Column
Header
Column Data
ERROR
Forward reading only. Trying to go back throws an error.

14. Sequential Access Only the current value is stored in client memory
Move only forward in the row
Get* methods for getting data
GetBytes, GetChars for BLOBs
Save to a stream by reading it in chunks
// get first column value that is int
rdr.GetInt32(0);
// get second column value that is binary
byte[] buffer = new byte[100];
// loop this to get the whole column out in chunks
rdr.GetBytes(0, 100, buffer, 0, buffer.Length);
// save the byte buffer to file or something

15. Sequential Access Full streaming support in .Net 4.5
GetStream, GetTextReader, GetXmlReader
No more chunking

16. Sequential Access
SqlCommand readCmd = // get varbinary(MAX) from source table
new SqlCommand("SELECT BinData FROM Source",
readConn);
SqlCommand writeCmd = // insert varbinary(MAX) into target table
new SqlCommand("INSERT INTO Target(BinData) VALUES
writeConn);
// Size is set to -1 to indicate "MAX"
SqlParameter streamParameter =
SqlDbType.Binary, -1);
SqlDataReader readerSource =
readCmd.ExecuteReader(CommandBehavior.SequentialAccess);
while (readerSource.Read()) {
// Grab a stream to the binary data in the source database
Stream dataStream = readerSource.GetStream(0);
// Set the parameter value to the stream source that was opened
streamParameter.Value = dataStream;
// send the data to the target database
writeCmd.ExecuteNonQuery(); }

17. Async SqlDataReader New exciting stuff!

18. Old “Run in Background” Ways
SqlCommand cmd = new SqlCommand("SELECT * FROM Table", conn);
// either this or a background worker
IAsyncResult result = cmd.BeginExecuteReader();
// for the reader to complete
while (!result.IsCompleted) {
// do UI operations without problems }
using (SqlDataReader reader = cmd.EndExecuteReader(result))
// show reader results
ShowDbOperationResult(reader);

19. What exactly is the new stuff then?
Make C# async programming easier
Since .Net 4.5
Supports easy cancellation
Uses the new async/await keywords
Reader’s *Async methods
private async void CustomEventHandler(object sender, EventArgs e) {
textBox.Text = "Start long operation." + Environment.NewLine;
// await returns control to the caller
// this means that the UI won't be blocked
// since CPU is free to do its stuff
double duration = await MyLongIOBoundOperation();
// finish op and show duration in UI
textBox.Text = "Completed in " + duration + " sec"; }

20. What is it good for? Make code async without complex threading code
No threads created by default
Use default for IO-bound workload
Use Task.Run to create background thread for CPU- bound workload
Supported by Entity Framework 6
For more in depth details on this, search for “Asynchronous Programming with Async and Await”

21. What to use when? Prefer NextResultAsync in either access modes
TDS tokens between result sets are processed in async
Result set
TDS
Tokens
Result set
TDS
Tokens
Result set
Async
Async

22. What to use when? Prefer ReadAsync in either access modes
TDS tokens between multiple packets are processed in async
Single Row
Data Packet
TDS
Tokens
Data Packet
TDS
Tokens
Data Packet
Async
Async

23. What to use when? Use IsDBNull and GetFieldValue<T>
If ReadAsync() in Default (Non-Sequential) mode
Since data is already in the client buffer
IsDBNullAsync and GetFieldValueAsync<T>
Sequential mode
target column is large
needed to read past large columns
rdr.GetStream(i).CopyToAsync(outputStream)
for huge columns in Sequential mode

24. DEMO

25. TransactionScope Consistency is the key!

26. TransactionScope using (TransactionScope txScope =
new TransactionScope()) {
using (SqlConnection conn2012AW =
new SqlConnection(_connString2012AW))
// do stuff }
txScope.Complete();

27. TransactionScope Easy to use Keeps code simple
Great transaction management out of the box
MSTDC escalation
Always, unless connecting to the same database with non concurrent connections
sys.dm_tran_active_transactions.transaction_uow
Default isolation level is SERIALIZABLE

28. DEMO

29. Connection Pooling Reduce, Reuse, Recycle!

30. Connection Pooling Connection pools created
per process
per application domain
per connection string
per Windows identity (if integrated security) 
Default max pool size = 100
Audit Login, Audit Logout Events
EventSubClass
1 = Nonpooled
2 = Pooled

31. Connection Pooling Resets
sp_reset_connection
Resets the connection for reuse
A long list of what it does on the next slide for reference
Does NOT reset the isolation level
Always explicitly specify the isolation level

32. Connection Pooling Resets
All error states and numbers (like
Stops all EC's (execution contexts) that are child threads of a parent EC executing a parallel query
Waits for any outstanding I/O operations that is outstanding
Frees any held buffers on the server by the connection
Unlocks any buffer resources that are used by the connection
Releases all allocated memory owned by the connection
Clears any work or temporary tables that are created by the connection
Kills all global cursors owned by the connection
Closes any open SQL-XML handles that are open
Deletes any open SQL-XML related work tables
Closes all system tables
Closes all user tables
Drops all temporary objects
Aborts open transactions
Defects from a distributed transaction when enlisted
Decrements the reference count for users in current database which releases shared database locks
Frees acquired locks
Releases any acquired handles
Resets all SET options to the default values
Resets the value
Resets the value
Resets any session level trace options using dbcc traceon()

33. DEMO

34. Batching Commands Make it go faster!

35. Batching Commands Optimizing network round trips
Consider Bulk Insert if possible
Streaming data insert
SqlBulkCopy Class
10k inserts = 10k round trips
1000 x 10 batches = 10 round trips
1000 CSV SQL statements in one command
Or… a hidden gem 

36. DEMO

37. Statistics What’s it doing?

38. Client Statistics Enable per connection Cumulative
Good for tracking the actual db access from app
Save to a queue, process off-peak
Analyze historical access data

39. DEMO

40. MARS Multiple interleaves?

41. Multiple Active Result Sets (MARS)
Statement multiplexing/interleaving
SELECT, FETCH
RECEIVE, READTEXT
BULK INSERT / BCP
Other statements must complete
Transaction savepoints aren’t allowed
Usually used for SELECT N+1 scenarios

42. DEMO

43. ?