Database Unit Testing Team Edition for Database Professionals Sachin Rekhi Program Manager Microsoft Corporation

Agenda Team Data Overview Data Generation Database Unit Testing ● Overview ● Designer Experience ● Test Validation ● Test Execution ● Exploring the Generated Code ● Data-driven Testing ● Custom Verification Logic ● Transaction Support ● Negative Testing ● Custom Test Conditions ● Integration with Application Unit Tests

3 Incorporate the Database Professional into the software lifecycle and provide them with a foundation for change management and process integration. Change Management ● Project Based Development Project Model that represents schema as objects providing a “personal sandbox” for offline development that lives within a Visual Studio Solution Team Collaboration with Work Item and Process Integration with Team Foundation Server ● Automated Change Support Rename Refactoring with the ability to preview pending changes prior to execution Comparison Tools (Schema & Data Compare) allow comparisons & synchronization of schema and data with design/test/production databases Version Control of all database objects with the ability to reverse engineer a database to bring it under Source Control ● Database Unit Testing Database Unit Testing ensures iterative changes don’t break existing functionality Data Generator provides Repetitive Dataset Generation for tests based on saved settings Build / Deployment ● MSBuild Integration for Database Deployments/Builds based on Projects ● Either Create a new Database at the target location or Update an Existing Schema

4 Visual Studio Team System Visual Studio Team Suite MSF Process and Guidance Visual Studio Team Foundation Server Visual Studio Industry Partners Software Architects Software Developers Software Testers Database Professionals Visual Studio Team Explorer Application Modeling Infrastructure and Deployment Modeling Code Analysis Performance Tuning Security Analysis Database Build & Deployment Database Change Management Database Testing Performance Testing Manual Testing Test Case Management Visual Studio Professional Edition Change Management Work Item Tracking Reporting Project Site Integration Services Project Management Load Test Agent Visio and UML Modeling Class Modeling Unit Testing Code Coverage New!

5 Database Projects Overview Team Data

6 To create a solid foundation for testing we support data generation Deterministic – always generate the same layout Matched to your schema and very customizable Easily update generation plan to reflect schema changes Bottom Line: Realistic values with representative distributions Data Generation Overview

Data Generation Design Time Setting up Data Generation implies defining: ● Which generator to use ● Which distribution to attach to the generator ● Changing setting on the generator & distribution ● The numbers of rows to generate ● Optionally defining ratios between related tables By default: ● Each column is bound to the generator matching the column data type FK columns are mapped to the Foreign Key generator Uniqueness is inferred from PK, UC constraints and indexes Using the Uniform distribution when not unique

Data Generation Design Time Value generators Simple generators for each data type ● Strings: ASCII and Unicode ((var)char, n(var)char, (n)text) ● Numbers: tinyint, smallint, int, bigint, real, float, decimal, numeric, money ● Binary ((var)binary, image) ● Date and Time ● UniqueIdentifier (GUID) ● Bit Complex generators ● Foreign Key, Regular Expression, Data Bound Distributions ● Uniform, Normal, Inverse Normal, Exponential, Inverse Exponential

Data Generation Work

Data Generation Design Time Understand domain constraints ● Check constraints (min/max) Table cardinality ● Enforce table ratios ● Ex) Orders \ OrderDetails

Data Generation Executing a Data Generation Plan Validation of ● Security requirements Fails when security requirements are not met! ● Target schema against DGEN definitions Fails the generation when bindings do not match! Optionally purge tables ● Required to guarantee repeatable data generation Spin up parallel streams of INSERT statements ● Based on relationships between tables ● Number of connections used is currently gated by the schema relationships Configurable Error Thresholds

Data Generation Customization & Extensibility Customization of value generation ● Regular Expression Generator ● Data Bound Generator Extensibility ● Custom Generator ● Custom Distribution

13 Creating & Configuring a Generation Plan Data Generation

Ensures that database changes do not break existing codeEnsures that database changes do not break existing code Generate automated tests for programmability objectsGenerate automated tests for programmability objects Develop a battery of tests to run with every database buildDevelop a battery of tests to run with every database build Design & customize your tests in the language of your choiceDesign & customize your tests in the language of your choice ● T-SQL, VB.NET\C# Builds on existing Team Test unit testing functionalityBuilds on existing Team Test unit testing functionality Integrate your database tests along side your application unit testsIntegrate your database tests along side your application unit tests Database Unit Testing Overview

Develop your database unit test through an easy to use database test designer Test Script Generation ● Automatically generate test script stub for Stored Procedures, Functions, Triggers Test Script ● SQL to exercise the object under test Pre\Post Test Scripts ● SQL executed prior to and after test script to setup & cleanup test Test Initialize\Cleanup Scripts ● Common SQL executed prior to and after every test Database Unit Testing Designer Experience

Two primary methods of validating database unit tests ● T-SQL Assertions RAISERROR ● Test Conditions Easily configured UI-based client side assertions Supported types –Empty ResultSet –Not Empty ResultSet –Row Count –Scalar Value –Execution Time –Inconclusive Database Unit Testing Test Validation

Automatic Deployment Integration ● Automatically deploy database project prior to running tests Data Generation Integration ● Automatically generate data based on generation plan prior to running tests Execution & Validation connection strings ● Validation connection can be higher privileged account Database Unit Testing Test Execution

Ways to execute database tests ● From Visual Studio Test Menu \ Test View \ Test Manager \ Test Results F5 Experience ● Command-line Mstest.exe ● Team Build Execute tests post Team Build Database Unit Testing Test Execution

19 Creating a database unit test Database Unit Testing

Database unit test designer round-trips generated VB.NET\C# code ADO.NET code to ● Execute SQL against target ● Validate test conditions Exposed code enables variety of customization scenarios Database Unit Testing Exploring the Generated Code

Have the ability to parameterize inputs in test script Feed inputs from a database table Easily test a variety of inputs in a single test case Requires customization of VB.NET\C# code Database Unit Testing Data-driven Testing

Add custom verification logic in VB.NET\C# Exposed ExecutionResult object ● Exposes resultset data returned from test script ● Exposes execution time & rows affected info Take advantage of Classic Team Test Assertions Database Unit Testing Custom Verification Logic

Data Generation ● Used to ensure consistent database state prior to test run Transaction Support ● Ensures consistent state during test run ● Two Methods Transactions in T-SQL System.Transactions in VB.NET\C# Database Unit Testing Transaction Support

Expected error cases are as important to test as main success scenarios 3 methods ● SQL – Try\Catch ● VB.NET\C# - Try\Catch ● VB.NET\C# - Expected Exception Attribute Database Unit Testing Negative Testing

Create re-usable advanced verification logic Fully supports test condition extensibility Author a new test condition that derives from TestCondition base class in any.NET language Complete integrated UI experience Database Unit Testing Custom Test Conditions

Create VS solution containing both app & db projects Support for heterogeneous test lists Run application & database tests side-by-side Ensure database changes don’t break data access through app tests Database Unit Testing Integration with Application Tests

Database Unit Testing ensures quality in the data tierDatabase Unit Testing ensures quality in the data tier Data Generation to support repeatable data generation for testingData Generation to support repeatable data generation for testing First class unit testing framework for database developersFirst class unit testing framework for database developers Integrate your database tests along side your application testsIntegrate your database tests along side your application tests Database Unit Testing Wrap Up

28 Pricing, Licensing and Availability Included in Team Suite at No Extra Cost Purchase as an individual Edition ● Same pricing as other Team System Editions Availability ● CTP 7 Available Today ● RTM by the end of 2006

29 Resources Product Website & CTP7 Download ● fault.aspx fault.aspx Product MSDN Forums ● 25&SiteID=1 25&SiteID=1 Blogs ● Wealth of blog posts, videos, and web casts on database unit testing tips, tricks, and best practices ● ●

30