1. Building Robust and Reliable Software Jason Anderson, Microsoft jasand@microsoft.com

2. Agenda Team System Overview Static Analysis Profiling Unit Testing Test Driven Development Code Coverage Team Developer and Team Foundation Server Questions

3. Development Teams Infrastructure Architect Solution Architect Project Manager Developer Tester End User

4. Visual Studio Team System Process and Architecture Guidance Visual Studio Industry Partners Dynamic Code Analyzer Visual Studio Team Architect Static Code AnalyzerCode ProfilerUnit TestingCode CoverageVisio and UML ModelingTeam Foundation Client (includes CAL)Visual Studio Professional EditionClass DesignerLoad TestingManual TestingTest Case ManagementApplication DesignerLogical Infra. DesignerDeployment Designer Visual Studio Team Developer Visual Studio Team Test Dynamic Code AnalyzerStatic Code Analyzer Code Profiler Unit TestingCode Coverage Change ManagementWork Item TrackingReportingProject Site Visual Studio Team Foundation Integration ServicesProject ManagementBig Build

5. Static Analysis

6. Find errors in your code before you run or deploy it Checks range from style to code correctness to security issues Integrated into the Team System Build Environment

7. Static Analysis The dev process without Static Analysis Compile Code Binary PreprocGrammarLinkRaiseErrors

8. Static Analysis The dev process with Static Analysis Code Compile StaticAnalysis AnalyzeCodeCheckRulesRaiseErrors Binary

9. Static Analysis Managed Code Example static void Initialize(string id) { if (id == "") return; if (id == "") return; try try { Utilities.OpenUserAccount(id); Utilities.OpenUserAccount(id); } catch(Exception ex) catch(Exception ex) { // user must not exist // user must not exist Utilities.CleanAccountState(id); Utilities.CleanAccountState(id); Console.WriteLine(ex); Console.WriteLine(ex); }} Compiles Static Analysis gives warnings

10. Static Analysis Managed Code Example static void Initialize(string id) { if (id == "") return; if (id == "") return; try try { Utilities.OpenUserAccount(id); Utilities.OpenUserAccount(id); } catch(Exception ex) catch(Exception ex) { // user must not exist // user must not exist Utilities.CleanAccountState(id); Utilities.CleanAccountState(id); Console.WriteLine(ex); Console.WriteLine(ex); }} e:\Logger.cs(39): 'Logger.Initialize(System.String)#System.Void' Modify the following catch clauses in 'Logger.Initialize' by catching a more specific exception type or rethrowing the exception ' catch(Exception) {...}' e:\logger.cs(36): 'Logger.Initialize(System.String)#System.Void' Replace all calls to "".Equals or String.Empty.Equals in 'Logger.Initialize(System.String)' with checks for Length == 0.

11. Static Analysis Managed Code Example static void Initialize(string id) { if (id.Length == 0) if (id.Length == 0) { throw new ArgumentExeception(“Bad A…”); } try try { Utilities.OpenUserAccount(id); Utilities.OpenUserAccount(id); } catch(Exception ex) catch(Exception ex) { // user must not exist // user must not exist Utilities.CleanAccountState(id); Utilities.CleanAccountState(id); throw ex; throw ex; }} Improve my code

12. Static Analysis

13. Profiling

14. Profiling Bad performance is expensive ● Customer perception ● Hardware ● Investigation

15. Profiling Instrumentation main AMethod BMethod main 001 AMethod0023 BMethod0098 ….. 0145 Adds instructions to your code to monitor

16. Profiling Sampling main AMethod BMethod Analyze without changing your app main 3 AMethod 6 BMethod 9 2

17. Profiler Able to do both managed and unmanaged Able to do both instrumentation and sampling Used by many internal teams and on customer engagements

18. Profiling

19. Unit Testing

20. Unit tests are tests written by programmers to check the functionality of production code ● Always automated ● Generally one-to-one mapping between Unit Test and production code Typical problems: ● Too much overhead for everyone to learn ● Too tedious ● Isn’t effective enough to warrant the cost ● Not process agnostic

21. Writing a Unit Test made Easy Low overhead, very few moving parts We do provide simple helper APIs and easy to understand attributes We don’t force inheritance, interfaces, etc

22. Unit Test Code Generation Human error sometimes accounts for missed methods and classes Code Generation can develop the skeleton of your unit tests ● Namespaces ● Classes ● Methods Generated code fails with result ‘Inconclusive’

23. Unit Testing

24. Test Driven Development

25. Think about the code before you write it, have a failing test before you write product code The Test Case is a specification ● User Scenarios ● Business Cases ● Minimal bar, boundary conditions, performance, etc Infrastructure can be painful ● Build Errors ● Managed your failures

26. Test Driven Development

27. Code Coverage

28. Test Authoring isn’t about quantity, it’s about quality Code coverage helps you monitor your tests effectiveness ● Team Members can analyze results at a high- level ● Can also analyze source for specific missed methods and branches

29. Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

30. Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

31. Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

32. Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

33. Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

34. Code Coverage void PurchaseItem(int itemID) { if (itemID == 0) { throw new Exception(); } else { ProcessOrder(itemID); } UnitTest() { PurchaseItem(1); }

35. Code Coverage

36. Team Developer and Team Foundation Server

37. Visual Studio Team System Process and Architecture Guidance Visual Studio Industry Partners Dynamic Code Analyzer Visual Studio Team Architect Static Code AnalyzerCode ProfilerUnit TestingCode CoverageVisio and UML ModelingTeam Foundation Client (includes CAL)Visual Studio Professional EditionClass DesignerLoad TestingManual TestingTest Case ManagementApplication DesignerLogical Infra. DesignerDeployment Designer Visual Studio Team Developer Visual Studio Team Test Dynamic Code AnalyzerStatic Code Analyzer Code Profiler Unit TestingCode Coverage Change ManagementWork Item TrackingReportingProject Site Visual Studio Team Foundation Integration ServicesProject ManagementBig Build

38. Visual Studio Team System Process and Architecture Guidance Visual Studio Industry Partners Change ManagementWork Item TrackingReportingProject SiteIntegration ServicesProject ManagementDynamic Code Analyzer Visual Studio Team Architect Static Code AnalyzerCode ProfilerUnit TestingCode CoverageVisio and UML ModelingTeam Foundation Client (includes CAL)Visual Studio Professional EditionLoad TestingManual TestingTest Case ManagementApplication DesignerLogical Infra. DesignerDeployment Designer Visual Studio Team Developer Visual Studio Team Test Change ManagementWork Item Tracking Reporting Visual Studio Team Foundation Class Designer Big Build

39. Roles Addressed by Team System Project ManagerArchitectDeveloperTester Change ManagementWork Item TrackingReportingProject Portal Visual Studio Team Foundation Integration ServicesProject Management VisualStudioIDE

40. Work Item Integration Project Management can assign work items and track across lifecycle of the project Developers and Testers can assign defects and these integrate with PM work items Artifacts (tests, product modules, source code) can be associated with Work Items

42. Reviews Find / Fix Rates…

43. ..and Daily Trends

44. …and Bugs Against Code Churn and Testing Activity

45. Team Developer Static Analysis Profiling Unit Testing Code Coverage AND MORE!!! Integrated with TFS ● Source Code Control ● Project Management ● Bug Tracking ● Reports All fully functional from the VSTS 2005

46. © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.