Peli de Halleux, Nikolai Tillmann Research in Software Engineering Microsoft Research, Redmond, USA

 1999… people packing groceries for the Y2k bug  How do you replace DateTime.Now ? if (DateTime.Now == new DateTime(2000,1,1)) throw Y2KBugException(); DateTime.Now = () => new DateTime(2000,1,1); DateTime.Now MDateTime.NowGet = () => new DateTime(2000,1,1); Moles

 Delegates naming convention  Lambda Expressions and Statements delegate void Action (T t); // void f(int i); delegate R Func (T t); // int f(string i); delegate void Action (T t); // void f(int i); delegate R Func (T t); // int f(string i); // C# 2.0 Func f = delegate(string s) {return 0; } // C# 2.0 Func f = delegate(string s) {return 0; } // C# 3.0 Func f = (s) => 0 Func f = (s) => { return 0; } Func f = _ => 0 // C# 3.0 Func f = (s) => 0 Func f = (s) => { return 0; } Func f = _ => 0

DEMO

Motivation

 A unit test is a small program with assertions  Tests a single (small) unit of code in isolation  Reality check: Real unit tests are not that simple! void ReadWrite() { var list = new List(); list.Add(3); Assert.AreEqual(1, list.Count); } void ReadWrite() { var list = new List(); list.Add(3); Assert.AreEqual(1, list.Count); }

 Components depend on other components  Hidden Integration Tests void FileExistsTest() { File.Write(“foo.txt”, “”); var result = IsFileEmpty(“foo.txt”) Assert.IsTrue(result); } void FileExistsTest() { File.Write(“foo.txt”, “”); var result = IsFileEmpty(“foo.txt”) Assert.IsTrue(result); } bool IsFileEmpty(string file) { var content = File.ReadAllText(file); return content.Length == 0; } bool IsFileEmpty(string file) { var content = File.ReadAllText(file); return content.Length == 0; } File.ReadAllText(file); File.Write(“foo.txt”, “”);

 Slow, complicated setup, non-deterministic tests  Solution: Replace by Simpler Environment (“mocking”)  Testable Design: Abstraction layer + Dependency Injection + Mocks for testing  Simply uses virtual methods  Hard-coded Design: No abstraction layer, static methods, sealed types.  Runtime rewriting needed

 Replace Any.NET method with A Delegate  Method can be overridden? Use Stubs  Interfaces, Abstract classes,  Virtual methods in non-sealed types  Method cannot be overridden? Use Moles  Static methods,  Sealed types,  Inline Constructor calls

 Introduce abstraction for external components  Replace them with something simpler, i.e. a Mock bool IsFileEmpty(IFileSystem fs, string file) { var content = fs.ReadAllText(file); return content.Length == 0; } bool IsFileEmpty(IFileSystem fs, string file) { var content = fs.ReadAllText(file); return content.Length == 0; } void FileExistsTest() { IFileSystem fs = ???; fs.Write(“foo.txt”, “”); var result = IsFileEmpty(fs,“foo.txt”) Assert.IsTrue(result); } void FileExistsTest() { IFileSystem fs = ???; fs.Write(“foo.txt”, “”); var result = IsFileEmpty(fs,“foo.txt”) Assert.IsTrue(result); } IFileSystem fs IFileSystem fs = ???; Mock, Stub, Double, Fake, …

 Replace Any.NET method with A Delegate var fs = new SIFileSystem() { ReadAllTextString = file => “”; }; var fs = new SIFileSystem() { ReadAllTextString = file => “”; }; file => “”; interface IFileSystem { string ReadAllText(string file); } class SIFileSystem : IFileSystem { Func ReadAllTextString; string IFileSystem.ReadAllText(string file) { return this.ReadAllTextString(file); }} // class SIFileSystem : IFileSystem { Func ReadAllTextString; string IFileSystem.ReadAllText(string file) { return this.ReadAllTextString(file); }} // Func ReadAllTextString; this.ReadAllTextString(file); string ReadAllText(string file);

DEMO

 Existing external components cannot be re-factored  SharePoint, Asp.NET, VSTO  Need mechanism to stub non-virtual methods  Static methods, methods in sealed types, constructors  MSIL code rewriting required  Other Tools provide this functionality

 Method redirected to user delegate, i.e. moled  Requires Code Instrumentation, e.g. via Profiler!  Pex provides [HostType(“Pex”)]  NUnit, xUnit, etc… also supported bool result = IsFileEmpty(“foo.txt”); Assert.IsTrue(result); bool result = IsFileEmpty(“foo.txt”); Assert.IsTrue(result); MFile.ReadAllTextString = file => “”;

File.ReadAllText(string name) { } File.ReadAllText(string name) { } mscorlib.dll File.ReadAllText(string name) { var d = GetDetour(); if (d != null) return d(); } File.ReadAllText(string name) { var d = GetDetour(); if (d != null) return d(); } push ecx push edx push eax push ecx push edx push eax.NET Runtime Just In Time Compiler.NET Runtime Just In Time Compiler ExtendedReflection

DEMO

 Lightweight Framework  Type Safe  Refactorable  Testable and “Hard-coded” Code  Overridable methods -> Stubs  Any other -> Moles  Delegate Based – use the language!

var list = new List(capacity); list.Add(item); var count = list.Count; var list = new List(capacity); list.Add(item); var count = list.Count; Assert.AreEqual(1, count); } Assert.AreEqual(1, count); }  A Unit Test has Three essential ingredients:  Data  Method Sequence  Assertions void Add() { int item = 3; int capacity = 4; void Add() { int item = 3; int capacity = 4; // for all item, capacity... void Add(int item, int capacity) { // for all item, capacity... void Add(int item, int capacity) { void List.Add(T item) { if (this.count >= this.Capacity) this.ResizeArray(); this.items[this.count++] = item; } void List.Add(T item) { if (this.count >= this.Capacity) this.ResizeArray(); this.items[this.count++] = item; } if (this.count >= this.Capacity) Capacity = 0 Test Case! Capacity = 0 Test Case!

 Automated White box Analysis does not ‘understand’ the environment  Isolate Code using Stubs and Moles if (DateTime.Now == new DateTime(2000,1,1)) throw new Y2KException(); if (DateTime.Now == new DateTime(2000,1,1)) throw new Y2KException(); DateTime.Now ??? Void Y2k(DateTime dt) { MDateTime.NowGet = () => dt... } Void Y2k(DateTime dt) { MDateTime.NowGet = () => dt... } MDateTime.NowGet = () => dt DateTime.Now == dt

DEMO

Future standalone download Future standalone download ExtendedReflection Runtime Code Instrumentation Source Code Generation ExtendedReflection Runtime Code Instrumentation Source Code Generation Moles Stubs Z3 Constraint Solver Z3 Constraint Solver Pex Test Generation Automated White box Analysis Pex Test Generation Automated White box Analysis

 Types  Methods  Properties Bar.IFoo -> Bar.Stubs.SIFoo void Foo(string v) -> FooString String Value {get;} -> ValueGet

 Types  Methods  Properties Bar.Foo -> Bar.Stubs.MFoo void Foo(string v) -> FooString string Value {get;} -> ValueGet

class Foo { static int StaticMethod() {…} int InstanceMethod() {…} } class Foo { static int StaticMethod() {…} int InstanceMethod() {…} } class MFoo : MoleBase { static Func StaticMethod { set; } Func InstanceMethod { set; } implicit operator Foo (MFoo foo); } class MFoo : MoleBase { static Func StaticMethod { set; } Func InstanceMethod { set; } implicit operator Foo (MFoo foo); }

 Compiler generates closures for us void Test(string content) { var fs = new SIFileSystem(); bool called = false; fs.ReadAllText = file => { called = true; return content; };... Assert.IsTrue(called); } void Test(string content) { var fs = new SIFileSystem(); bool called = false; fs.ReadAllText = file => { called = true; return content; };... Assert.IsTrue(called); } bool called = false; called = true; called

 For free with Object Initializers interface IBar { IFoo Foo {get;} } interface IFoo { string Value {get;} } interface IBar { IFoo Foo {get;} } interface IFoo { string Value {get;} } var bar = new SIBar { FooGet = () => new SIFoo { ValueGet = () => “hello” } }; var bar = new SIBar { FooGet = () => new SIFoo { ValueGet = () => “hello” } }; IBar bar = … if(bar.Foo.Value == “hello”)... IBar bar = … if(bar.Foo.Value == “hello”)... new SIBar().Foo.Value

 For free with Object Initializers class Bar { public Foo Foo {get;} } class Foo { public string Value {get;} } class Bar { public Foo Foo {get;} } class Foo { public string Value {get;} } MBar.Constructor = me => { new Mbar(me) => { FooGet = () => new MFoo { ValueGet = () => “hello” }}} MBar.Constructor = me => { new Mbar(me) => { FooGet = () => new MFoo { ValueGet = () => “hello” }}} if(new Bar().Foo.Value == “hello”)... new Bar().Foo.Value

 It just works! class Bar { public Foo Foo {get;} } interface IFoo {string Value {get;} } class Bar { public Foo Foo {get;} } interface IFoo {string Value {get;} } MBar.Constructor = (me) => { new Mbar(me) => { FooGet = () => new SIFoo { ValueGet = () => “hello” }}} MBar.Constructor = (me) => { new Mbar(me) => { FooGet = () => new SIFoo { ValueGet = () => “hello” }}} if(new Bar().Foo.Value == “hello”)... new Bar().Foo.Value

 Bind all methods of an interface at once class Foo : IEnumerable {...} int[] values = {1,2,3}; var foo = new MFoo().Bind(values); // bind all methods of // Ienumerable int[] values = {1,2,3}; var foo = new MFoo().Bind(values); // bind all methods of // Ienumerable

 Set a trap to flag any call to a type  Iteratively build the mole sequence MFoo.FallbackToNotImplemented();

 Dispatching moles per instance class Foo { public int Value {get;} } var foo = new MFoo { ValueGet = () => 1 }; var bar = new MFoo { ValueGet = () => 2 }; Assert.IsTrue(foo.Value != bar.Value); var foo = new MFoo { ValueGet = () => 1 }; var bar = new MFoo { ValueGet = () => 2 }; Assert.IsTrue(foo.Value != bar.Value);

 Attach Mole when Contructor is run class Foo { public Foo() {} public int Bar() {…} } class Foo { public Foo() {} public int Bar() {…} } MFoo.Constructor = me => { var foo = new MFoo(me) { Bar = () => 10 }; MFoo.Constructor = null; // only 1 instance }; MFoo.Constructor = _ => new MFoo(_) { Bar = () => 10 }; MFoo.Constructor = me => { var foo = new MFoo(me) { Bar = () => 10 }; MFoo.Constructor = null; // only 1 instance }; MFoo.Constructor = _ => new MFoo(_) { Bar = () => 10 };

 Stubs inherited from class may call base implementation abstract class FooBase { public virtual string Value {get;} } var foo = new SFooBase() { CallBase = true; } // call base class if no stub provided var value = foo.Value; var foo = new SFooBase() { CallBase = true; } // call base class if no stub provided var value = foo.Value; CallBase = true;

 Defines behavior when stub not provided  Default throws exception interface IFoo { string Value {get;} } StubFallbackBehavior.Current = StubFallbackBehavior.Default; var foo = new SIFoo(); var value = foo.Value; // returns null StubFallbackBehavior.Current = StubFallbackBehavior.Default; var foo = new SIFoo(); var value = foo.Value; // returns null

 Defines behavior when mole not provided  Default throws exception class Foo { string Value {get;} } var foo = new MFoo() { InstanceFallbackBehavior = MoleFallbackBehavior.Default }.Instance; var value = foo.Value; //returns null var foo = new MFoo() { InstanceFallbackBehavior = MoleFallbackBehavior.Default }.Instance; var value = foo.Value; //returns null

 Pex automatically detects and uses Stubs  Pex provides return values interface IFoo { string Value {get;} } [PexMethod] void Test(IFoo foo) { // pex uses SIFoo if (foo.Value == “foo”) throw...; // pex chooses value [PexMethod] void Test(IFoo foo) { // pex uses SIFoo if (foo.Value == “foo”) throw...; // pex chooses value