2. Bart J.F. De Smet bartde@microsoft.com http://blogs.bartdesmet.net/bart Software Development Engineer Microsoft Corporation Session Code: DTL304

3. The many faces of “dynamic” Source: www.coolopticalillusions.comwww.coolopticalillusions.com Generate code at runtime Changing types at runtime No types at all Simplified deployment

4. Dynamic programming languages In the broadest sense of the word Dynamic programming language is a term used broadly in computer science to describe a class of high-level programming languages that execute at runtime many common behaviors that other languages might perform during compilation, if at all. These behaviors could include extension of the program, by adding new code, by extending objects and definitions, or by modifying the type system, all during program execution. Source: Wikipedia Isn’t this what the CLR is all about?

5. The traditional view of the world Dynamic Languages Simple and succinctImplicitly typedMeta-programmingNo compilation Static Languages RobustPerformanceIntelligent toolsBetter scaling

6. Common Language = CLR + DLR Today’s view of the world Dynamic Languages Simple and succinctImplicitly typedMeta-programmingNo compilation Static Languages RobustPerformanceIntelligent toolsBetter scaling

7. Python binder Python binder Ruby binder COM binder JavaScript binder Object binder Dynamic Languages on.NET Dynamic Language Runtime Expression Trees Dynamic Dispatch Call Site Caching IronPython IronRuby C# VB.NET Others…

8. The advent of the DLR “Project 7” Reality check for the CLR Not the biggest success for dynamic languages Jim Hugunin Jython implementation Python on the JVM Surprising success story “Why the CLR is a terrible platform for dynamic languages” Not! www.codeplex.com/dlr

9. Standard Pystone Benchmark IronPython0.1 Python2.3 IronPython1.0Python2.5 Python2.1 46K 80K 90K 50K 36K Higher numbers are better

10. How to get the DLR Today: www.codeplex.com/dlrwww.codeplex.com/dlr, release 0.91 Microsoft Public License (MS-PL) Implementation of IronPython and IronRuby Lots of samples Tomorrow: DLR infrastructure integral part of.NET 4.0 System.Linq.Expressions, System.Dynamic Base for C# 4.0 “dynamic”

11. Implementation Running On.NET True language implementation True to the language True to the community True to the experience Excellent performance Great integration with.NET Easy to use.NET libraries Easy to use other.NET languages Easy to use in.NET hosts Easy to use with.NET tools IronWhat?

12. IronPython and IronRuby on.NET 4.0 Bart J.F. De Smet Software Development Engineer Microsoft Corporation

13. Identity crisis Python Objects C-Based Objects C-Based Objects py1 c2.NET Objects py1 c2 Different heaps CLR’s GC heap Common Language Runtime

14. .NET 4.0.NET 3.5 DLR2 LINQ DLR Common Language Runtime++ Language features Expression trees Expression trees v2 Expression trees v2 Call site caching Dynamic dispatch Dynamic dispatch Silverlight.NET 2.0 Generics Fast delegates Fast delegates Dynamic methods Dynamic methods Script hosting Compiler as a Service Compiler as a Service ???.NET 1.0 GC BCL Reflection JIT Code generation Verifier sandbox Verifier sandbox

15. C# 4.0 and dynamic Calculator calc = GetCalculator(); int sum = calc.Add(10, 20); Calculator calc = GetCalculator(); int sum = calc.Add(10, 20); object calc = GetCalculator(); Type calcType = calc.GetType(); object res = calcType.InvokeMember("Add", BindingFlags.InvokeMethod, null, new object[] { 10, 20 }); int sum = Convert.ToInt32(res); object calc = GetCalculator(); Type calcType = calc.GetType(); object res = calcType.InvokeMember("Add", BindingFlags.InvokeMethod, null, new object[] { 10, 20 }); int sum = Convert.ToInt32(res); ScriptObject calc = GetCalculator(); object res = calc.Invoke("Add", 10, 20); int sum = Convert.ToInt32(res); ScriptObject calc = GetCalculator(); object res = calc.Invoke("Add", 10, 20); int sum = Convert.ToInt32(res); dynamic calc = GetCalculator(); int sum = calc.Add(10, 20); dynamic calc = GetCalculator(); int sum = calc.Add(10, 20); Statically typed to be dynamic Dynamic method invocation Dynamic conversion

16. Python binder Python binder Ruby binder COM binder JavaScript binder Object binder Dynamic dispatch Dynamic Language Runtime Expression Trees Dynamic Dispatch Call Site Caching IronPython IronRuby C# VB.NET Others…

17. Multi-language dynamic dispatch Object binder Python binder Ruby binder COM binder Dynamic Language Runtime GetMember Name=“Foo”, IgnoreCase=false GetMember IronPython x.Foo IronPython x.Foo IronRuby x.Foo IronRuby x.Foo C# x.Foo C# x.Foo VB.NET x.Foo VB.NET x.Foo Others $x{Foo} Others $x{Foo} Your binder GetMember Name=“Foo”, IgnoreCase=true GetMember

18. ActionPythonRubyC#VB.NET GetMember x.Foo SetMember x.Foo = y DeleteMember del d.Foo x.send :remove_instance_variable :@foo No syntax UnaryOperation -x BinaryOperation x + y Convert No syntax (Foo)xCType(x,Foo) InvokeMember x.Foo(a,b) 2 x.Foo(a,b) Invoke x(a,b)x.call(a,b) 1 x(a,b) CreateInstance X(a,b)X.new(a,b)No syntax GetIndex x[a,b] x(a,b) SetIndex x[a,b] = y X(a,b) = y DeleteIndex del x[a,b]No syntax Common actions

19. DynamicObject public abstract class DynamicObject : IDynamicMetaObjectProvider { public DynamicMetaObject GetMetaObject(Expression parameter); public virtual bool TryGetMember(GetMemberBinder b, out object result); public virtual bool TrySetMember(SetMemberBinder b, object value); public virtual bool TryDeleteMember(DeleteMemberBinder b); public virtual bool TryUnaryOperation(UnaryOperationBinder b, out object result); public virtual bool TryBinaryOperation(BinaryOperationBinder b, object arg, out object result); public virtual bool TryConvert(ConvertBinder b, out object result); public virtual bool TryInvokeMember(InvokeMemberBinder b, object[] args, out object result); public virtual bool TryInvoke(InvokeBinder b, object[] args, out object result); public virtual bool TryCreateInstance(CreateInstanceBinder b, object[] args, out object result); public virtual bool TryGetIndex(GetIndexBinder b, object[] indices, out object result); public virtual bool TrySetIndex(SetIndexBinder b, object[] indices, object value); public virtual bool TryDeleteIndex(DeleteIndexBinder b, object[] indices); }

20. C# 4.0 dynamic and DynamicObject Bart J.F. De Smet Software Development Engineer Microsoft Corporation

21. Python binder Python binder Ruby binder COM binder JavaScript binder Object binder Expression trees Dynamic Language Runtime Expression Trees Dynamic Dispatch Call Site Caching IronPython IronRuby C# VB.NET Others…

22. Expression Trees revisited Func twiceD = x => x * 2; // Same as: // delegate (int x) { return x * 2; } int two = twiceD(1); Expression > twiceE = x => x * 2; // Same as: // var x = Expression.Parameter(typeof(int)); // var twiceE = Expression.Lambda >( // Expression.Multiply(x, Expression.Constant(2)), // x); Func twiceDfromE = twiceE.Compile(); int four = twiceDFromE(two); Plain old delegate Code as data (ET) Compiler as a Service

23. Expression Trees, take two System.Linq.Expressions v3.5 Assignment (simple, compound) Control flow (loops, if, return, exceptions) Dynamic dispatch (for operations from table) System.Linq.Expressions v4.0 Support for full method bodies public List GetPrimes(int to) { List res = new List (); for (int n = 2; n <= to; n++) { bool found = false; for (int d = 2; d <= Math.Sqr(n); d++) { if (n % d == 0) { found = true; break; } } if (!found) res.Add(n); } return res; } public List GetPrimes(int to) { List res = new List (); for (int n = 2; n <= to; n++) { bool found = false; for (int d = 2; d <= Math.Sqr(n); d++) { if (n % d == 0) { found = true; break; } } if (!found) res.Add(n); } return res; } public List GetPrimes(int to) { List res = new List (); for (int n = 2; n <= to; n++) { bool found = false; for (int d = 2; d <= Math.Sqr(n); d++) { if (n % d == 0) { found = true; break; } } if (!found) res.Add(n); } return res; } public List GetPrimes(int to) { List res = new List (); for (int n = 2; n <= to; n++) { bool found = false; for (int d = 2; d <= Math.Sqr(n); d++) { if (n % d == 0) { found = true; break; } } if (!found) res.Add(n); } return res; }

24. Dynamic is contagious System.Linq.Expressions v3.5 Assignment (simple, compound) Control flow (loops, if, return, exceptions) Dynamic dispatch (for operations from table) System.Linq.Expressions v4.0 public List GetPrimes(int to) { List res = new List (); for (dynamic n = 2; n <= to; n++) { bool found = false; for (int d = 2; d <= Math.Sqr(n); d++) { if (n % d == 0) { found = true; break; } } if (!found) res.Add(n); } return res; } public List GetPrimes(int to) { List res = new List (); for (dynamic n = 2; n <= to; n++) { bool found = false; for (int d = 2; d <= Math.Sqr(n); d++) { if (n % d == 0) { found = true; break; } } if (!found) res.Add(n); } return res; }

25. Factorial in C# Return If Parameter n Parameter n Constant 0 Constant 0 BinaryOp - BinaryOp - BinaryOp == BinaryOp == MethodCall Program.fact MethodCall Program.fact BinaryOp * BinaryOp * Constant 1 Constant 1 Parameter n Parameter n Parameter n Parameter n Constant 1 Constant 1 Return static int fact(int n) { if (n == 0) { return 1; } else { return n * fact(n - 1); }

26. Factorial in C# with dynamic Return If Parameter n Parameter n Constant 0 Constant 0 Dynamic CSharpOp[-] Dynamic CSharpOp[-] Dynamic CSharpOp[==] Dynamic CSharpOp[==] MethodCall Program.fact MethodCall Program.fact Dynamic CSharpOp[*] Dynamic CSharpOp[*] Constant 1 Constant 1 Parameter n Parameter n Parameter n Parameter n Constant 1 Constant 1 Return static dynamic fact(dynamic n) { if (n == 0) { return 1; } else { return n * fact(n - 1); }

27. def fact(n): if n == 0: return 1 else: return n * fact(n - 1) Factorial in Python Return If Parameter n Parameter n Constant 0 Constant 0 Dynamic PythonOp[-] Dynamic PythonOp[-] Dynamic PythonOp[==] Dynamic PythonOp[==] Dynamic PythonInvoke Dynamic PythonInvoke Dynamic PythonOp[*] Dynamic PythonOp[*] Constant 1 Constant 1 Parameter n Parameter n Parameter n Parameter n Constant 1 Constant 1 Return Property Value Property Value Field $global.fact Field $global.fact

28. def fact(n) if n == 0 1 else n * fact(n - 1) end Factorial in Ruby If Parameter n Parameter n Constant 0 Constant 0 MethodCall Ruby.IsTrue MethodCall Ruby.IsTrue Dynamic RubyOp[-] Dynamic RubyOp[-] Dynamic RubyOp[==] Dynamic RubyOp[==] Dynamic RubyCall[fact] Dynamic RubyCall[fact] Dynamic RubyOp[*] Dynamic RubyOp[*] Parameter self Parameter self Constant 1 Constant 1 Parameter n Parameter n Parameter n Parameter n Constant 1 Constant 1

29. The story with expression trees Shared for dynamically and statically typed code “dynamic” as a first-class type Islands of different typing disciplines Built on CLR 2.0 (and higher) dynamic methods A.k.a. System.Reflection.Emit Fully collectable by the GC Analyzable, transformable, composable Recall original use with LINQ New ExpressionVisitor class

30. Expression trees illustrated Bart J.F. De Smet Software Development Engineer Microsoft Corporation

31. Python binder Python binder Ruby binder COM binder JavaScript binder Object binder Call sites Dynamic Language Runtime Expression Trees Dynamic Dispatch Call Site Caching IronPython IronRuby C# VB.NET Others…

32. Old Idea: Polymorphic Inline Cache Implemented with delegates and generics No changes in CLR runtime engine (today) Major Addition: Multiple languages on CLR Interop for sharing objects across languages Customization to work for each language Customization for library writers System.Runtime.CompilerServices Call sites

33. CallSite static CallSite > _site = …; if (_site.Target(_site, x, 0)) { … } if (x == 0) { … } static bool _0(Site site, object x, int y) { return site.Update(site, x, y); //tailcall } As strongly typed as possible Cache is learning

34. CallSite static CallSite > _site = …; if (_site.Target(_site, x, 0)) { … } if (x == 0) { … } static bool _1(Site site, object x, int y) { if (x is int) { return (int)x == y; } else { return site.Update(site, x, y); //tailcall } }

35. CallSite static CallSite > _site = …; if (_site.Target(_site, x, 0)) { … } if (x == 0) { … } static bool _2(Site site, object x, int y) { if (x is int) { return (int)x == y; } else if (x is BigInteger) { return BigInteger.op_Equality((BigInteger)x, y); } else { return site.Update(site, x, y); //tailcall }

36. Update Target Test Site targets built from trees Parameter x Parameter x BinaryOp == BinaryOp == TypeIs int TypeIs int Cast int Cast int Parameter y Parameter y Parameter x Parameter x If Field Update Field Update MethodCall Invoke MethodCall Invoke Parameter site Parameter site Parameter x Parameter x Parameter y Parameter y

37. Update Target Test Update = simple tree transform Parameter x Parameter x BinaryOp == BinaryOp == TypeIs int TypeIs int Cast int Cast int Parameter y Parameter y Parameter x Parameter x If Field Update Field Update MethodCall Invoke MethodCall Invoke Parameter site Parameter site Parameter x Parameter x Parameter y Parameter y Parameter x Parameter x MethodCall Op_Equality MethodCall Op_Equality TypeIs BigInteger TypeIs BigInteger Cast BigInteger Cast BigInteger Parameter y Parameter y Parameter x Parameter x If

38. Call sites in a nutshell Bart J.F. De Smet Software Development Engineer Microsoft Corporation

39. Static objects are incredibly fast Test Parameter x Parameter x TypeIs List TypeIs List Target Property Count Property Count Cast List Cast List Parameter x Parameter x Test Parameter x Parameter x TypeIs DynamicObject TypeIs DynamicObject Target MethodCall GetMember MethodCall GetMember Cast DynamicObject Cast DynamicObject Constant GetMemberBinder(“Count”) Constant GetMemberBinder(“Count”) Parameter x Parameter x x.Count But DynamicObject is not

40. DynamicMetaObject The role of DynamicMetaObject Target Test Parameter x Parameter x BinaryOp == BinaryOp == TypeIs int TypeIs int Cast int Cast int Parameter y Parameter y Parameter x Parameter x public abstract class DynamicObject : IDynamicMetaObjectProvider { public DynamicMetaObject GetMetaObject(Expression parameter); Dynamically provides a value at runtime

41. DynamicMetaObject public class DynamicMetaObject { public BindingRestrictions Restrictions { get; } public Expression Expression { get; } public bool HasValue { get; } public object Value { get; } public Type RuntimeType { get; } public virtual DynamicMetaObject BindGetMember(GetMemberBinder b); public virtual DynamicMetaObject BindSetMember(SetMemberBinder b, DynamicMetaObject value); public virtual DynamicMetaObject BindDeleteMember(DeleteMemberBinder b); // Other bind operations… } Target Test Parameter x Parameter x Property Slot0 Property Slot0 TypeIs FastBag TypeIs FastBag Cast FastBag Cast FastBag Parameter x Parameter x

42. Implementing DynamicMetaObject public class Bag : IDynamicMetaObjectProvider { private Dictionary _data = new Dictionary (); public object GetMember(GetMemberBinder binder) { return _data[binder.Name]; } public void SetMember(SetMemberBinder binder, object value) { _data[binder.Name] = value; } public DynamicMetaObject GetMetaObject(Expression parameter) { return new BagMetaObject(this, parameter); } } Providing the underlying meta-object

43. Implementing DynamicMetaObject public class BagMetaObject : DynamicMetaObject { public BagMetaObject(Bag bag, Expression ex) : base(ex, BindingRestrictions.GetTypeRestriction(ex, typeof(Bag))) { } public override DynamicMetaObject BindGetMember(GetMemberBinder binder) { return new DynamicMetaObject( Expression.Call( Expression.Convert(this.Expression, typeof(Bag)), typeof(Bag).GetMethod("GetMember"), Expression.Constant(binder) ), BindingRestrictions.GetTypeRestriction( this.Expression, typeof(Bag) ) ); } } DynamicMetaObject Target Test Expression MethodCall GetMember MethodCall GetMember TypeIs Bag TypeIs Bag Cast Bag Cast Bag Constant binder Constant binder Expression

44. The sky is the limit Immutable objects Cache values If changes are rare Add invalidation logic Track shared layout Concept of “hidden classes” (Self, V8) Statically optimized code for loops Function mapping, numeric arrays, … Explore IronPython and IronRuby source code!

45. A simple dynamic meta-object Bart J.F. De Smet Software Development Engineer Microsoft Corporation

46. Python binder Python binder Ruby binder COM binder JavaScript binder Object binder Summary Dynamic Language Runtime Expression Trees Dynamic Dispatch Call Site Caching IronPython IronRuby C# VB.NET Others…

48. www.microsoft.com/teched Sessions On-Demand & Community http://microsoft.com/technet Resources for IT Professionals http://microsoft.com/msdn Resources for Developers www.microsoft.com/learning Microsoft Certification & Training Resources Resources

49. Related Content Breakout Sessions (session codes and titles) Interactive Theater Sessions (session codes and titles) Hands-on Labs (session codes and titles)

50. Track Resources Resource 1 Resource 2 Resource 3 Resource 4

51. Complete an evaluation on CommNet and enter to win! Required Slide

52. © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. Required Slide