1. © 2006 MetaCase 1 Agile Development with Domain-Specific Languages 19 th June 2006 Aali Alikoski, Microsoft Steven Kelly, MetaCase

2. © 2006 MetaCase 2 DSM with full code generation: the ultimate refactoring tool? 19 th June 2006 Aali Alikoski, Microsoft Steven Kelly, MetaCase

3. © 2006 MetaCase 3 Schedule Part I: What’s it all about? 14:00-14:15 Welcome & Introductions 14:15-14:45 Agile & DSM introduction & definitions 14:45-15:05 Software Factories & DSL Tools demo 15:05-15:15 MetaEdit+ demo Part II: OK, I can do that! 15:15-15:30 Interactive TV Applications Domain 15:30-16:00 Coffee 16:00-17:00 You build your first DSM Language! 17:00-17:20 Final topics, Conclusions & Wrap-up

4. © 2006 MetaCase 4 Contents Introduction What is domain-specific modeling (DSM) DSM examples & case studies Benefits of DSM How to implement domain-specific methods –Modeling language –Generators

5. © 2006 MetaCase 5 How has productivity improved? "The entire history of software engineering is that of the rise in levels of abstraction" Grady Booch New programming languages have not increased productivity UML and visualization of code have not increased productivity Abstraction of development can be raised above current level...... and still generate full production code (and ignore it!) *Software Productivity Research & Capers Jones, 2002

6. © 2006 MetaCase 6 Domain Idea Finished Product Solve problem in domain terms Assembler Map to code, implement UML Model Map to UML Generate, Add bodies Components Model in DSM language Generate calls to components No need to map! Code Map to code, implement Modeling domain vs. modeling code

7. © 2006 MetaCase 7 What is domain-specific modeling Captures domain knowledge (as opposed to code) –Raise abstraction from implementation world –Uses domain abstractions –Applies domain concepts and rules as modeling constructs –Narrow down the design space –Focus on single range of products Lets developers design products using domain terms  Apply familiar terminology  Solve the RIGHT problems  Solve problems only ONCE! – directly in models, not again by writing code, round-trip etc.

8. © 2006 MetaCase 8 Domain Idea Finished Product Example: Digital wristwatch Components Model in DSM language Generate calls to components No need to map! Solve problem in domain terms Assembler Map to code, implement UML Model Map to UML Generate, Add bodies Code Map to code, implement Product family –Models: His, Hers, Sport, Kid, Traveler, Diver… Reusable component applications –Time, Alarm, Timer, WorldTime, StopWatch… Hide complexity from modeler –Model-View-Controller separation –Separate thread for real-time display Implementation in Java

9. © 2006 MetaCase 9 Case1: Insurance products & eCommerce Developing portal for insurances and financial products Need to specify several hundred financial products Insurance experts specify visually insurance products and generate code to the portal Comparison to writing directly Java after first 30 products = DSM at least 3 times faster

10. © 2006 MetaCase 10

11. © 2006 MetaCase 11

12. © 2006 MetaCase 12 Telecom services and their configuration Users visually specify new configuration models Generate various configurations from single design –One model –Multiple outputs Reusable component library Code generators refers to external files Case2: Configuration of services

13. © 2006 MetaCase 13

14. © 2006 MetaCase 14

15. © 2006 MetaCase 15 Web application for e-commerce; product catalogs, events, press releases, and discussion forums Core components and basic functionality available for reuse and customization needs Each customer can specify own data content, behavioral logic and user interface Code generators produce running Java applets, stylesheets and xml files Generation of documents for both internal and external use Case3: Web application

16. © 2006 MetaCase 16

17. © 2006 MetaCase 17

18. © 2006 MetaCase 18 Case4: VoiceMenu for micro controller Voice VoiceMenu for micro controller based home automation system Remote control for lights, heating, alarms, etc. VoiceMenus are programmed straight to the device with assembler-like language (8bit) Modeling language to define overall menu structure and individual voice prompts Code generator produces 100% of menu implementation Development time for a feature from a week to a day!

19. © 2006 MetaCase 19

20. © 2006 MetaCase 20

21. © 2006 MetaCase 21 Case5: Enterprise apps in Smartphone Symbian/Series 60 for enterprise application development Platform provides basic services Modeling language to define application logic using basic widgets and services Code generator produces 100% of implementation Complete chain from model to running app Libraries possible to integrate

22. © 2006 MetaCase 22

23. © 2006 MetaCase 23

24. © 2006 MetaCase 24 Specify services than can run safely on Internet telephony servers Designs can be considered valid and well-formed already at the design stage Language use concepts familiar to the service developer – Switches, Locations and Signaling actions etc. Generate full service from the model There are also cases where the language has been extended to cover also domain extensions and new requirements e.g. for Java and VoiceXML. Case6: Call Processing Language

25. © 2006 MetaCase 25

26. © 2006 MetaCase 26

27. © 2006 MetaCase 27 Nokia; Mobile Phone product line Bell Labs / AT&T / Lucent; 5ESS telecommunications switch, Honeywell; embedded software architectures ORGA; SIM toolkit & JavaCard Pecunet; B2B E-Business: insurance LexiFi; mlFi, financial contracts DuPont; Activity Modeling NASA; Architecture Definition Language NASA ASE group; Amphion NASA JPL; embedded measurement systems USAF; Message Transformation and Validation … See more at www.DSMForum.org Let’s look at industry experience: Some reported cases

28. © 2006 MetaCase 28 DSM Case Study: Nokia * MetaCase, Nokia case study, 1999 DSM and related code generators for mobile phone* Order of magnitude productivity gains (10x) –"A module that was expected to take 2 weeks... took 1 day from the start of the design to the finished product" Focus on designs rather than code –Domain-oriented method allows developers to concentrate on the required functionality Training time was reduced significantly –“Earlier it took 6 months for a new worker to become productive. Now it takes 2 weeks”

29. © 2006 MetaCase 29 DSM Case Study: Nokia Tetra DSM for Tetra terminals* –Earlier development was based on SDL and manual coding Faster development –”MetaEdit+ has clearly boosted development speed due to its capabilities for supporting extensive reuse and managing variation” Quality of product code improved –”DSM rules out errors, eliminating them already in the design stage” Hiding complexity made development easier –”Developers with less C code experience can effectively develop features using DSM” * MetaCase, Nokia case study, 2003

30. © 2006 MetaCase 30 DSM Case Study: Lucent * D. Weiss et al, Software Product-Line Engineering, Addison-Wesley 5ESS Phone Switch and several DSMs * Reported productivity improvements of about 3-10 times –From several cases –From several DSMs Shorter intervals between product releases Improved consistency across product variants –“DSM should always be used if more than 3 variants”

31. © 2006 MetaCase 31 DSM Case Study: USAF Development of message translation and validation system (MTV)* Declarative domain-specific language + code generators and customization of components Compared DSM against component-based development: DSM is 3 times faster than code components DSM leads to fewer errors: about 50% less DSM gives “superior flexibility in handling a greater range of specifications” than components * Kieburtz et al., A Software Engineering Experiment in Software Component Generation, ICSE’96

32. © 2006 MetaCase 32 Why Domain-Specific Modeling? Development becomes faster –less to ”write” as abstraction is raised –no need to model and then code too Development becomes easier –lower level details hidden (cf. Assembler to C) –models easier to read, understand and remember Expertise can be leveraged –domain to code mappings made only once – by the expert developer Routine tasks minimized –Developers can focus on the problem, not its implementation details

33. © 2006 MetaCase 33 Why it is possible (now)? Need to fit only one company’s requirements! Modeling is Domain-Specific –Works for one application domain, framework, product family etc. –Language has concepts people already are familiar with –Models used to solve the problem, not to visualize code Generator is Domain-Specific –Generate just the code needed from models Efficient full code No manual coding afterwards No reason for round-tripping –Generator links to existing primitives/components/platform services etc. –Can produce Assembler, 3GL, object-oriented, XML, etc.

34. © 2006 MetaCase 34 Where to apply? Repetitive development tasks –Large portion of the work similar to earlier products (or several products made in parallel) Domain expertise needed –Non-programmers can participate These normally include: –Product Family –Platform-based development –Configuration –Business rule definitions –Embedded devices

35. © 2006 MetaCase 35 Domain Idea Finished Product Solve problem in domain terms Assembler Map to code, implement UML Model Map to UML Generate, Add bodies Code Map to code, implement Components Model in DSM language Generate calls to components No need to map! Modeling domain vs. modeling code

36. © 2006 MetaCase 36 Domain Idea Finished Product Components Model in DSM language Generate calls to components Easy! Expert (few) Normal (many ) Done a few times before! Code generator DSM language Domain framework How to implement DSM

37. © 2006 MetaCase 37 Tool support is essential Building DSM must be fast, cheap and easy A variety of tools available –Lex & Yacc for textual DSLs –Customizable IDE (e.g. Eclipse, Visual Studio) –MetaCASE tools (e.g. MetaEdit+) 5 ways to get the tools 1.Write own tool from scratch 2.Write own tool based on frameworks 3.Metamodel, generate tool skeleton, add code 4.Metamodel, generate full tool 5.Integrated modeling and metamodeling environment

38. © 2006 MetaCase 38 Schedule Part I: What’s it all about? 14:00-14:15 Welcome & Introductions 14:15-14:45 Agile & DSM introduction & definitions 14:45-15:15 Software Factories & DSL Tools demo 15:05-15:15 MetaEdit+ demo Part II: OK, I can do that! 15:15-15:30 Interactive TV Applications Domain 15:30-16:00 Coffee 16:00-17:00 You build your first DSM Language! 17:00-17:20 Final topics, Conclusions & Wrap-up

39. © 2006 MetaCase 39 Difference from MDA? Same idea on using models and transformations, but... DSM is always full code direct from models –Not OMG MDA (elaborationist) –Simpler in terms of versioning and management DSM = domain-specific language and generators –MDA is UML-based* No reverse- or round-trip engineering in DSM –We want a real lift in the level of abstraction –How often do you reverse engineer assembler to code? Separation of concerns –You are the experts in your domain and code (not the vendor) DSM is agile: as much or as little as you want * official definition, www.omg.org

40. © 2006 MetaCase 40 How to implement DSM Expert developer defines the DSM, others apply it –expert always defines the domain better than less- experienced developers –Always better to define the concepts and mappings once, rather than let everyone do it all the time Identify your current concepts and rules –Formalize them –No need to learn new semantics and notations –Models easier to read, remember and check Map to implementation framework –Generators: –Call component interfaces –Fill template code –Map to different target platforms

41. © 2006 MetaCase 41 Implementation covers Initial: –Metamodel: concepts and rules of the language –Notation: symbols and their behavior –Tool behavior: dialogs, icons, browsers etc. –Generators: for code, checking, inspection, docs etc. –Language help –Connectivity with other tools Continuously: –DSM language or tool sharing –DSM updates (of metamodel, rules, notation) –Generator updates –Model updates based on changed language –…often in multi developer settings

42. © 2006 MetaCase 42 Concepts Symbols Generators Rules 12 3 4 Steps for implementing DSM

43. © 2006 MetaCase 43 1. Design domain concepts Map modeling concepts accurately to domain concepts –What information is stored with each concept Concentrate on semantics! –Add extensions for software production later Design to make reuse precise and easy

44. © 2006 MetaCase 44 Concepts  Graph  Object  Property  Role  Relationship Rules, e.g.  Max 1  Min 1  Unique, etc Ports for connections Links between models Reuse rules Metamodeling with GOPRR

45. © 2006 MetaCase 45 2. Define domain rules Define semantics and rules as they exist in the domain Examples of rule types: –Links between concepts –Layering abstractions –Reuse rules –etc.

46. © 2006 MetaCase 46 3. Draw symbols (notation) Define symbols illustrating as well as possible the corresponding domain concept’s natural ”visualization” –e.g. end-user’s notation, customer’s notation

47. © 2006 MetaCase 47 4. Implement generators Reporting language fast and easy to use: no programming Cost of developing generators defrayed over a few users Write report definitions based on domain concepts: –Component use– Configuration data –Code generation– Testing and analysis –Automated build– Reports that make reports –Documentation– Review

48. © 2006 MetaCase 48 Apply in software production Develop applications using the DSM infrastructure –5-10x faster! Cf. NASA, Nokia, Lucent, USAF, Pecunet Continuously evolve your DSM –Domain & platforms evolve –Models update automatically

49. © 2006 MetaCase 49 Code generator principle Generator translates the models into a required output 1.Crawl through the models  navigation according to metamodel (modeling concepts and rules available) 2.Extract required information  access data in models 3.Translate it as the code  translation semantics and rules 4.Using some output format  possibility to define output format

50. © 2006 MetaCase 50 Python for smartphones Symbian/Series 60 application development Platform provides basic services Modeling language to define application logic using basic widgets and services Code generator produces 100% of implementation Complete chain from model to running app Libraries possible to integrate

51. © 2006 MetaCase 51 Generator definition Function calls – Series 60/Python def Note3_2227(): appuifw.note(u"Registration made", 'conf') return Stop3_983 def Note3_6109(): appuifw.note(u“Registration cancelled", ‘info') return Stop3_983 def Note3_2543(): appuifw.note(u“Conference registration: Welcome", ‘info') return Popup_menu3_2520 def Stop3_983(): # This applications stops here return appuifw.app.set_exit... Generator output Report '_Note' /* Produces Note code */ 'def '; type;oid; '():'; ‘ appuifw.note(u"';:Text or code; '", '''; :Note type; ''')'; subreport; '_next element'; run; endreport Report '_next element' /* reports next flow element*/ do ~From>Flow { do :Function used {''; :Function name;} '('; :Code; ')'; do ~To.() {‘ return '; subreport; '_Internal name'; run;} } endreport Generator definition

52. © 2006 MetaCase 52 C for wristwatch applications Typical for embedded devices –e.g. medical devices, diving instruments, car infotainment Modeling language to define application logic using basic domain concepts, using extended state machine Uses services from the domain-specific platform –e.g. in case of digital wristwatch: time, display, button etc Code generator produces 100% of variant implementation

53. © 2006 MetaCase 53 Switch-Case – Watch/C C from the extended state machine; Watch typedef enum { Start, EditHours, EditMinutes, Show, Stop } States; typedef enum { None, Mode, Set } Buttons; Generator ouput int state = Start; int button = None; /* pseudo-button for following buttonless transitions */ void runWatch() { while (state != Stop) { handleEvent(); button = getButton(); /* waits for and returns next button press */ } void handleEvent() { switch (state) { case EditHours: switch (button) { case Set: state = EditHours; increment_and_roll(tempOffset, HOURS, UP); break; case Mode: icon (editHours, OFF); icon (editMinutes, ON); state = EditMinutes; break; default: break; }

54. © 2006 MetaCase 54 How to design a generator, 1 Make generation process complete, target 100% generation output –Never modify the generated code want to change assembler after compiling? –Correct the generator or framework instead no round-trip-related problems do you want to edit Assembler and keep C in synch with it? Use modeling languages that provide required support for generation Use modeling languages to raise abstraction Don’t visualize code Generating a class from a diagram to a class in code helps very little, if not at all… Put domain rules up-front to the language Generator definition becomes easier when the input is correct Models should be impossible to draw wrongly for generation (difficult with universal languages, e.g. in UML OCL is written freely to model elements)

55. © 2006 MetaCase 55 How to design a generator, 2 Make generator for your situation only –Trying to make general purpose generator often fails Keep generator as simple as possible –Raise variation to the specification language –Push low-level common implementation issues to the framework Keep generator modular to reflect changes –e.g. structure generator based on modeling concepts –e.g. use common generator subroutines for common needs Try to generate as little code as possible –Glue code only Support code already made –Best to put to the framework

56. © 2006 MetaCase 56 Generator degrees of freedom Different levels of generators: modular / tree structure 1.Generator per file to be generated 2.Generator per section in a file 3.Generator per metamodel element Different Model of Computation implementations –Sequential –Function calls –Switch-case structure –Transition tables, etc. Different levels of code that generated code can call or subclass –Other generated code –Domain framework components –Platform functions Different generation options for different runs –Different top-level generators –Top-level graph for generation options

57. © 2006 MetaCase 57 Summary Domain-specific modeling radically improves productivity (5-10x) DSM leverages expert developers’ abilities to empower other developers in a team MetaCASE tools provide a cost-effective way to create DSM infrastructure Building DSM is great fun –The ultimate refactoring! More info: –www.dsmforum.org –www.metacase.com –www.microsoft.com

58. © 2006 MetaCase 58 For question and comments, please contact us MetaCase Ylistönmäentie 31 FI-40500 Jyväskylä, Finland Phone +358 14 4451 400, Fax +358 14 4451 405 www.metacase.com Thank you!