Demystifying Domain Specific languages & Software Factories Govind Kanshi Architect Advisor Microsoft Corporation

Agenda Modeling and Methods Industrializing Software Domain Specific Languages A Software Factory Schema MDA and UML Wrap Up

Agenda Modeling and Methods Industrializing Software Domain Specific Languages A Software Factory Schema MDA and UML Wrap Up

Questions We Hear… What types of systems can I build? What’s the architecture of each type? How do I go from requirements to deployment? What artifacts do I need to build? How are they related? What are the key decisions that need to be made? Why are methodologies so abstract? Why can’t I get concrete guidance for my project? Why isn’t modeling more effective? Why can’t tools generate production quality code? Why don’t models stay synchronized with code? Why don’t models fit my file-oriented environment?

Agenda Modeling and Methods Industrializing Software Domain Specific Languages A Software Factory Schema MDA and UML Wrap Up

Software Development as Craftsmanship Labor Intensive Generic Tools Generic Processes One off applications Hand stitched from scratch Minimal reuse Overruns, defects, security holes, project failures

MassCustomizationCraftsmanship MassProductionContinuousImprovement Industrialization Product change Process change Stable Dynamic

Exploiting Commonality We already exploit economies of scale to automate production Stamping out many identical copies of a prototype Used to produce CDs/DVDs Does nothing to help development

Exploiting Commonality We can also exploit economies of scope Reuse designs & components Build many similar but distinct prototypes Key is supporting variability Define only the unique pieces of each system

Software Factories Domain-specific process Domain-specific tools & languages Domain-specific content Automate rote and menial tasks General-purpose IDEs become domain-specific software factories

What Is A Software Factory? “A set of integrated tool, process and content assets delivered to VS users in context to accelerate life cycle tasks for a specific type of software deliverable” “A set of integrated tool, process and content assets delivered to VS users in context to accelerate life cycle tasks for a specific type of software deliverable” Software Factories exploit innovations in Software product lines Service oriented architecture and orchestration Model driven development

Examples A set of assets for developing a User Interface Process, including: The PAG UIP block A DSL based UIP designer A recipe that adds a UIP project to an existing solution from a project template Help and documentation based process guidance for building User Interface Processes A set of assets spanning the entire life cycle for B2C extranet ecommerce systems for small to medium sized businesses, including Feature models of requirements A baseline architecture A set of assemblies containing prebuilt components for all major subsystems A tailored development process A configurable deployment topology A set of test cases and data sets A set of build scripts DSL based tools for development and deployment tasks Recipes that create the initial solution, and then build out project clusters for major components on demand

Agenda Modeling and Methods Industrializing Software Complexity in developing software Using Abstraction to Reduce Complexity A Software Factory Schema MDA and UML Wrap Up

Automate business processes Value chains with partners Customer and supplier self service Improve ROI of IT Lower operating costs Cut time to market New Business Requirements

New Application Architecture Automate business processes Improve ROI of IT Lower operating costs Cut time to market Value chains with partners Customer and supplier self service Service oriented architecture Web Services, messaging, contracts Security Caching and state management Deployment policies & constraints Heterogeneous platforms

Evolving Platform Technology Automate business processes Improve ROI of IT Lower operating costs Cut time to market Value chains with partners Customer and supplier self service Service oriented architecture Messaging and protocols Security Caching and state management Deployment policies & constraints Heterogeneous platforms Application Servers and standards Schemas and databases Code and attributes Configuration files Project structures Physical server configurations

Complexity Automate business processes Value chains with partners Customer and supplier self service Improve ROI of IT Lower operating costs Cut time to market Service oriented architecture Web Services, messaging, contracts Security Caching and state management Deployment policies & constraints Heterogeneous platforms Application servers & standards Schemas and databases Code and attributes Configuration files Project structures Physical server configurations ? ?

Where Does Complexity Arise? ? ? package com.microsoftpearl circle.auction.auc tionmanager.busine ss.auctionmanager; import java.rmi.RemoteExc eption; import java.util.Vector; package com.microsoftpearl circle.auction.auc tionmanager.busine ss.auctionmanager; import java.rmi.RemoteExc eption; import java.util.Vector; High Level Concepts «entity» Auction + categoryUid : String + creditCardAccount : String + creditCardExpiration : Long + creditCardName : String + creditCardType : String + endTime : Long + itemUid : String + minimumBidIncrement : Float + sellerUid : String + startingPrice : Float + startTime : Long + status : String + timezoneUid : String + uid : String + setValues ( ) + getValues ( ) «entity» Bid + amount : Float + auctionUid : String + buyerUid : String + cancelExplanation : String + date : Long + status : String + uid : String «entity» Seller «entity» User + address : String + city : String + country : String + String + groupId : String + password : String + secretNumber : Integer + state : String + status : String + uid : String + userId : String + zip : String «entity» Buyer «entity» Credit Card + accountNumber : String + cardType : String + expirationDate : Long + nameOnCard : String + uid : String + userUid : String + setValues ( ) + getValues ( ) * «derived» Low-level Implementation languages

Agenda Modeling and Methods Industrializing Software Complexity in developing software Using Abstraction to Reduce Complexity A Software Factory Schema MDA and UML Wrap Up

Abstraction Reduces Complexity Assembler, Pcode, CLR IL Bytecode or machine languages General Purpose Abstractions C#, Java, etc Domain Specific Abstractions

Frameworks Reduce Translation Complexity Assembler, Pcode, CLR IL Bytecode or machine languages General Purpose Abstractions C#, Java, etc Domain Specific Abstractions Frameworks E.g. ASP.Net Domain Specific Frameworks

Domain-specific language A language specific to a domain horizontal – e.g.: * GUI * GUI * web * web * business logic * business logic * database * database * deployment * deployment vertical – e.g.: * telephone bills * telephone bills * insurance * insurance * plane tickets * plane tickets * hardware construction * hardware construction PictorialflowchartsUML Textual C#, Java OCL

Domain Specific Modeling Language Domain Specific Abstractions Narrow Scope Graphical Highly Expressive Coarse-grained abstractions May be translated into General purpose programming languages Specific framework completion Other modeling languages

 C(x) h    C(x)  t 2m  x  ih = – Languages : Modelling and Programming TextualPictorial Descriptive Prescriptive class Magnox : NuclearPowerStation { public void dropRods () jo biscuits 24 green pat cakes 32 pink Employee name address promote Job description pay assign 0..* a>b && c==d call Call Record call length  base rate: £/s store

Telephone billing scheme call calendar month Call Record call length  base rate: £/s call length  friend discount rate: £/s store friends calls other calls - bill billing period store   + -

Understanding Web Services Activities Services Web services MessagesContractsSequencesSecurityReuse Wire-level protocols etc

Where Does Complexity Arise? ? ? package com.microsoftpearl circle.auction.auc tionmanager.busine ss.auctionmanager; import java.rmi.RemoteExc eption; import java.util.Vector; package com.microsoftpearl circle.auction.auc tionmanager.busine ss.auctionmanager; import java.rmi.RemoteExc eption; import java.util.Vector; High Level Concepts Activities Messages Security Reuse Services Contracts High Level Concepts Activities Messages Security Reuse Services Contracts «entity» Auction + categoryUid : String + creditCardAccount : String + creditCardExpiration : Long + creditCardName : String + creditCardType : String + endTime : Long + itemUid : String + minimumBidIncrement : Float + sellerUid : String + startingPrice : Float + startTime : Long + status : String + timezoneUid : String + uid : String + setValues ( ) + getValues ( ) «entity» Bid + amount : Float + auctionUid : String + buyerUid : String + cancelExplanation : String + date : Long + status : String + uid : String «entity» Seller «entity» User + address : String + city : String + country : String + String + groupId : String + password : String + secretNumber : Integer + state : String + status : String + uid : String + userId : String + zip : String «entity» Buyer «entity» Credit Card + accountNumber : String + cardType : String + expirationDate : Long + nameOnCard : String + uid : String + userUid : String + setValues ( ) + getValues ( ) * «derived» Low-level Implementation languages

.asmx files.asmx code behindssource code Statements and attributes XML Configuration Files and schemas (e.g. WSDL files) Project Files and templates Deployment Policies And Packages ? ? Scattered Concepts …

.asmx files.asmx code behindssource code Statements and attributes XML Configuration Files and schemas (e.g. WSDL files) Project Files and templates Deployment Policies And Packages …Replaced By A Holistic View Web Service Interaction DSL

Need A Designer Infrastructure For a specific aspect of overall development Define a metamodel for the abstractions Define a graphical notation Define a synchronization engine

Complexity ?? Automate business processes Value chains with partners Customer and supplier self service Improve ROI of IT Lower operating costs Cut time to market Service oriented architecture Web Services, messaging, contracts Security Caching and state management Deployment policies & constraints Heterogeneous platforms Application servers & standards Schemas and databases Code and attributes Configuration files Project structures Physical server configurations

Simplify Using DSLs Assess DesignImplement PlanRun Business IT StrategicObjectivesStrategicObjectives TechnologyArchitectureTechnologyArchitecture Business Processes and Entities Business Processes and Entities Logical Data Center Logical Physical servers and segments Services, Messages, Contracts, Schedules Services, Messages, Contracts, Schedules XML, Database, Classes, Code Code Manual Procedures Each work product can be a separate modeling language Mappings are defined between modeling languages

What’s Inside A Viewpoint? Assess DesignImplement PlanRun Business IT StrategicObjectivesStrategicObjectives TechnologyArchitectureTechnologyArchitecture Business Processes and Entities Business Processes and Entities Logical Data Center Logical Physical servers and segments Services, Messages, Contracts, Schedules Services, Messages, Contracts, Schedules XML, Database, Classes, Code Code Manual Procedures DSLsPatternsProcessesFrameworksComponentsAspectsTransformsConstraints

What’s Inside A Viewpoint? Assess DesignImplement PlanRun Business IT StrategicObjectivesStrategicObjectives TechnologyArchitectureTechnologyArchitecture Business Processes and Entities Business Processes and Entities Logical Data Center Logical Physical servers and segments Services, Messages, Contracts, Schedules Services, Messages, Contracts, Schedules XML, Database, Classes, Code Code Manual Procedures

Agenda Complexity in Connected Systems Using Abstraction to Reduce Complexity Domain Specific Modeling Languages and Patterns Scenario 1 - Web Services & Design for Operations Scenario 2 – Implementing Business Processes Microsoft Strategy and Tools for Visual Studio Whidbey

Web Service design & operations Assess DesignImplement PlanRun Business IT StrategicObjectives TechnologyArchitecture Business Processes and Entities Business Processes and Entities Logical Data Center Logical Physical servers and segments Services, Messages, Contracts, Schedules Services, Messages, Contracts, Schedules XML, Database, Classes, Code Code Manual Procedures

Dynamic Systems Initiative (DSI) Network Topology & Operating Systems Hardware System Definition Model Application Hosts Applications

Whidbey Solutions… A logical data center designer An application designer Application Hosts Applications A system hosting designer System Definition Model

A DSL In Use Today

Web Service Interaction Patterns Service Interface GatewayFaçade Layered Architecture AspectsTransactionsSecurityTransformations Validate against deployment infrastructure Transform into implementation artifacts

Logical System Architecture Logical System Architecture Patterns “Hardened IIS” – metabase settings MSA Patterns IDC, EDC Tiered Architecture Patterns Aspects Firewall Settings Protocol Settings Transformations Validate against Web service application design Transform into implementation SDM

Applying A Service Façade Pattern Policy Management Customer Management Customer Self-Service Service Facade Pattern Facade Messages Service Selected Messages Project Structures References Default XSD schemas Security Policy Access Policy Pass-thru Logic & Filters Customer Mgmt Facade

Aspect Weaving – Instrumentation Policy Management Customer Management Customer Self-Service Instrumentation Aspect Framework Call Structure Pointcut Code Aspect

Designing Business process Assess DesignImplement PlanRun Business IT StrategicObjectives TechnologyArchitecture Business Processes and Entities Business Processes and Entities Logical Data Center Logical Physical servers and segments Services, Messages, Contracts, Schedules Services, Messages, Contracts, Schedules XML, Database, Classes, Code Code Manual Procedures

Invoice To Payment

How Do We Describe BPs? Business Process DSL Activities Business Event Dependencies Business Entities DocumentsEtc.

Patterns And Transforms Process Patterns User Process Scheduled Process Case Process Rules Process Transformations Transform into Web service collaborations

Build One Business Process DSL? Assess DesignImplement PlanRun Business IT StrategicObjectivesStrategicObjectives TechnologyArchitectureTechnologyArchitecture Business Processes and Entities Business Processes and Entities Logical Data Center Logical Physical servers and segments Services, Messages, Contracts, Schedules Services, Messages, Contracts, Schedules XML, Database, Classes, Code Code Manual Procedures Huge Variation Few Platforms

Building Applications With DSLs Choose a specific aspect of overall development Define a metamodel for the abstractions OR extend an existing one Define a graphical notation Define a synchronization engine Define Patterns Define Aspects

Agenda Complexity in Connected Systems Using Abstraction to Reduce Complexity Domain Specific Modeling Languages and Patterns Scenario 1 - Web Services & Design for Operations Scenario 2 – Implementing Business Processes Microsoft Strategy and Tools for Visual Studio Whidbey

Making Use Of Metadata Metadata collected in DSLs can be made available across the life cycle Source control systems Debuggers Testing Tools Compilers and language editors

Enterprise Design Tools Build an initial set of model-driven development tools supporting DSLs Focused on specific development tasks With hi-fidelity, continuously synchronized mappings to code or other models Expand and integrate model data throughout life cycle tools Productize and promote underlying tool- building frameworks and tools Goal: Simplify design and deployment of SOA’s

DSL’s Supported in Whidbey Rich support for designing and constructing connected applications from Web services Designing WS for operational deployment Logical infrastructure requirements considered early – inform and constrain developers Verify application settings against logical infrastructure

Standards: UML and MDA UML has been very successful as a source of vocabulary and diagrammatic conventions for OO design UML specification is overly complex, difficult to understand and full of ambiguities and inconsistencies UML (as specified) is “one-size-fits-all” – difficult to extend and not a good foundation for domain specific languages UML does not map effectively onto CLR without significant alterations Properties, fields, indexers Type system CLR attributes

Standards: UML and MDA MDA refers specifically to model-driven development based on the use of OMG’s modeling technologies - UML and MOF Extending the reach of MDA to other areas is difficult MDA does not address broader issues involved in using integrated models, patterns, frameworks, and tools synergistically to support software product lines

Review Modeling languages can help reduce complexity Domain-specific abstractions Holistic view of scattered concepts Microsoft is working to improve Web service design and deployment scenarios Implementing DSLs can be made effective with the right underlying engine Tool Extensibility is very important

A Design DSL In Use Today

E.g., Issues in web service dev The concept of the web service is spread out Developer must keep files in sync Refactoring engines ignorant of web services Cannot process files like schemas and.config files Debugging, intellisense, editors all ignorant of web service concepts Source control, work items, tests all ignorant of Web Service concepts

.asmx files.asmx code behindSource code.config siles and schemas (e.g. WSDL files) Projects and templates Deployment policies and packages ? ? Scattered Concepts …

.asmx files.asmx code behindsource code.config files and schemas (e.g. WSDL files) Projects and templates Deployment policies and packages …a Holistic View Web Service Connectivity DSL

XML, Projects, Configs, Classes, Code Deployment Units packaged into Services, Messages, Applications, Endpoints Abstraction/refinement System Design

Web Service Connectivity DSL Services, Messages, Applications, Endpoints Services, Messages, Applications, Endpoints XML, Projects, DBs, Classes, Code XML, Projects, DBs, Classes, Code Physical servers & segments DeploymentUnitsDeploymentUnits Abstraction/ Refinement packaged into deployed on

Web Service Designer

IT Operations doesn’t understand application requirements Development IT Operations Developers don’t understand operations policy Plan/Design DevelopmentTestingDeploymentOperations

Design for Operations Services, Messages, Applications, Endpoints Services, Messages, Applications, Endpoints XML, Projects, DBs, Classes, Code XML, Projects, DBs, Classes, Code Logical Data Center Logical Physical servers & segments DeploymentUnitsDeploymentUnits Abstraction/ Refinement Abstraction/ Refinement Constraints packaged into deployed on

XML, Projects, Configs, Classes, Code Physical Servers and Network Segments Deployment Units packaged into deployed on Services, Messages, Applications, Endpoints Logical Data Center Host Software Abstraction/refinement Data Center Design

Providing Traceability Applications Application Hosting Logical Machines & Network Topology Hardware System Definition Model

Constraining design to deployment

Code Visualization Services, Messages, Applications, Endpoints Services, Messages, Applications, Endpoints XML, Projects, DBs, Classes, Code XML, Projects, DBs, Classes, Code Logical Data Center Logical Physical servers & segments DeploymentUnitsDeploymentUnits Abstraction/ Refinement Abstraction/ Refinement Constraints packaged into deployed on

Class designer

DBDefinitions XML, Projects, Configs, Classes, Code Physical Servers and Network Segments Deployment Units packaged into deployed on Services, Messages, Applications, Endpoints Logical Business Entities, Operations, Rules Implementable Business Processes and Entities User Interface Process Logical Data Center Host Software User Interface Navigation

DBDefinitions XML, Projects, Configs, Classes, Code Physical Servers and Network Segments Deployment Units packaged into deployed on Services, Messages, Applications, Endpoints LogicalBusiness Entities, Operations, Rules Biztalk Schedules & Rules Implementable Business Processes and Entities User Interface Process Business Processes and Entities Manual Processes BusinessCapabilities Logical Data Center Host Software Business Entities

DBDefinitions XML, Projects, Configs, Classes, Code Physical Servers and Network Segments Deployment Units packaged into deployed on Services, Messages, Applications, Endpoints Logical Business Entities, Operations, Rules Implementable Business Processes and Entities User Interface Process Business Processes and Entities BusinessCapabilities Logical Data Center Host Software Business Process Modeling

DBDefinitions XML, Projects, Configs, Classes, Code Physical Servers and Network Segments Deployment Units packaged into deployed on Services, Messages, Applications, Endpoints LogicalBusiness Entities, Operations, Rules Biztalk Schedules & Rules Implementable Business Processes and Entities User Interface Process Business Processes and Entities Manual Processes BusinessCapabilities Logical Data Center Host Software Business Process Implementation

Agenda Modeling and Methods Industrializing Software Domain Specific Languages A Software Factory Schema MDA and UML Wrap Up

DB Definitions XML, Projects, Configs, Classes, Code Physical Servers and Network Segments Deployment Units Services, Messages, Applications, Endpoints Logical Business Entities Rules, Operations BizTalk Schedules & Rules Implementable Business Processes and Entities User Interface Process Business Processes and Entities Manual Processes Business Capabilities Logical Data Center Host Software A graph of interrelated viewpoints Software Factory Schema

DB Definitions XML, Projects, Configs, Classes, Code Physical Servers and Network Segments Deployment Units Services, Messages, Applications, Endpoints Logical Business Entities Rules, Operations BizTalk Schedules & Rules Implementable Business Processes and Entities User Interface Process Business Processes and Entities Manual Processes Business Capabilities Logical Data Center Host Software A recipe for a family of applications Software Factory Schema ModelsPatternsBlocksFrameworksComponentsProcess Test Cases Tools

DB Definitions XML, Projects, Configs, Classes, Code Physical Servers and Network Segments Deployment Units Services, Messages, Protocols, Endpoints Logical Business Entities and Operations BizTalk Schedules & Rules Implementable Business Processes and Entities User Interface Process Business Processes and Entities Manual Processes Business Capabilities Logical Data Center Host Software A graph of interrelated viewpoints A Software Factory Schema

DB Definitions XML, Projects, Configs, Classes, Code Physical Servers and Network Segments Deployment Units Services, Messages, Protocols, Endpoints Logical Business Entities and Operations BizTalk Schedules & Rules Implementable Business Processes and Entities User Interface Process Manual Processes Logical Data Center Host Software A Software Factory Schema Supported in Visual Studio 2005 Team Architect Business Processes and Entities Business Capabilities

Agenda Modeling and Methods Industrializing Software Domain Specific Languages A Software Factory Schema MDA and UML Wrap Up

UML and DSLs Simple Questions How many diagrams exist in UML 2 How many do we end up using How do we use it (sketch,compiler,language) According to MF(Steve cook) 99% people use it as sketch Can one trace across one diagram to another with ease Can one model(diagram) auto synchronize with another Can model capture the patterns/tools/patterns for specific situation Can we use UML for GUI design, compilers, embedded systems

UML Is Great For Sketching… Set of popular and successful notations 5 of 13 UML 2 diagrams widely recognized Developed to unify competing notations Booch, Rumbaugh, Jacobson, and others Created for Object Oriented Analysis & Design Where models are a form of documentation Repository for modeling concepts Contains many useful common abstractions Basis for many domain modeling efforts From choreographies (RosettaNet) to Health Care (HL7)

…But Not For Development Not designed to support automation Addresses only a few domains well Class, interaction, use case, process, states Hard to extend to additional domains Stereotypes and tags do not carry behavior

DSL: UML and More Use generic UML for documentation Use DSLs for model driven development Reduce cost and effort of building DSLs Use UML notational styles for DSLs Transform UML profile definitions into DSLs Transform UML profile based models into corresponding DSL based models

What About MDA? A way to build applications in UML using A platform-independent UML model (PIM) Many platform-specific UML models (PSM) Its primary goal is platform independence Real needs are interoperability and productivity What doesn’t MDA address? How to make models precise enough to compile What to model for given type of system How to integrate models with requirements, architecture, frameworks, patterns and code How to fit models into the development process How to use models across the life cycle

MDA and More… DSLs are better languages for development Platform independence is nice, but… Interoperability, productivity and predictability also matter Other practical aspects of software development What types of systems can be developed What is the architecture of each type of system What to model for given type of system How to use models with frameworks, patterns, code How to use models in file oriented environments How to use models for requirements, deployment, testing, operations, maintenance

Agenda Modeling and Methods Industrializing Software Separating Concerns Domain Specific Languages A Software Factory Schema MDA and UML Wrap Up

Support For Building Factories Guidance for building Software Factories With Visual Studio 2005 Team System Integration with Visual Studio 2005 Methodology Templates, Dynamic Help Visual tools for building Software Factories DSL Designer, Feature Modeler, Schema Designer, Template Designer, Guidance Package Generator, Process Designer Possibilities we are considering

Support For Using Factories Guidance for using Software Factories Integration with MSF Agile and Formal Guidance tuned for specific types of systems Organized according to viewpoint Visual tools for using Software Factories DSLs, Guidance Packages, Configuration Editor, Template Installer Possibilities we are considering

Why Software Factories Consolidate implicit business and system development knowledge into specialized tools, process, and content Increase productivity and predictability by better organizing and automating the development process Reduce cost and risk by distributing the software life cycle across networks of interdependent groups and partners

Resources Books Software Factories by Jack Greenfield and Keith Short Websites p

Your Feedback is Important! Please Fill Out a Survey for This Session on CommNet

© 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.