Software Factories : Assembling Applications with Patterns, Models, Frameworks and Tools Steve Cook Architect Visual Studio

Agenda Software Industrialization Using DSLs to Reduce Complexity Scenario 1 - Web Services & Design for Operations Scenario 2 – Visualizing Code Scenario 3 – Implementing Business Processes Software Factories

The Software Crisis (ca. 2004) Development lags platform technology Globally distributed business processes constructed line by line $250B/yr in US (average $430K to $2.3M per project) 16% on time and budget but deliver less than planned (avg 42%) 53% overrun (avg 189%) 31% are canceled, losing $140B/yr Global demand will vastly exceed capacity this decade Apprenticeship will not scale to level of demand Other industries industrialized to solve similar problems

Industrialization Configure, adapt and assemble components to produce variants Standardize, integrate and automate production processes Develop and configure extensible tools for rote or menial tasks Markets Standards Processes Continuous improvement Simple & manageable “small- scale” interfaces Localization of complexity

Agenda Software Industrialization Using DSLs to Reduce Complexity Scenario 1 - Web Services & Design for Operations Scenario 2 – Visualizing Code Scenario 3 – Implementing Business Processes Software Factories

Raising the Level of Abstraction The history of programming is an exercise in hierarchical abstraction. In each generation, language designers produce constructs for lessons learned in the previous generation, and then architects use them to build more complex and powerful abstractions. (Smith and Stotts)

 C(x) h    C(x)  t 2m  x  ih = – Modelling and Programming TextualPictorial Declarative Procedural class Magnox : NuclearPowerStation { public void dropRods () jo biscuits 24 green pat cakes 32 pink Employee name address promote Job description pay assign 0..* a>b && c==d call Call Record call length  base rate: £/s store

Domain Specific Languages (DSLs) make problems “Small Scale” Small scale = Not much design beyond existing platform  easy to understand agile small team early prototypes easy maintenance changes inexpensive

Finding prime numbers is Small Scale … If we have a computer that can: do multiplication and division do things repeatedly _*_ _/_ _%_ while (…) {…} C, Pascal, Fortran,… Von Neumann computer with conventional CPU language: platform:

Finding parts lists is Small Scale … If we have a computer that can: keep large lists of parts & scan them quickly combine and filter the lists efficiently CREATE … JOIN … … WHERE … JOIN … … WHERE … SELECT … Relational Database SQL platform: language:

Developing a GUI is Small Scale … If we have a computer that can: display combinations of windows and widgets show controls and interact with user in them Visual GUI builder System.Windows.Forms platform: language:

if we have a language of phone billing and a platform to run it on Phone bill systems are Small Scale … Phone Billing Engine call calendar month Call Record call length  base rate: £/s call length  friend discount rate: £/s store friends calls other calls - bill billing period store   Telephone Billing Scheme

Anatomy of a DSL Notation (concrete syntax) May reuse predefined patterns eg box-line, channels, grids, maps, layouts Concepts (abstract syntax) May reuse predefined patterns eg states, flows, structures Well-formedness rules (grammar rules) Serialization format(s) Typically XML Interactive behaviour Drag/drop, copy/paste, select/edit, etc Mappings and Transformations

DSL & Framework Pattern CustomPartialClasses FrameworkPartialClasses FrameworkClasses Framework Config Files OtherFrameworkArtifacts uses uses completes FrameworkTools DSLEditor DSLDefinition generates OtherEditors edit / build generates

Agenda Software Industrialization Using DSLs to Reduce Complexity Scenario 1 - Web Services & Design for Operations Scenario 2 – Visualizing Code Scenario 3 – Implementing Business Processes Software Factories

.asmx files.asmx code behindssource code Statements and attributes XML Configuration Files and schemas (e.g. WSDL files) Project Files and templates Deployment Policies And Packages ? ? Scattered Concepts …

Issues Concept of Web service is “lost” among many files Developer must keep files in sync Refactoring engines ignorant of Web service concepts Can not process non-code files like schemas and config files Cannot undo refactoring changes Debugging, Intellisense, editors all ignorant of Web service concepts Source Control, Work Item, Tests all ignorant of Web Service concepts

A DSL In Use Today

.asmx files.asmx code behindssource code Statements and attributes XML Configuration Files and schemas (e.g. WSDL files) Project Files and templates Deployment Policies And Packages …A Holistic View Web Service Connectivity DSL

Services, Messages, Applications, Endpoints Services, Messages, Applications, Endpoints XML, Projects, DBs, Classes, Code XML, Projects, DBs, Classes, Code Physical servers & segments DeploymentUnitsDeploymentUnits Abstraction/ Refinement packaged into deployed on

IT Operations doesn’t understand application requirements Development IT Operations Developers don’t understand operations policy Plan/Design DevelopmentTestingDeploymentOperations

Design for Operations Services, Messages, Applications, Endpoints Services, Messages, Applications, Endpoints XML, Projects, DBs, Classes, Code XML, Projects, DBs, Classes, Code Logical Data Center Logical Physical servers & segments DeploymentUnitsDeploymentUnits Abstraction/ Refinement Abstraction/ Refinement Constraints packaged into deployed on

Agenda Software Industrialization Using DSLs to Reduce Complexity Scenario 1 - Web Services & Design for Operations Scenario 2 – Visualizing Code Scenario 3 – Implementing Business Processes Software Factories

Code Visualization Services, Messages, Applications, Endpoints Services, Messages, Applications, Endpoints XML, Projects, DBs, Classes, Code XML, Projects, DBs, Classes, Code Logical Data Center Logical Physical servers & segments DeploymentUnitsDeploymentUnits Abstraction/ Refinement Abstraction/ Refinement Constraints packaged into deployed on

Agenda Software Industrialization Using DSLs to Reduce Complexity Scenario 1 - Web Services & Design for Operations Scenario 2 – Visualizing Code Scenario 3 – Implementing Business Processes Software Factories

DSL-to-DSL Mappings Services, Messages, Applications, Endpoints Services, Messages, Applications, Endpoints XML, Projects, DBs, Classes, Code XML, Projects, DBs, Classes, Code Logical Data Center Logical Physical servers & segments DeploymentUnitsDeploymentUnits Abstraction/ Refinement Abstraction/ Refinement Constraints packaged into deployed on Business Processes and Entities Business Processes and Entities Abstraction/ Refinement Reconciliation

Invoice To Payment

Agenda Software Industrialization Using DSLs to Reduce Complexity Scenario 1 - Web Services & Design for Operations Scenario 2 – Visualizing Code Scenario 3 – Implementing Business Processes Software Factories

A Set of Interrelated Models BusinessCapabilitiesBusinessCapabilities ManualProceduresManualProcedures TechnologyArchitectureTechnologyArchitecture Constraints Reconciliation Services, Messages, Applications, Endpoints Services, Messages, Applications, Endpoints XML, Projects, DBs, Classes, Code XML, Projects, DBs, Classes, Code Logical Data Center Logical Physical servers & segments DeploymentUnitsDeploymentUnits Abstraction/Refinement Constraints packaged into deployed on Business Processes and Entities Business Processes and Entities Reconciliation Abstraction/Refinement

… and related artifacts BusinessCapabilitiesBusinessCapabilities ManualProceduresManualProcedures TechnologyArchitectureTechnologyArchitecture Constraints Reconciliation Services, Messages, Applications, Endpoints Services, Messages, Applications, Endpoints XML, Projects, DBs, Classes, Code XML, Projects, DBs, Classes, Code Logical Data Center Logical Physical servers & segments DeploymentUnitsDeploymentUnits Abstraction/Refinement Constraints packaged into deployed on Business Processes and Entities Business Processes and Entities Reconciliation Abstraction/Refinement DSLsPatternsProcessesFrameworksComponentsTransformsConstraints Test Cases etc.

…defines a Software Factory This is called a software factory schema Like a recipe for a specific type of application A set of viewpoints related by mappings that support transformation, validation, traceability Lists artifacts required to build application type and explains how to combine them A software factory template is content Configures a development environment for application type Projects, patterns, frameworks, guidance Configured development environment is software factory Integrates tools, process and content for that type of application Domain specific editing, rendering, compilation, debugging, refactoring

A Software Factory Defines A Domain-Specific Methodology Domain specific guidance for application type Integrates tools, content and process Associates process fragments with viewpoints Fragments are specific to their associated viewpoints Work flow is non-deterministic but constrained What to model to build an application How to capture and analyze requirements, and map them to design and implementation Identify types of models involved and show how they relate to each other and to other source artifacts How models fit into the development process How to version them, debug them, generate tests from them

Building A Factory Build software factory schema Define target application type and constituent viewpoints Viewpoints layer by major life cycle artifact Requirements, Architecture, Implementation, Deployment, Testing, Management, Maintenance Define life cycle process for target application type Process fragments plus constraints on viewpoint transitions Build software factory template Delivered as nesting parameterized install packages Refactor software factory as applications are built Based on new requirements and user feedback

Using A Factory Define application to be developed Only unique requirements – common requirements assumed Specifying application customizes schema and template Adds, removes or changes viewpoints Customizes tool configurations and predefined content Customizes process guidance Use customized factory to develop application artifacts Features outside factory scope custom developed Refactor application as it evolves Capture changes in requirements in factory configuration Factory configuration defines delivered application Simplifies maintenance and enhancement Makes impact of changes easier to understand Changes propagate through factory

The End Software Industrialization Using DSLs to Reduce Complexity Scenario 1 - Web Services & Design for Operations Scenario 2 – Visualizing Code Scenario 3 – Implementing Business Processes Software Factories

© 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.