1. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 The Little-JIL Process Definition Language Leon J. Osterweil (ljo@cs.umass.edu) Lab. For Advanced SE Research (LASER) http://laser.cs.umass.edu University of Massachusetts Amherst, MA USA USC Center for Software and Systems Engineering 17 March 2008

2. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Good Communication Effective Coordination Greater Efficiency Effective use of Automation Better Education and Training Continuous Improvement Reinvention thru Understanding Different People “Do Process” in Different Domains for Different Reasons

3. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Good Communication Effective Coordination Greater Efficiency Effective use of Automation Better Education and Training Continuous Improvement Reinvention thru Understanding Different People “Do Process” in Different Domains for Different Reasons Our main interests

4. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Yields two complementary sets of issues Macro view –What does it do/how does it behave Micro view –How does it effect its behavior Each benefits the other

5. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Process as Object Process Outputs Artifacts Effects Resources: People Money Tools Time Input Artifacts Other Behaviors Cost Time Errors

6. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Macro Focus Process Outputs Artifacts Effects Resources: People Money Tools Time Input Artifacts Other Behaviors Cost Time Errors

7. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Micro-Process Focus Process Outputs Artifacts Effects Resources: People Money Tools Time Input Artifacts Other Behaviors Cost Time Errors

8. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Each Benefits the Other Process Outputs Artifacts Effects Resources: People Money Tools Time Input Artifacts Other Behaviors Cost Time Errors

9. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Bridging Micro- and Macro- Explain how behaviors are produced Suggest changes, predict their effects Validate changes before they are made Each has interests in all of these Each knows it needs the other’s approach

10. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Analogy to other disciplines Economics Physics –Thermodynamics –Electricity Medicine/biology

11. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 What we learn from the analogies Macro- approach comes first Limited success in engineering Micro- approach/theory follows Facilitates more effective engineering –Improved predictability –Reduced uncertainty –Fewer surprises

12. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Overview of Our Micro Approach Processes are Software Engineer them the way we engineer software –Define them in a rigorously defined language –Reason about them using software analysis –Execute them –Employ continuous process improvement Starts with need for language(s) –To materialize processes –To provide execution semantics –Make them amenable to analysis and improvement

13. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Language Desiderata Precise Detailed Broad in scope Clear

14. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Language Desiderata Precise Detailed Broad in scope Clear Diagrams and pictures won’t do

15. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Some Key Process definition issues Blending proactive and reactive control Coordinating human and automated agents –Without favoring either Dealing with exceptions Specification of resources Real time specification Assignment of agents Scaling Reuse Novel transaction notions –Long, nested

16. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 The Little-JIL Process Language Vehicle for exploring language abstractions for –Reasoning (rigorously defined) –Automation (execution semantics) –Understandability (visual) Supported by –Visual-JIL graphical editor –Juliette interpreter Evaluation by application to broad domains A third-generation process language A “work in progress”

17. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Four parts to a Little-JIL Process Coordination diagram Artifact space Resource repository Agents

18. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Little-JIL Language Architecture Coordination Resources Actors Artifacts Activities Agents Steps

19. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Hierarchy, Scoping, and Abstraction in Little-JIL Process definition is a hierarchical decomposition Think of steps as procedure invocations –They define scopes –Copy and restore argument semantics Encourages use of abstraction –Eg. process fragment reuse

20. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Hierarchy, Scoping, and Abstraction in Little-JIL Process definition is a hierarchical decomposition Think of steps as procedure invocations –They define scopes –Copy and restore argument semantics Encourages use of abstraction –Eg. process fragment reuse These have all been very successful

21. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 The “Step” is the central Little-JIL abstraction TheStepName Interface Badge (parameters, resources, agent) Prerequisite Badge Postrequisite Badge Substep sequencing Handlers X Artifact flows Exception type continuation

22. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 An Example: Open Cry Auction

23. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Iteration usually through recursion Alternation using pre/post requisites Proactive Flow Specified by four Sequencing Kinds Sequential –In order, left to right Parallel –Any order (or parallel) Choice –Choose from Agenda –Only one choice allowed Try –In order, left to right

24. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Pre- and Post-requisites Steps guarded by (optional) pre- and post- requisites Are steps themselves Can throw exceptions May be executed by different agents –From each other –From the main step

25. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Exception Handing: A Strong Feature of Little-JIL Steps may have one or more exception handlers Handlers are steps themselves –With parameter flow React to exceptions thrown in descendent steps –By Pre- or Post-requisites –Or by Agents DevelopInterfaceFiles InterfaceFilesCompile InterfaceFiles Don’tCompile

26. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Four different continuations on exception handlers Complete –Handler was a “fixup”; substep is completed Continue –Handler cleaned up; parent step is completed Restart –Handler cleaned up; repeat substep (deprecated) Rethrow –Rethrow to parent step

27. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Artifact flow Primarily along parent-child edges –As procedure invocation parameters –Passed to exception handlers too –Often omitted from coordination diagrams to reduce visual clutter This has been shown to be inadequate –Artifacts also need to flow laterally –And subtasks need to communicate with each other

28. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Data Channels and Lateral flow Data Channel is like a queue in some ways Can specify step(s) that can add artifacts And steps that can take them All artifacts must be of the same type Queue semantics are only ones allowed Generalizations of these are needed

29. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Resources Entities needed in order to perform step Step specifies resource needed as a type –Perhaps with attributes, qualifiers Resource instances bound at runtime Exception when “resource unavailable”

30. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Resources Entities needed in order to perform step Step specifies resource needed as a type –Perhaps with attributes, qualifiers Resource instances bound at runtime Exception when “resource unavailable” Much research is needed here

31. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Examples of Resources Access to artifacts: shared document, locks on databases People: various kinds with varying skills Tools: compilers, CASE tools Agents: Each step has a distinctly identified unique resource responsible for execution of the step (and all of its substeps) May be complex relations among them

32. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Resource Request Example IdentifyRelationships SpecifyRelationshipsRefineRelationships Agent: OODDesigner;expert tool: ClassDiagramEditor artifact: DiagramReposLock Resource request is a query on the Resource specification repository

33. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Agents Collection of all entities that can perform a step –Human or automated Process definition is orthogonal to assignments of agents to steps –Path to automation of process Have freedom to execute leaf steps in any way they want

34. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Try and Step Kinds support human (agent) flexibility Implement Look_for_Inheritance Look_for_Objects_to_Delegate_to Look_for_Parameterized_Class Custom_Implementation Reuse_Implementation

35. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 An Articulate Process Can Help Answer These Questions Requirements Low-Level Design Code Test High-Level Design Where does output go? What causes this rework? What portion of activity should be done? How do we break this cycle? What to do when reviews fail?

36. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 High-Level Process

37. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Trivial Example Elaboration of Requirements Step

38. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Trivial Example Elaboration of Design Step

39. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Requirements Rework

40. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Requirements Rework Invocation of step originally defined as substep of Requirements

41. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Requirements Rework Invocation of step originally defined as substep of Requirements Same exception thrown

42. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Requirements Rework Invocation of step originally defined as substep of Requirements Same exception thrown Different invocation context -> different response

43. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008

44. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Timing Step has (optional) deadline specification Exception when deadline exceeded Parent can proceed –Child may be unaware of this

45. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Timing Step has (optional) deadline specification Exception when deadline exceeded Parent can proceed –Child may be unaware of this Much research needed here: A major oversight in most process languages

46. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Transaction Semantics Needed There is a need to specify atomic tasks And to deal with the need for –Rollback –Compensation Nested Transactions Long Transactions

47. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Verification of Four Key Characteristics Precise –From FSA definitions of step semantics Detailed –From hierarchical elaboration capability Broad in Scope –Exceptions, abstraction, timing, resources, concurrency, channels, artifact flow, etc. Clear –Visual –Use of abstraction

48. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Using Little-JIL to drive User Interfaces Process execution used to control multiple graphical user interfaces (GUIs) Involves communication between Little-JIL and user’s screen Communication done through a “Dispatcher” capable of interfacing to different types of GUI platforms. Dispatcher controlled by a binding script Script is capable of describing multiple types of GUI platforms (desktop or web) with a high-level syntax

49. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Architecture Dispatcher User Interfaces Executing Little-JIL Process screen changes user interactions process directives process updates Binding Script

50. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 High Level Design

51. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Issues from Barry

52. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Process V&V Rigorous Little-JIL semantics support definitive analyses Automatic generation of flowgraph Finite state verification for process defects –Propel used to define properties –FLAVERS used for analysis Fault tree generated from Little-JIL –FTA and FMEA analysis of Fault Tree Automatic generation of discrete event simulations Applied in medical process analyses, election process analyses

53. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Ambiguity Tolerance Little-JIL program has four “factors” –Activities –Agents –Resources –Artifacts Each can be developed at a different rate Leaf steps can be left unelaborated Resources are left as types, not instances –Late-bound at run time Some analyses of incomplete definitions Different simulations with different resource loadings

54. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Little-JIL Scalability Largest process definitions so far: –Chemotherapy Process »426 total steps »381 step declarations + 45 references »270 leaf step declarations –Brainstorm »109 step declarations »55 step references »51 leaf step declarations Generate flowgraphs with tens of thousands of nodes and edges

55. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Extensibility Processes are quite extensible, but language extensibility is much harder Process extensibility by –Elaboration of leaf steps »E.g. by reference to existing steps/modules –Augmenting resource model and repository Language extensibility –By modification of FSAs that define step kind semantics –Resource model and request language are orthogonal and more easily extensible

56. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Coverage As a management vehicle –Management and Operations process are defined as being in parallel »Report travel across channels »Management decisions and resources too –Reporting data gathered from channels Step cost, schedule annotations can be basis for projections using static analysis Steps are modules –Facilitates reuse –Facilitates extensibility –Facilitates comprehension

57. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Dynamism Exception handling is a key feature –React to runtime exceptions Facility for catching and reacting to signals/events Resources are late bound –Resource instances can be added/removed during execution Analysis support continuous process improvement

58. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Processes in Other Domains Mechanical Engineering Ecommerce –Processes such as auctions –Online license renewal Robotics Digital Government –Labor/management negotiation –Elections Medical/nursing processes Distributed scientific statistical data processing –The Analytic Web Project

59. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Some Observations Process engineering is important, feasible Effective process languages are possible –Borrowing from programming languages helps »Abstraction, scoping, exception management, concurrency, etc. –Transactions and Real-time are needed too Analysis is feasible for detecting defects –Basis for systematic process improvement Process guided execution has value –Needs process guided user interface management

60. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Little-JIL Process Development Process Definition Various Editors Resources Definition Artifacts Definition Process Programmer

61. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Editors Coordination Diagram Editing –Visual JIL (for version 1.3) is available –Visual JIL/Eclipse (for version 1.5) soon (?) –Textual “viewer” »Also “outline form” views Resource Editor –Is under development Artifact Editor –Currently use Java Beans

62. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Little-JIL Execution Architecture Process Definition Various Editors Resources Definition Artifacts Definition Execution Engine (Juliette) Agents Agendas Coordination Structure Resource Repository Artifact Repository Agenda Manager Process Programmer

63. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Execution Engine Version 1.3 –In use –Prototype, undistributable Version 1.4/1.5 –In late development –Expected to be distributed

64. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Static Analysis of Little-JIL Process Process Definition Various Editors Resources Definition Artifacts Definition Coordination Structure Resource Repository Artifact Repository Agenda Manager Analyzers Properties Flavers Simulator Fault Tree Analyzer Process Programmer

65. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Analysis/Reasoning is Key Goal Before execution, be sure the process is “suitable” Various analyses for various kinds of suitability No limit to possible analysis Integration of capabilities needed

66. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Analysis Examples Finite State Verification –To find process defects Fault Tree Analysis –To understand consequences of incorrect performance Failure Mode Effects Analysis (FMEA) –To determine ripple effects Discrete Event Simulation –To evaluate different execution strategies (eg. resource loading decisions) And more…..

67. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Interfacing with Little-JIL Agents Process Definition Various Editors Resources Definition Artifacts Definition Execution Engine (Juliette) Agents Agendas Coordination Structure Resource Repository Artifact Repository Agenda Manager User Interface Manager Process Programmer

68. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Little-JIL Environment Architecture Process Definition Various Editors Resources Definition Artifacts Definition Execution Engine (Juliette) Agents Agendas Coordination Structure Resource Repository Artifact Repository Agenda Manager Analyzers Properties Flavers Simulator Fault Tree Analyzer User Interface Manager Process Programmer

69. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Juliette The Execution Engine for Little-JIL processes Implements coordination among agents –Human –Machine –Internal components Moves artifacts around Acquires resources as needed

70. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Status Little-JIL –Version 1.5 defined –Version 1.4 editor working Juliette –Version 1.3 interpreter works »Not distributable –Version 1.4 working now »Eventually distributable Resource Manager –Crude prototype only User Interface Dispatcher –Prototype works –Eventually distributable

71. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008

72. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Engineering: Partial Gear Train Design Process Allows reworking previous design steps with new constraints Drives novice design/training tool Based on concrete design activity

73. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Gear Design Optimization Process

74. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Robot Coordination Process

75. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 The Future: Full Process programming Environment Editors for creating processes Execution engine Analyzers of all kinds Powerful and comfortable user interfaces Superior integration of all of the above

76. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Major Components Agents –The actors in processes Agenda Management System –Manages communication between agents and processes Step Interpreters –Enacts processes Resource Manager –Optimizes allocation of agents and other contended entities Artifact Manager –Enables the creation and manipulation of work products

77. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Juliette Runtime Architecture

78. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Resource Management Resource Model –Represent structure of resources –Their interrelations –Their constraints Resource Manager –Maintain a repository of resource instances –Accept requests for resources –Provide (optimal?) instances –Dynamic state management

79. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Resource Research Issues What is a resource? How to model resources? How to represent resource constraints? How to request resources? How to do resource allocation?

80. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Proposed Architecture Consult/ Update Consult Resource Allocation Component Update Consult Processed Request Resource Modeling Component Resource Request Resource Request Interpreter Resource Model 1 Match maker Resource Model m... Resource Object Repository Scheduler System Using the Resource Manager User Specify Allocation Table Assignment Resource Constraints

81. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 ODR Process –Process Variation Examples »Levels of Anonymity »Role of the Mediator

82. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Process Fragment: Administer

83. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Process Fragment: Change Phase

84. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Process Fragment: Add Option Performed by a human agent. Performed by a software agent. Example of a sub-process with leaf steps.

85. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Binding Script Example agent: Participant * item: Contribute Items.POSTED action: Contribute Items.START action: setCurrentPage(“ParticipantBrainstormPage”) action: showComponent(“ContribueItemComponent”) item: View Items.POSTED action: View Items.START action: displayArtifactInComponent(“state BrainstormState”, “ContributeItemComponent”)

86. Copyright L. Osterweil, all rights reserved USC CSSE 17 March 2008 Little-JIL Environment Architecture Process Definition Various Editors Resources Definition Artifacts Definition Execution Engine (Juliette) Agents Agendas Coordination Structure Resource Repository Artifact Repository Agenda Manager Analyzers Properties Flavers Simulator Fault Tree Analyzer User Interface Manager Process Programmer