1. Architecture Tutorial 1 Overview of Today’s Talks Provenance Data Structures Recording and Querying Provenance –Break (30 minutes) Distribution and Scalability Security Methodology

2. Architecture Tutorial PrIMe : Provenance Incorporating Methodology Steve Munroe (sjm@ecs.soton.ac.uk)

3. Architecture Tutorial 3 Overview of Talk Introducing PrIMe Stepping through PrIMe –Step 1. Provenance use cases –Step 2. Information items –Step 3. Identifying actors –Step 4. Actor interactions –Step 5. Knowledgeable actors –Step 6. Adaptations Summary Conclusions

4. Architecture Tutorial 4 Introducing PrIMe A Methodology for making applications provenance-aware Provenance use cases involving documentation of identified past processes Therefore, processes and how they are enacted must be known before PrIMe is applicable –We suggest that, for new applications, use of PrIMe is interleaved with standard development methodologies Application DevelopmentProvenance Incorporation

5. Architecture Tutorial 5 Introducing PrIMe: Key aims Provide guidelines for: identifying and expressing provenance use cases identifying the kinds of information items that are required to satisfy use cases identifying actors and the interactions between them in order to effect the recording of process documentation identifying the set of adaptations that integrate the provenance architecture with the application to expose documentation for querying Aim to expose only factual information, i.e. no inferences are made at this point

6. Architecture Tutorial 6 Introducing PrIMe: Overview Application Structure

7. Architecture Tutorial 7 Introducing PrIMe: Overview Application Structure Step 1. Use Cases

8. Architecture Tutorial 8 Introducing PrIMe: Overview Application Structure Step 1. Use Cases Step 2. Information Items

9. Architecture Tutorial 9 Introducing PrIMe: Overview Application Structure Step 3. Actors Step 1. Use Cases Step 2. Information Items

10. Architecture Tutorial 10 Introducing PrIMe: Overview Application Structure Step 3. ActorsStep 4. Interactions Step 1. Use Cases Step 2. Information Items

11. Architecture Tutorial 11 Introducing PrIMe: Overview Step 5. Knowledgeable Actors Application Structure Step 3. ActorsStep 4. Interactions Step 1. Use Cases Step 2. Information Items

12. Architecture Tutorial 12 Introducing PrIMe: Overview Step 6. Adaptations Step 5. Knowledgeable Actors Application Structure Step 3. ActorsStep 4. Interactions Step 1. Use Cases Step 2. Information Items

13. Architecture Tutorial 13 Step 1: Provenance Use Cases Application Structure Step 1. Use Cases

14. Architecture Tutorial 14 Step 1: Provenance Use Cases We distinguish two types of provenance use case –A core provenance use case is a use case known when PrIMe is applied. –A future provenance use case is a use case that is not considered until after a process in the application is enacted but uses documentation of that process Core provenance use cases help inform the designers of the granularity of the processes to be considered and the critical information to expose. Future provenance use cases cannot be known by the designer, but can be anticipated by ensuring that the application is designed to capture potentially useful documentation.

15. Architecture Tutorial 15 Step 1: Provenance Use Cases Gathering Core Use Cases It is not always obvious to users what use cases they could expect the provenance architecture to support. We provide a simple requirements elicitation process to help designers collect the core provenance use cases –Give definitions of provenance –Give examples of the general questions that can be answered using the architecture –How to express provenance use cases

16. Architecture Tutorial 16 Step 1: Provenance Use Cases Definition of Provenance The provenance of a result is the process that produced that result. The provenance of an item of data is the process that generated it.

17. Architecture Tutorial 17 Step 1: Provenance Use Cases The OTM Application In the OTM application, use case questions relate to specific objects within the application, i.e.: –Recipients of organs –Organs –Organ Donors –Decisions

18. Architecture Tutorial 18 Step 1: Provenance Use Cases OTM Use Case Questions Below are questions that have been taken from the OTM application. –Retrieve data linked to all actions / events associated with a patient (recipient or donor) –What decisions were made for a particular case? –What is the medical analysis tree for a given organ? –Determine if any deviation took place from the standard workflow for a given organ

19. Architecture Tutorial 19 Step 1: Provenance Use Cases Eliciting Use Cases: We are looking to elicit use cases of the form: –(1) Actor A does something. –(2) Actor B does something else etc. –(3) Actor C determines the answer to a question about the provenance of data (such as a specific example of one of those above).

20. Architecture Tutorial 20 Step 1: Provenance Use Cases Elicitation Steps 4 Important steps: –Step (1) Describe something that already happens in the application. –Step (2) Describe a specific provenance- related use case question that cannot be answered (easily), but our functionality could help to achieve. –Step (3) Identify the relevant services required for answering the use case. –Step (4) Identify the relevant information items.

21. Architecture Tutorial 21 Step 1: Provenance Use Cases Example use case User interface Donor organ diagnoser Donor data collector Electronic health care records Testing laboratory Donor A’s organs are screened for potential donation. What is the provenance of the donor’s organ diagnosis?

22. Architecture Tutorial 22 Step 1: Provenance Use Cases Form-Based Capture Donor A’s organs are screened for potential donation. What is the provenance of the donor’s organ diagnosis? Application DescriptionDonor A’s organs are screened for potential donation Use case questionWhat is the provenance of the donor’s organ diagnosis Relevant servicesDonor data collector, Donor organ diagnoser, EHCR, Testing Lab Relevant information itemsPID, Patient records, Blood analysis result, Analysis decision

23. Architecture Tutorial 23 Step 2: Information items Application Structure Step 1. Use Cases Step 2. Information Items

24. Architecture Tutorial 24 Step 2: Information Items Overview The kinds of information that will answer your use case May be one piece or many pieces of information –E.g. a given result, or a sequence of decisions For each core provenance use case, identify the information items required to satisfy the use case. –For each process in the system, identify any additional items of information that could be exposed and may be useful in future provenance use cases.

25. Architecture Tutorial 25 Step 2: Information Items Examples Information items may be : –Data items, i.e. the result of some calculation, decision (contained in state). –Whole or part processes, e.g. the sequence of decisions that led to a donor’s organ being rejected for donation. –Relationships, e.g. what were the causal determinants of a given decision.

26. Architecture Tutorial 26 Step 2: Information Items Capture Information items are to be captured by process documentation, i.e. p-assertions –Data items: Interaction or actor state p- assertions –Processes: Interaction and relationship p- assertions –Relationships: Relationship and interaction p- assertions

27. Architecture Tutorial 27 Step 3: Actors Step 3. Actors Step 1. Use Cases Step 2. Information Items Application Structure

28. Architecture Tutorial 28 Step 3: Actors Description An actor is an entity within the application that performs actions, e.g. Web Services, components, machines, people etc. and interacts with other actors. –One actor may be seen as being composed of other actors. A primitive actor is one for which the designers do not know the other actors of which it is composed (or do, but the decomposition is deemed to be too detailed to be relevant). A role is a place-holder for an actor performing a particular function, where we cannot know exactly which actor will perform that function during the application's execution. –Roles can be composite or primitive as with actors.

29. Architecture Tutorial 29 Step 3: Actors Roles in a provenance architecture Asserting Actors – assert p-assertions Recording Actors – record p-assertions Querying Actors – retrieve p-assertions Managing Actors – maintain provenance stores

30. Architecture Tutorial 30 Step 3: Actors Identification Heuristics Identify the components that receive information. E.g. a component/service in a workflow, a script command, the GUI/desktop application into which a user enters information. Identify the components that provide the information in each interaction. These could be, for example, a workflow engine, a script executor, a user. –Each may also be intensionally defined, i.e. “the component that is the receiver in this interaction”.

31. Architecture Tutorial 31 Step 3: Actors OTM Example User request (M1) Diagnosis result (M8) User interface Donor organ diagnoser Donor data collector Electronic health care records Testing laboratory Get Donor info (M2) Request p ID (M3) Return p ID (M4) Request blood test (M5) Return result (M6) Return result (M7)

32. Architecture Tutorial 32 Step 3: Actors Information Message IDData itemReceiver ID M3 M5 M7 Q1 Pid r1 EHCR Testing Lab Diagnoser Donor data collector Sending Receiving Message IDData itemSender ID M2 M4 M6 q1 pid R1 Diagnoser EHCR Testing lab Actor : Donor data collector

33. Architecture Tutorial 33 Step 4: Interactions Application Structure Step 3. ActorsStep 4. Interactions Step 1. Use Cases Step 2. Information Items

34. Architecture Tutorial 34 Step 4: Interactions Information exchange

35. Architecture Tutorial 35 Step 4: Interactions Tracking processes A common information item required for provenance use cases is the process to which documentation refers Interaction p-assertions Relationship p-assertions Tracers

36. Architecture Tutorial 36 Step 4: Interactions Session Tracer Testing Lab Donor Data Collector Donor Organ Diagnoser Testing Lab Donor Data Collector Donor Organ Diagnoser Run 1 Run 2 Tracers

37. Architecture Tutorial 37 Step 4: Interactions Tracer terminology A computational activity –Actors cooperating on some work Superiors –Any actor sending requests to other actors Inferiors –Any actor receiving requests from other actors Tasks –An independent computation within an actor, delimited by a request to the actor and a subsequent response from the actor

38. Architecture Tutorial 38 Step 4: Interactions Session Tracer Semantics Generation rule –An actor must generate a new session tracer at the start of each task and add the tracer to all requests within that task Propagation rule (to inferior) –An actor must add any session tracers received from a superior to all requests it makes to inferiors within the task started by the superior’s request Propagation rule (to superior) –An inferior must add the session tracers supplied by its superior to its response to its superior

39. Architecture Tutorial 39 Step 4: Interactions Other Tracers Other application specific tracers possible –e.g. In the medical domain, a tracer could be used to identify all interactions belonging to a particular case.

40. Architecture Tutorial 40 Step 5: Knowledgeable Actors Step 5. Knowledgeable Actors Application Structure Step 3. ActorsStep 4. Interactions Step 1. Use Cases Step 2. Information Items

41. Architecture Tutorial 41 Step 5: Knowledgeable Actors Description A knowledgeable actor is an actor that has access to an information item. The primary knowledgeable actor for an information item is the primitive actor who first becomes aware of that information, for one of the following reasons. –The actor creates the item. –The actor receives or observes the item from outside the application.

42. Architecture Tutorial 42 Step 5: Knowledgeable Actors Who knows what? Hospital EHCRS Testing lab

43. Architecture Tutorial 43 Step 5: Knowledgeable Actors OTM Example User interface Donor organ diagnoser Donor data collector Electronic health care records Testing laboratory Hospital

44. Architecture Tutorial 44 Step 3,4,5: Knowledgeable Actors Repeat as necessary –Step 3: Identify actors –Step 4: Identify interactions –Step 5: Identify knowledgeable actors

45. Architecture Tutorial 45 Step 5: Knowledgeable Actors Recording Process Documentation

46. Architecture Tutorial 46 Step 6: Adaptations Step 6. Adaptations Step 5. Knowledgeable Actors Application Structure Step 3. ActorsStep 4. Interactions Step 1. Use Cases Step 2. Information Items

47. Architecture Tutorial 47 Step 6: Adaptations Provenance Functionality The provenance wrapper exposes an actor’s input and output data, and aspects of the actor’s state.

48. Architecture Tutorial 48 Step 6: Adaptations The Client Side Library A collection of functions –To allow provenance-aware applications to communicate with provenance store services –An implementation of the actor side library should contain at least one of the query library, the record library and the management library –Helps application developers follow architecture rules

49. Architecture Tutorial 49 Step 6: Adaptations CSL Layered Approach Client Side Library Applications Provenance Store Server Application API Utilities Server API

50. Architecture Tutorial 50 Step 6: Adaptations Modifying actors A non-knowledgeable actor may be modified so that it exposes for documentation an information item that is part of its state. A non-knowledgeable actor may be modified so that it gains access to information items not currently available to itself or other actors in the system.

51. Architecture Tutorial 51 Step 6: Adaptations Actor Introduction A new actor can be introduced to the application to help in the answering of use cases

52. Architecture Tutorial 52 Step 6: Adaptations Interaction Extension An interaction in the application can be extended to exchange more information between a knowledgeable actor and a non- knowledgeable actor, making the latter knowledgeable. Actor Before Actor After a a,b

53. Architecture Tutorial 53 Step 6: Adaptations Interaction Introduction A new interaction between actors can be introduced into the application in which a knowledgeable actor sends the information item to another actor, which then becomes knowledgeable. Actor Before Actor After b

54. Architecture Tutorial 54 Summarising PrIMe Step 1: Identify the provenance use cases Step 2: Identify relevant information items –Step 3: Identify actors –Step 4: Identify interactions –Step 5: Identify knowledgeable actors Step 6: Make necessary adaptations Granularity

55. Architecture Tutorial 55 Conclusions PrIMe provides a clear and easy guide to make applications provenance-aware Crucial in the adoption of the Provenance Architecture Ongoing process Development requires collaboration and input from users –Tomorrow’s case studies

56. Architecture Tutorial 56 Questions? Steve Munroe sjm@ecs.soton.ac.uk