Mining Constraints for Artful Processes Claudio Di Ciccio and Massimo Mecella Claudio Di Ciccio 15 th International Conference on Business Information Systems (BIS 2012) Tuesday, May the 22 nd, Vilnius, Lithuania

Process Mining Process Mining [Aalst2011.book], also referred to as Workflow Mining, is the set of techniques that allow the extraction of process descriptions, stemming from a set of recorded real executions (logs).Aalst2011.book ProM [AalstEtAl2009] is one of the most used plug-in based software environment for implementing workflow mining (and more) techniques.AalstEtAl2009 The new version 6.0 is available for download at Definition BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

Process Mining Process Mining involves: Process discovery Control flow mining, organizational mining, decision mining; Conformance checking Operational support We will focus on the control flow mining Many control flow mining algorithms proposed α [AalstEtAl2004] and α ++ [WenEtAl2007]AalstEtAl2004WenEtAl2007 Fuzzy [GüntherAalst2007]GüntherAalst2007 Heuristic [WeijtersEtAl2001]WeijtersEtAl2001 Genetic [MedeirosEtAl2007]MedeirosEtAl2007 Two-step [AalstEtAl2010]AalstEtAl2010 … Definition BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

Artful processes [HillEtAl06]HillEtAl06  informal processes typically carried out by those people whose work is mental rather than physical (managers, professors, researchers, engineers, etc.) “knowledge workers” [ACTIVE09]ACTIVE09 Knowledge workers create artful processes “on the fly” Though artful processes are frequently repeated, they are not exactly reproducible, even by their originators, nor can they be easily shared. Artful processes and knowledge workers A different context BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

A different context In collaborative contexts, knowledge workers share their information and outcomes with other knowledge workers  E.g., a software development mgr. Typically, by means of several conversations  conversations are actual traces of running processes that knowledge workers adhere to conversations BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

From the collection of messages, you can extract the processes that lay behind  Related conversations are traces of their runs Valuable advantages for users  Automated discovery of formal representations with no effort for knowledge workers  Tidy organization for naïve best practices kept only in mind  Opportunity to share and compare the knowledge on methodologies  Automated discovery of bottlenecks, delays, structural defects from the analysis of previous runs conversations are a kind of semi-structured text  this approach is not tailored to the electronic mail it can be extended to the analysis of other semi-structured texts Processes from conversations A different context BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

MailOfMine  MailOfMine is the approach and the implementation of a collection of techniques, the aim of which is to is to automatically build, on top of a collection of messages, a set of workflow models that represent the artful processes laying behind the knowledge workers’ activities.  [DiCiccioEtAl11]DiCiccioEtAl11  [DiCiccioMecella/TR12]DiCiccioMecella/TR12 What is MailOfMine? BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

From the archive to key parts Mail archiv DatabaseConversations Key Parts Multi-format mail storage plug-in based crawlers [ZardettoEtAl10]-based clustering algorithm [CarvalhoEtAl04] -based filter The MailOfMine approach BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

From key parts to processes Activity indicium Tasks Key Parts Concatenation [ZardettoEtAl10]-based Processes [CohenEtAl04, SakuraiEtAl05] -based task extractor BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

MINERful Tasks Processes BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

MINERful  MINERful is a workflow mining algorithm  Its input is a collection of strings T and an alphabet Σ T  Each string t is a trace  Each character is an event (enacted task)  The collection represents the log  Its output is a declarative process model  What is a declarative process model? The mining algorithm in MailOfMine BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

On the visualization of processes The imperative model Represents the whole process at once The most used notation is based on a subclass of Petri Nets (namely, the Workflow Nets) BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

On the visualization of processes Rather than using a procedural language for expressing the allowed sequence of activities, it is based on the description of workflows through the usage of constraints the idea is that every task can be performed, except the ones which do not respect such constraints this technique fits with processes that are highly flexible and subject to changes, such as artful processes BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma) The declarative model If A is performed, B must be perfomed, no matter before or afterwards (responded existence) Whenever B is performed, C must be performed afterwards and B can not be repeated until C is done (alternate response) The notation here is based on [AalstEtAl06, MaggiEtAl11] (DecSerFlow, Declare)AalstEtAl06 MaggiEtAl11

On the visualization of processes Imperative vS declarative Imperative Declarative Declarative models work better in presence of a partial specification of the process scheme BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

A real discovered process model “Spaghetti process” [ Aalst2011.book ] Aalst2011.book BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

Constraint templates as Regular Expressions (REs) Declare constraint templates BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

Constraint templates as Regular Expressions (REs) Declare constraint templates BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

MINERful by example  A project meeting is scheduled  We suppose that a final agenda will be committed (“confirmAgenda”) after that requests for a new proposal (“requestAgenda”), proposals themselves (“proposeAgenda”) and comments (“commentAgenda”) have been circulated.  Shortcuts for tasks (process alphabet):  p (“proposeAgenda”)  r (“requestAgenda”)  c (“commentAgenda”)  n (“confirmAgenda”) Scenario BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

MINERful by example  Existence constraints 1. Participation(n) 2. Uniqueness(n) 3. End(n)  Relation constraints 4. Response(r,p) 5. RespondedExistence(c,p) 6. Succession(p,n)  The agenda 1. must be confirmed, 2. only once: 3. it is the last thing to do.  During the compilation: 4. the proposal follows a request; 5. if a comment circulates, there has been / will be a proposal; 6. after the proposal, there will be a confirmation, and there can be no confirmation without a preceding proposal. Constraints on tasks BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

MINERful by example  In order to validate the algorithm  We translate constraints into REs The overall process is expressed by the intersection of REs  We use a RE-driven random string builder [Xeger] for creating a test- and-validation setXeger  We analyze the result and evaluate the performances  In order to see how it works now  We follow a run of MINERful over a string built by Xeger: r r p c r p c r c p c n Testing by replay BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

MINERful by example pp  p occurred 3 times in 1 string γ p (3) = 1 For each m ≠ 3 γ p (m) = 0  p did not occur as the first nor as the last character g i (p) = 0 g l (p) = 0 nn  γ n (1) = 1  For each m ≠ 1, γ n (m) = 0  n occurred as the last character in 1 string g i (n) = 0 g l (n) = 1 Building the “ownplay” of p and n BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma) r r p c r p c r c p c n

MINERful by example  With respect to the occurrence of p, n occurred… i.Never before: 3 times δ p,n (-∞) = 3 ii.2 char’s after: 1 time δ p,n (2) = 1 iii.6 char’s after: 1 time δ p,n (6) = 1 iv.9 char’s after: 1 time δ p,n (9) = 1 v.Alternating: i.Onwards: 2 times b → p,n = 2 ii.Backwards: never b ← p,n = 0  Looking at the string i. r r p c r p c r c p c n ii. r r p c r p c r c p c n iii. r r p c r p c r c p c n iv. r r p c r p c r c p c n v. i. r r p c r p c r c p c n ii. r r p c r p c r c p c n Building the “interplay” of p and n BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

MINERful by example -∞ δ r,p Building the “interplay” of r and p b → r,p = 1 b ← r,p = 0 r r p c r p c r c p c n BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

MINERful by example  Interplay and ownplay constitute the Knowledge Base of MINERful  The KB construction is such that each new string adds information  The algorithm does not need to read the strings more than once each  Constraints are determined by the evaluation of boolean queries on the KB  This allows the discovery of constraints with a faster procedure on a smaller set than the whole input  MINERful is a two-step algorithm Workflow discovery by constraints inference BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

MINERful by example  RespondedExistence(c,p)  ¬(δ r,p (0) > 0) There is no string where p does not occur, if r is read  Response(r,p)  RespondedExistence(r,p) ∧ ¬(δ r,p (+∞) > 0) RespondedExistence(r,p) holds and there is no string where p does not follow r  Precedence(r,p)  RespondedExistence(p,r) ∧ ¬(δ r,p (-∞) > 0) RespondedExistence(p,r) holds and there is no string where p does not precede r  Succession(p,n)  Response(p,n) ∧ Precedence(p,n) …… Some queries for inferring constraints     BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

MINERful by example  Response(c, n)  RespondedExistence(c, p)  NotSuccession(n, c), NotSuccession(n, p), NotSuccession(n, r)  Participation(p)  AlternatePrecedence(p, n)  Succession(p, n)  AlternatePrecedence(r, c)  Response(r, p) Other inferred constraints BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

Relation constraint templates subsumption  E.g.,  A trace like a b a b c a b c c satisfies (w.r.t. b and a ): RespondedExistence(a, b), RespondedExistence(b, a), CoExistence(a, b), CoExistence(b, a), Response(a, b), AlternateResponse(a, b), ChainResponse(a, b), Precedence(a, b), AlternatePrecedence(a, b), ChainPrecedence(a, b), Succession(a, b), AlternateSuccession(a, b), ChainSuccession(a, b)  The mining algorithm would show the most strict constraint only ( ChainSuccession(a, b) )  MINERful faces and solves this issue, by refining queries on the basis of the subsumption hierarchy of constraints Constraint templates are not independent of each other Mining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma) BIS 2012, Vilnius

Relation constraint templates subsumption Constraint templates are not independent of each other Mining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma) BIS 2012, Vilnius

Conclusions  MailOfMine is a system designed for mining artful processes out of collections  MINERful is the worflow mining algorithm designed for MailOfMine  MINERful is  Independent on the formalism used for expressing constraints  Modular (two-phase)  Capable of eliminating redundancy in the process model BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma) Recap

Conclusions  Linear w.r.t. the number of strings in the testbed |T|  Quadratic w.r.t. the size of strings in the testbed |t max |  Quadratic w.r.t. the size of the alphabet |Σ T |  Hence, polynomial in the size of the input O(|T|·|t max | 2 ·|Σ T | 2 ) BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma) On the asymptotic complexity of MINERful

References [Aalst2011.book] van der Aalst, W.M.P.: Process Mining: Discovery, Conformance and Enhancement of Business Processes. Springer (2011). [AalstEtAl2009] van der Aalst, W.M.P., van Dongen, B.F., Güther, C.W., Rozinat, A., Verbeek, E., Weijters, T.: Prom: The process mining toolkit. In de Medeiros, A.K.A., Weber, B., eds.: BPM (Demos). Volume 489 of CEUR Workshop Proceedings., CEUR-WS.org (2009) [AalstEtAl2004] van der Aalst, W.M.P., Weijters, T., Maruster, L.: Workflow mining: Discovering process models from event logs. IEEE Trans. Knowl. Data Eng. 16(9) (2004) 1128–1142. [WenEtAl2007] Wen, L., van der Aalst, W.M.P., Wang, J., Sun, J.: Mining process models with non-free-choice constructs. Data Min. Knowl. Discov. 15(2) (2007) 145–180. [GüntherEtAl2007] Günther, C.W., van der Aalst, W.M.P.: Fuzzy Mining - Adaptive Process Simplification Based on Multi-perspective Metrics. BPM 2007: [WeijtersEtAl2001] Weijters, A., van der Aalst, W.: Rediscovering workflow models from event- based data using little thumb. Integrated Computer-Aided Engineering 10 (2001) [MedeirosEtAl2007] Medeiros, A.K., Weijters, A.J., Aalst, W.M.: Genetic process mining: an experi- mental evaluation. Data Min. Knowl. Discov. 14(2) (2007) 245–304. [AalstEtAl2010] van der Aalst, W., Rubin, V., Verbeek, H., van Dongen, B., Kindler, E., Gnther, C.: Process mining: a two-step approach to balance between underfitting and overfitting. Software and Systems Modeling 9 (2010) 87– /s z. Cited articles and resources, in order of appearance BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

References [HillEtAl06] Hill, C., Yates, R., Jones, C., Kogan, S.L.: Beyond predictable workflows: Enhancing productivity in artful business processes. IBM Systems Journal 45(4), 663–682 (2006) [ACTIVE09] Warren, P., Kings, N., et al.: Improving knowledge worker productivity - the active integrated approach. BT Technology Journal 26(2), 165–176 (2009) [DiCiccioEtAl11] Di Ciccio, C., Mecella, M., Catarci, T.: Representing and Visualizing Mined Artful Processes in MailOfMine. USAB 2011:83-94 [DiCiccioMecella12] Di Ciccio, C., Mecella,M.: Mining constraints for artful processes. In W. Abramowicz, D. Kriksciuniene, V.S., ed.: 15th International Conference on Business Information Systems. Volume 117 of Lecture Notes in Business Information Processing., Springer (2012) (to appear). [DiCiccioMecella/TR12] Di Ciccio, C., Mecella, M.: MINERful, a mining algorithm for declarative process constraints in MailOfMine. Technical report, Dipartimento di Ingegneria Infor- matica, Automatica e Gestionale “Antonio Ruberti” – SAPIENZA, Universita` di Roma (2012). [AalstEtAl06] van der Aalst, W.M.P., Pesic, M.: Decserflow: Towards a truly declarative service flow language. Proc. WS-FM 2006 [MaggiEtAl11] Maggi, F.M., Mooij, A.J., van der Aalst, W.M.P.: User-guided discovery of declar- ative process models. In: CIDM, IEEE (2011) 192–199 [Xeger] Cited articles and resources, in order of appearance BIS 2012, VilniusMining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)