1. Twin Cities BizTalk User Group Rule Engines: BRE vs WF Rules November 2006 Stephen Kaufman

2. Agenda RETE Algorithm WF Rules Processing BRE Rules Processing Compare/Contrast Forward Chaining Iterating Overview

3. To RETE or Not to RETE Is that a question we should care about? Which rules engine implements RETE? What is RETE –It is an optimization algorithm –It reduces the number of predicate evaluations by remembering the most recent value and re-using it on subsequent re- evaluations You do not need a RETE engine to implement forward chaining

4. RETE There are times that rule sets are structured so that the optimization would make no difference or even hurt performance –If each rule evaluates independent expressions against different facts, the RETE algorithm will provide no improvement. –The RETE algorithms intrinsic overhead may result in a net loss of performance

5. Agenda RETE Algorithm WF Rules Processing BRE Rules Processing Compare/Contrast Forward Chaining Iterating Overview

6. WF Rules Processing WF provides a policy class –This policy class uses an internal executor class to execute the rule set The policy activity sends the rule set to the executor which creates a sorted list of RuleState objects –Each RuleState presents an individual rule and its associated state –The RuleState objects are sorted by priority

7. WF Rules Processing –Each rule is executed in sequence If two or more rules have the same priority the execution ends up based on an implementation dependent choice – currently on rule name (however, that is not something to count on) Each rule can have two sets of actions –One of these sets is executed depending on the overall evaluation of the rule condition – thus allowing if/then/else functionality The executor detects when actions perform changes to properties

8. WF Rules Processing The executor detects when actions perform changes to properties which can lead to forward chaining

9. Agenda RETE Algorithm WF Rules Processing BRE Rules Processing Compare/Contrast Forward Chaining Iterating Overview

10. BRE Rule Processing The BRE uses a RETE engine to process the rule execution –The RETE engine is a compiled representation of a rule set The BRE executes it RETE algorithm by asserting a number of facts to the engine These facts are turned into rule objects which may end up on the agenda

11. BRE Rule Processing All rule objects are evaluated based on their priority –Once rules are placed on the agenda they are then processed in order The BRE follows the Match-Resolve-Act phases. –This is discussed on pages 314-316 in the book “Artificial Intelligence – a modern approach”, v1 by Russell and Norvig

12. BRE Rule Processing The agenda is created with the rule objects in the order they will run As rules are pulled off the agenda and executed they may; –cause conditions to be re-evaluated and the agenda being modified before the next rule is pulled off the agenda –update existing facts, –assert new facts, –re-assert existing facts –Remove facts from the agenda

13. Agenda RETE Algorithm WF Rules Processing BRE Rules Processing Compare/Contrast Forward Chaining Iterating Overview

14. Compare/Contrast - Agenda WF does not use an agenda nor does it have the concept of a vocabulary BRE uses an agenda and the developer has the ability to modify how the engine operates at run time

15. Compare/Contrast - Assert WF acts against one object –At run-time there is no mechanism for adding or removing individual items –It is only possible to update the values – which may lead to a re-evaluation of the rules (forward chaining) With the BRE you can assert as many facts as necessary (short term and long term) as well as retract facts –You need to manually remove certain types

16. Compare/Contrast - Assert Retracting Facts Manually –This is necessary if you manually add facts during rule processing and don’t want to reuse that fact in future processing –These facts can be preserved if you are using the same instance of the engine and want to maintain the state across execution calls –Typically manually added facts are used for specific instances of rule processing and therefore need to be retracted

17. Compare/Contrast - Verbs WF’s Update verb is used in rule actions to invoke forward chaining where the value of a property has not been changed. BRE’s Update verb works differently in that you are telling the engine to re- evaluate what should be added to the agenda WF provides both explicit and implicit updates whereas the BRE only provides explicit updates.

18. Compare/Contrast - Verbs Implicit Update –Each rule is analyzed and relationships between rules are identified automatically. –The engine ensures that the rule will be re- evaluated whenever a dependant rule is executed Explicit Update –The developer tell the engine to re-evaluate rules with a specified condition

19. Compare/Contrast - Instances BRE requires you think and design in a non procedural fashion One rule may result in multiple facts of the same type being added to the agenda –The BRE allows you to associate an instance number with the terms that reference a given fact type factType(1).item == factType(2).item This will cause every possible combination of one instance being evaluated against every possible combination of the other instance

20. Multi Instance Support What is it? –BRE rules are expressed against types, not instances or variables. At execution time, the engine matches instances that are asserted against the rules written against their type. –This makes it possible to assert as many instances of a given type as needed and have them be processed independently. Be Careful.

21. Multi Instance Support Why should I be careful? Example IF Customer.AccountType == “Residential” AND Order.Value > 500 THEN Order.Discount = 5 If two instances of Customer type and Order were asserted, then the outcome would be : –Instance 1 of Customer and Instance 1 of Order –Instance 1 of Customer and Instance 2 of Order –Instance 2 of Customer and Instance 1 of Order –Instance 2 of Customer and Instance 2 of Order

22. Multi Instance Support Why should I be careful? Example IF Customer.AccountType == “Residential” AND Order.Value > 500 THEN Order.Discount = 5 If two instances of Customer type and Order were asserted, then the outcome would be : –Instance 1 of Customer and Instance 1 of Order –Instance 1 of Customer and Instance 2 of Order –Instance 2 of Customer and Instance 1 of Order –Instance 2 of Customer and Instance 2 of Order You need to add an additional condition to the rules IF Customer.AccountType == “Residential” AND Order.Value > 500 AND Order.CustomerID = Customer.ID THEN Order.Discount = 5

23. Types vs Instances There are times when writing rules against instances have advantages over types –Scenario An employee is requesting a higher security clearance. If the employees manager has authorization to grant the requested level or a higher level and the employee is within one level of the requested security clearance then grant the request

24. Types vs Instances WF IF this.manager.AuthorizationLevel >= this.accessRequest.RequestedLevel AND this.employee.SecurityLevel == this.accessRequest.RequestedLevel – 1 AND this.accessRequest.IsManagerApproved == true THEN this.employee.SecurityLevel = this.accessRequest.RequestedLevel IF this.employee.SecurityLevel > this.accessRequest.RequestedLevel – 1 AND this.accessRequest.IsManagerApproved == true THEN this.director.RequestSecurityClearance(this.employee, this.accessRequest) this.employee, this.manager and this.director are all objects of type Employee

25. Types vs Instances BRE IF Employee.AuthorizationLevel >= AccessRequest.RequestedLevel AND Employee.SecurityLevel == AccessRequest.RequestedLevel – 1 AND AccessRequest.IsManagerApproved == true THEN Employee.SecurityLevel = AccessRequest.RequestedLevel IF Employee.SecurityLevel > AccessRequest.RequestedLevel – 1 AND AccessRequest.IsManagerApproved == true THEN Employee.RequestSecurityClearance(Employee, AccessRequest)

26. Types vs Instances BRE IF Employee.AuthorizationLevel >= AccessRequest.RequestedLevel AND Employee.SecurityLevel == AccessRequest.RequestedLevel – 1 AND AccessRequest.IsManagerApproved == true THEN Employee.SecurityLevel = AccessRequest.RequestedLevel IF Employee.SecurityLevel > AccessRequest.RequestedLevel – 1 AND AccessRequest.IsManagerApproved == true THEN Employee.RequestSecurityClearance(Employee, AccessRequest) Will this work? Is there something missing?

27. Types vs Instances BRE IF Employee.AuthorizationLevel >= AccessRequest.RequestedLevel AND Employee.SecurityLevel == AccessRequest.RequestedLevel – 1 AND AccessRequest.IsManagerApproved == true THEN Employee.SecurityLevel = AccessRequest.RequestedLevel IF Employee.SecurityLevel > AccessRequest.RequestedLevel – 1 AND AccessRequest.IsManagerApproved == true THEN Employee.RequestSecurityClearance(Employee, AccessRequest) Will this work? Is there something missing? You need to distinguish between the various instances of Employee

28. Types vs Instances BRE IF Employee(0).AuthorizationLevel >= AccessRequest.RequestedLevel AND AccessRequest.EmployeeID == Employee(0).ID AND AccessRequest.IsManagerApproved == true AND Employee(1).SecurityLevel == AccessRequest.RequestedLevel – 1 AND Employee(0).ManagerID == Employee(1).ID THEN Employee(1).SecurityLevel = AccessRequest.RequestedLevel IF Employee(0).SecurityLevel > AccessRequest.RequestedLevel – 1 AND AccessRequest.EmployeeID == Employee(0).ID AND AccessRequest.IsManagerApproved == true AND Employee(1).Title == “Director” THEN Employee(1).RequestSecurityClearance(Employee(0), AccessRequest)

29. Types vs Instances There are also times when writing rules against types have advantages over instances –The BRE algorithm lets you decide the manager relationship in rules rather in the calling code –If you wanted to change the rule so that the manager’s manager can be used when the manager is out of the office you could do this entirely in the rules with BRE

30. Types vs Instances –With WF you would need to change the calling code You would need to add an extra property You would also need to get the additional employee object All in all they really are doing different things. –BRE is executing over a collection of employee objects, –WF is executing over a single object with 2 properties.

31. Agenda RETE Algorithm WF Rules Processing BRE Rules Processing Compare/Contrast Forward Chaining Iterating Overview

32. Forward Chaining Explained In forward chaining, rule actions operate against the object state where rule actions change the value of items in the object state This causes the engine to automatically re-evaluate the relevant rules against the new values which in turn continues until no additional re-evaluation is needed.

33. Forward Chaining - WF Rules run directly on the sorted list of the RuleState objects and execute the then or else action The RuleState object has a pending flag which starts off as true –As each rule is processed the pending flag is set to false –The flag is set no matter which action may, or may not, have fired

34. Forward Chaining - WF When the then or else is processed, the engine analyzes the action and determines if they invoke code which will have side effects. –A side effect is code that may alter the state of the object in a way that could affect the evaluation of conditions in other rules

35. Forward Chaining - WF If side effects are encountered, the engine searches the collection of RuleState objects looking for rules whose conditions are affected AND whose pending flag is set to false If the engine finds an affected rule with a higher priority than the rule which caused the side effect, the higher priority rule is evaluated and its actions are processed

36. Forward Chaining - WF Execution continues from the higher priority rule that just ran If there are no higher priority rules affected then the engine continue to process the rules in order from the rule that caused the side effect So, forward chaining is tied to prioritization and can cause a loop back to higher priority rules

37. Forward Chaining - WF For simpler rule sets, forward chaining can sometimes be eliminated by re- prioritizing the rules so that they are evaluated in a different order. –Or for rule sets with no prioritization, creating the rules in a different order MAY also be a possibility to eliminate forward chaining –Remember that rules are evaluated arbitrarily

38. Forward Chaining Example What does this rule do? IF Truck.ShippingCharge < 2.5 THEN Truck.ShippingCharge = 1;

39. Forward Chaining - Looping WF provides the RuleReevaluationBehavior at the rule level as well as the RuleChainingBehavior public enum RuleChainingBehavior { Full,// default; implicit, explicit and attribute-based chaining None,// do not chain; purely sequential execution UpdateOnly,// chain only on explicit Update statements } BRE provides the Maximum Execution Loop Depth at the ruleset level (default of 65536)

40. Agenda RETE Algorithm WF Rules Processing BRE Rules Processing Compare/Contrast Forward Chaining Iterating Overview

41. Iterating in the BRE Given the following XML, how will the BRE loop through the item nodes? Manifest Pro BizTalk 2006 by Apress Essential Windows Workflow by Addison-Wesley Default Composer Settings –XPath Field: *[local-name()='Item' and namespace-uri()=''] –XPath Selector: /*[local-name()='Package' and namespace-uri()='']/*[local- name()='Items' and namespace-uri()='']

42. Iterating in the BRE Manifest Pro BizTalk 2006 by Apress Essential Windows Workflow by Addison-Wesley The developer must manually modify the properties in the Composer –XPath Field: '.' (without the single quotes) –XPath Selector: /*[local-name()='Items' and namespace-uri()='']/*[local- name()='Item' and namespace-uri()='']'

43. Iterating in WF Rules WF requires additional methods/properties be placed on the object that is submitted. It is possible to iterate using the WF rules when your source is an xml document i.e., this.document.SelectNodes()

44. Iterating in WF Rules Create ruleset with at least 4 rules as well as decorate your class with extra properties Create an enumerator property of type System.Collections.Generic.IEnumerator or System.Collections.IEnumerator Create an items collection to contain your collection of items which can be of type List or anything that supports IEnumerator Create a property to hold the current item

45. Iterating in WF Rules Rules Iteration Pattern - Four Rules –Rule 1 (Priority = 2) //Always execute this rule once to create the enumerator. IF 1==1 THEN this.enumerator = this.myCollection.GetEnumerator() –Rule2 (Priority = 1) IF this.enumerator.MoveNext() THEN this.currentInstance = this.enumerator.Current –Rules 3-N (Priority = 0).... //Additional rules written against this.currentInstance

46. Iterating in WF Rules Rules Iteration Pattern - Four Rules –Rule N+1 (Priority = -1) // can be any condition as long as it is evaluated every time; // this.currentInstance will be evaluated each time this.currentInstance changes, whereas // "1==1" would only be evaluated once. IF this.currentInstance == this.currentInstance THEN... Update("this/enumerator") //this will cause Rule 2 to be reevaluated ELSE... Update("this/enumerator") This pattern works for a single collection –Repeat this as needed or in a hierarchy for multiple collections

47. Agenda RETE Algorithm WF Rules Processing BRE Rules Processing Compare/Contrast Forward Chaining Iterating Overview

48. Tools BRE Provides –An in-process execution engine that runs on the server –The ability to collect references into a concise library to provide domain specific terms – a vocabulary. This is a feature that will be added in future version of WF rules –A centralized database for the storage and management of rulesets. This includes audit information on ruleset changes –A windows service to monitor changes and provide ruleset deployment –A tracking infrastructure when called from BizTalk. This tracking data is available through the BTS tracking DB as well as the Health and Activity Tracking tool (HAT) and also file based. There is also a public interface to write your own tracking interceptor

49. Tools WF Provides –A rules UI that can be re-hosted in your own application –An in-process execution engine that runs on the client –The ability to track the rules –Rules located and executed on the client

50. BRE vs. Workflow Rules BizTalk Rules –Rete Based Analysis –XML/DB support in addition to the CLR Type support –If/Then –Vocabulary –File and DB storage of policies –Dynamic Policy-level updates –Caching Architecture –Implicit multi-instance support –Working Memory (stateful) –Rules Tracking –Server Based –Security Workflow Rules –Forward Execution Algorithm –Rules written against workflow variables – CLR Types –If/Then/Else –Short circuiting evaluation –Implicit chaining –More chaining control –Client based

51. Questions?