Presentation on theme: "The Instant Replay MA for FIM"— Presentation transcript:
1 The Instant Replay MA for FIM Bob Bradley, MVP, MCTS, FIM Team founding member
2 Background Bob Bradley, FIM MVP 2012, 2013 Work for UNIFY Solutions in Australia with Carol Wapshere and other colleagues in “The FIM Team”Specialize in event-driven FIM solutions (you will see this in demo)Began working with MIIS full time in 2004 (then ILM2007)Have worked full time on FIM since mid 2009Worked closely for 2 solid years with the MCS Identity Management Practice lead in Australia on the biggest FIM sites in Australia, neither of which were “OOTB”In working with FIM I came up with a couple of unique ideas – including this one
3 Presentation Outline Inspiration and Concept Construction and Demo Use Case Scenarios and DemoAdditional use case: Maintaining ReferencesAdvanced Implementation and DemoConclusion
4 InspirationThe Replay MA was inspired by limitations encountered using the FIM MA …The FIM MA is very different from any other typeAdditional rules apply, e.g.only one instance of the FIM MA allowed per sync serviceonly one FIM service connected to a single sync serviceone-to-one “like with like” attribute mappings onlyonly direct flows configurable in the MA wizard onlyno manual precedence allowed when FIM MA contributes an attribute value to the MVConstraints such as the ones above can impose solution limitations … ones that we might find ourselves looking for ways around The FIM Portal and FIM Sync Engine are connected by a special management agent (the FIM MA). Significant restrictions are in place to ensure that the FIM MA is only used within certain (sometimes inhibiting) constraints, such as the following:There can only ever be one instance of the FIM MA per sync server instanceThere can only ever be one FIM Service connected to a single FIM Synchronization ServiceObjects can only ever be mapped between the FIM MA CS and MV on a 1-1 basisAttribute flows for the FIM MA between the MV and CS can only ever be direct (no rules extensions allowed), and must be configured in the MA wizardManual precedence rules cannot be defined to include attributes contributed to the Metaverse by the FIM MAThe above restrictions mean that there is effectively no real flexibility around how to design configurations to achieve common synchronization requirements. Specifically there are several emerging use cases (which I will come to shortly) which can NOT be achieved using the synchronization engine and can only be approximated using custom workflow activities. As a result, there is no documented means of using the FIM Synchronization engine to achieve certain desirable outcomes.
5 ConceptThe Replay MA is a very low-cost option (in terms of development as well as processing overhead) for providing the FIM Metaverse with an additional feed of the same objects already present in an existing MA (connected or not)In the special case of the FIM MA, this provides added benefits, including restoring the advanced flow rule and manual precedence options otherwise denied for the FIM MA, seemingly leaving you no option but to implement “equal precedence”
6 Concept (continued)This session will walk you through how to create a standard text file MA that works alongside the FIM MA, allowing you to overcome advanced flow and precedence restrictions.You will also see how, with a more advanced configuration, you can also achieve that sought-after flexibility with reference attributes.
7 Construction Export the configuration of your target MA (FIM MA) Run ReplayLDIF-GenerateSchema.ps1 to transform the DSML file to an LDIF file template for a new text MACreate a new LDIF file MA from the templateBe selective in your attribute flows, flowing only those objects and properties that you want toConfigure enhanced precedence and advanced flow rules as necessaryThe Instant Replay MA is a concept which leverages standard FIM Sync Engine features in a way not considered before to allow certain configuration options that are often not possible otherwise. Using what is essentially a read-only clone of any existing MA (including the FIM MA), any attribute can be contributed by the cloned MA in lieu of the original MA, thereby allowing for standard and extended options involving these attribute flows.Here’s how …I've created a generic mechanism to generate a DSML template file for a new DSML Text File FIM management agent from the audit drop file (also DSML but in an incompatible format). The components of the solution are as follows:A powershell script to transform the audit drop file (DSML) into LDIF formatAn XSLT stylesheet which performs the transformationOperational software to coordinate the standard MA run profile with the replay MA run profile (Event Broker)
8 Basic ImplementationConfigure audit drop files for your target MA, and use these as the source for your Replay MAConfigure the ReplayLDIF-GenerateData.ps1 script to transform the DSML drop file into LDIF formatTest and refineUse automation to orchestrate run profile sequencing on the back of the source run profile sequenceThe Instant Replay MA is a concept which leverages standard FIM Sync Engine features in a way not considered before to allow certain configuration options that are often not possible otherwise. Using what is essentially a read-only clone of any existing MA (including the FIM MA), any attribute can be contributed by the cloned MA in lieu of the original MA, thereby allowing for standard and extended options involving these attribute flows.Here’s how …I've created a generic mechanism to generate a DSML template file for a new DSML Text File FIM management agent from the audit drop file (also DSML but in an incompatible format). The components of the solution are as follows:A powershell script to transform the audit drop file (DSML) into LDIF formatAn XSLT stylesheet which performs the transformationOperational software to coordinate the standard MA run profile with the replay MA run profile (Event Broker)
10 Use Case Scenarios Avoid using equal precedence Derive multiple/alternative import mappings from the same FIM Portal propertySelectively import reference valuesImport reference values as strings (not just FIM MA) using direct or advanced flow rulesImplement manual precedence for import flows involving the FIM PortalThe following are 3 specific scenarios where my colleague Carol has used the Replay MA idea for her clients …Use cases for this MA include the following:Eliminate the need to configure "equal precedence" for scenarios where there is no alternative when involving the FIM MAThere are several scenarios here (e.g. group membership for migrated groups should become authoritative in the portal post migration) which are presently not achievable without configuring equal precedence. This is always problematic and would be good to avoid by introducing a 3rd authoritative source for group membership which can trump the others.Provide a means for FIM portal attributes to be used to derive additional columns (incl. in advanced attribute flows).The FIM MA allows only direct 1-1 attribute flows between like object classes in the FIM Portal and the FIM Metaverse using fixed class schema. One scenario is where you wish to join on something other than the mv GUID – e.g. on the manager attribute so as to enable flow of the manager display name (redundantly) to the subordinate.Provide a means for FIM portal attributes to be used to be treated as different attribute types (incl. in advanced attribute flows).The FIM MA allows only direct 1-1 attribute flows between like object classes in the FIM Portal and the FIM Metaverse using fixed class schema. This prevents the use of advanced flow rules in such cases as only flowing reference attributes based on the value of another attribute of the same identity, or flowing reference types as strings to allow for advanced flow rules. * Note: there is a documented alternative (advanced) for this scenario when working with Portal sync rules.Provide a means to define MANUAL precedence by enabling advanced attribute flows (rules extensions) from the FIM PortalThe FIM MA allows only direct 1-1 attribute flows, and as a result any attribute contributed by the FIM Portal cannot be included in a “manual precedence rule” when the FIM MA is the only means of sourcing this attribute from the FIM PortalThe following 3 slides are graphics of specific examples of the above use cases …
11 Use Case: FIM MA Not Precedent End DateHR Precedent for StaffPortal Precedent for ContractorsTermination WF in PortalHRTermDateADFIM MASkipped: Not PrecedentMetaverseEmployeeEndDateaccountExpiresemployeeEndDateDirect FlowsHR Precedent over Replay MAReplay MAFIM MA precedentEqual precedenceManual precedenceEmployeeEndDate
12 Use Case: Advanced Import Flow Unique IDGenerate Person IDMust not changeOther usesApplicationIDADFIM MAPersonIDMetaverseObjectIDemployeeIDpersonIDReplay MAOnly flow ifnot presentNew portal object changes personIDObjectIDpersonID does not change
13 Use Case: Selective Reference Flow Staged Group MigrationSome precedent in PortalSome precedent in NotesMust be identical in Notes and ADNotesMembersADFIM MAFIMAuthoratativeMetaverseMembermembermemberSelective JoinReplay MAMemberFIMAuthoratativeEqual precedenceFIM RTM – no scoped SRsReplay MA precedent over Notes
15 FIM Back-links Use Case #1 Leveraging relative-to-resource MPRs Relative-to-resource idea saves on set/MPR proliferation, which is a known cause for FIM performance degradationThis style of MPR comes with the hidden cost of maintaining the referencesMulti-value reference must be maintained in sync with each collection of administrators for a locationHigh processing overhead in maintaining this via workflowNeed Housekeeping to ensure integrity (topic for another time!)Sync option is far more attractive … just need to support deltas!Use case #1A multi-value reference attribute locationAdministrators on PERSON must be maintained in sync with the PersonID assigned to each LOCATION object in the FIM Portal. In this scenario each user in any given location must be updated with the current set of Location Administrators when location changes. This is to allow a (delegation style) policy "Org admins can manage the users in their assigned Orgs" to be implemented.This example involves the updating of all user objects in the same location as the person administering a location.In my FIM Housekeeping talk I discussed the way workflows are often used to maintain such “back links”, and the inspiration for the Housekeeping idea was from just this kind of scenario. Often there is no alternative but to go down this route … but in many cases the FIM Replay MA offers a pleasing alternative …There are several significant drawbacks to implementing intra-FIM sync style workflows, but they all amount to effectively the same thing - a lack of state management capability. This is where the Sync Engine excels. Data issues and environment failures can lead to workflow exceptions, and the only way to address this problem is to mitigate it with housekeeping type functionality (i.e. calculate what the backlinks should be and compare this to what they actually are ... and do this iterating over the entire user base on a regular maintenance cycle).One of the biggest drains on FIM performance is the number of dynamic sets defined, and the number of policies (MPRs) which are defined on these sets. The reason for the above design ("relative to resource" style MPR) is that only one set, workflow and policy triple is required to implement such Location policy this way, as opposed to one MPR/Set/Workflow triple for each and every location in existence (and these invariably are added and removed over time, meaning that other policy would be required to generate these triples to achieve the equivalent zero maintenance of the "relative to resource" idea). However, the cost of this MPR style is entirely determined by the overhead of maintaining these multi-value reference attributes such as the ones on PERSON in the above example. So the workflow overhead is effectively the "lesser of two evils".A synchronisation approach to the same problem has none of these problems, with ongoing integrity assessable at a glance. Although the workflow approach has no doubt been replicated many times throughout the FIM world by now, it is a (presently necessary) weakness in the design, and this will only happen more and more as new ideas emerge with how to use FIM. A sync approach is far more streamlined, simple and consistent (when references are synthesized by a tool such as Identity Broker and not hand-crafted XSLT to support deltas), and I expect that not only are the overheads much less, but the confidence in the integrity is priceless (especially when we're talking about access control here).
16 FIM Back-links Use Case #2 Set definitions on derived references to support MPRsMaintain Person.memberOf multi-value property derived from group.memberADUC console in AD shows a user’s group membership in the “Member Of” tab … however this is just a run-time inversion of the Group’s “Member” property, and cannot be synchronisedCould support set transition or request MPRs such as “All new users in the TEC2012 group are notified of their membership”, or simply “All users are notified of set membership changes”
17 Advanced Implementation Configure audit drop files for your target MAUse extended XSLT to transform the DSML file into an LDIF fileConfigure additional derived “back link” MA propertiesBe selective in your attribute flows, flowing only those objects and properties that you want toConfigure enhanced precedence and advanced flow rules as necessary, as well as derived “back-link” flowsUse an automation tool to orchestrate run profile sequencingA more advanced variation on the FIM Replay MA involves the computing of “back-link” reference properties, by applying transformations on the data read from the DSML drop file.Here we take the basic concept, but instead of loading directly into FIM via an LDIF MA, we extend the MA to incorporate additional properties. Why do I do this?To demonstrate what can be done, I’ll apply an extended XSLT transformation (additional code) to the FULL IMPORT DSML FILE to derive additional reference properties in the generated LDIF file. We’ll then see what happens when a delta import DSML file is processed under this model, thereby demonstrating what is required to support delta imports from your Replay MA.
18 Maintaining References What’s involved in enforcing referential integrity in FIM?Think of all the possible use casesIdentify all the relevant setsConstruct action workflowsConstruct set transition MPRsCross your fingers and hope nothing breaks …Here the FIM Replay MA can give you that peace of mind you need …So what can you do to ensure FIM has the best chance possible of maintaining this referential integrity:Consider all possible use cases which would result in a change to the target (reference) property (e.g. person.roles, or person.locationAdministrators)Identify changes to sets which together cover all the above use cases, and from which workflows can be run to reinforce referential integrityCreate action workflows to reinforce referential integrity to cover each set changeDefine set transition MPRs to fire workflows for each setApply MPRs retrospectively... with the work required above it is easy to see how scenarios can be missed, and how you can be left with some doubt that at any given time you can be confident you have full referential integrity=> here is where the HouseKeeping Fairy can save you …
20 ConclusionThe FIM Replay MA is a very simple, low cost option of providing the FIM Metaverse with an additional feed of the same data already present in an existing MA.In the special case of the FIM MA, this provides added benefits, including avoiding having to go down the “equal precedence” route.
21 More Info My blog: bobbradley1967.wordpress.com LinkedIn: au.linkedin.com/in/bradleybobTwitter: twitter.com/unificator (#FIM2010)The FIM Team: thefimteam.comMy Company:FIM Forum: social.technet.microsoft.com/Forums/en-US/ilm2Bob Bradley:
22 PostScriptOn 09/18/12 9:52 PM, Jason Bell wrote (LinkedIn): I didn't realize you had it posted yet... It is funny, shortly after TEC - I was inspired by the Replay MA concept and developed a ECMA 2.0 that let you select the MA to replay from a dynamic drop down list. Does a full dynamic schema discovery. I got it all up and running for single value attributes and forgot about it... then the other day I needed it and got to thinking that I should share it with you when I get it fully functional. I have been working on various ECMA 2.0 Management Agents to easily perform tasks that have historically required out-of-band processes. The Replay MA idea fit into this category. So anyway, when I get it done - I would like to show it to you and make sure you get due credit. Hopefully we will see you again at TEC in Keep up the great Blogs!