Presentation on theme: "IJIS Industry Summer Briefing – July 2011"— Presentation transcript:
1 IJIS Industry Summer Briefing – July 2011 NIEMTestModelDataDeployRequirementsBuildExchangeGenerateDictionaryUsing NIEM EIEMDictionaries for BetterInformationAlignmentExchange DevelopmentPeter Doolan, GVP & Chief Technologist, Oracle Public Sector David Webber, Information Architect, Oracle Public SectorIJIS Industry Summer Briefing – July 2011
2 OverviewThe NIEM NTAC has introduced the use of Enterprise Information Exchange Models (EIEM) and dictionariesHow can your organization leverage these, what are the technical advantages, business benefits and challenges?Understand lessons learned from existing experience creating EIEM dictionaries and models and what are the resources and assets needed to do this for an enterprise.Creating IEPD exchanges using EIEM assets can enable rapid leveraging of NIEM for organizations; however, managing the process and stakeholders for the EIEM then provides additional needs.
3 Introduction What is the big picture of EIEM for federal enterprises? How does government acquire and manage these assets?How can you leverage your existing infrastructure?What roles do semantics and rules play?EIEM collaboration and alignmentLessons learned and examplesWhat tools are available?Initial support provided in GTRI dictionary serverCAM toolkit dictionary support
4 Transitioning from NIEM past/present to NIEM future EIEM – the big picture
5 NIEM Past and Present Challenges Vertical domain vocabulary for DHS and DOJOne dozen domain dictionariesTechnology limited - built using XSD schemaCore components highly contextual to DHS/DOJSurprisingly – no actual logical models of information!Gap with semantic technologies integrationLimited data content rules and code listsMapping automation support missingMulti-year development life cycles
6 NIEM Future?Abstract dictionary representations with strong semantics, rules and code lists supportSchema, models, documentation are generated from dictionary automaticallyContextual mechanisms allow linear number of true core components – not exponential growthEasier for practitioners to leverage with more predictable and repeatable resultsEnable domain use horizontally across governmentETL mapping automation supportContinuous collaboration environment
8 Why do we need EIEM dictionaries? Allows domains to manage their components librariesProvides consistency for project development teamsSets of NIEM consistent XML exchange componentsAligned to enterprise data storesOptimized for reuse and interoperabilitySave time and effort across the enterprisePerennial question for developers – when should I use NIEM components, and when our own local ones?Are there components already available for that purpose?Provide formal mechanisms and procedures to share components and collaborate across SDLC processProvide external parties consistent data views
9 How to create EIEM dictionaries Top down - directly from information modelsEnterprise modelling toolsSQL database physical table definitionsBottom upHarvesting from existing data assetsHarvesting from XML exchange structures and schemaUsing neutral abstract standards based representations
10 Conceptual Information Model Canonical Components DictionaryXMLFollowsNaming and Design Rule (NDR) principles and guidelinesEach compound componentParent (ABIE)ItemParent (ABIE)ItemParent (ABIE)ItemParent (ABIE)ItemEach atomic componentAttribute(ASBIE)Child(BBIE)ItemChild(BBIE)ItemChild(BBIE)ItemChild(BBIE)ItemAttribute(ASBIE)Shows overall components in the dictionary. Parents contain references to children. Children may exist atomically without parent. One child element may be referenced by multiple parents. Attributes can be associated with children or parent.Attribute(ASBIE)ebXML CCTS terms (ABIE, BBIE, ASBIE)Parent = Aggregate Business Information EntityChild = Basic Business Information EntityAttribute = Association Business Information EntityAttribute(ASBIE)Optional attributes of component* CCTS – Core Components Technical Specification
11 Example – Person Name Person Name (ABIE) Verified Details? (ASBIE)First Name (BBIE)Middle Name (BBIE)Last Name (BBIE)Previous Name? (ASBIE)Language Code (BBIE)Language Code may exist independently of Person NameVerified Details and Previous Name are flags that denote additional information about the entity they are associated withThere are three component items aspects:structure relationships; content rules; definitions
12 Top Down - ICE EIEM dictionary generation Production RulesICE Enterprise Data Model (EDM)ERwin Refactoring,Compress,Renaming ToolNIEM NDR Evaluations123OASIS CAM templateModel ComponentsXSDschemaEleExportImportApplyDef54DDLCodeListsCodeListsExcel SpreadsheetERwin modelCompleted CAM templateAnalyst ReviewBENEFITS:Will reduce ICE development time and effortPre-built NIEM consistent XML componentsXML consistent with ICE enterprise SQL dataRemoves scope for definition errorsMaximizes reuse potential across ICEOpen public standards based approachOpen technology infrastructureERwinClient76XML+XSDCanonical Dictionary of ICE exchange componentsGenerateStandardComponents DictionaryXMLWriteebXML CCTS compatible (ABIE, BBIE, ASBIE)
13 Renaming Tool Expands abbreviations (has built-in lists with context) Corrects common typos and spelling errors (200+)Adjusts names to camel case with no separators (_)Retains acronymsAdds name suffix according to NIEM representation terms and type of content, e.g. date, number, code, etcOptionally refactor rules by their type for interoperability; make nillable optional; relax DB length restrictionsCan optionally insert namespace prefixCan be configured by editing the lists in: word-utils.xslRefactoring of rules by type removes inconsistent length constraints often found in databases. These are not consistent with XML approach of having unconstrained content.Two approaches possible. 1# - leave all constraints “as is” in the dictionary – control their use by using Expander tool options, or 2# fix with renaming tool.
14 Bottom Up – OASIS/NIST EML EML – Election Markup LanguageState and Local voting management systems data standardOption 2 – Derive from existing exchange XSD schemaImport each XSD and merge into CAM dictionary234CAM templateNDREvaluation,Refactor,RenamingToolMerge & Generate DictionaryXMLExchange XSD schemaImportCAM templateXMLExchange XSD schemaImportOASIS CAM templateDictionary of canonical componentsExchange XSD schemaImportAnalyst Review1ebXML CCTS compatible (ABIE, BBIE, ASBIE)EML Core XSD schema5OASIS CIQ schemaGenerate model diagramsUML/XMIMind mapsOASIS GIS schema
15 N-DEx Dictionary and Model Dictionary viewer with N-Dex componentsFreemind Interactive Model(Harvested from N-DEx schema – ndexia.xsd and ndexibp2.xsd)
16 Leveraging EIEM dictionaries Where can we use EIEM assets?What formats do we need to use?Are there applicable standards / guidelines?
17 Visual Designer with Dictionary Collection Domain dictionaryIndustry dictionary1Component DefinitionsExchangeDesignerComponent Definitions2SearchToolsInsertDictionaryParentComponentsDrag n’ Drop Tool345Completed Exchange Template
18 Alien Transfer Manifest – Mock Up dictionariesXMLICE EDM dictionaryNIEM Immigration dictionaryEID extensions dictionaryICE components referencedEID pre-defined collectionsbased on NIEM + ICE + newImmigration componentNamespaces of dictionary componentsDefinitions stored as syntax neutral canonical XMLNIEM componentsCollectionThis slide shows a conceptual prototype for an alien transfer exchange for repatriation purposes. The exchange structure is built using a selection of components from ICE, NIEM and domain defined assemblies. The dictionaries provide the structural and content rule details for those items. New items and collections in the EID dictionary eventually will be promoted to ICE EDM during the SLM process.Technical Note: Inline child elements for ice: components shown for illustrative purposes only; actually are type reuse references (paperclip icon on parent node).
19 Challenges and Opportunities Learning curve for application development teamsTeams used to manually crafting XML Schema and Java code in developer XML IDE toolsParadigm shiftIntegration into SDLC and IEPD processesIntegration with existing SOA stack environmentsCurrently working with SAIC and State DepartmentAutomatic generation of WSDL and Java code assetsUnit test XML instance generationXML test framework and validation services
20 Vision: Top Down Exchange Assembly ComponentsCanonical XML Components Dictionary1XMLUser InsertsNavigation and Query ToolsRelationship LookupsComponent Associations and Couplings2Domains Reuse LibraryExchange Templates and Rules3Artefact relationshipsCatalogTemplateAutomatic Rendering ToolsPresentationW3C Schema and Model Representations4SchemaXMLCollaboration ServicesModelsComponentsDelivery Control, Messaging, SecurityDeployment Environments and Middleware5Testing Workbench, Rules Engine, Data Samples, Integration ETLImplementation Artifacts and Examples6
21 Exchange Delivery Lifecycle Component DictionaryDrag and Drop Visual EditorTemplateUpdatesDesignMonitorAuthorTestCAMTemplateXML SamplesDeployXSD SchemaProductionLog ResultsXMI / UML ModelsCompiledTemplatesDocumentation
23 CAM V2.0 Release FeaturesDictionary “Drag and Drop” of components to build exchangesEnhanced visual editor and performance - 3x faster, smaller memory footprintCross-platform deploy - Windows, Ubuntu/LINUX, Mac; 32/64bitMind map models – integration with Freemind toolUML/XMI export – import to UML tools / ArgoUMLCode list table lookups – rule based and selectableThis allows people to fully integrate their exchange development from dictionaries through to working deployed exchange schema.NIEM Team - Public Sector Sales
24 Available XML Dictionaries LEXS 3.1.4 dictionaryOASIS EDXL dictionaryOASIS EML dictionaryNIEM 2.1 dictionariesCBRN dictionaryEmergency dictionaryFamily dictionaryImmigration dictionaryInfrastructure dictionaryIntelligence dictionaryJustice dictionaryMaritime dictionaryScreening dictionaryTrade dictionaryImmigration blueprintNIEM core dictionaryPackaged with CAM editorsee dictionary folder of install+ spreadsheet+ sample control filesXMLXMLXMLNote: Those marked in bold are model style dictionaries with recursive components.Available from download sitedirect link:+ includes spreadsheets and sample modelsSampling of available dictionaries. Those marked in bold are model style dictionaries with recursive components.XMLXMLXMLXMLXMLXML
25 EIEM Next Steps Align CAM dictionary tools with GTRI dictionary server Integrate to other modelling tools – not just ERwinProvide extended reporting tools for analysis across dictionary of components and between dictionary collections – report onInconsistencyCommon componentsExtensions