1. Oracle Data Integrator Transformations: Adding More Complexity5
Oracle Data Integrator Transformations: Adding More Complexity
4-1

2. Multiple Sources
You can add more than one source datastore to an interface.
These datastores must be linked using joins.
Two ways to create joins:
References in the models automatically become joins in the diagram.
Joins must be manually defined in the diagram for isolated datastores.
4-2

3. Manually Creating a Join
Drag and drop a column from one datastore onto a column in another datastore.
A join linking the two datastore appears in the diagram.
In the join code box, an expression joining the two columns also appears.
Modify the join expression to create the required relation.
You can use the expression editor.
Check the expression’s syntax if possible.
Test the join if possible.
4-3

4. Setting up a Join
Joins can be defined across technologies (here a database table and a flat file)
The number of joins per interface is not limited
SQL join expression (technology dependant)
Validate expression
Execution location
Expression editor
Join type
Inner/Outer, Left/Right.
Save expression
Join order (ISO-92 Syntax)
Use ISO-92 syntax
Automatically calculate order
4-4

5. Types of Joins The following type of joins exist: 4-5 Cross Join
Cartesian Product. Every combination of any Customer with any Order, without restriction.
Inner Join
Only records where a customer and an order are linked.
Left Outer Join
All the customers combined with any linked orders, or blanks if none.
Right Outer Join
All the orders combined with any linked customer, or blanks if none.
Full Outer Join
All customers and all orders.
4-5

6. Options for Filters, Joins and Mappings
Active Mapping
When unchecked, the filter, join or mapping is disabled for this interface
Enable mapping for update and/or insert
Allows mappings to only apply to updates or inserts.
By default, both insert and update are enabled
Choose the update key by selecting the Key checkbox
Change the execution location of the filter, join or mapping.
4-6

7. Setting Options for Filters, Joins and Mappings
Activate/Deactivate
For mappings, filters or joins
Execution Location
For mappings, filters or joins
Part of the Update Key
For target columns (mappings)
Active Mapping
When unchecked, the filter, join or mapping is disabled for this interface
Enable mapping for update and/or insert
Allows mappings to only apply to updates or inserts.
By default, both insert and update are enabled
Choose the update key by selecting the Key checkbox
Change the execution location of the filter, join or mapping.
Insert/Update
For mappings
4-7

8. Note – Update Keys for Flow Control
To perform Updates or use Flow Control, you must define an update key for the interface
4-8

9. What is an Update Key? An update key:
is a set of columns capable of uniquely identifying one row in the target datastore
is used for performing updates and flow control
can be:
one of the primary/unique keys defined for the datastore
defined specially for the interface
4-9

10. How to Define the Update Key
Go to the Diagram tab of the interface
Select the Target Datastore.
Select the Update Key in the properties panel.
To define a new key in the Interface only
Choose <Undefined> for the update key.
Select one target column to make part of the update key.
Check the Key checkbox in the properties panel.
Repeat for each column in the update key.
To define a new key for the table that could be used in other interfaces
Go back in the Model
Expand the table
Right-click on Constraints and add a new key (more on this in a later chapter)
4-10

11. How to Change the Execution Location
For mappings, filters and joins, you can choose where the operation will take place: source database, staging area or target database (mappings only, and for the mappings, only literals and database functions)
Go to the interface’s Diagram tab
Select the filter, join or mapping to edit.
Select an execution location from the properties panel.
Not every execution location is always possible.
Must be set to Active first.
4-11

12. Why Change the Execution Location?
You may need to change the execution location if:
The technology at the current location does not have the features required
Files, JMS, etc do not support transformations
A required function is not available
The current location is not available for processing
The machine can’t handle any more demand
ODI does not allow this location
It is not possible to execute transformations on the target.
4-12

13. What is the Flow?
Flow – The path taken by data from the sources to the target in an ODI interface.
The flow determines where and how data will be extracted, transformed, then integrated into the target.
4-13

14. What Defines the Flow? Three factors:
Where the staging area is located
On the target, on a source or on a third server
How mappings, filters and joins are set up
Execution location: Source, target or staging area
Whether transformations are “active”
Choice of Knowledge Modules
LKM: Loading Knowledge Module
IKM: Integration Knowledge Module
4-14

15. A Data Integration Scenario
Filter
- ORDERS.STATUS=‘CLOSED’ …
Mapping
- SALES = SUM(LINES.AMOUNT) + CORRECTION.VALUE.
- SALES_REP = ORDERS.SALES_REP_ID …
Source Sybase
ORDERS
LINES
Target Oracle
SALES
CORRECTIONS
File
Join
- ORDERS.ORDER_ID = LINES.ORDER_ID …
4-15

16. The Basic Process
Sequence of operations with or without an integration tool
Source: Sybase
ORDERS
LINES
Target: Oracle 5
Transform & Integrate
C$_0 1
Extract/Join/Transform
SALES
I$_SALES 3
Join/Transform
CORRECTIONS
File
C$_1 2
Extract/Transform
4-16

17. What Is the Staging Area?
Staging Area – A separate, dedicated area in an RDBMS where ODI creates its temporary objects and executes some of your transformation rules.
By default, ODI sets the staging area on the target data server.
4-17

18. Case Study: Placing the Staging Area
The Staging Area may be located:
On the target database (default).
On a third RDBMS database or the Sunopsis Memory Engine.
On the source database.
The Staging Area cannot be placed on non relational systems (Flat files, ESBs, etc.)
4-18

19. How to change the Staging Area
Go to the interface’s Definition tab of your Interface.
To choose the Staging Area, check the Staging Area Different From Target option, then select the logical schema that will be used as the Staging Area.
To leave the Staging area on the target, uncheck the Staging Area Different From Target option
Go to the Flow tab. You can now see the new flow.
4-19

20. Case #1 in ODI 4-20 Staging Area in the Target Source Sets
Staging Area + Target
This is how this arrangement looks in ODI.
First, you uncheck the Staging Area Different From Target checkbox.
Now, on the Flow tab you can see that the SALES target is in a box labelled Target+Staging Area (ORACLE). The two source servers are shown as Source Sets. A source set is a group of joined source datastores located in one server. Here, the Sybase server is a source set, as it contains the joined ORDERS and ORDERS_LINES source datastores. The file datastore is another source set, as it is stored on a different server.
4-20

21. Case #2 in ODI 4-21 Staging Area is the Sunopsis Memory Engine Target
Source Sets
Staging Area
Target
To create this kind of setup in ODI, first set the Staging Area Different From Target checkbox on the Definition tab. Then select a third server. As previously mentioned, we’re using the built-in Sunopsis Memory Engine.
On the Flow tab, we now see the three separate phases of the integration. You can see that the join between the two orders tables is performed on the source, Sybase, server.
The data from this these tables and the data from the corrections file are then loaded onto the staging area. These source sets, or temporary tables, are shown as the yellow cogs SS_0 and SS_1. The inner join is then performed on this staging area, which is shown as the box labelled Staging Area (SUNOPSIS_MEMORY_ENGINE).
Lastly, in the integration phase there is a cross-database copy to the Oracle target.
4-21

22. Case #3 in ODI 4-22 Staging Area in the Source Target
To create this scenario, set the Staging Area Different From Target checkbox, then select the source server from the drop down list. We see that the data from the file source is now being transferred to the Sybase server into the source set SS_0. However, there is no loading phase for the Orders tables, since they are already on the staging area. Hence they are displayed as datastores, rather than temporary tables. Unlike the previous situations, we now see that both the joins take place on the same server. As usual, there is a final integration phase onto the target.
Source Sets
4-22