1. Data Warehousing Group Teradata Tools & Utilities
Last Updated : 27th April 2004
Center of Excellence
Data Warehousing Group
Teradata Tools & Utilities

2. Teradata Tools & Utilities
Utilities exploits the capabilities provided by Teradata parallel architecture.
All Teradata utilities communicate with Teradata database via Call Level Interface (CLI).
CLI can establish multiple sessions with DBS to send multiple task to Teradata at the same time.
Utilities are supported on Mainframe as well as Workstation platforms.

3. Teradata Load/Unload Utilities
BTEQ
Fastload
Multiload
FastExport
Tpump

4. Client – Server Connectivity
Parser Engine
Message Passing Layer
AMP
Vdisk
Gateway S/W
Ethernet Adapter
TDP
CLI
Client Req
MTDP
Client Req.
MOSI
Channel Adapter
Bus Adapter
Channel
attached
system
Network

5. Bteq For submitting SQL requests to Teradata database.
Also used widely for reporting purposes.
Can read input records from flat file and apply DMLs (INSERT, UPDATE, DELETE) to the Teradata database.
Can also export data to client system from Teradata database.

6. Data Import Using Bteq
Bteq .IMPORT can be used to load data from a client system to Teradata Database.
{DATA}
.IMPORT {INDICDATA} FILE = data.dat;
{REPORT}
{VARTEXT}
USING col1(integer), col2(char(10)
INSERT INTO table1 VALUES(:col1, :col2);
Use REPEAT * if you want all input records to be inserted.

7. Data Export Using Bteq
Bteq .EXPORT command can be used to export data records from Teradata database to a client system.
{DATA}
.EXPORT {INDICDATA} FILE = data.dat;
{REPORT}
SELECT * from table1 where col1 = 10;
.EXPORT RESET;

8. Multiple Sessions
Multiple sessions can be established by a client utility to perform multiple tasks in parallel.
Multiple sessions are useful for only those types access which use row hash locking (UPI,NUPI,USI).
Avoid establishing multiple sessions where these is a full table lock is involved.

9. FastLoad Utility High speed data load tool for large amount of data.
Can load an empty table only.
It has restart capability.
Data can be loaded in multiple stages.
Can not load a table with Secondary Indexes, RIs and Join Indexes defined.
INMOD exit routines can be used to load from external data sources.

10. FastLoad Utility Phase 1 Establishes the sessions for FastLoad task.
Host sends data blocks through the sessions.
PEs receive the data blocks and directly send them to AMPs.
AMPs redistributes the data rows to their target AMPs.
AMPs writes the blocks (unsorted) to the disk.

11. FastLoad Utility Phase 2
Phase 2 start when END LOADING command received.
AMPs read their blocks and sort them based on row hash sequence.
Write the sorted data blocks back to the disk.
Fallback copy of the data is created.

12. FastLoad Utility AMPs Data R1 PE R2 R3 Vdisk R4 R5 R6 R7 M R8 P R9 L

13. Example LOGON tdpid/userid,password; BEGIN LOADING Customers
ERRORFILES Custerr1, Custerr2;
DEFINE Cust_Id (INTEGER)
,Cust_Name (VARCHAR(25))
FILE = custdata.dat;
INSERT INTO Customers VALUES
(:Cust_Id,
:Cust_Name);
END LOADING;
LOGOFF;
Input data definition
Phase 1 Starts
Phase 2 Starts

14. FastLoad Error Tables FastLoad uses two error tables.
Error table 1 captures the information of the rows which can not be loaded due to data conversion errors.
Error Table 2 captures the information of the rows which can not be applied due to duplicate UPIs.
FastLoad discards empty error tables after successful completion of load task.

15. Restartability
If CHECKPOINT option is used, a FastLoad job can be restarted from the last checkpoint.
No need to re-run the entire FastLoad job.
As taking checkpoint is a time consuming task it recommended to have bigger checkpoint intervals for larger tables.
By default Fastload take checkpoint at every records.

16. Loading Multiple Data Files
LOGON tdpid/userid,password;
BEGIN LOADING Customers
ERRORFILES Custerr1, Custerr2;
DEFINE Cust_Id (INTEGER)
,Cust_Name (VARCHAR(25))
FILE = custdata1.dat;
INSERT INTO Customers VALUES
(:Cust_Id,
:Cust_Name);
END LOADING;
LOGOFF;
LOGON tdpid/userid,password;
BEGIN LOADING Customers
ERRORFILES Custerr1, Custerr2;
DEFINE Cust_Id (INTEGER)
,Cust_Name (VARCHAR(25))
FILE = custdata2.dat;
INSERT INTO Customers VALUES
(:Cust_Id,
:Cust_Name);
END LOADING;
LOGOFF;

17. FastLoad Limitations Table should be empty. Only INSERT allowed.
Can not load table with secondary Indexes defined.
Loads only one table in one job task.

18. Multiload Used for high volume data maintenance
Supports INSERTs, UPDATEs and DELETEs.
Loads up to five tables in a single job.
Tables may not necessarily be empty.
Block level data maintenance.
Input data rows can be applied conditionally to DML requests.
Supports UPSERTs.
Can not load tables with USI and RI defined.
Tables with NUSI can be loaded using Multiload.
INMOD exit routines can be used to load from external data sources.

19. Advantage of Multiload over normal Inserts/Updates
Input data buffers are sent directly to AMPs which minimizes PE utilization.
All AMPs works in parallel to apply the change.
No transient journal and permanent journal overhead.
Update up to 5 tables in one Multiload Task.

20. Example .LOGTABLE Logtable001_ml; .LOGON tdp3/user2,tyler;
.BEGIN MLOAD TABLES Employee, Employee_History;
.LAYOUT Employee_Trans;
.FILLER in_Transcode 1 CHAR(3);
.FIELD in_EmpNo * SMALLINT;
.FIELD in_DeptNo * SMALLINT;
.FIELD in_Salary * DECIMAL (8,2);
.DML LABEL Payroll DO INSERT FOR MISSING UPDATE ROWS ;
UPDATE Employee SET Salary = :in_Salary
WHERE EmpNo = :in_EmpNo;
INSERT INTO Employee (EmpNo, Salary)
VALUES (:in_EmpNo, :in_Salary);
.DML LABEL Terminate ;
DELETE FROM Employee WHERE EmpNo = :in_EmpNo;
INSERT INTO Employee_History (EmpNo, DeptNo)
VALUES (:in_EmpNo, :in_DeptNo);
.IMPORT INFILE infile1
LAYOUT Employee_Trans
APPLY Payroll WHERE in_Transcode = 'PAY'
APPLY Terminate WHERE in_Transcode = 'DEL';
.END MLOAD;
.LOGOFF;

21. Two Distinct Multiload Tasks
Import Task
Multiple combinations of INSERTs/UPDATEs/DELETEs/UPSERTs are supported.
Where condition must be an equality condition on primary Index.
Delete Task
Can delete large number of rows in parallel.
No equality condition on Primary Index value.

22. Phases of Multiload Import Task
Preliminary Phase
Start Multiload sessions
Create temporary tables.
Apply utility locks (disallow DDLs) on the tables.
DML Transaction Phase
Send DML requests, USING clause and Match-Tags to server.
Acquisition Phase
Client sends data buffers to server
AMPs receives the data blocks are redistributes (by row hash) each data records to its target AMPs.
Data rows are accumulated in work tables of corresponding target table.
At the end of acquisition phase it puts write locks on the target tables.

23. Phases of Multiload Import Task
Application Phase
Apply work table changes to target table.
Clean-up Phase
Release locks
Drop empty error tables, work table and log table.
Report load statistics
Logoff sessions.

24. Restartability
Multiload reinitiated automatically after a database recovery.
Multiload job terminated due to a client failure can be resubmitted to continue from the point of failure.
Multiload uses Checkpoint and log table mechanism for restart recovery.
No user interaction is required.

25. FastExport
To export large volume of data from Teradata database to client.
Takes advantage of multiple sessions.
Fastexport sorts data rows before sending them through sessions.
Can be used with INMOD/OUTMOD feature for external data sources and destinations.
Restartable feature supported.

26. Example .LOGTABLE sample_log; .LOGON tdpid/username,password;
.BEGIN EXPORT SESSIONS 4;
.EXPORT OUTFILE out.dat;
SELECT * FROM employees
WHERE loc_name = ‘MGRD’
ORDER BY 1;
.END EXPORT;
.LOGOFF;
.LOGTABLE sample_log;
.LOGON tdpid/username,password;
.BEGIN EXPORT SESSIONS 4;
.EXPORT OUTMOD out.dat;
SELECT * FROM employees
WHERE loc_name = ‘MGRD’
ORDER BY 1;
.END EXPORT;
.LOGOFF;

27. Tpump
Allows near real-time updates from transactional systems into the warehouse.
Best fit for low volume data maintenance.
Unlike Multiload, Tpump uses row hash lock allows concurrent updates on the same table.
INSERT,UPDATE, DELETE supported.
No restrictions applied for tables with SI, RI, triggers etc.
No limit on number of concurrent sessions.
Speed can be tuned dynamically.

28. Example .LOGTABLE Logtable001_ml; .LOGON tdp3/user2,tyler;
.BEGIN LOAD SESSIONS 4 ERRORTABLE ET_Employee;
.LAYOUT Employee_Trans;
.FILLER in_Transcode 1 CHAR(3);
.FIELD in_EmpNo * SMALLINT;
.FIELD in_DeptNo * SMALLINT;
.FIELD in_Salary * DECIMAL (8,2);
.DML LABEL Payroll DO INSERT FOR MISSING UPDATE ROWS ;
UPDATE Employee SET Salary = :in_Salary
WHERE EmpNo = :in_EmpNo;
INSERT INTO Employee (EmpNo, Salary)
VALUES (:in_EmpNo, :in_Salary);
.DML LABEL Terminate ;
DELETE FROM Employee WHERE EmpNo = :in_EmpNo;
INSERT INTO Employee_History (EmpNo, DeptNo)
VALUES (:in_EmpNo, :in_DeptNo);
.IMPORT INFILE infile1
LAYOUT Employee_Trans
APPLY Payroll WHERE in_Transcode = 'PAY'
APPLY Terminate WHERE in_Transcode = 'DEL';
.END LOAD;
.LOGOFF;

29. Teradata Warehouse Builder
Parallel Load/Unload environment that reduces the cost of building and maintaining your warehouse.
Load data from multiple and dissimilar sources simultaneously
Read, transform and load a table in a single job
Extensible environment allow users to integrate custom transformations through open APIs
Jobs restart at the last checkpoint if a DBS reset is encountered
Executes multiple instances of operators
Eliminates the need of intermediate storage.

30. TWB Infrastructure TWB Teradata Warehouse Builder Processor Script
Teradata Warehouse Builder Operator Interface
INMOD
Routine
OUTMOD
Access
Modules
Inmod
Adapter
Operator
Filter
Data
Source/
Destination
Files,
Devices,
Databases
TWB
Script
Teradata
Database
Outmod
Adapter
Operator
Consumer
Operator
Data
Connector
Operator
Producer
Operator

31. Data Modes and Data Formats
Indicators Mode
Data Formats
FastLoad Format
Binary Format
Unformatted
Text
Vartext
Data
Indic bits Data
2 Bytes Data OA/0D
Data OA/0D
Data
Datadatadatadatadata (Fixed length)
Data|data|data|data|data ( Variable length)

32. Questions ?