1 EM413 Using Adaptive Server Anywhere’s Remote Data Access Feature Mark Culp Software Development Manager iAnywhere Solutions

2 Goals of this presentation Enable you to access remote data from your ASA database Show you how to migrate your data to ASA Help you improve your cross database join performance What should you not do when using ASA's remote data access feature

3 Contents What is “Remote Data Access” ? Basics – How to get started What if it doesn't work? – Some helpful hints Data migration tools Cross database joins Limitations – What to avoid Questions

4 What is “Remote Data Access” ?

5 Allows an ASA server to access data from a remote source as if it were local Remote data may be : iAnywhere ASA, Sybase ASE Oracle, IBM DB2, MS SQL Server Other ODBC data sources E.g. MS Access, MS Excel

6 Why use “Remote Data Access” ? Cross database joins Maintain real time access to remote data Integration to other projects and other data sources Migration of legacy data Quickly set up a remote ASA database for Mobilink

7 ASA Server ASA Store Query Execution Engine Client Comm. What is … - Architecture Remote Store Other ODBC ASE jConnect ASA Remote Server

8 What is … - Platforms Available on Win32 platforms: 95,98,ME,NT,2000 UNIX platforms (since ASA 7.0.2): Linux and SPARC Solaris Netware (since ASA 7.0.2)

9 What is … – Also known as … Remote Data Access capability is also known as: OMNI - OmniConnect (from ASE 10.5) CIS - Component Integration Services Proxy Tables

10 Basics – How to get started

11 Basics Three steps to accessing remote data 1.Define your remote server 2.Map your external login 3.Create your remote proxy tables Then use the proxy tables in the same way you would use any other table in your query

12 ASA needs to know: What you will call the remote server How to connect to the remote server The capabilities of the remote server Basics – Step 1: Define your remote server SQL statement: create server class using

13 Step 1 – Create server … class Connection class: JDBC (jConnect) classes: asajdbc, asejdbc ODBC classes: asaodbc, aseodbc – ASA and ASE db2odbc – IBM DB2 mssodbc – MS SQL Server oraodbc – Oracle server (version 8.0 or later) odbc – any other ODBC data source

14 Step 1 – Create server … location (JDBC) Remote location (JDBC): : [ / ] Location appended to JDBC URL specification: 'jdbc.sybase.TDS.' / only required if more than one database loaded on remote server

15 Step 1 – Create server … example (JDBC) Example 1 (using jConnect): create server DemoServerViaJconnect class 'asajdbc' using 'localhost:2638/asademo' jConnect asajdbc DemoServer ViaJconnect ASA server: asademo localhost :2638 /asademo Remote server:

16 Step 1 – Create server … location (ODBC) Remote location (ODBC): [ ; ] DSN (data source name) normally specifies parameters on how to find remote server Driver name Driver specific parameters: E.g. ServerName Connection is in context of the ASA server

17 Step 1 – Create server … location (ODBC) Example 2 (using an ODBC DSN): create server DemoServer class 'asaodbc' using 'demoDSN' DemoServer ASA server: asaodbc ODBC Driver Manager ODBC Driver demoDSN Note: You can use dbdsn to create ASA DSNs: dbdsn –w demoDSN –c “eng=asademo” asademo Eng= asademo Remote server:

18 Step 1 – Create server … location (ODBC) Example 3 (connect to an Excel Spreadsheet): create server ExcelData class 'odbc' using 'ExcelDataDSN' Note 'Read Only' Check Box

19 Step 1 – Create server … location (ODBC) Example 4 (no ODBC DSN): create server DemoServerNoDSN class 'asaodbc' using 'driver=Adaptive Server Anywhere 8.0; ServerName=asademo' DemoServer NoDSN ASA server: asaodbc ODBC Driver Manager ODBC Driver Driver=… asademo Server Name= asademo Remote server:

20 Step 1 – Create server … location (UNIX) Remote location (UNIX ODBC): [ ; driver= ] There is no standard ODBC driver manager on UNIX, so ASA needs to know where to find the driver (manager) that you want to use. You must specify a driver-path either: In your location string, or In your file DSN specification

21 Step 1 – Create server … location (UNIX) Example 5 (UNIX - no driver manager): create server DemoServerDirect class 'asaodbc' using 'demoDSN' Contents of ~/.odbc.ini: [demoDSN] driver=/opt/sybase/SYBSsa8/lib/dbodbc8_r.so servername=asademo

22 Step 1 – Create server … location (UNIX) Example 5 continued (UNIX - no driver manager): Since no "driver=" found in location string, ASA looks for location of driver in the DSN asademo DemoServer Direct asaodbcdemoDSN ASA server: ODBC Driver dbodbc8_r.so Remote server:

23 Step 1 – Create server … location (UNIX) Example 6 (UNIX - with driver manager): create server DemoServerUsingManager class 'asaodbc' using 'driver= ; DSN=demoDSN' Contents of ~/.odbc.ini: [demoDSN] driver=/opt/sybase/SYBSsa8/lib/dbodbc8_r.so servername=asademo

24 Step 1 – Create server … location (UNIX) Example 6 continued (with driver manager): ASA loads ODBC driver manager ODBC driver manager loads ODBC driver dbodbc8_r.so asademo DemoServer UsingManager asaodbc.odbc.ini driver=… ASA server: ODBC Driver Manager Server Name= asademo ODBC Driver Remote server:

25 Basics – Step 2: Map your external login ASA may need to know: User id and password to use when connecting to the remote server SQL statement: create externlogin to remote login [ identified by ]

26 Step 2 – When to create externlogin By default, ASA uses the names and passwords of its clients whenever it connects to a remote server on behalf of those clients. CREATE EXTERNLOGIN statement assigns an alternate login name and password to be used when communicating with a remote server

27 Step 2 – Map your external login Example 7: create externlogin dba to DemoServer remote login dba identified by sql

28 Basics – Step 3: Create proxy tables ASA needs to know how to make the mapping from a local proxy table to its remote data: If the remote table already exists, use the CREATE EXISTING statement. If the remote table is to be created, use the CREATE … AT statement.

29 Step 3 – Create existing table To map an existing remote table: SQL Statement: create existing table [.] [ ( ) ] at ; This statement defines the proxy table for an existing table on the remote server.

30 Step 3 – Location clause location_clause specifies where to locate the remote table, and is of the form: ' ; ; ; ' where: server - name of server as known by current database from create server command database- depends on type of remote server owner - owner of remote table table – name of the remote table

31 Step 3 – Create existing example Example 8: create existing table dba.p_employee at 'DemoServer;;dba;employee'; dba. p_employee ASA local database:DemoServer: dba. employee proxy table:remote table:

32 Step 3 – Create new table To create a new remote table: SQL Statement: create table [.] ( ) at ; This statement creates a new table on the remote server, and also defines the proxy table for that table.

33 Step 3 – Create new example Example 9: create table p_employee_sheet ( emp_id numeric(9,0), emp_fname varchar(30) ) at 'ExcelData;;;employee_sheet' Copy data into the remote table using: insert into p_employee_sheet select emp_id, emp_fname from p_employee

34 Some Helpful Hints

35 Hints – Dropping proxy tables To drop a proxy table, use: SQL Statement: drop table [.] ; If the proxy table was created using CREATE EXISTING, only the proxy table is dropped If the proxy table was created using CREATE … AT statement, then the proxy table and the remote table are both dropped.

36 Hints – Dropping proxy tables example Example 10: drop table p_employee Since this proxy table was created using CREATE EXISTING, this statement drops the proxy table only; the remote table is not affected. drop table p_employee_sheet Since this proxy table was created using CREATE TABLE … AT, this statement drops the proxy table, AND will also drop the remote table at 'ExcelData;;;employee_sheet'

37 Remote Data Access passthru mode To send a statement directly to the remote server, use: SQL statement: forward to ; forward to; The remote_statements text is sent to the remote server as given Text is terminated by the "forward" and "to" tokens

38 Hints – Useful system tables sys.sysservers – list of remote servers select * from sys.sysservers

39 Hints – Useful system tables sys.systable – list of proxy tables select table_name, remote_location, existing_obj from systable where remote_location is not NULL;

40 Hints – Useful built-in procedures sp_remote_tables server_name [, table_name ] [, table_owner ] [, table_qualifier ] [, with_table_type ] Lists tables available at remote server sp_remote_columns server_name, table_name [, owner ] [, database] Lists columns in remote table

41 Remote Data Access trace CIS option You diagnose remote data access problems by using the cis_option connection property SQL statement: set temporary option cis_option = 7; ASA will display messages in console log. Set cis_option value to 0 (zero) to turn off.

42 Trace CIS option example Example 11: set temporary option cis_option=7; select * from p_employee;

43 Data Migration Tools

44 Data migration tools - Availability You can import remote data using the sa_migrate set of stored procedures Available in ASA 7.0.2, and 8.0 Source data may be from one of: iAnywhere ASA Sybase ASE Oracle, IBM DB2, MS SQL Server, MS Access

45 Data migration tools - Basics Basic methodology: create base table(s) and proxy table(s) copy data from remote table(s) create foreign keys proxy table ASA local database:Remote data source: remote table base table

46 Migration – Single step process To migrate entire remote database in one step, use stored procedure: dbo.sa_migrate( IN local_table_ownerVARCHAR(128), IN server_nameVARCHAR(128), IN table_nameVARCHAR(128) DEFAULT NULL, IN owner_nameVARCHAR(128) DEFAULT NULL, IN database_nameVARCHAR(128) DEFAULT NULL, IN migrate_dataBIT DEFAULT 1, IN drop_proxy_tablesBIT DEFAULT 1 )

47 Migration – Single step example Example 12: call dbo.sa_migrate( 'dba', 'DemoServer', 'Customer', 'dba' ) This migrates the customer table definition and data from DemoServer to my database by: creating customer table in my database creating customer_et proxy table copying the customer data dropping the customer_et proxy table

48 Migration – Single step example results

49 Migration – Multi-step process To have more control over what is migrated, use: 1.sa_migrate_create_remote_table_list( … ) 2.sa_migrate_create_tables( … ) 3.sa_migrate_data( … ) 4.sa_migrate_create_remote_fks_list( … ) 5.sa_migrate_create_fks( … ) 6.sa_migrate_drop_proxy_tables( … )

50 Migration – Step 1: create table list 1. Create a list of tables to migrate using dbo.sa_migrate_create_remote_table_list( IN server_namevarchar(128), IN table_namevarchar(128) default NULL, IN owner_namevarchar(128) default NULL, IN database_namevarchar(128) default NULL ) this populates the dbo.migrate_remote_table_list table delete any rows that you do not want

51 Migration – Step 2: create proxy tables 2.Create proxy and base tables using: dbo.sa_migrate_create_tables( IN local_table_ownervarchar(128) ) uses rows in dbo.migrate_remote_table_list creates a proxy table and a base table for each remote table listed also creates all primary key indexes

52 Migration - Step 3: migrate data 3.Migrate data into base tables using: dbo.sa_migrate_data ( IN local_table_ownervarchar(128) ) uses rows in dbo.migrate_remote_table_list populates base tables with rows from proxy tables

53 Migration – Step 4: create fks list 4.Create list of foreign keys to migrate: dbo.sa_migrate_create_remote_fks_list( IN server_namevarchar(128) ) uses rows in dbo.migrate_remote_table_list creates dbo.migrate_remote_fks_list table delete any rows that you do not want

54 Migration – Step 5: create foreign keys 5.Create the foreign keys using: dbo.sa_migrate_create_fks( IN local_table_ownervarchar(128) ) uses rows in dbo.migrate_remote_fks_list creates a foreign key for each listed

55 Migration - Step 6: drop proxy tables 6.Drop all of the proxy tables: dbo.sa_migrate_drop_proxy_tables( IN local_table_ownervarchar(128) ) uses rows in dbo.migrate_remote_table_list drops the proxy table (created in step 2) for each remote table listed

56 Migration – Multi-step example Example 13: call dbo.sa_migrate_create_remote_table_list ( 'DemoServer', NULL, 'dba' ) select * from dbo.migrate_remote_table_list delete from dbo.migrate_remote_table_list where table_name = 'customer' or table_name like 'sales_%' or table_name like 'fin%' call dbo.sa_migrate_create_tables( 'dba' ) call dbo.sa_migrate_data( 'dba' ) call dbo.sa_migrate_create_remote_fks_list( 'DemoServer' ) select * from dbo.migrate_remote_fks_list call dbo.sa_migrate_create_fks( 'dba' ) call dbo.sa_migrate_drop_proxy_tables( 'dba' )

57 Migration – In case of a failure Migration may fail if a foreign key referenced table does not exist You may choose to update the dbo.migrate_remote_fks_list table (delete the offending entry) and continue Minimally you must run dbo.sa_migrate_drop_proxy_tables to delete the temporary proxy tables that were created

58 Cross Database Joins

59 Cross database joins – What is it? A "Cross Database Join" is a join between two or more tables where the tables reside on two or more different servers Table A (ASA) Join Table B (remote) Example A: Table A (remote a) Join Table B (remote b) Example B:

60 Cross database joins – Implementation ASA handles cross database joins by: Parse the query and identify remote tables Push predicates down to remote server when possible Reads remote rows using forward-only cursor Note: 6.0.x and 7.0.x uses only naïve nested loop join operator

61 Cross database joins – Remote Join When the join is on tables from the same server, the join is pushed to the remote Table A (remote a) Join Table B (remote a) Example C:

62 Cross database joins – Example 14 Example 14: R is remote table, S is ASA local table Select R.* from R join S where R.k = S.k For each row r in R if exists( select 1 from S where S.k = r.k ) match row

63 Cross database joins – Example 15

64 Cross database joins – Example 16

65 Cross Database Joins – Why is it slow? If ASA uses a plan that has the remote table on the right side of the join operator, then the remote data will be hauled across the communication link multiple times – once for each left hand side row. For each row s in S for each row r in ( select * from R ) if s.k = r.k match row

66 Cross Joins – How to make it go faster? Only include remote columns that are absolutely required by the query Run the query using dbisql, and check the plan that is being used Simplify query to have fewer remote tables Consider using local temporary tables to hold intermediate results Migrate data to ASA and perform join on local table

67 Limitations: What should you avoid ?

68 Limitations Features not supported on remote data: ALTER TABLE statement against remote tables Triggers defined on proxy tables will not fire SQL Remote Java data types Foreign keys that refer to remote tables are ignored Positioned UPDATE and DELETE UPDATE and DELETE requiring an intermediate temporary table Backwards scrolling on cursors

69 What to avoid … No cycles 1.Don't create a proxy table that refers back to the same database. Reason: Deadlock can occur if remote access waits for a lock caused by the same connection. Corollary: Don't create cycles in your databases' remote access definitions.

70 What to avoid … Separate Servers 2.Don't create a proxy table that refers back to the same server. Reason: Deadlock will occur when more than one concurrent remote access is attempted. Work around: increase –gx parameter Solution: Use separate servers to host the local and remote databases.

71 What to avoid … Join Large Tables 3.Don't join two large remote tables from different remote servers. Reason: Slow response Solution: Migrate one of the tables into an ASA table first.

72 What to avoid … N-way joins 4.Don't create complicated joins containing many remote tables from many different servers. Reason: Slow response Solution: Migrate table(s) into ASA first.

73 Summary ASA's Remote Data Access feature is a powerful tool to access your remote data from your ASA database: Easy to migrate your legacy data to ASA, or Use your remote data in place as if it were part of the ASA database More detailed information found in: ASA SQL User's Guide 15. Accessing Remote Data

74 Thank you Questions ?