Presentation is loading. Please wait.

Presentation is loading. Please wait.

Copyright 2000-2006 Steven Feuerstein - Page 1 OPP 2007 February 28 – March 1, 2007 San Mateo Marriott San Mateo, California An ODTUG SP* Oracle PL/SQL.

Similar presentations

Presentation on theme: "Copyright 2000-2006 Steven Feuerstein - Page 1 OPP 2007 February 28 – March 1, 2007 San Mateo Marriott San Mateo, California An ODTUG SP* Oracle PL/SQL."— Presentation transcript:

1 Copyright Steven Feuerstein - Page 1 OPP 2007 February 28 – March 1, 2007 San Mateo Marriott San Mateo, California An ODTUG SP* Oracle PL/SQL Programming Conference * SP – Seriously Practical Conference For more information visit or call ODTUG Kaleidoscope June 18 – 21, 2007 Pre-conference Hands-on Training - June 16 – 17 Hilton Daytona Beach Oceanfront Resort Daytona, Florida WOW-Wide Open World, Wide Open Web!

2 Copyright Steven Feuerstein - Page 2 Everything you need to know about collections, but were afraid to ask Steven Feuerstein PL/SQL Evangelist Quest Software

3 Copyright Steven Feuerstein - Page 3 Ten Years Writing Ten Books on the Oracle PL/SQL Language

4 Copyright Steven Feuerstein - Page 4 How to benefit most from this seminar Watch, listen, ask questions. Download the training materials and supporting scripts: – –"Demo zip": all the scripts I run in my class available at Use these materials as an accelerator as you venture into new territory and need to apply new techniques. Play games! Keep your brain fresh and active by mixing hard work with challenging games –MasterMind and Set ( filename_from_demo_zip.sql

5 Copyright Steven Feuerstein - Page 5 PL/SQL Collections Collections are single-dimensioned lists of information, similar to 3GL arrays. They are an invaluable data structure. –All PL/SQL developers should be very comfortable with collections and use them often. Collections take some getting used to. –They are not the most straightforward implementation of array-like structures. –Advanced features like string indexes and multi- level collections can be a challenge.

6 Copyright Steven Feuerstein - Page 6 What we will cover on collections Review of basic functionality Indexing collections by strings Working with collections of collections MULTISET operators for nested tables Then later in the section on SQL: –Bulk processing with FORALL and BULK COLLECT –Table functions and pipelined functions

7 Copyright Steven Feuerstein - Page 7 What is a collection? A collection is an "ordered group of elements, all of the same type." (PL/SQL User Guide and Reference) –That's a very general definition; lists, sets, arrays and similar data structures are all types of collections. –Each element of a collection may be addressed by a unique subscript, usually an integer but in some cases also a string. –Collections are single-dimensional, but you can create collections of collections to emulate multi-dimensional structures. abcdefsfqrrrswq

8 Copyright Steven Feuerstein - Page 8 Why use collections? Generally, to manipulate in-program-memory lists of information. –Much faster than working through SQL. Serve up complex datasets of information to non- PL/SQL host environments using table functions. Dramatically improve multi-row querying, inserting, updating and deleting the contents of tables. Combined with BULK COLLECT and FORALL.... Emulate bi-directional cursors, which are not yet supported within PL/SQL.

9 Copyright Steven Feuerstein - Page 9 Three Types of Collections Associative arrays (aka index-by tables) –Can be used only in PL/SQL blocks. –Similar to hash tables in other languages, allows you to access elements via arbitrary subscript values. Nested tables and Varrays –Can be used in PL/SQL blocks, but also can be the datatype of a column in a relational table. –Part of the object model in PL/SQL. –Required for some features, such as table functions –With Varrays, you specify a maximum number of elements in the collection, at time of definition.

10 Copyright Steven Feuerstein - Page 10 About Associative Arrays Unbounded, practically speaking. –Valid row numbers range from -2,147,483,647 to 2,147,483,647. –This range allows you to employ the row number as an intelligent key, such as the primary key or unique index value, because AAs also are: Sparse –Data does not have to be stored in consecutive rows, as is required in traditional 3GL arrays and VARRAYs. Index values can be integers or strings (Oracle9i R2 and above). assoc_array_example.sql

11 Copyright Steven Feuerstein - Page 11 About Nested Tables No pre-defined limit on a nested table. –Valid row numbers range from 1 to 2,147,483,647. Part of object model, requiring initialization. Is always dense initially, but can become sparse after deletes. Can be defined as a schema level type and used as a relational table column type. nested_table_example.sql

12 Copyright Steven Feuerstein - Page 12 About Varrays Has a maximum size, associated with its type. –Can adjust the size at runtime in Oracle10g R2. Part of object model, requiring initialization. Is always dense; you can only remove elements from the end of a varray. Can be defined as a schema level type and used as a relational table column type. varray_example.sql

13 Copyright Steven Feuerstein - Page 13 How to choose your collection type Use associative arrays when you need to... –Work within PL/SQL code only –Sparsely fill and manipulate the collection –Take advantage of negative index values Use nested tables when you need to... –Access the collection inside SQL (table functions, columns in tables) –Want to perform set operations Use varrays when you need to... –If you need to specify a maximum size to your collection –Access the collection inside SQL (table functions, columns in tables).

14 Copyright Steven Feuerstein - Page 14 Wide Variety of Collection Methods Obtain information about the collection –COUNT returns number of rows currently defined in collection. –EXISTS returns TRUE if the specified row is defined. –FIRST/LAST return lowest/highest numbers of defined rows. –NEXT/PRIOR return the closest defined row after/before the specified row. –LIMIT tells you the max. number of elements allowed in a VARRAY. Modify the contents of the collection –DELETE deletes one or more rows from the index-by table. –EXTEND adds rows to a nested table or VARRAY. –TRIM removes rows from a VARRAY.

15 Copyright Steven Feuerstein - Page 15 Useful reminders for PL/SQL collections Memory for collections comes out of the PGA or Process Global Area –One per session, so a program using collections can consume a large amount of memory. Use the NOCOPY hint to reduce overhead of passing collections in and out of program units. Encapsulate or hide details of collection management. Don't always fill collections sequentially. Think about how you need to manipulate the contents. Try to read a row that doesn't exist, and Oracle raises NO_DATA_FOUND. mysess.pkg sess2.sql nocopy*.*

16 Copyright Steven Feuerstein - Page 16 Function PGA Data Caching with PL/SQL Tables First access Subsequent accesses PGA Function Database Not in cache; Request data from database Pass Data to Cache Application Requests Data Data retrieved from cache Data returned to application Application Requests Data Data returned to application Data retrieved from cache Database Data found in cache. Database is not needed. emplu.pkg emplu.tst

17 Copyright Steven Feuerstein - Page 17 New indexing capabilities for associative arrays Prior to Oracle9iR2, you could only index by BINARY_INTEGER. You can now define the index on your associative array to be: –Any sub-type derived from BINARY_INTEGER –VARCHAR2(n), where n is between 1 and –%TYPE against a database column that is consistent with the above rules –A SUBTYPE against any of the above. This means that you can now index on string values! (and concatenated indexes and...) Oracle9i Release 2

18 Copyright Steven Feuerstein - Page 18 Examples of New TYPE Variants All of the following are now valid TYPE declarations in Oracle9i Release 2 –You cannot use %TYPE against an INTEGER column, because INTEGER is not a subtype of BINARY_INTEGER. DECLARE TYPE array_t1 IS TABLE OF NUMBER INDEX BY BINARY_INTEGER; TYPE array_t2 IS TABLE OF NUMBER INDEX BY PLS_INTEGER; TYPE array_t3 IS TABLE OF NUMBER INDEX BY POSITIVE; TYPE array_t4 IS TABLE OF NUMBER INDEX BY NATURAL; TYPE array_t5 IS TABLE OF NUMBER INDEX BY VARCHAR2(64); TYPE array_t6 IS TABLE OF NUMBER INDEX BY VARCHAR2(32767); TYPE array_t7 IS TABLE OF NUMBER INDEX BY employee.last_name%TYPE; TYPE array_t8 IS TABLE OF NUMBER INDEX BY types_pkg.subtype_t; Oracle9i Release 2

19 Copyright Steven Feuerstein - Page 19 Working with string-indexed collections Specifying a row via a string takes some getting used to, but if offers some very powerful advantages. DECLARE TYPE population_type IS TABLE OF NUMBER INDEX BY VARCHAR2(64); country_population population_type; continent_population population_type; howmany NUMBER; BEGIN country_population ('Greenland') := ; country_population ('Iceland') := ; howmany := country_population ('Greenland'); continent_population ('Australia') := ; END; assoc_array*.sql assoc_array_perf.tst

20 Copyright Steven Feuerstein - Page 20 Ra p id access to data via strings One of the most powerful applications of this features is to construct very fast pathways to static data from within PL/SQL programs. –If you are repeatedly querying the same data from the database, why not cache it in your PGA inside collections? Emulate the various indexing mechanisms (primary key, unique indexes) with collections. Demonstration package: assoc_array5.sql Comparison of performance of different approaches: vocab*.* Generate a caching package: genaa.sql genaa.tst

21 Copyright Steven Feuerstein - Page 21 The String Tracker package (V1) Another example: I need to keep track of the names of variables that I have already used in my test code generation. –Can't declare the same variable twice. CREATE OR REPLACE PACKAGE BODY string_tracker IS TYPE used_aat IS TABLE OF BOOLEAN INDEX BY maxvarchar2_t; g_names_used used_aat; FUNCTION string_in_use ( value_in IN maxvarchar2_t ) RETURN BOOLEAN IS BEGIN RETURN g_names_used.EXISTS ( value_in ); END string_in_use; PROCEDURE mark_as_used (value_in IN maxvarchar2_t) IS BEGIN g_names_used ( value_in ) := TRUE; END mark_as_used; END string_tracker; string_tracker1.*

22 Copyright Steven Feuerstein - Page 22 Multi-level Collections Prior to Oracle9i, you could have collections of records or objects, but only if all fields were scalars. –A collection containing another collection was not allowed. Now you can create collections that contain other collections and complex types. –Applies to all three types of collections. The syntax is non-intuitive and resulting code can be quite complex. Oracle9i

23 Copyright Steven Feuerstein - Page 23 String Tracker Version 2 The problem with String Tracker V1 is that it only supports a single list of strings. –What if I need to track multiple lists simultaneously or nested? Let's extend the first version to support multiple lists by using a string-indexed, multi- level collection. –A list of lists....

24 Copyright Steven Feuerstein - Page 24 The String Tracker package (V2) CREATE OR REPLACE PACKAGE BODY string_tracker IS TYPE used_aat IS TABLE OF BOOLEAN INDEX BY maxvarchar2_t; TYPE list_of_lists_aat IS TABLE OF used_aat INDEX BY maxvarchar2_t; g_list_of_lists list_of_lists_aat; PROCEDURE mark_as_used ( list_in IN maxvarchar2_t, value_in IN maxvarchar2_t, case_sensitive_in IN BOOLEAN DEFAULT FALSE ) IS l_name maxvarchar2_t := CASE case_sensitive_in WHEN TRUE THEN value_in ELSE UPPER ( value_in ) END; BEGIN g_list_of_lists ( list_in ) ( l_name) := TRUE; END mark_as_used; END string_tracker; string_tracker2.*

25 Copyright Steven Feuerstein - Page 25 Other multi-level collection examples Multi-level collections with intermediate records and objects. Emulation of multi-dimensional arrays –No native support, but can creates nested collections to get much the same effect. –Use the UTL_NLA package (10gR2) for complex matrix manipulation. Four-level nested collection used to track arguments for a program unit. –Automatically analyze ambiguous overloading. multidim*.* ambig_overloading.sql OTN: OverloadCheck multilevel_collections.sql

26 Copyright Steven Feuerstein - Page 26 Encapsulate these complex structures! When working with multi-level collections, you can easily and rapidly arrive at completely unreadable and un-maintainable code. What' s a developer to do? –Hide complexity -- and all data structures -- behind small modules. –Work with and through functions to retrieve contents and procedures to set contents. cc_smartargs.pkb: cc_smartargs.next_overloading cc_smartargs.add_new_parameter

27 Copyright Steven Feuerstein - Page 27 Nested Tables unveil their MULTISET-edness Oracle10g introduces high-level set operations on nested tables (only). –Nested tables are multisets, meaning that theoretically there is no order to their elements. This makes set operations of critical importance for manipulating nested tables.. You can now… –Check for equality and inequality –Perform UNION, INTERSECT and MINUS operations –Check for and remove duplicates Oracle10g

28 Copyright Steven Feuerstein - Page 28 Check for equality and inequality Just use the basic operators…. Oracle10g DECLARE TYPE clientele IS TABLE OF VARCHAR2 (64); group1 clientele := clientele ('Customer 1', 'Customer 2'); group2 clientele := clientele ('Customer 1', 'Customer 3'); group3 clientele := clientele ('Customer 3', 'Customer 1'); BEGIN IF group1 = group2 THEN DBMS_OUTPUT.put_line ('Group 1 = Group 2'); ELSE DBMS_OUTPUT.put_line ('Group 1 != Group 2'); END IF; IF group2 != group3 THEN DBMS_OUTPUT.put_line ('Group 2 != Group 3'); ELSE DBMS_OUTPUT.put_line ('Group 2 = Group 3'); END IF; END; 10g_compare.sql 10g_compare2.sql 10g_compare_old.sql

29 Copyright Steven Feuerstein - Page 29 UNION, INTERSECT, MINUS Straightforward, with the MULTISET keyword. Oracle10g BEGIN our_favorites := my_favorites MULTISET UNION dad_favorites; show_favorites ('MINE then DAD', our_favorites); our_favorites := dad_favorites MULTISET UNION my_favorites; show_favorites ('DAD then MINE', our_favorites); our_favorites := my_favorites MULTISET UNION DISTINCT dad_favorites; show_favorites ('MINE then DAD with DISTINCT', our_favorites); our_favorites := my_favorites MULTISET INTERSECT dad_favorites; show_favorites ('IN COMMON', our_favorites); our_favorites := dad_favorites MULTISET EXCEPT my_favorites; show_favorites ('ONLY DAD''S', our_favorites); END; 10g_setops.sql 10g_string_nt.sql 10g_favorites.sql 10g*union*.sql

30 Copyright Steven Feuerstein - Page 30 Turbo-charged SQL with BULK COLLECT and FORALL Improve the performance of multi-row SQL operations by an order of magnitude or more with bulk/array processing in PL/SQL! CREATE OR REPLACE PROCEDURE upd_for_dept ( dept_in IN employee.department_id%TYPE,newsal_in IN employee.salary%TYPE) IS CURSOR emp_cur IS SELECT employee_id,salary,hire_date FROM employee WHERE department_id = dept_in; BEGIN FOR rec IN emp_cur LOOP UPDATE employee SET salary = newsal_in WHERE employee_id = rec.employee_id; END LOOP; END upd_for_dept; Conventional binds (and lots of them!)

31 Copyright Steven Feuerstein - Page 31 Oracle server PL/SQL Runtime Engine SQL Engine PL/SQL block Procedural statement executor SQL statement executor FOR rec IN emp_cur LOOP UPDATE employee SET salary =... WHERE employee_id = rec.employee_id; END LOOP; Performance penalty for many context switches Conventional Bind

32 Copyright Steven Feuerstein - Page 32 Enter the Bulk Bind: FORALL Oracle server PL/SQL Runtime Engine SQL Engine PL/SQL block Procedural statement executor SQL statement executor FORALL indx IN list_of_emps.FIRST.. list_of_emps.LAST UPDATE employee SET salary =... WHERE employee_id = list_of_emps(indx); Much less overhead for context switching

33 Copyright Steven Feuerstein - Page 33 Use the FORALL Bulk Bind Statement Instead of executing repetitive, individual DML statements, you can write your code like this: Things to be aware of: –You MUST know how to use collections to use this feature! –Only a single DML statement is allowed per FORALL. –New cursor attributes: SQL%BULK_ROWCOUNT returns number of rows affected by each row in array. SQL%BULK_EXCEPTIONS... –Prior to Oracle10g, the binding array must be sequentially filled. –Use SAVE EXCEPTIONS to continue past errors. PROCEDURE upd_for_dept (...) IS BEGIN FORALL indx IN list_of_emps.FIRST.. list_of_emps.LAST UPDATE employee SET salary = newsal_in WHERE employee_id = list_of_emps (indx); END; bulktiming.sql bulk_rowcount.sql

34 Copyright Steven Feuerstein - Page 34 Use BULK COLLECT INTO for Queries DECLARE TYPE employees_aat IS TABLE OF employees%ROWTYPE INDEX BY BINARY_INTEGER; l_employees employees_aat; BEGIN SELECT * BULK COLLECT INTO l_employees FROM employees; FOR indx IN 1.. l_employees.COUNT LOOP process_employee (l_employees(indx)); END LOOP; END; bulkcoll.sql Declare a collection of records to hold the queried data. Use BULK COLLECT to retrieve all rows. Iterate through the collection contents with a loop.

35 Copyright Steven Feuerstein - Page 35 Limit the number of rows returned by BULK COLLECT CREATE OR REPLACE PROCEDURE bulk_with_limit (deptno_in IN dept.deptno%TYPE) IS CURSOR emps_in_dept_cur IS SELECT * FROM emp WHERE deptno = deptno_in; TYPE emp_tt IS TABLE OF emps_in_dept_cur%ROWTYPE; emps emp_tt; BEGIN OPEN emps_in_dept_cur; LOOP FETCH emps_in_dept_cur BULK COLLECT INTO emps LIMIT 100; EXIT WHEN emps.COUNT = 0; process_emps (emps); END LOOP; END bulk_with_limit; Use the LIMIT clause with the INTO to manage the amount of memory used with the BULK COLLECT operation. WARNING! BULK COLLECT will not raise NO_DATA_FOUND if no rows are found. Best to check contents of collection to confirm that something was retrieved. bulklimit.sql

36 Copyright Steven Feuerstein - Page 36 Tips and Fine Points Use bulk binds in these circumstances: –Recurring SQL statement in PL/SQL loop. Oracle recommended threshold: five rows! Bulk bind rules: –Can be used with any kind of collection; Collection subscripts cannot be expressions; The collections must be densely filled (pre-10gR2). Bulk collects: –Can be used with implicit and explicit cursors –Collection is always filled sequentially, starting at row 1. emplu.pkg cfl_to_bulk*.*

37 Copyright Steven Feuerstein - Page 37 The Wonder Of Table Functions A table function is a function that you can call in the FROM clause of a query, and have it be treated as if it were a relational table. Table functions allow you to perform arbitrarily complex transformations of data and then make that data available through a query. –Not everything can be done in SQL. Combined with REF CURSORs, you can now more easily transfer data from within PL/SQL to host environments. –Java, for example, works very smoothly with cursor variables

38 Copyright Steven Feuerstein - Page 38 Building a table function A table function must return a nested table or varray based on a schema-defined type, or type defined in a PL/SQL package. The function header and the way it is called must be SQL-compatible: all parameters use SQL types; no named notation. –In some cases (streaming and pipelined functions), the IN parameter must be a cursor variable -- a query result set.

39 Copyright Steven Feuerstein - Page 39 Simple table function example Return a list of names as a nested table, and then call that function in the FROM clause. CREATE OR REPLACE FUNCTION lotsa_names ( base_name_in IN VARCHAR2, count_in IN INTEGER ) RETURN names_nt IS retval names_nt := names_nt (); BEGIN retval.EXTEND (count_in); FOR indx IN 1.. count_in LOOP retval (indx) := base_name_in || ' ' || indx; END LOOP; RETURN retval; END lotsa_names; tabfunc_scalar.sql SELECT column_value FROM TABLE ( lotsa_names ('Steven', 100)) names; COLUMN_VALUE Steven 1... Steven 100

40 Copyright Steven Feuerstein - Page 40 Streaming data with table functions You can use table functions to "stream" data through several stages within a single SQL statement. –Example: transform one row in the stocktable to two rows in the tickertable. CREATE TABLE stocktable ( ticker VARCHAR2(20), trade_date DATE, open_price NUMBER, close_price NUMBER ) / CREATE TABLE tickertable ( ticker VARCHAR2(20), pricedate DATE, pricetype VARCHAR2(1), price NUMBER) / tabfunc_streaming.sql

41 Copyright Steven Feuerstein - Page 41 Streaming data with table functions - 2 In this example, transform each row of the stocktable into two rows in the tickertable. CREATE OR REPLACE PACKAGE refcur_pkg IS TYPE refcur_t IS REF CURSOR RETURN stocktable%ROWTYPE; END refcur_pkg; / CREATE OR REPLACE FUNCTION stockpivot (dataset refcur_pkg.refcur_t) RETURN tickertypeset... BEGIN INSERT INTO tickertable SELECT * FROM TABLE (stockpivot (CURSOR (SELECT * FROM stocktable))); END; / tabfunc_streaming.sql

42 Copyright Steven Feuerstein - Page 42 Use pipelined functions to enhance performance. Pipelined functions allow you to return data iteratively, asynchronous to termination of the function. –As data is produced within the function, it is passed back to the calling process/query. Pipelined functions can be defined to support parallel execution. –Iterative data processing allows multiple processes to work on that data simultaneously. CREATE FUNCTION StockPivot (p refcur_pkg.refcur_t) RETURN TickerTypeSet PIPELINED

43 Copyright Steven Feuerstein - Page 43 Applications for pipelined functions Execution functions in parallel. –In Oracle9i Database Release 2 and above, use the PARALLEL_ENABLE clause to allow your pipelined function to participate fully in a parallelized query. –Critical in data warehouse applications. Improve speed of delivery of data to web pages. –Use a pipelined function to "serve up" data to the webpage and allow users to being viewing and browsing, even before the function has finished retrieving all of the data.

44 Copyright Steven Feuerstein - Page 44 Piping rows out from a pipelined function CREATE FUNCTION stockpivot (p refcur_pkg.refcur_t) RETURN tickertypeset PIPELINED IS out_rec tickertype := tickertype (NULL, NULL, NULL); in_rec p%ROWTYPE; BEGIN LOOP FETCH p INTO in_rec; EXIT WHEN p%NOTFOUND; out_rec.ticker := in_rec.ticker; out_rec.pricetype := 'O'; out_rec.price := in_rec.openprice; PIPE ROW (out_rec); END LOOP; CLOSE p; RETURN; END; tabfunc_setup.sql tabfunc_pipelined.sql Add PIPELINED keyword to header Pipe a row of data back to calling block or query RETURN...nothing at all!

45 Copyright Steven Feuerstein - Page 45 Enabling Parallel Execution The table function's parameter list must consist only of a single strongly-typed REF CURSOR. Include the PARALLEL_ENABLE hint in the program header. –Choose a partition option that specifies how the function's execution should be partitioned. –"ANY" means that the results are independent of the order in which the function receives the input rows (through the REF CURSOR). {[ORDER | CLUSTER] BY column_list} PARALLEL_ENABLE ({PARTITION p BY [ANY | (HASH | RANGE) column_list]} )

46 Copyright Steven Feuerstein - Page 46 Table functions – Summary Table functions offer significant new flexibility for PL/SQL developers. Consider using them when you... –Need to pass back complex result sets of data through the SQL layer (a query); –Want to call a user defined function inside a query and execute it as part of a parallel query.

47 Copyright Steven Feuerstein - Page 47 Collections – don't start coding without them. It is impossible to write modern PL/SQL code, taking full advantage of new features, unless you use collections. –From array processing to table functions, collections are required. Today I offer this challenge: learn collections thoroughly and apply them throughout your backend code. –Your code will get faster and in many cases much simpler than it might have been (though not always!).

48 Copyright Steven Feuerstein - Page 48 OPP 2007 February 28 – March 1, 2007 San Mateo Marriott San Mateo, California An ODTUG SP* Oracle PL/SQL Programming Conference * SP – Seriously Practical Conference For more information visit or call ODTUG Kaleidoscope June 18 – 21, 2007 Pre-conference Hands-on Training - June 16 – 17 Hilton Daytona Beach Oceanfront Resort Daytona, Florida WOW-Wide Open World, Wide Open Web!

Download ppt "Copyright 2000-2006 Steven Feuerstein - Page 1 OPP 2007 February 28 – March 1, 2007 San Mateo Marriott San Mateo, California An ODTUG SP* Oracle PL/SQL."

Similar presentations

Ads by Google