1. Oracle Advanced Queuing: An Overview
Hamid R. Minoui
Fritz, a UPS Company

2. Messaging & Queuing
An essential part of Distributed Application Development (DAD)
A key piece of a middle-ware known as Message-Oriented Middleware (MOM)
Motto: Every DAD needs a MOM

3. What does MOM provide?
Time-independent responses in a client/server environment
Help in passing information between servers and clients
A facility that allows general-purpose messages to be exchanged in a client/server system using message queues

4. Communication with Messages
Applications communicate over networks by putting messages in queues (ENQUEUE) and by getting messages from queues (DEQUEUE) using simple high-level API

5. Store-and-forward Communication
Characteristics and benefits
Less vulnerable to network, machine, and application failures.
No need for a real-time logical or dedicated connection
Connections can be established later with no loss
Applications can run independently at different speeds

6. Application Communication models
Two Application Models
Synchronous (online, connected)
Program sends a request to another program and waits for the reply
Asynchronous (deferred,disconnected)
Producer programs place messages in a queue and continue
Consumer programs retrieve messages (requests) from the queue and act on them

7. A sample synchronous model
A credit needs to be approved before a loan application can be processed
The load processing module awaits the completion of the credit approval model before it can do any work

8. A sample Asynchronous model
An order entry system places orders in appropriate order queues to be processed later. A complete order.
The order queues are read and handled one-at-a-time, in specific sequence , by the order processing systems

9. Oracle AQ Oracle Advance Queuing is Oracle’s message queuing facility
Available as of Oracle 8.0.3
First database-integrated messaging system in the industry

10. What AQ Addresses End-to-end Tracking Automatic Correlation
Automatic Auditing
Automatic Exception Handling
Message Warehousing
End-to-End Tracking Advanced Queuing can determine the current state and history o f every message. Every message carries with it the history of
where it has been.
Automatic Correlation Advanced Queuing automatically tracks the relationship between messages like Cause/Effect, Request/Reply, and Original/Copy relationships.
Automatic Auditing In many environments, the messages that flow between various applications represent important business transactions. It is required, and in some cases legally mandated, that every message be logged in its original form for auditing. Advanced Queuing provides automatic auditing of all messages.
Automatic Exception Handling
All messages that are not consumed by the specified recipients
within the specified time interval, are automatically moved to
the exception queue, where they will not get lost.
Message Warehousing As with data, messages, like data, accumulated over a period of time constitute a valuable asset. Message warehouses can be used for a variety of purposes like Business Process Analysis and Dispute Resolution.

11. Overview of AQ Features
Querying
Retention
Propagation
Exception Handling
Subscription List
Multiple Recipients
Time Specification
Statistics
Optional Transaction Protection
Priority & Ordering

12. SQL-based access SQL-based access
Messages are placed in normal rows in a database table.
They can be queried using standard SQL.
SQL can be used to access
Message properties
Message history
Payload
Indexes can be used to optimize access

13. Database-level Operational support
Standard database facilities: backup, recovery,export, import, OEM
AQ queues also benefit from high availability, scalability and reliability features of the Oracle server

14. Structured Payload
Object types can be used to structure and manage message content (or payload)
Support of strongly type content enables these features:
Content-based routing
Content-based subscription
Querying which enables message warehousing

15. Retention & History Retention Message History
AQ users can specify messages to be retained after consumption
Administrators can specify the retention period
Message History
History information like enqueue/dequeue time and the identification of transactions that executed each request is stored by AQ

16. Tracking & Journaling Retained messages can be related to each other
Users can track sequences of related messages and produce event journals automatically

17. Other Features Integrated Transactions
Management and development is simplified by the integration of control information and data payload
Queue Level Access Control (oracle 8i)
Queue level privileges can be granted or revoked by queue owners
DBAs can grant or revoke AQ system level privileges to database users
AQ administration can also be granted or revoked

18. Non-persistent Queues
AQ supports the delivery of non-persistent messages to subscribers
Persistent and non-persistent messages are supported with a common API
Persistent messages can be event-driven

19. AQ Administrator AQ Administrator created by DBA: Additional grants:
Create the user (aqadm)
Grant AQ_ADMINISTRATOR_ROLE
Grant connect, resource to aqadm;
Additional grants:
Execute on dbms_aqadm
Execute on dbms_aq

20. AQ User AQ_USER_ROLE granted by AQ Administrator
Has execute privilege on DBMS_AQ

21. Enqueue Options Correlation Identifier Subscription & Recipient Lists
Message Ordering & Prioritization
Propagation
Sender Identification
Message Grouping
Time specification & Scheduling
Rule-based Subscribers
Asynchronous Notification

22. Correlation Identifier
Users can assign an identifier to each message.
A specific enqueued message may be retrieved later by its identifier

23. Publish and Subscribe
Allows the establishment of a publish and subscribe mechanisms that include:
Rule-based subscription
Message propagation
Listening for incoming messages
Notification capabilities

24. Subscription & Recipient Lists
Multiple consumers can consume a single message from a queue.
A queue administrator can specify the list of subscribers who can retrieve messages from a queue
Different queues can have different subscribers

25. Subscribers & Recipients (more)
A consumer program can subscribe to more than one queue
Specific messages in a queue can be directed towards specific recipients who may not be in the subscribers list for the queue

26. Prioritization & Ordering
Enqueued messages can be assigned priority
The queue position of enqueued messages can be specified

27. Prioritizing & Ordering Options
Three Consuming Options
A sort order specifies properties used for ordering messages in a queue
A priority can be assigned to each message
A sequence deviation by allowing to position a message in relation to the others

28. Propagation Features
Allows coordination of ENQUEUE and DEQUEUE operations
Allows for location independence between the senders and recipients
Propagation agents automatically ENQUEUE from a local queue to another local or remote queue using database links

29. Starting Propagation Process
Set JOB_QUEUE_PROCESSES according to the number of queues involved in propagation
Set COMPATIBLE to or higher
Schedule propagation
DBMS_AQADM.SCHEDULE_PROPAGATION
Remove propagation schedules
DBMS_AQADM.UNSCHEDULE_PROPAGATION

30. Message Grouping
Messages for a queue can be grouped in the queue to form a set to be consumed by one user at a time
All messages in a group are created in one transaction
Queue table of the queue must be enabled for message grouping

31. Sender Identification
Applications can mark the messages they send
Oracle identifies the queue from which a message was dequeued
Applications can track the pathway of a propagated message

32. Time Specification & Scheduling
A enqueued message allows execution window by specifying:
Delay interval
Expiration time
A message is available after the delay and before the expiration

33. AQ_TM_PROCESSES An INIT.ORA parameter for managing the time:
In which messages are available for dequeuing
After which messages are expired
Values between 1 and 10 creates that many Queue Monitor background processes to monitor messages in queues
Processes are named ora_aqtm<oracle_sid>

34. Queue Monitor To start or stop the Queue Monitor
DBMS_AQADM.START_TIME_MANAGER
DBMS_AQADM.STOP_TIME_MANAGER

35. Rule-based subscribers
Users can subscribe to receive only messages of specified properties or contents
They define a rule-based subscription for a given queue to receive only messages of interest

36. Dequeue Options Multiple Recipients Local & Remote Recipients
Navigation of Messages in Dequeing
Dequeuing Modes
Dequeue Message header/ No Payload
Optimization of Waiting for the Arrival of Messages
Retries with Delays
Transaction Protection
Exception Handling
Wait on Multiple Queues

37. Multiple Recipients & Navigation
Multiple recipients can DEQUEUE the same message
Users can DEQUEUE in several ways:
Retrieve the first message
Establish a position & retrieve relative to that position
Select based on an order
Use the message identifier

38. DEQUEUE modes Browse Remove Locked
Will stay in the queue for more processing
Remove
No longer available for other dequeue requests
Locked
Messages are locked for other dequeuing request for the duration of the transaction

39. Fanning out Messages
Distribute messages to a large number of recipients.
Other queues would be recipients
Agents are defined as subscribers

40. Funneling In Messages
Concentrate messages for many queue to a single queue, also called composing
For example to get confirmation from a broadcast message

41. Basic AQ elements Message Queue Queue Table Agent Queue Monitor
Recipient List
Producer
Consumer
Message ID
Message group

42. Basis Steps Create a queue table
Create one or more queues in the queue table
Start the queue manager
Enqueue/dequeue to/from queue

43. Programming Interfaces
Users Programming Interface
DBMS_AQ
enqueue/dequeue
AQ Administrator Interface
DBMS_AQADM
Create/drop queue, queue table
Start/Stop queue manager

44. Message The smallest unit of work in the queue Contains
Metadata (control information)
Payload (supplied data)
Created by DBMS_AQ.ENQUEUE

45. Queue Data structure for messages Two types of queues can be created
Users Queues (normal queues)
Used by standard message processing
Exception Queues
Used by AQ for messages indicative of failed DEQUEUE attempts, or expired messages
Managed by DBMS_AQADM

46. Queue Table A database table that holds one or more queues
A queue table also contains a default exception queue

47. Simple AQ Examples Define a message object type
Create TYPE message_type AS OBJECT
(title VARCHAR2(30),
text VARCHAR2(2000));

48. Create A Queue Table AQADM creates a queue table called msg:
EXEC DBMS_AQADM.CREATE_QUEUE_TABLE
(queue_table => ‘msg’,
queue_payload_type => ‘message_type’);

49. Create & start a queue
AQADM creates a queue named msgqueue in msg and start it:
EXEC DBMS_AQADM.CREATE_QUEUE
(queue_name => ‘msgqueue’,
queue_table => ‘msg’;
EXEC DBMS_AQADM.START_QUEUE
(queue_name => ‘msgqueue’);

50. Message Properties Default message property record
TYPE DBMS_AQ.MESSAGE_PROPERTIES_T IS RECORD
(priority BINARY_INTEGER DEFAULT 1,
delay BINARY_INTEGER DEFAULT DBMS_AQ.NODELAY,
expiration BINARY_INTEGER DEFAULT DBMS_AQ.NEVER,
correlation VARCHAR2(128) DEFAULT NULL,
attempts BINARY_INTEGER,
recipient_list DBMS_AQ.AQ$_RECIPIENT_LIST_T,
exception_queue VARCHAR2(51) DEFAULT NULL,
enqueue_time DATE,
state BINARY_INTEGER);
Priority: Priority of the enqueued message. Smallest number => highest priority Delay: Delay of the enqueued message in seconds before being available for dequeue operation.
NO_DELAY = Immediately available
time unit = Place in WAITING state until delay expires then placed in READY state.
Requires QUEUE MONITOR be started.
Expiration: Specifies time in second when messages expires upon which it will no longer be available for dequeueing.
It is originally placed in the READY state. If it expires, it will be moved to the exception queue in the EXPIRED state.
Correlation: Free form identifier supplied by producer at enqueue.It will be identified when dequeuing.
Attempts: Number of attempts to dequeue. Maintained by AQ.
Recipient_list: List of agents.
Exception_queue: An exception queue other than the default.
Enqueue time: Time when enqueued available when dequeued.
State: State at dequeue time.
Values:
DBMS_AQ.WAITING => Delay has not been reached (value=1)
DBMS_AQ.READY => Ready(value=0)
DBMS_AQ.PROCESSED => Processed(value=3)
DBMS_AQ.EXPIRED => Moved to exception queue as expired (value=4)

51. Set message properties
Example code:
DECLARE
msg_prop DBMS_AQ.MESSAGE_PROPERIES_T;
BEGIN
msg_prop.priority := -100 /*high*/
msg_prop.delay:= 60*60*24 /* 1 day */
msg_prop.expiration:= 60*60 /* 1 hour after delay */
…..

52. Enqueue Options Record Type
Specify options associated with the message when you enqueue it
TYPE DBMS_AQ.ENQUEUE_OPTIONS_T IS RECORD
(visibility BINARY_INTEGER
DEFAULT DBMS_AQ.ON_COMMIT,
relative_msgid RAW(16)
DEFAULT NULL,
sequence_deviation BINARY_INTEGER
DEFAULT NULL);
Visibility: Transactional behavior of the enqueue request.
Possible values:
DBMS_AQ.ON_COMMIT: The enqueue is treated as part of the current transaction, completes only when transaction commits.
DBMS_AQ.IMMEDIATE: The enqueue is not part of the current transaction, enqueue acts as its own transaction.
Relative_msgid:
Valid is the next field is BEFORE. Specifies the message ID of the message referenced in the sequence deviation operation
Sequence_deviation
Specifies if message being enqueued should be dequeued before other messages already in the queue.
Valid options:
DBMS_AQ.BEFORE : Message is enqueued ahead of the message specified by relative_msgid
DBMS_AQ.TOP: Message is enqueued ahead of all others in the queue
NULL: No deviation

53. Set enqueue options Code example:
Have the message placed at the top of the queue and immediately visible.
DECLARE
queue_opts DBMS_AQ.ENQUEUE_OPTIONS_T;
BEGIN
queue_opts.visibility := DBMS_AQ.IMMEDIATE;
queue_opts.sequence_deviation := DBMS_AQ.TOP

54. ENQUEUE the message DECLARE queue_opts DBMS_AQ.ENQUEUE_OPTIONS_T;
msg_props DBMS_AQ.MESSAGE_PROPERTIES_T;
msg_id aq.msgid_type; /* set in a user package */
the_message message_type;
BEGIN
the_message := message_type (‘First msg’, ‘more to come..’);
DBMS_AQ.ENQUEUE (‘msgqueue’, queue_opts, msg_props,
the_message, msg_id);
END; /
CREATE OR REPLACE PACKAGE AQ IS
v_msgid RAW(16);
SUBTYPE msgid_type IS v_msgid%TYPE;

55. Change Message Properties & Options
Delay first message by 2 days
msg_props.delay := 2 * 60 * 60 * 24
Request one message to be dequeued before another
DECLARE msg_id1, msg_id2 := aq.msgid_type; ….
queue_opts.sequence_deviation := DBMS_AQ.BEFORE;
queue_opts.relative_msgid := msg_id1;
DBMS_AQ.ENQUEUE (‘msg_queue’, queue_opts, msg_props, another_message, msg_id2;

56. DEQUEUE Specification
PROCEDURE DBMS_AQ.DEQUEUE
(queue_name IN VARCHAR2,
dequeue_options IN DBMS_AQ.DEQUEUE_OPTIONS_T,
message_properties OUT DBMS_AQ.MESSAGE_PROPERTIES_T,
payload OUT <type_name>,
msgid OUT RAW);

57. DEQUEUE Options Record Type
TYPE DBMS_AQ.DEQUEUE_OPTIONS_T IS RECORD
(consumer_name VARCHAR2(30)
DEFAULT NULL,
dequeue_mode BINARY_INTEGER
DEFAULT DBMS_AQ.REMOVE,
navigation BINARY_INTEGER DEFAULT
DBMS_AQ.NEXT_MESSAGE,
visibility BINARY_INTEGER
DEFAULT DBMS_AQ.ON_COMMIT,
wait BINARY_INTEGER
DEFAULT DBMS_AQ.FOREVER,
msgid RAW(16) DEFAULT NULL,
correlation VARCHAR2 (128) DEFAULT NULL);
Consumer_name:
Name of the consumer of the message so that only the messages associated with the name are accessed only when the queue is setup for multiple consumer, otherwise NULL.
dequeue_mode:
Mode behavior associated with the dequeue operation.
Valid options:
DBMS_AQ.BROWSE – Read only, like a query “readers never block writers”
DBMS_AQ.LOCKED – Read and obtain a write lock on the message during transaction, like SELECT FOR UPDATE
DBMS_AQ.REMOVE – Read the message and update or delete after. Default.
Navigation:
Position of the message that will be retrieved next.
Steps after a dequeue:
The position of the message in the queue is determined
The search search criteria specified by this and other fields is applied
The appropriate message is retrieved
DBMS_AQ.NEXT_MESSAGE > Retrieve the next message in the queue that matches the search criteria
DBMS_AS.NEXT_TRANSACTION -> Skip the reminder of the current transaction group and retrieve the 1st in the next T group
DBMS_AS.FIRST_MESSAGE -> Retrieve the 1st available message matching the search criteria. Resets position to beginning of the queue.
Visibility:
Specifies whether the new message is dequeued as part of the current transaction, except for BROWSE mode.
Valid values:
DBMS_AQ.ON_COMMIT
DBMS_AQ.IMMEDIATE
Wait: No of seconds to wait for incoming messages matching the search criteria.
Values:
FOREVER => Default
NO_WAIT => No wait at all
any other values => seconds to wait
Correlation: Search pattern, correlation id. Follows SQL wildcard rule.

58. DEQUEUE the message Code example: DECLARE
queue_opts DBMS_AQ.ENQUEUE_OPTIONS_T;
msg_props DBMS_AQ.MESSAGE_PROPERTIES_T;
msg_id aq.msgid_type; /* set in a user package */
the_message message_type;
BEGIN
DBMS_AQ.DEQUEUE (‘msgqueue’, queue_opts, msg_props,
the_message, msg_id);
DBMS_OUPUT.PUT_LINE (‘Dequeued text: ’ ||
the_message.text);
END; /

59. Stopping & Dropping Operations
On QUEUES
DBMS_AQADM.STOP_QUEUE
DBMS_AQADM.DROP_QUEUE
On QUEUE TABLES
After all queues in the table are dropped
DBMS_AQADM.DROP_QUEUE_TABLE

60. Data Dictionary Views DBA_QUEUE_TABLES USER_QUEUE_TABLES DBA_QUEUES
USER_QUEUES
AQ$<queue_table>

61. AQ Statistics Views
Basic queue statistics are available via the V$AQ view
This view can be queried to see the number of messages in waiting, ready or expired state for each queue
Total and average wait time for all ‘READY’ messages in the queue is also recorded

62. The End of The Queue ANY QUESTIONS ????
This concludes the AQ overview presentation
Thank you for your attention
ANY QUESTIONS ????