1. How Autonomous is the Oracle Autonomous Data Warehouse?
Christian Antognini / Dani Schnider
@chrisantognini
antognini.ch/blog
@dani_schnider
danischnider.wordpress.com

2. Agenda Introduction Set Up Connectivity Loading Data ETL Performance
Query Performance
Monitoring Performance
Miscellaneous
Conclusion
How Autonomous is the Oracle Autonomous Data Warehouse?

3. Introduction
How Autonomous is the Oracle Autonomous Data Warehouse?

4. Larry Ellison, ATP Announcement, 7th August 2018
“There is nothing to learn, there is nothing to do”
How Autonomous is the Oracle Autonomous Data Warehouse?

5. Autonomous Data Warehouse Cloud
Automated Database Administration «…you do not need to configure or manage any hardware, or install any software. Autonomous Data Warehouse handles creating the data warehouse, backing up the database, patching and upgrading the database, and growing or shrinking the database.»
Automated Performance Tuning
«When you use Autonomous Data Warehouse, no tuning is necessary. You do not need to consider any details about parallelism, partitioning, indexing, or compression. The service automatically configures the database for high-performance queries.»
Source: Using Oracle Autonomous Data Warehouse, Chapter 1
How Autonomous is the Oracle Autonomous Data Warehouse?

6. Set Up
How Autonomous is the Oracle Autonomous Data Warehouse?

7. Creating Name Number of CPU (1..128) Storage (1..128 TB)
PDB admin password
License type
Tags (optional)
How Autonomous is the Oracle Autonomous Data Warehouse?

8. Scale Up/Down
At any moment, without downtime, it’s possible to scale up/down the number of CPU and the available storage
How Autonomous is the Oracle Autonomous Data Warehouse?
grow.html#GUID-4D3A72BB-C43A-490C-901C-496F82903C8C

9. Initialization Parameters That Can Be Modified
APPROX_FOR_AGGREGATION APPROX_FOR_COUNT_DISTINCT APPROX_FOR_PERCENTILE AWR_PDB_AUTOFLUSH_ENABLED1 OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES2 OPTIMIZER_IGNORE_HINTS OPTIMIZER_IGNORE_PARALLEL_HINTS
PLSCOPE_SETTINGS PLSQL_CCFLAGS PLSQL_DEBUG PLSQL_OPTIMIZE_LEVEL PLSQL_WARNINGS TIME_ZONE2 NLS_*
1System level only 2Session level only
How Autonomous is the Oracle Autonomous Data Warehouse?
database-users.html#GUID-7CF648C ED1-6F5719D6779E
alter session set approx_for_aggregation = FALSE;
alter session set approx_for_count_distinct = FALSE;
alter session set approx_for_percentile = none;
alter session set awr_pdb_autoflush_enabled = TRUE;
alter session set optimizer_capture_sql_plan_baselines = FALSE;
alter session set plscope_settings = 'IDENTIFIERS:NONE';
alter session set plsql_ccflags = '';
alter session set plsql_debug = FALSE;
alter session set plsql_optimize_level = 2;
alter session set plsql_warnings = 'DISABLE:ALL';
alter session set time_zone = '+5:0';
alter system set approx_for_aggregation = FALSE;
alter system set approx_for_count_distinct = FALSE;
alter system set approx_for_percentile = none;
alter system set awr_pdb_autoflush_enabled = TRUE;
alter system set optimizer_capture_sql_plan_baselines = FALSE;
alter system set plscope_settings = 'IDENTIFIERS:NONE';
alter system set plsql_ccflags = '';
alter system set plsql_debug = FALSE;
alter system set plsql_optimize_level = 2;
alter system set plsql_warnings = 'DISABLE:ALL';
alter system set time_zone = '+5:0';

10. Resource Manager
A plan (DWCS_PLAN) with three consumer groups is pre-configured
Consumer Group
CPU
Session Pool
PX Server Limit
DOP Limit
HIGH 4
Unlimited 50
CPU_COUNT1
MEDIUM 2 84
LOW 1
2*CPU_COUNT1
OTHER_GROUPS
1When several instances are used, CPU_COUNT <> “CPU Core Count”
How Autonomous is the Oracle Autonomous Data Warehouse?
with-oracle-database-resource-manager.html#GUID-3BDCC024-22DD-479D-A7C3- 92E7145C854F
SQL> col name format a12
SQL> show parameter cpu_count
SQL> select count(*) from gv$instance;
SQL> select group_or_subplan name, mgmt_p1, active_sess_pool_p1, parallel_server_limit, parallel_degree_limit_p1
2 from dba_rsrc_plan_directives
3 where plan = 'DWCS_PLAN';
NAME TYPE VALUE
cpu_count integer 1
COUNT(*) 1
NAME MGMT_P1 ACTIVE_SESS_POOL_P1 PARALLEL_SERVER_LIMIT PARALLEL_DEGREE_LIMIT_P1
HIGH
MEDIUM
LOW
OTHER_GROUPS
cpu_count integer 2
HIGH
LOW 2
cpu_count integer 4
HIGH
LOW
cpu_count integer 8
HIGH
LOW
cpu_count integer
HIGH
LOW
NAME TYPE VALUE
cpu_count integer 16
HIGH
LOW
cpu_count integer 24
HIGH
LOW

11. Resource Manager – Run Away Queries
Except for OTHER_GROUPS, two thresholds to configure when queries are cancelled can be set
Run time in seconds
Amount of disk I/O in MB
Remark:
0 = detection disabled
How Autonomous is the Oracle Autonomous Data Warehouse?

12. Connectivity
How Autonomous is the Oracle Autonomous Data Warehouse?

13. How to Connect to Autonomous Data Warehouse Cloud
Tool
Purpose
Connection Method
Service Console
service management
browser
Apache Zeppelin
Oracle Machine Learning
SQL Developer
development,
ad-hoc queries
JDBC (automatic configuration)
SQL*Plus, SQLcl, Toad, ...
OCI, JDBC, ODBC
(manual configuration)
ETL Tools (ODI, 3rd party)
data integration services
BI Tools (OBIEE, 3rd party)
BI services
How Autonomous is the Oracle Autonomous Data Warehouse?

14. Service Management via Browser
Used for
Service Console
Start/Stop DB
Scale up/down
Restore
Management of credential
How Autonomous is the Oracle Autonomous Data Warehouse?

15. Oracle Machine Learning
Notebook style application for advanced SQL users
Apache Zeppelin
Interactive data analysis
Graphical reports
Typical Users
Data scientists
Developers
Business users
How Autonomous is the Oracle Autonomous Data Warehouse?

16. Connection with SQL Developer
Download client credential file
Via Service Console
Create new connection
Type “Cloud PDB”
Import credential file
Enter keystore password (only for versions < 18.2)
Select service level (LOW, MEDIUM, HIGH)
How Autonomous is the Oracle Autonomous Data Warehouse?

17. Manual Connection Configuration
Connections using Oracle Net Services (SQL*Net)
For SQL*Plus, SQLcl, Toad, ...
ETL and BI tools, 3rd party tools
Oracle Cloud
Oracle Call Interface (OCI)
Wallet / Keystore
Wallet / Keystore
Encrypted SSL connection
Autonomous
Data Warehouse
Cloud
JDBC OCI
JDBC Thin Client
TCP/IP
ODBC
How Autonomous is the Oracle Autonomous Data Warehouse?

18. Loading Data
How Autonomous is the Oracle Autonomous Data Warehouse?

19. Loading Data Oracle Object Storage Oracle SQL*Loader
cloud
client computer
object storage service
Oracle Object Storage
Oracle SQL*Loader
SQL*Net
HTTPS / REST
Amazon Simple Storage Service (S3)
3rd party application
SQL*Net
Microsoft Azure Blob Storage
command-line interface
HTTPS / REST
How Autonomous is the Oracle Autonomous Data Warehouse?

20. Object Storage Service (OSS)
ADW supports the following object storage services
Oracle’s Object Storage
Amazon Simple Storage Service (S3)
Microsoft’s Azure Blob Storage
Those services store data as objects within buckets (containers)
Objects are identified with an URI
The access to the objects stored within buckets is protected
Credentials managed through DBMS_CLOUD are used
How Autonomous is the Oracle Autonomous Data Warehouse?
Objects are identified with an URI having the following format:
Oracle Cloud:
AWS:
Azure:

21. Command Line Interface (CLI)
Oracle provides a CLI to work with Oracle Cloud Infrastructure (OCI) objects and services The CLI is built on Python and runs on Windows, Linux and Mac The CLI makes calls to the OCI API Among other things, the CLI can be used to manage buckets and objects stored in the supported OSS
How Autonomous is the Oracle Autonomous Data Warehouse?
General info about the CLI:
Object storage service command reference: cli/latest/oci_cli_docs/cmdref/os.html

22. DBMS_CLOUD It provides features to manage credentials to access an OSS
manage external tables that can be used to query data stored in an OSS
handle objects stored in an OSS
handle files stored on ADW in DATA_PUMP_DIR
Data Pump 12.2 supports importing (but not exporting) a file stored in an OSS
ALTER DATABASE PROPERTY SET default_credential = 'ADMIN.CHRIS'
impdp dumpfile=default_credentials:
How Autonomous is the Oracle Autonomous Data Warehouse?
Official doc: cloud/user/dbmscloud-reference.html#GUID-52C9974C-D95E-4E41-AFBD-0FC4065C029A

23. Example – Loading Data into a 1TB TPC-DS Schema (1)
Step 1: Upload the text files from the file system of a server hosted in the cloud (same region) to the Oracle’s object storage
By default, such a statement uses 10 parallel threads
The load took 6 hours
Step 2: Load the data into the database through DBMS_CLOUD
The number of allocated CPU cores determines how fast data is loaded
With 32 cores it took 67 minutes
oci os object bulk-upload --bucket-name tpcds
--src-dir /data/tpcds
--include "*.dat"
How Autonomous is the Oracle Autonomous Data Warehouse?

24. Example – Loading Data into a 1TB TPC-DS Schema (2)
Example of scalability: load 22GB into the CATALOG_RETURNS table
source:
How Autonomous is the Oracle Autonomous Data Warehouse?

25. ETL Performance
How Autonomous is the Oracle Autonomous Data Warehouse?

26. Prepared for High ETL Performance
Good ETL performance is possible
Setup is optimized for DWH and ETL
But details must be considered
Parallel DML Execution
Direct-Path Load Operations
Online Statistics Gathering
No Indexes
Hybrid Columnar Compression
How Autonomous is the Oracle Autonomous Data Warehouse?

27. Parallel DML Execution and Direct-Path Operations
Parallel DML (PDML) is enabled by default
Only if CPU core count > 1 and consumer group is MEDIUM or HIGH
Hint /*+ parallel */ can be added (usually not needed)
Set optimizer_ignore_parallel_hints = FALSE (default: TRUE)
Direct-Path INSERT is used
For parallel DML
If hint /*+ append */ is added
How Autonomous is the Oracle Autonomous Data Warehouse?

28. Parallel DML Execution and Consumer Groups
| Id | Operation | Name | TQ |IN-OUT| PQ Distrib |
| 0 | INSERT STATEMENT | | | | |
| 1 | LOAD AS SELECT | CUSTOMERS_1 | | | |
| 2 | PX COORDINATOR | | | | |
| 3 | PX SEND QC (RANDOM) | :TQ | Q1,00 | P->S | QC (RAND) |
| 4 | OPTIMIZER STATISTICS GATHERING | | Q1,00 | PCWC | |
| 5 | PX BLOCK ITERATOR | | Q1,00 | PCWC | |
| 6 | TABLE ACCESS STORAGE FULL | CUSTOMERS | Q1,00 | PCWP | |
- automatic DOP: Computed Degree of Parallelism is 8
HIGH
MEDIUM
| Id | Operation | Name |
| 0 | INSERT STATEMENT | |
| 1 | LOAD TABLE CONVENTIONAL | CUSTOMERS_1 |
| 2 | TABLE ACCESS STORAGE FULL| CUSTOMERS |
- automatic DOP: Computed Degree of Parallelism is 1 because of no expensive parallel operation
- PDML disabled because object is not decorated with parallel clause
- Direct Load disabled because no append hint given and not executing in parallel
LOW
How Autonomous is the Oracle Autonomous Data Warehouse?

29. Parallel DML / Direct-Path and Constraints
| Id | Operation | Name |
| 0 | INSERT STATEMENT | |
| 1 | LOAD TABLE CONVENTIONAL | CUSTOMERS_1 |
| 2 | PX COORDINATOR | |
| 3 | PX SEND QC (RANDOM) | :TQ |
| 4 | PX BLOCK ITERATOR | |
| 5 | TABLE ACCESS STORAGE FULL| CUSTOMERS |
Note
-----
- automatic DOP: Computed Degree of Parallelism is 8
- PDML disabled because parent referential constraints
are present
Restrictions must be considered:
If FK constraints are defined, PDML / direct-path is disabled
Conventional load is used
Recommendation:
Define reliable constraints
ALTER TABLE customers_1
ADD FOREIGN KEY (country_id) REFERENCES countries
RELY DISABLE NOVALIDATE
How Autonomous is the Oracle Autonomous Data Warehouse?

30. Online Statistics Gathering for Direct-Path Loads
Statistics are gathered automatically
Unlike 12c, this works also
for non-empty tables
for histograms
Two new undocumented parameters
_optimizer_gather_stats_on_load_all (default: TRUE)
_optimizer_gather_stats_on_load_hist (default: TRUE)
| Id | Operation | Name |
| 0 | INSERT STATEMENT | |
| 1 | LOAD AS SELECT | TARGET |
| 2 | PX COORDINATOR | |
| 3 | PX SEND QC (RANDOM) | :TQ10000 |
| 4 | OPTIMIZER STATISTICS GATHERING | |
| 5 | PX BLOCK ITERATOR | |
| 6 | TABLE ACCESS STORAGE FULL | SOURCE |
How Autonomous is the Oracle Autonomous Data Warehouse?

31. Statistics Gathering for Conventional Loads
Attention: statistics are not gathered automatically Call DBMS_STATS with default values Automatic Statistics Gathering job is enabled, but maintenance windows are disabled
How Autonomous is the Oracle Autonomous Data Warehouse?

32. No Indexes Indexes are not allowed in ADW
Good! Avoids many ETL performance issues
Indexes are not needed for (most) analytical queries
Exception: indexes are created for primary key and unique constraints
CREATE INDEX s_order_item_delivery_date_idx ON s_order_item (delivery_date); ORA-01031: insufficient privileges
How Autonomous is the Oracle Autonomous Data Warehouse?
Official doc: cloud/user/dbmscloud-reference.html#GUID-52C9974C-D95E-4E41-AFBD-0FC4065C029A

33. Index Maintenance (PK and Unique Constraints)
Index maintenance after direct-path load
Very time-consuming, cannot be parallelized
Workaround: create PK with RELY DISABLE NOVALIDATE
How Autonomous is the Oracle Autonomous Data Warehouse?
The demo schemas in ADW have only disabled PK and FK constraints, or no constraints at all

34. Hybrid Columnar Compression
All tables are automatically HCC compressed, with row-level locking enabled
SELECT table_name, compression, compress_for
FROM user_tables ORDER BY table_name
TABLE_NAME COMPRESS COMPRESS_FOR
H_BEER ENABLED QUERY HIGH ROW LEVEL LOCKING
H_BOTTLING ENABLED QUERY HIGH ROW LEVEL LOCKING
H_BREW ENABLED QUERY HIGH ROW LEVEL LOCKING
H_CUSTOMER ENABLED QUERY HIGH ROW LEVEL LOCKING
H_MALT ENABLED QUERY HIGH ROW LEVEL LOCKING
H_ORDER ENABLED QUERY HIGH ROW LEVEL LOCKING …
How Autonomous is the Oracle Autonomous Data Warehouse?

35. Example: Loading Data Vault Schema
Data Vault Model of Craft Beer Brewery
25 target tables (Hubs, Links, Satellites)
20 beers, 272K customers
16M orders, 173M order items
“Daily loads” for 8 months
Random data generator
Data Vault load patterns
Delta detection and versioning
1.4M order items / day
How Autonomous is the Oracle Autonomous Data Warehouse?

36. Example: Loading Data Vault Schema
4 CPUs
8 CPUs 1 2
How Autonomous is the Oracle Autonomous Data Warehouse?

37. Query Performance
How Autonomous is the Oracle Autonomous Data Warehouse?

38. Result Cache It is enabled by default on ADW RESULT_CACHE_MODE = FORCE
For queries being re-executed, it can lead to a tremendous performance improvement
To avoid caching, use the NO_RESULT_CACHE hint
Even if OPTIMIZER_IGNORE_HINTS = TRUE (default on ADW)
How Autonomous is the Oracle Autonomous Data Warehouse?

39. Improve Query Performance
What cannot be done
Partition tables
Create materialized views
Use In-Memory Column Store
Create indexes (except for PK/UK)
What can be done
Scale up the number of CPU cores
Use different service
Use constraints to enable query transformations (e.g. join elimination)
How Autonomous is the Oracle Autonomous Data Warehouse?

40. How Are Queries on a Star Schema Optimized?
Since no index is automatically created, the star transformation is not available
Instead, the query optimizer can use the vector transformation
Introduced in for In-Memory Aggregation
In many situations, faster than star transformation
Works even if tables are not populated in IMCS
In-Memory is enabled (but not used) in ADW
INMEMORY_SIZE =
Note
-----
- vector transformation used for this statement
How Autonomous is the Oracle Autonomous Data Warehouse?
= 1 GB

41. Example – TPC-DS Queries
Service
How Autonomous is the Oracle Autonomous Data Warehouse?
Query 38
select /*+ no_result_cache */ * from (select i_product_name
,i_brand
,i_class
,i_category
,avg(inv_quantity_on_hand) qoh
from inventory
,date_dim
,item
where inv_date_sk=d_date_sk
and inv_item_sk=i_item_sk
and d_month_seq between 1184 and
group by rollup(i_product_name
,i_category)
order by qoh, i_product_name, i_brand, i_class, i_category
) where rownum <= 100;
I run the query 4 times for every combination of HIGH/MEDIUM/LOW and 2/4/8/16 CPU cores. The chart shows the fastest executions.
HIGH
2 CPU Cores: 174 / 175 / 178 / 178
4 CPU Cores: 90 / 93 / 94 / 93
8 CPU Cores: 55 / 55 / 57 / 55
16 CPU Cores: 31 / 32 / 31 / 31
MEDIUM
2 CPU Cores: 175 / 185 / 177 / 178
4 CPU Cores: 92 / 91 / 92 / 92
8 CPU Cores: 90 / 91 / 90 / 91
16 CPU Cores: 90 / 91 / 90 / 90
LOW
2 CPU Cores: 311 / 305 / 305 / 310
4 CPU Cores: 305 / 309 / 311 / 307
8 CPU Cores: 312 / 311 / 312 / 312
16 CPU Cores: 312 / 309 / 303 / 304

42. Example – Sample Star Schema Benchmark (SSB)
2 millions
2556
6 billions
2 millions
30 millions
How Autonomous is the Oracle Autonomous Data Warehouse?

43. Example – Sample Star Schema Benchmark (SSB)
How Autonomous is the Oracle Autonomous Data Warehouse?
Query 1
select sum(lo_extendedprice*lo_discount) as revenue
from ssb.lineorder, ssb.dwdate
where lo_orderdate = d_datekey
and d_yearmonthnum =
and lo_discount between 4 and 6
and lo_quantity between 26 and 35;

44. Example – Sample Star Schema Benchmark (SSB)
How Autonomous is the Oracle Autonomous Data Warehouse?
Query 14
select d_year, s_city, p_brand1, sum(lo_revenue - lo_supplycost) as profit
from ssb.dwdate, ssb.customer, ssb.supplier, ssb.part, ssb.lineorder
where lo_custkey = c_custkey
and lo_suppkey = s_suppkey
and lo_partkey = p_partkey
and lo_orderdate = d_datekey
and c_region = 'AMERICA'
and s_nation = 'UNITED STATES'
and (d_year = 1997 or d_year = 1998)
and p_category = 'MFGR#14'
group by d_year, s_city, p_brand1 order by d_year, s_city, p_brand1;

45. Automated Tuning? Result cache? Exadata storage indexes?
(in addition to disk I/O caching)
Exadata storage indexes?
Not observed 
source:
How Autonomous is the Oracle Autonomous Data Warehouse?

46. Monitoring Performance
How Autonomous is the Oracle Autonomous Data Warehouse?

47. Monitoring Performance and SQL Statements
Service Console with monitoring capabilities:
Activity and utilization
Storage and CPU usage
Real-time or time period
Running SQL statements
Time & wait statistics, I/O statistics
Runtime execution plan
Parallel processes
Downloadable real-time SQL Monitoring report
Default 8 days, can be changed with AWR settings
How Autonomous is the Oracle Autonomous Data Warehouse?

48. Ev. Live demo instad of screen shots
How Autonomous is the Oracle Autonomous Data Warehouse?
Ev. Live demo instad of screen shots

49. Monitoring Performance with AWR and ASH Reports
AWR reports can be generated (only on PDB level)
With DBMS_WORKLOAD_REPOSITORY
With SQL Developer reports
SELECT output FROM TABLE(
DBMS_WORKLOAD_REPOSITORY.AWR_REPORT_HTML ( , 7, 3524, 3535) )
ASH reports can be generated based on
V$ACTIVE_SESSION_HISTORY / DBA_HIST_ACTIVE_SESS_HISTORY
SQL Developer reports
How Autonomous is the Oracle Autonomous Data Warehouse?

50. Miscellaneous
How Autonomous is the Oracle Autonomous Data Warehouse?

51. Backup & Recovery Automatic (incremental) backups take place daily
The start time cannot be set
The retention period for automatic backups is 60 days
Manual backups can be initiated through the console
They are stored in the object store
The DEFAULT_CREDENTIAL and DEFAULT_BUCKETS database properties must be set
Recovery at any point-in-time can be initiated through the console
How Autonomous is the Oracle Autonomous Data Warehouse?
restore.html

52. Patching Oracle is patching the service on a regular basis
No announcement
No downtime
There is no way to “schedule” when the installation of the patches takes place
How Autonomous is the Oracle Autonomous Data Warehouse?

53. Oracle Support
As soon as a problem cannot be solve because of missing privileges, an SR has to be opened
Support has (almost) no visibility
E.g. no access to the alert.log of the PDB
E.g. no OS access
Support relies on Operations to fix things
The issues we experienced were fixed in a time frame going from one days to one week
How Autonomous is the Oracle Autonomous Data Warehouse?
SR : Cannot create new service because of duplicate database name (2 days)
SR : Directory object DATA_PUMP_DIR not found (1 week)
SR : After restore unable to access Data Pump export file (2 days)
SR : Cannot connect after recovery (2 days)

54. Conclusion
How Autonomous is the Oracle Autonomous Data Warehouse?

55. ✔ ✗ ! Conclusion Appropriate setup for DWH
Better than many manual configured DWHs
Easy (limited) administration, ready to use
No partitioning, no In-Memory (yet)
Support in case of a service request can take too long
Sometimes “shaky” (no control about patches / changes)
Not as simple as it looks at first sight
Knowhow about physical DB design still important ✗ !
How Autonomous is the Oracle Autonomous Data Warehouse?

56. Further Information in Blog Posts
DBMS_CLOUD Package – A Reference Guide
Which Privileges Are Required to Use the ADWC Service Console?
Observations About the Scalability of Data Loads in ADWC
External Tables in Autonomous Data Warehouse Cloud
Gathering Statistics in the Autonomous Data Warehouse Cloud
10 Tips to Improve ETL Performance – Revised for ADWC
Star Schema Optimization in Autonomous Data Warehouse Cloud
How Autonomous is the Oracle Autonomous Data Warehouse?

57. Questions and Answers…
Christian Antognini Dani Schnider
@chrisantognini
antognini.ch/blog
@dani_schnider
danischnider.wordpress.com
How Autonomous is the Oracle Autonomous Data Warehouse?