Presentation is loading. Please wait.

Presentation is loading. Please wait.

Distance checker Make sure you see these select a value which is visible from last rows at 18 This is a positive highlighted value This is a negative highlighted.

Similar presentations


Presentation on theme: "Distance checker Make sure you see these select a value which is visible from last rows at 18 This is a positive highlighted value This is a negative highlighted."— Presentation transcript:

1 Distance checker Make sure you see these select a value which is visible from last rows at 18 This is a positive highlighted value This is a negative highlighted value select a value which is visible from last rows at 14 This is a positive highlighted value This is a negative highlighted value

2 Working with Automatic PGA All graphs are taken from the document SQL Memory Management in Oracle9i from Benoît Dageville and Mohamed Zait from Oracle Corporation Christo Kutrovsky The Pythian Group 2005 Feb

3 What is PGA ? Program Global Area i.e. Process Global Area Why is PGA different from SGA What goes into PGA Sort Hash PL/SQL variables and cursor memory

4 SGA Oracle memory model P1 P2 P3 P4 PGA SORT HASH PGA BIG SORT cursors pl/sql var. cursors pl/sql var. PGA Sort cursors pl/sql var. PGA Hash cursorspl/sql var.

5 Virtual memory mapping P1 P2 SGA 32/64 bit addressing space PGA Sort cursors pl/sql var. PGA Hash cursorspl/sql var. 0 gb1 gb2 gb3 gb4 gb

6 The need for more memory To make things go faster ! The fast = true parameter Not really More memory has different effects on sorts and hash joins

7 Effects on Sorting Sorting has ONLY 3 modes optimal or cache (all in memory) 1 pass (the entire result set is written once to disk) multi-pass (the entire result set is written multiple times to disk)

8 Sorting response time Response time Memory used Optimal 1 pass

9 Crossing optimal => 1 pass SQL> select count(*) from (select * from TBLSESSION t where rownum<= order by ses_ip); … Executed in seconds SQL> select count(*) from (select * from TBLSESSION t where rownum<= order by ses_ip); … Executed in seconds SQL> select count(*) from (select * from TBLSESSION t where rownum<= order by ses_ip); … Executed in seconds SQL> select count(*) from (select * from TBLSESSION t where rownum<= order by ses_ip); … Executed in seconds

10 Crossing optimal => 1 pass select substr(sql_text,57,15) as sql, operation_type as op, operation_id as id, policy, round(estimated_optimal_size/1024/1024,2) as e_opt, round(estimated_onepass_size/1024/1024,2) as e_one, round(last_memory_used/1024/1024,2) as l_mem, last_execution as last, total_executions as tot, optimal_executions as opt, onepass_executions as one, multipasses_executions as mult, round(active_time/ ,2) as sec, round(max_tempseg_size/1024/1024,2) as tmp_m, round(last_tempseg_size/1024/1024,2) as tmp_L from v$sql_workarea swa, v$sql sq where swa.address = sq.address and swa.hash_value = sq.hash_value and sql_text like 'select count(*) from ( select * from TBLSESSION% order by sql; SQL OP ID E_OPT E_ONE L_MEM LAST TOT OPT ONE MULT SEC TMP_M TMP_L <= SORT OPTIMAL <= SORT OPTIMAL <= SORT PASS <= SORT PASS

11 Other nuances of SORTs When sorting, you are sorting the ENTIRE result set. Thus you need as much memory as ALL your select columns So yes, by adding an extra field (even a fixed string) you could just pass the limit and switch to one-pass sort select * is even worse

12 Affecting sort memory needs Adding a fixed string: SQL> select count(*) from (select t.*,'extra' from TBLSESSION t where rownum<= order by ses_ip); … Executed in seconds Narrowing the select list SQL> select count(*) from (select ses_id, ses_start, ses_refkey from TBLSESSION t where rownum<= order by ses_ip); … Executed in seconds

13 Switching to multi-pass Multipass is a long way down the road, compared to optimal => 1 pass TYPE POLICY SID A_SEC WSIZE EXP ACT MAX PASSES TEMP SORT AUTO

14 Top N queries Remember top N queries? SQL> select count(*) from (select * from TBLSESSION t where rownum<= order by ses_ip ) where rownum <=1000; … Executed in seconds TYPE POLICY SID A_SEC WSIZE EXP ACT MAX PASSES TEMP SORT AUTO

15 Effect on HASH joins Hash joins have the same 3 modes: optimal (cache, in memory), 1 pass and multi-pass Hash joins benefit from additional memory between optimal and one-pass execution That is, according to Oracle, in my testing there was no measurable benefit

16

17 Nuances HASH joins If the optimizer estimates it will go above the bound, it will use temp right away The estimated value is re-learned from the last execution Thus a 2 nd execution, without any changes, may not use temp If the estimate is bypassed, then it will write to temp as needed, same way as a sort

18 Automatic PGA Now to the automatic word

19

20

21 Global memory manager Updates the global memory bound every 3 seconds Reacts to over allocation by lowering the bound further

22 Some facts Oracles SQL operators can adjust the memory consumption in mid-execution Sorts that switch from optimal to 1-pass will release the extra memory immediately (depending on disk speed) Background jobs processes will not free-up the virtual memory, if sufficient memory is available

23

24 Drift A drift of 10% is allowed When the limit is passed, a foreground bound recompute occurs

25

26 Monitoring PGA activity Overview Details Advisors

27 Monitoring - Overview SQL> select * from v$pgastat; NAME VALUE UNIT aggregate PGA target parameter bytes aggregate PGA auto target bytes global memory bound bytes total PGA inuse bytes total PGA allocated bytes maximum PGA allocated bytes total freeable PGA memory bytes PGA memory freed back to OS bytes total PGA used for auto workareas 0 bytes maximum PGA used for auto workareas bytes total PGA used for manual workareas 0 bytes maximum PGA used for manual workareas bytes over allocation count 0 bytes processed bytes extra bytes read/written bytes cache hit percentage percent

28 Monitoring - Overview – no bytes select name,round(value/1024/1024,1) as Mb from v$pgastat where unit = 'bytes'; NAME MB aggregate PGA target parameter 2000 aggregate PGA auto target global memory bound 100 total PGA inuse 22.5 total PGA allocated maximum PGA allocated total freeable PGA memory 4 PGA memory freed back to OS total PGA used for auto workareas 0 maximum PGA used for auto workareas total PGA used for manual workareas 0 maximum PGA used for manual workareas 0.5 bytes processed extra bytes read/written

29 Monitoring – getting the details 1 When the query is running v$sql_workarea_active select operation_type as type, policy, sid, round(active_time/ ,2) as a_sec, round(work_area_size/1024/1024,2) as wsize, round(expected_size/1024/1024,2) as exp, round(actual_mem_used/1024/1024,2) as act,round(max_mem_used/1024/1024,2) as max, number_passes as passes, round(tempseg_size/1024/1024,2) as temp from v$sql_workarea_active; TYPE POLICY SID A_SEC WSIZE EXP ACT MAX PASSES TEMP SORT AUTO

30 Monitoring – getting the details 2 When it has finished (or for another query) v$sql_workarea select operation_type as op, operation_id as id, policy, round(estimated_optimal_size/1024/1024,2) as e_opt, round(estimated_onepass_size/1024/1024,2) as e_one, round(last_memory_used/1024/1024,2) as l_mem, last_execution as last, total_executions as tot, optimal_executions as opt, onepass_executions as one, multipasses_executions as mult, round(active_time/ ,2) as sec, round(max_tempseg_size/1024/1024,2) as tmp_m, round(last_tempseg_size/1024/1024,2) as tmp_L from v$sql_workarea where address = ' E23CF0' and hash_value = ' '; OP ID POLICY E_OPT E_ONE L_MEM LAST TOT OPT ONE MULT SEC TMP_M TMP_L SORT 1 AUTO OPTIMAL HASH-JOIN 2 AUTO OPTIMAL HASH-JOIN 9 AUTO OPTIMAL

31 Monitoring – getting the details 3 You can link workareas to steps from the EX.PLAN v$sql_plan select rpad(' ', depth*3)||operation||' '||options||nvl2(object_name, ' -> ','')||object_name||decode(search_columns,0,NULL,' ('||search_columns||')') as OP,cost, cardinality as CARD, bytes, id as "id",access_predicates as "ACCESS", filter_predicates as filter,round(temp_space/1024/1024) as TMP_MB,partition_start ||nvl2(partition_start, ' - ', '')||partition_stop as P, partition_id, other, other_tag, cpu_cost, io_cost, distribution, object_owner, parent_id,optimizer from ( select * from V$SQL_PLAN where address = hextoraw(' E23CF0') and hash_value = ' ' and child_number = 0) t connect by prior id = parent_id start with id = 0 order by id, position; OP id ACCESS TMP_MB SELECT STATEMENT 0 SORT ORDER BY 1 HASH JOIN 2 "VID"="VID" NESTED LOOPS 3 TABLE ACCESS INDEX ROWID -> TAG_SYM 4 INDEX RANGE SCAN -> TS_VSID (1) 5 "VSID"=:1 TABLE ACCESS BY INDEX ROWID -> VEN_TAG 6 INDEX UNIQUE SCAN -> VT_TAG_PK (1) 7 "TAG"="TAG" VIEW 8 HASH JOIN 9 ROWID=ROWID INDEX FAST FULL SCAN -> VI_NAME_PK 10 INDEX FAST FULL SCAN -> VI_VID_UK 11

32 Monitoring – getting the details 4 Summaries – overall per workarea size v$sql_workarea_histogram SELECT LOW_OPTIMAL_SIZE/1024/1024 low_mb,(HIGH_OPTIMAL_SIZE+1)/1024/1024 high_mb, optimal_executions,onepass_executions,multipasses_executions FROM v$sql_workarea_histogram WHERE total_executions != 0 and (low_optimal_size/1024/1024 >= 8 or total_executions > optimal_executions) ; LOW_MB HIGH_MB OPTIMAL_EXECUTIONS ONEPASS_EXECUTIONS MULTIPASSES_EXECUTIONS

33 Monitoring – getting the details 5 Summaries – finding the bad ones select operation_type as op, operation_id as id, policy, round(estimated_optimal_size/1024/1024,2) as e_opt, round(estimated_onepass_size/1024/1024,2) as e_one, round(last_memory_used/1024/1024,2) as l_mem, last_execution as last, total_executions as tot, optimal_executions as opt, onepass_executions as one, multipasses_executions as mult, round(active_time/ ,2) as sec, round(max_tempseg_size/1024/1024,2) as tmp_m, round(last_tempseg_size/1024/1024,2) as tmp_L from v$sql_workarea where max_tempseg_size is not null; OP ID POLICY E_OPT E_ONE L_MEM LAST TOT OPT ONE MULT SEC TMP_M TMP_L HASH-JOIN 4 AUTO PASS

34 The Advisors – been really lazy Similar to v$db_cache_advice v$pga_target_advice Shows estimate temp IO usage v$pga_target_advice_histogram can be used to estimate opt/1pass/multi executions for different workarea sizes

35 v$pga_target_advice select round(pga_target_for_estimate/1024/1024) as est_mb,pga_target_factor as factor, round(bytes_processed/1024/1024) as p_mb,round(estd_extra_bytes_rw/1024/1024) as extra_mb, estd_pga_cache_hit_percentage as hit_ratio,estd_overalloc_count as est_over from v$pga_target_advice; EST_MB FACTOR P_MB EXTRA_MB HIT_RATIO EST_OVER …

36

37 To make it work And now what you need to make it work

38 Documented parameters pga_aggregate_target thats it

39 Before we continue The following slides will show undocumented parameters Do not use them, unless instructed by Oracle support Bugs you should be aware off

40 Patches/Bugs that are required corruption with auto PGA, allow more then 1gb per workarea the above 2 (as they conflict) Check with Oracle what is required for your patch level

41 Maximum workarea size The maximum PGA workarea is hard limited to 5% of pga_aggregate_target or a max of 100Mb Can be controlled with _smm_max_size Value is in KB !

42 Maximum PGA per process One process can have many workareas Max total size can be controlled with _pga_max_size Value is in bytes, default 200Mb CAUTION: virtual address space limitation on 32 bit platforms

43 So whats possible? Put really BIG sorts in memory ! A single workarea is still limited at 4gb In addition due to steps-allocations, its spilling much earlier, around 3.5 gb A process however is not limited to 4gb, at least on 64 bit system

44 The system Solaris with 64 bit Oracle Gb RAM 16 Gb SGA pga_aggregate_target = 8000m _pga_max_size = _smm_max_size =

45 Monitoring – The adjusted system SQL> select name,round(value/1024/1024,1) as Mb from v$pgastat where unit = 'bytes'; NAME MB aggregate PGA target parameter 8000 aggregate PGA auto target global memory bound total PGA inuse 67.4 total PGA allocated maximum PGA allocated total freeable PGA memory PGA memory freed back to OS total PGA used for auto workareas 0 maximum PGA used for auto workareas total PGA used for manual workareas 0 maximum PGA used for manual workareas 0.5 bytes processed extra bytes read/written

46 A big sort 1 - running select operation_type, policy, sid, round(active_time/ ,2) as a_sec, round(work_area_size/1024/1024,2) as wsize, round(expected_size/1024/1024,2) as exp, round(actual_mem_used/1024/1024,2) as act, round(max_mem_used/1024/1024,2) as max, number_passes as passes, round(tempseg_size/1024/1024,2) as temp from v$sql_workarea_active; OPE POLICY SID A_SEC WSIZE EXP ACT MAX PASSES TEMP SOR AUTO

47 A big sort 2 - spilling select operation_type, policy, sid, round(active_time/ ,2) as a_sec, round(work_area_size/1024/1024,2) as wsize, round(expected_size/1024/1024,2) as exp, round(actual_mem_used/1024/1024,2) as act, round(max_mem_used/1024/1024,2) as max, number_passes as passes, round(tempseg_size/1024/1024,2) as temp from v$sql_workarea_active; OPER POLICY SID A_SEC WSIZE EXP ACT MAX PASSES TEMP SORT AUTO

48 A big sort 3 - spilled (first fetch) select operation_type, policy, sid, round(active_time/ ,2) as a_sec, round(work_area_size/1024/1024,2) as wsize, round(expected_size/1024/1024,2) as exp, round(actual_mem_used/1024/1024,2) as act, round(max_mem_used/1024/1024,2) as max, number_passes as passes, round(tempseg_size/1024/1024,2) as temp from v$sql_workarea_active; OPER POLICY SID A_SEC WSIZE EXP ACT MAX PASSES TEMP SORT AUTO

49 Is it seriously using 3.5gb of ram? Oh yes, even more ! select round(pga_alloc_mem/1024/1024,1) as alloc_mb, round(pga_used_mem/1024/1024,1) as used_mb, round(pga_max_mem/1024/1024,1) as max_mb, pid, spid, serial#, program, traceid, background, pga_freeable_mem from v$process order by alloc_mb desc ALLOC_MB USED_MB MAX_MB SPID PROGRAM PGA_FREEABLE_MEM (TNS V1-V3) (DBW0) (TNS V1-V3) (TNS V1-V3) (NSV0) (TNS V1-V3) (TNS V1-V3)

50 Checking doubts on the OS side I dont believe that, thats just oracle telling us /home/oracle> pmap -xs 16291|head 16291: oracleprod (LOCAL=NO) Address Kbytes RSS Anon Locked Pgsz Mode Mapped File K r-x-- oracle r-x-- oracle … /home/oracle> pmap -xs 16291|tail 7FFFFFFF7FE K r-x-- ld.so.1 7FFFFFFF7FF2C K rwx-- ld.so.1 FFFFFFFF7FFE K rw--- [ stack ] total Kb The above values are in Kb. ~16 gb sga + ~5 gb anonymous memory, which is PGA = total 21 gb resident in use

51 Not used doesnt mean returned select round(pga_alloc_mem/1024/1024,1) as alloc_mb, round(pga_used_mem/1024/1024,1) as used_mb, round(pga_max_mem/1024/1024,1) as max_mb, pid, spid, username, serial#, program, round(pga_freeable_mem/1024/1024,1) as freeable_mb from v$process order by alloc_mb desc; ALLOC_MB USED_MB MAX_MB PID SPID USERNAME SERIAL# PROGRAM FREEABLE_MB oracle 2 … (TNS V1-V3) oracle 1 … (DBW0) oracle 10 … (TNS V1-V3) 0

52 Wrap-up – what can be achieved Use more memory for sorting/hashing Yet controlled automatically to avoid swapping Resulting in improved response time and less disk IO Resulting in more IO capacity available for other tasks Basically fast = true

53 The End Thank you, Questions? Christo Kutrovsky The Pythian Group 2005 Feb


Download ppt "Distance checker Make sure you see these select a value which is visible from last rows at 18 This is a positive highlighted value This is a negative highlighted."

Similar presentations


Ads by Google