1. The Burden of Proof Jonathan Lewis www.jlcomp.demon.co.uk

2. NoCOUG KeynoteThe Burden of Proof 2Jonathan Lewis © 2002-2003 Agenda Who am I ? Why do we need proof ? What is proof ? Constructing proofs Where next ? Q & A

3. NoCOUG KeynoteThe Burden of Proof 3Jonathan Lewis © 2002-2003 Who am I ? Independent Consultant. 18 years experience. Design, Strategy, Reviews, Briefings, Seminars, Tutorials, Trouble-shooting www.jlcomp.demon.co.uk

4. NoCOUG KeynoteThe Burden of Proof 4Jonathan Lewis © 2002-2003 Why do we need proof ? How does an architect design a house ? –Soil Characteristics –Material Strengths –Heating requirements –Illumination Why are so many Cathedrals standing after 900 years ? Because they are the second, third or fourth attempt. Industry Standard Formulae

6. NoCOUG KeynoteThe Burden of Proof 6Jonathan Lewis © 2002-2003 "Proof" by intuition C =  D C = 2  r  = 3.1415926 …..  3 C  6r To add 36 inches to the circumference, you have to add 6 inches to the radius.

7. NoCOUG KeynoteThe Burden of Proof 7Jonathan Lewis © 2002-2003 "Proof" by intuition Problem: One transaction takes 0.1 seconds, which is good The overnight batch takes 3 hours, which is bad. The "batch" emulated 100,000 transactions ! 100,000 * 0.1 = 10,000 seconds. 10,000 / 3,600 = 2 hours 47 minutes You have to 'fix' the transaction. The manual says: "redo log space requests should be close to zero".

8. NoCOUG KeynoteThe Burden of Proof 8Jonathan Lewis © 2002-2003 Skinner Boxes Which pig does all the work ?

9. NoCOUG KeynoteThe Burden of Proof 9Jonathan Lewis © 2002-2003 "Proof" by anthropomorphism Pigs are not people. The Oracle database engine is not a person. How would you handle this query: selectmax(partitioning_col) frompartitioned_table;

10. NoCOUG KeynoteThe Burden of Proof 10Jonathan Lewis © 2002-2003 "Proof" by anthropomorphism Quote from Metalink forum: I would have thought Oracle was smart enough to know that it just has to go to the top partition to get max(pt_col)." What if the top partition is empty ? Smart code for max() in indexes only appeared in 8.1. One day partition code might start from the top down. How will you discover that this new feature has appeared.

11. NoCOUG KeynoteThe Burden of Proof 11Jonathan Lewis © 2002-2003 "Proof" by self-deception X The truth is out there somewhere

12. NoCOUG KeynoteThe Burden of Proof 12Jonathan Lewis © 2002-2003 "Proof" by self-deception select count(*) from t1; select count(1) from t1; select count(pk_column) from t1; SORT (AGGREGATE) BITMAP CONVERSION (COUNT) BITMAP INDEX (FAST FULL SCAN) OF T1_BIT

13. NoCOUG KeynoteThe Burden of Proof 13Jonathan Lewis © 2002-2003 The Dodo argument

14. NoCOUG KeynoteThe Burden of Proof 14Jonathan Lewis © 2002-2003 An Oracle Dodo I had I/O problems and saw lots of sorts (disk) in v$sysstat. My I/O problems went away when I increased my sort_area_size. The Dodo says: If you have I/O problems and lots of sorts to disk, increase your sort_area_size. The scientist says: What I/O stopped happening, and why ?

15. NoCOUG KeynoteThe Burden of Proof 15Jonathan Lewis © 2002-2003 Proving or Testing Why do we need proofs ? Why do we test ? What's the difference ? We test for our own benefit. A proof is what we pass on to others. The difference is in the publication.

16. NoCOUG KeynoteThe Burden of Proof 16Jonathan Lewis © 2002-2003 Testing. Why do we test ? It avoids expensive mistakes. How do we test ? As cost-effectively as possible.

17. NoCOUG KeynoteThe Burden of Proof 17Jonathan Lewis © 2002-2003 Why do we test ? The Manuals Here be dragons To learn anything other than the stuff you find in books, you need to be able to experiment, to make mistakes, to accept feedback and to try again. Charles Handy

18. NoCOUG KeynoteThe Burden of Proof 18Jonathan Lewis © 2002-2003 Case Study (1) create table t1 ( idnumberprimary key, v1varchar2(20) ) organization index partition by range (id) subpartition by hash (id) ( partition p1 values less than (10) ); ORA-25198: only range partitioning...

19. NoCOUG KeynoteThe Burden of Proof 19Jonathan Lewis © 2002-2003 How do we test ? Don't even try to be realistic to start with - test the concept, not the detailed target. If you want to find out if 'feature X' works: imagine the simplest possible case where it is most likely to happen If you need 'feature X' to work: don't design a test to see if it works - imagine the simplest possible case that breaks it.

20. NoCOUG KeynoteThe Burden of Proof 20Jonathan Lewis © 2002-2003 Designing tests is hard Try drawing a 'random' triangle

21. NoCOUG KeynoteThe Burden of Proof 21Jonathan Lewis © 2002-2003 Case study (2) create view daily_sales_pv as select s.* from t_1998_01_01 s, dates d where s.day_code = d.day_code union all... union all select s.* from t_1998_12_13 s, dates d where s.day_code = d.day_code ; The designer thought this was a partition view, and tested it with 1,000 items of data. Response time was good.

22. NoCOUG KeynoteThe Burden of Proof 22Jonathan Lewis © 2002-2003 How do we test ? Know how to measure the results –sql_trace –tkprof –netstat –iostat –v$ snapshots Not the clock ! Know how to generate data cheaply –dbms_random –trunc(), mod(), sequences

23. NoCOUG KeynoteThe Burden of Proof 23Jonathan Lewis © 2002-2003 Proving or Testing A Test becomes a Proof when: it has a specification the results are reproducible alternative explanations have been eliminated it is published it survives peer-group review

24. NoCOUG KeynoteThe Burden of Proof 24Jonathan Lewis © 2002-2003 Example of test/proof Is memory 14,000 times faster than disk ? Assume disks can do 100 I/Os per sec So the hypothesis claims that you could do 1.4M logical I/Os per sec. in memory

25. NoCOUG KeynoteThe Burden of Proof 25Jonathan Lewis © 2002-2003 System used to test O/SWindows 2000 CPU Speed700 MHz Oracle 8.1.7.4 db_block_size8K create table t1 (n, v, primary key(n)) organization index as select rownum n, to_char(rownum) v from all_objects where rownum <= 21;

26. NoCOUG KeynoteThe Burden of Proof 26Jonathan Lewis © 2002-2003 Peer Group review declare m_v varchar2(32); begin for i in 1..140000 loop select v into m_v from t1 where n = 1; end loop; end; Consistent Gets = 140,000 The hypothesis predicts 0.01 CPU seconds Actual CPU used by this session = 16.18 seconds What's wrong with the test ?

27. NoCOUG KeynoteThe Burden of Proof 27Jonathan Lewis © 2002-2003 Second attempt select count(*) X from t1 connect by n > prior n start with n = 1; CR gets1,000,000 Buffer is pinned 500,000 Total1,500,000 The hypothesis predicts a little over 1 CPU second Actual CPU used by this session:15.08 seconds

28. NoCOUG KeynoteThe Burden of Proof 28Jonathan Lewis © 2002-2003 Warning: Is memory 14,000 times faster than disk ? (No) Does it matter ? Was it worth the effort of proving the point ? Is it a generally applicable observation ? Be careful about the line between pure research and practical investigation.

29. NoCOUG KeynoteThe Burden of Proof 29Jonathan Lewis © 2002-2003 Relevance declare m_v varchar2(32); begin for i in 1..140000 loop select v into m_v from t1 where n = 1; end loop; end; Does a small table need an index ? Should a small table BE an index (IOT) ? Or a single table hash cluster ?

30. NoCOUG KeynoteThe Burden of Proof 30Jonathan Lewis © 2002-2003 Quality Designing detailed tests is hard work. Covering all the options is difficult. The best results come from group efforts. Thorough proofs tend to be cumulative –but technology changes push proofs out of date

31. NoCOUG KeynoteThe Burden of Proof 31Jonathan Lewis © 2002-2003 Where next ? Always ask for some proof –Books and magazines –FAQs –User Group websites Questions to ask of authors: –How would you demonstrate that ? –Under what circumstances is this true ? –Do you have a sample script ?

32. NoCOUG KeynoteThe Burden of Proof 32Jonathan Lewis © 2002-2003 Where next ? Document your own tests Publish your findings - especially problems. –www.jlcomp.demon.co.uk/faq/ind_faq.html –www.dbazine.com Lobby Oracle Corp. for realistic examples –They are getting much better –There are still too many single table examples

33. NoCOUG KeynoteThe Burden of Proof 33Jonathan Lewis © 2002-2003 The Benefits You are not the first ! You can build on previous examples It enhances your skills and understanding You avoid problems earlier and earlier

34. NoCOUG KeynoteThe Burden of Proof 34Jonathan Lewis © 2002-2003 Final Thoughts When you have eliminated the impossible, whatever remains, however improbable, must be the truth. Sir Arthur Conan Doyle If I have seen further than others, it is because I was standing on the shoulders of the giants who went before. Sir Isaac Newton Please provide a reproducible test case. Oracle Support