2. Bringing Internals to the Surface Internals Concepts for Highly Effective Engines Mark Scranton Principal Consultant/Trainer Informix Denver www.markscranton.com

3. user.conference Informix 2 Who Am I? Principal Consultant/Trainer, Informix Denver I do engines - XPS and IDS 5+ yrs with Informix website - www.markscranton.com tips/tricks sql/scripts documents, presentations Principal Consultant/Trainer, Informix Denver I do engines - XPS and IDS 5+ yrs with Informix website - www.markscranton.com tips/tricks sql/scripts documents, presentations

4. user.conference Informix 3 Presentation Overview This presentation will identify “internals” concepts that appear to be trivia But they could be useful daily for maintenance and support of IDS A strong working knowledge of IDS is assumed More info available in the “IDS Internal Architecture” class This presentation will identify “internals” concepts that appear to be trivia But they could be useful daily for maintenance and support of IDS A strong working knowledge of IDS is assumed More info available in the “IDS Internal Architecture” class

5. user.conference Informix 4 Topics Physical pages vs. logical pages? Discussion about the value of knowing the difference The dog has swallowed it’s tail...or the tale of physical log overflow PARTNUMs - and the story of getting to data We’ve all seen them…what does it really mean? Physical pages vs. logical pages? Discussion about the value of knowing the difference The dog has swallowed it’s tail...or the tale of physical log overflow PARTNUMs - and the story of getting to data We’ve all seen them…what does it really mean?

6. user.conference Informix 5 Topics The maximum number of extents for a table? Is there REALLY a max? Where did my row go? The long tale of the forward pointer The last topic The maximum number of extents for a table? Is there REALLY a max? Where did my row go? The long tale of the forward pointer The last topic

7. user.conference Informix 6 Topic 1: Logical Pages vs. Physical Pages Understanding the difference can help in: Interpretation of msg log assertion failures Interpretation of the extent list of a partition page Lock level identification with respect to rowid(s) Oncheck usage A bunch of other stuff Understanding the difference can help in: Interpretation of msg log assertion failures Interpretation of the extent list of a partition page Lock level identification with respect to rowid(s) Oncheck usage A bunch of other stuff

8. user.conference Informix 7 Physical Pages Always with respect to a chunk Numbering starts with 0 Format: 0xCCCPPPPP ccc - chunk number ppppp - page offset into the chunk Always with respect to a chunk Numbering starts with 0 Format: 0xCCCPPPPP ccc - chunk number ppppp - page offset into the chunk

9. user.conference Informix 8 Physical Pages 30023c30023d 300243300242300241300240 30023f30023e 300247300246300245300244 Chunk 3 0xCCCPPPPP c - chunk # p - page offset

10. user.conference Informix 9 Logical Pages Always with respect to a tablespace Numbering starts with 0 Format simply 0 through x Always with respect to a tablespace Numbering starts with 0 Format simply 0 through x

11. Logical Pages 30023c30023d 300243300242300241300240 30023f30023e 300247300246300245300244 extent 1 for sparky 0 54 21 3 logical page num create table sparky... physical page num

12. user.conference Informix 11 Usage Examples - Extent List extent list on a partition page (oncheck -pt skippy:sparky) chunk 0x4page 0x33

13. user.conference Informix 12 Usage Examples - Bad Page

14. user.conference Informix 13 Usage Examples - Lock Level onstat -k rowid format: 0xLLLLLLSS L - logical page in the table S - slot/row # on the page oncheck -pp 0x100016 2

15. user.conference Informix 14 Topic 2: Physical Log Overflow Physical Log File 1st before images of modified cache pages 75% full: checkpoint request is raised what happens now? back to the beginning, and keep writing!

16. user.conference Informix 15 Physical Log Overflow Physical Log File 75% full: checkpoint request is raised Two potential scenarios: the chkpt completes, and we give the physical log a new logical beginning. In that case, we don’t care about the overwritten pages if the engine falls over, you’ll most likely get stuck in fast recovery when trying to come back up

17. user.conference Informix 16 Topic 3: Partnums Three areas focused on here: What is a partnum? How do we use it to get to stuff What is a tablespace tablespace? Three areas focused on here: What is a partnum? How do we use it to get to stuff What is a tablespace tablespace?

18. user.conference Informix 17 Definitions Tablespace tablespace Keeps track of tablespaces, or partitions, in a dbspace Each dbspace has one Typical allocation is 50 pages for each extent Partition page A page in a tablespace tablespace Typically one page per tablespace It “describes” the tablespace Tablespace tablespace Keeps track of tablespaces, or partitions, in a dbspace Each dbspace has one Typical allocation is 50 pages for each extent Partition page A page in a tablespace tablespace Typically one page per tablespace It “describes” the tablespace

19. user.conference Informix 18 partnum DDDLLLLL dbspace number (starting at 1) Logical page num within the tablespace tablespace (starting at 0) 1 1/2 bytes (3 nibbles, or 3 hex digits) 2 1/2 bytes (5 nibbles, or 5 hex digits) 4 bytes 0x

20. user.conference Informix 19 dbspace4 rootdbs systables namepartnum database sparky 0020000311-11-1999informixsparky database tablespace partnumflagscreatedownername 0020003c mud table mud extents for mud tablespace page 3c CREATE DATABASE sparky IN dbspace4; CREATE TABLE mud; DATABASE sparky; SELECT * FROM mud;

21. user.conference Informix 20 Topic 4: Max Number of Extents What’s the maximum number of extents for a tablespace? Consider the previous topic, and the use of the partition page for a tablespace What’s the maximum number of extents for a tablespace? Consider the previous topic, and the use of the partition page for a tablespace

22. user.conference Informix 21 The Partition Page

23. user.conference Informix 22 The Partition Page oncheck -pt skippy:sparky

24. user.conference Informix 23 Oh by the Way... Do you really want to know the max? Fragmentation changes the max for a logical table 4K vs. 2K pages Do you really want to know the max? Fragmentation changes the max for a logical table 4K vs. 2K pages

25. user.conference Informix 24 Topic 5: Forward Pointers Forward pointer - a 4-byte number that “points” to the remainder pieces of a row that spans pages Used with rows longer than a page Format is same as a rowid, although it’s NOT the rowid for the row Used with varchars or really long rows (rows longer than a page typically) We’ll focus on varchars... Forward pointer - a 4-byte number that “points” to the remainder pieces of a row that spans pages Used with rows longer than a page Format is same as a rowid, although it’s NOT the rowid for the row Used with varchars or really long rows (rows longer than a page typically) We’ll focus on varchars...

26. user.conference Informix 25 Forward Pointers w/ varchars Problem - you can easily end up with a “chain” of forward pointer/remainder portions of rows This can happen when you’ve updated a varchar column and increased it’s size The new row size won’t fit into it’s old slot, and so we: Move the row - all or some of it - to another page Leave back a 4-byte forward pointer on the home page Problem - you can easily end up with a “chain” of forward pointer/remainder portions of rows This can happen when you’ve updated a varchar column and increased it’s size The new row size won’t fit into it’s old slot, and so we: Move the row - all or some of it - to another page Leave back a 4-byte forward pointer on the home page

27. user.conference Informix 26 Forward Pointers w/ varchars This can happen as many times as it needs Every move adds an additional 8 bytes to the row length 4 bytes for the forward pointer in the home page 4 bytes for the slot table entry on the new/remainder page This can happen as many times as it needs Every move adds an additional 8 bytes to the row length 4 bytes for the forward pointer in the home page 4 bytes for the slot table entry on the new/remainder page

28. user.conference Informix 27 Forward Pointers

29. user.conference Informix 28 What Can I Do? To rid yourself of the “chaining” effect: Unload/reload the table If the row will fit onto a single page, then there is no need for the forward pointer If it’s grown to be larger than a page (actual row length), then you’ll have at least one. This can’t be avoided - the row is now longer than a page Alter index to cluster Same caveats as above To rid yourself of the “chaining” effect: Unload/reload the table If the row will fit onto a single page, then there is no need for the forward pointer If it’s grown to be larger than a page (actual row length), then you’ll have at least one. This can’t be avoided - the row is now longer than a page Alter index to cluster Same caveats as above

30. user.conference Informix 29 Topic 7: How Do You Find Out More? Attend the “IDS Internal Architecture” class 5 days - very intense use of oncheck, and topics covered in this presentation Taught in Denver, Chicago, Menlo Park most often Watch TechNotes for a full white paper on this topic Send me email - scranton@informix.com Checkout the website - www.markscranton.com Attend the “IDS Internal Architecture” class 5 days - very intense use of oncheck, and topics covered in this presentation Taught in Denver, Chicago, Menlo Park most often Watch TechNotes for a full white paper on this topic Send me email - scranton@informix.com Checkout the website - www.markscranton.com

31. user.conference Informix 30 Thanks... For listening. Hope you learned something See you in class For listening. Hope you learned something See you in class