1. © 2011 IBM Corporation 11 April 2011 IDS Architecture

2. © 2011 IBM Corporation 2 Agenda  Architecture overview  Process model  Memory model  Storage model

3. © 2011 IBM Corporation 3 Architecture Overview

4. © 2011 IBM Corporation 4 IDS Server components  Process component: –One or more processes that do the tasks requested by the database server instance.  Shared memory component: –Shared memory segments used to cache table data from disk and maintain and control resources needed by processes.  Disk component: –Disk is a storage facility to hold table data and system information necessary to maintain the server.

5. © 2011 IBM Corporation 5 PROCESS MODEL

6. © 2011 IBM Corporation 6 Process based database server  Each process receives a time-slice of CPU time, represented by “CPU Usage Over Time”. Notice that the oninit processes fall “in line” between other processes (p1, p2, p4, p5…). As more users connect, the number of processes increase and the demand for resources does as well. * Informix database server processes are called oninit

7. © 2011 IBM Corporation 7 Process based database server cont…  Process-based database servers are highly inefficient! 0 100 Can waste 50% or more of CPU resources!! CPU Usage CPU being used CPU idle, not working on process

8. © 2011 IBM Corporation 8 Dynamic Scalable Architecture (DSA)  1993 – Informix creates Dynamic Scalable Architecture and IDS is born!  Multi-threaded from the core out.  Custom libraries, not Posix – Resource efficient. – Not O/S dependent: Easily ported.  Integrated parallelism: – Dynamic. – Scalable.

9. © 2011 IBM Corporation 9 Dynamic Scalable Architecture (DSA) cont…  Processes –Each database server process (oninit) is known as a “virtual processor” because it schedules and runs its own threads. –Virtual Processors (VPs) are divided into classes based on functionality: E.g. Writing to logical or physical log, reading data from disk, performing administrative tasks.  Threads –Services requests from client application (user threads): E.g. Select query issued by a client is serviced by an user thread. –Accomplish internal tasks like database I/O, logging I/O, page cleaning, administrative functions (internal threads). –A thread can run on any VP in its class. –To run a thread, VP retrieves thread’s data and environment stack from ready queue and executes request.

10. © 2011 IBM Corporation 10 Dynamic Scalable Architecture (DSA) cont… Provides fan-in and fan-out parallelism Fan-inFan-out Physical Processor Virtual Processor Physical Processor 1 Physical Processor 2 Physical Processor 3 Virtual Processor 1 Virtual Processor 2 Virtual Processor 3

11. © 2011 IBM Corporation 11 DSA - Thread based database server Each thread receives a “chance” to run as a part of the oninit process on a physical processor. Notice the bottom axis shows “Virtual Processor Usage Over Time”. This represents time the thread is running on a pre-existing oninit process As more users connect, the number of processes remains static as each connection represents a request to run a “thread”, not a “process”

12. © 2011 IBM Corporation 12 DSA - Thread based database server Result: almost complete usage of system resources and significantly better scalability! 0 100 CPU Usage CPU being used CPU idle, not working on process

13. © 2011 IBM Corporation 13 DSA – Putting it all together Serial aka Single threaded Scan Join Sort Time to Process Parallel Parallelized DSA processes tasks concurrently DSA breaks tasks into subtasks Write (Two table join)

14. © 2011 IBM Corporation 14 DSA - Putting it all together A single query example: select * from flights order by flightno; Introduction to Informix Dynamic Server 14 Disk Scan Scan Thread Sort Thread

15. © 2011 IBM Corporation 15 DSA – Putting it all together A more complex example using a parallel tree Exchange Threads Hash Join Threads Scan Threads (flightleg) Scan Threads (ticket) Exchange Threads Sort Threads select count(*) from tickets a, flightleg b where a.flno = b.flno and orig=“DFW” order by a.class

16. © 2011 IBM Corporation 16 MEMORY MODEL

17. © 2011 IBM Corporation 17 Shared Memory  O/S feature that allows database server processes to share data by sharing access to pools of memory.  Reduces disk I/O by caching data from disk.  Provides the fastest method of inter-process communication.  Provides communication channels for local client applications that use IPC communication.

18. © 2011 IBM Corporation 18 Shared Memory Segments  Resident segment –Contains the buffer pool and other system information: Buffer pool caches data from table. –Can be configured to remain resident in main memory. –Fixed size.  Virtual segment: –Contains information about the threads and sessions, and the data used by them. –Could be paged out to disk by the operating system. –Expandable.  Message segment: –Holds the message buffers used in client-server communication if shared memory communication is configured.

19. © 2011 IBM Corporation 19 Shared Memory Structure

20. © 2011 IBM Corporation 20 STORAGE MODEL

21. © 2011 IBM Corporation 21 Disk Page  The basic unit of storage in a server.  All database and system information is stored on pages.  The minimum unit of I/O in a server is a page.  The size of a page is by default 2KB on most UNIX systems and 4KB on AIX and Windows.  Page size can also be configured (max of 16KB).

22. © 2011 IBM Corporation 22 Extents and Tablespaces  Extent: –Collection of contiguous pages. –Space for table is allocated in units of extents.  Tablespace: –Logical collection of extents. –Storage for table. –A table can have multiple tablespaces. Extent Tablespace

23. © 2011 IBM Corporation 23 Chunks  Largest unit of contiguous disk dedicated to database server data storage.  Chunk can be a raw device (character-special device), a piece of a raw device, or a UNIX file (cooked file).  Maximum size of chunk is 4TB.  Maximum allowable chunks is 32767.

24. © 2011 IBM Corporation 24 Dbspaces  Logical collection of one or more chunks.  Can have between 1 and 32767 chunks.  Maximum allowable dbspaces is 2047.

25. © 2011 IBM Corporation 25 Storage Model Chunk Dbspace Tablespace Page Extents

26. © 2011 IBM Corporation 26 Blobspaces  Special dbspace that stores simple large objects (TEXT and BYTE data).  Basic unit of storage in blobspace is a blobpage.  Blobpage size can be configured as a multiple of the database server page size.  Database server writes data stored in a blobspace directly to disk (not buffered).

27. © 2011 IBM Corporation 27 Sbspaces  Special dbspace that stores smart large objects (BLOB and CLOB data) and user defined data.  Pages in sbspace are called sbpages.  Sbpage size is same as the database server page size and is not configurable.  Basic unit of allocation in sbspace is an extent.  Writes to sbspace are buffered by default: –Buffering can be turned off at sbspace creation time.

28. © 2011 IBM Corporation 28 Mirroring  Process of automatically writing same data to two disks.  Eliminates data loss due to disk failure: –If one disk fails, the data is still available on the other disk.  Costs: –Additional disk space. –Performance cost of having to write to two locations.

29. © 2011 IBM Corporation 29 Reading and Caching data  When a client issues a query, the table data is read from disk into buffers in shared memory.  I/O is performed in page units.  Subsequent access to the same data is from buffer pool.  Changes made to the data are visible to all database server process.

30. © 2011 IBM Corporation 30 Physical and Logical Logging  Physical logging: –Process of storing before-images of pages that are being modified. –Before-images are stored in physical log. –Physical log is a collection of contiguous pages on disk. –Required for recovery purpose in event of a system failure.  Logical logging: –Process of recording transaction details. –Transaction records are stored in logical log. –Logical log is comprised of logical log files, which are collections of contiguous pages on disk. –Required for transaction rollback and recovery purpose in event of a system failure.

31. © 2011 IBM Corporation 31 Checkpoints and Recovery  Checkpoints: –Periodic system event during which all modified buffers are written to disk. –Establishes a consistent state for database server. –Information on checkpoint event is recorded in the system pages and logical log (needed for recovery purposes).  Recovery: –If a system failure occurs, the database server restarts at that established point. –Before-images of pages modified since checkpoint are restored from physical log. –Transactions since checkpoint are then replayed from the logical log to get the server to the consistent state just before the time of failure.

32. © 2011 IBM Corporation 32 Resources  The Online IDS Information Center –http://publib.boulder.ibm.com/infocenter/idshelp/v115/index.jsphttp://publib.boulder.ibm.com/infocenter/idshelp/v115/index.jsp  IBM Informix DeveloperWorks Technical Articles –http://www.ibm.com/developerworks/db2/products/informix/index.htmlhttp://www.ibm.com/developerworks/db2/products/informix/index.html  IBM DeveloperWorks IDS Blogs –http://www-128.ibm.com/developerworks/blogs/page/roundrep (IDS Replication)http://www-128.ibm.com/developerworks/blogs/page/roundrep –http://www-128.ibm.com/developerworks/blogs/page/gbowerman (IDS Application Development)http://www-128.ibm.com/developerworks/blogs/page/gbowerman –http://www-128.ibm.com/developerworks/blogs/page/idsteam (IDS Experts Blog)http://www-128.ibm.com/developerworks/blogs/page/idsteam