Simply Powerful – Informix xC1 thru xC4

Simply Powerful – Informix 12.10.xC1 thru xC4
Scott Pickett WW Informix Technical Sales For questions about this presentation, to: © 2014 IBM Corporation

Forward The multitenancy portion of this presentation is thru xC5, it did make sense to split multitenancy into two different presentations. The Time Series and JSON stories are quite large for and as such are split off separately in their own presentations. Pricing and Packaging will be found in its own presentation on the CTP web site. As of June 23, 2016, the current release is xC7 Next release is xC8 and is due November 2016. There are sub-releases of xC7 due in September 2016. These will have Encryption at Rest as a main feature © 2014 IBM Corporation

Agenda Autonomics & Server Admin. Improvements Primary Storage Manager
Multitenancy REST Table Compression features Index Compression features New SQL Informix Warehouse Accelerator HA, ER, SDS, Flexible Grid, Connection Manager Spatial Enhancements © 2014 IBM Corporation

Informix Dynamic Server - Autonomic Tuning & Server Admin Improvements
Scott Pickett WW Informix Technical Sales For questions about this presentation contact: © 2014 IBM Corporation

Overview Problem Statement:
We want the Informix database server to be as invisible as possible. After the initial install of IDS and uptime, we need the server to be able to adapt to any workload that comes its way. Prior to 12.10xC2, several key server resources were not dynamic This could and did lead to artificial performance ceilings, and an increase in downtime to fix, or degradation in performance until the issue was detected by human interface. Not everyone knows how to detect ……. and if they do, getting the downtime from management is an issue …. always. The improvements are in several different categories….. Memory buffers were not dynamic. Physical log could run out of space because it did not sole occupy the dbspace it was in. There could be other objects within that space. So the Log could not expand. Many times we have recommended that the physical log be in its own space, for just this reason. It is a tuning impossibility to predict in advance how many logical logs and dbspaces to house them would be required to handle a given user workload. Given that these constantly over time, people simply guess in many cases and allocate to a perceived maximum workload, without really knowing what it will be. Sometimes …. this is gotten right …. Sometimes …. It is not. Adding a cpu vp is a difficult challenge to get right when it is needed. It is rare that you are literally sitting there and monitoring everything that is ongoing and get the bits of data correct that would require an additional cpu vp. It hasn’t been dynamically tunable in the past either. If your ready queue is backed up constantly ….. That’s one sign. And a good one. At that point you’ve crossed the threshold from non-need to need. Not enough cpu resource to drive the workload. These would be four of the biggest reason why you would have to take the server down to get more resource. Which is impossible in the age of 24 x 7 operations and in a web connected world. © 2014 IBM Corporation

Requirement Categories
Physical Log Logical Log Bufferpool CPUs These will follow similar rules guiding automatic tuning: How to detect when insufficient resources for each are causing a performance impact How to increase the resource with minimal impact to the application How to control the increase of the resource so it adapts in a timely manner… not too much, not too little, just right © 2014 IBM Corporation

Physical Log Expansion
Purpose: To help avoid performance loss associated with an undersized physical log caused by frequent server checkpoints which can block user transactions, located within an already constrained storage space with other objects therein, such as database and server objects. Expansion We support the changing of the physical log location already. We want to make sure there is only sole occupancy in the space for the physical log; nothing else hindering its successful expansion and requested size: A new single Physical Log Storage Space (PLOGspace) per instance, Has a single physical chunk containing only the server physical log with the physical page size of the host O/S. Physical log expansion is triggered by any of the following A checkpoint blocked user transactions because the physical log is too small. 5 checkpoints triggered by physical log activity: Since the last time the physical log was extended. And within the last 20 checkpoints. For years, we have stated in countless places that the physical log should be in a dbspace by itself, with no other objects within that space, to make sure that there were no obstacles to the expansion of the physical log if needed. We never enforced those spoken guidelines/rules, as many users were disk space constrained in the beginning and cost was a factor then. Today disk space is cheap and has plenty of capacity, so those concerns are alleviated. There is only one physical log per server instance and that is in one dbspace only. The physical log cannot span more than one physical chunk of storage, so there is only one chunk in a dbspace. The page size for this chunk and dbspace must be the page size of the operating system the instance is on. This new PLOGSpace takes all of those rules and formalizes them in the new structure. You can size the physical log to any size you like within the new PLOGspace, and the log will expand within the PLOGspace as needed. You will potentially still get an administrative checkpoint when the physical log gets 75% full though. This is all being done for performance, to eliminate as much as possible checkpoint blocking on transactions which can occur when the physical log is too small. Typically to get the most performance out of the physical log make your dbspace as large as possible and the physical log largest possible. Then the threshold for a blocking checkpoint on transactions for most operations becomes 75% of your physical log being full. As long as you don’t do any administrative checkpoints such as those performed by onmode –c. © 2014 IBM Corporation

Physical Log Space Creation – onspaces
General Syntax: onspaces -c –P <name> -p <path> -o <offset> -s <size> [-m <mpath> <offset>] Examples: onspaces -c –P plogdbs -p /dev/chk1 -o 0 -s 40000 onspaces -c –P pdbs1 -p /dev/pchk1 -o 500 -s m /dev/mchk1 0 Note If a PLOGspace already exists: Creating a new one will create the new PLOGspace, Move the physical log to that space, Drop the old PLOGspace, All of the above done automatically. New syntax for creation of a physical log in the new PLOGspace with onspaces. © 2014 IBM Corporation

Physical Log Space Creation – SQL Admin API
Creating a PLOGspace with SQL Admin API General Syntax: create plogspace, <name>, <path>, <size>, <offset>, <mir path>, <mir offset> Examples: execute function task(“create plogspace”,”plogdbs”,”/dev/chk1”,30000,0); execute function task(“create plogspace”,”plogdbs”,”/dev/chk1”,30000,0,”/dev/mchk1”,0); Via the SQL Admin API. Not shown but is valid, is execute function admin. © 2014 IBM Corporation

Physical Log Improvements - onspaces
Dropping a PLOGspace with onspaces The same space-dropping command (onspaces –d) may be used to drop this new type of space, Only if the space is empty (53 pages, oncheck –pe to verify) General Syntax: onspaces -d <PLOGspace name> [-y] Example: onspaces -d plogdbs -y Dropping the new PLOGspace. If necessary. Do this judiciously and cautiously. Should only be 53 pages on a 2K page in the space, not one page more, when you do this. That’s the overhead in the dbspace. It fails to drop if the space is not empty. © 2014 IBM Corporation

Physical Log Improvements – SQL Admin API
Dropping a PLOGspace with SQL Admin One must use a new “drop plogspace” command with SQL Admin API. “drop dbspace” will return an error. “drop plogspace” takes no arguments, since there can be at most one PLOGspace in the instance. The PLOGspace must be empty to be dropped. General Syntax: drop plogspace Example: execute function task(“drop plogspace”); Dropping via the SQL ADMIN API. © 2014 IBM Corporation

Warnings on Insufficient PLOGspace
In the message log file: Thru the onstat –m command line utility: You will see a message saying “Physical Log too small ……” Thru onstat –g ckp utility: Look for a warning at the top about the physical log being too small or a “Trigger” of “Plog” with “Block Time” > 0.0 The presence of long transaction rollbacks. A “Trigger” of “LongTX” with “Block Time” > 0.0 can be a too small physical log or too little Logical Log space. Onstat –g ckp only reports information on the last 20 checkpoints ……… so what you are looking for may not be here ….. message log file should have it if it exists. © 2014 IBM Corporation

Logical Log Improvements
Purpose To improve performance, having sufficient logical log space helps prevent frequent checkpoints from blocking transactions and also from having long or too frequent checkpoints. Knowing an appropriate amount of logical log space for a dynamic workload is difficult to supply in advance without first knowing and estimating the transaction load. Configuring a server with too much logical log space in advance can limit the out-of-box experience by requiring substantial disk space. We will therefore allocate logical log space dynamically as needed and therefore dynamically tunable. Detection The best way to detect if the logical log is a bottleneck is to look at the recent checkpoint activity: The server keeps track of the last 20 checkpoints. If a substantial portion of the recent checkpoints were triggered because of the logical log, then we should increase the logical log space by adding another logical log. If a long transaction or checkpoint blocking occurs because of the logical log, we should also add another logical log to the log space. This is part of our overall strategy to make Informix autonomic. Making logical logs expand in quantity by having the server respond and adapt logical log resources to dynamic, real time changes in workload transaction levels without real time human intervention is part of this. © 2014 IBM Corporation

Expansion We already have the adding logical logs dynamically feature: DYNAMIC_LOGS 2 in the configuration file defined by $ONCONFIG environment variable. Typically executes only when the server runs out of logical log space. The server already contains the logic to extend a dbspace while adding a logical log. There is a new, additional Logical Log expansion parameter in the configuration file called AUTO_LLOG Enables the automatic addition of logical logs when the database server detects that adding logical logs improves performance. If enabled, log expansion occurs: When 5 of the last 20 checkpoints were caused by logical logs filling up When a logical log causes a blocking checkpoint When a logical log causes a long checkpoint AUTO_LLOG & DYNAMIC_LOGS do not interact DYNAMIC _LOGS will only kick in once the server has actually run out of logical logs being available. AUTO_LLOG expands for performance reasons, there just simply are not enough logical logs to handle the workload. © 2014 IBM Corporation

Configuration The new AUTO_LLOG configuration parameter specifies Dbspace for new logical log files Maximum size of all logical log files. Estimation of maximum logical log space requirements: Depending on the number of concurrent users accessing your database server……. : MB : MB : GB More than 1000: GB Based on typical user needs over a long period of time and experience. Of course, your mileage may vary; every user system and workload is different. No two are alike. For really heavy transaction environments, Informix does recommend that logs be spread round-robin across 3 dbspaces and logs to be fairly large in size. In this configuration, the logical log backup threads (from the backup utilities ontape or onbar) will always be one dbspace behind the dbspace containing the log presently being filled; therefore these will not conflict with each other. Setting this up is still a dba operation. This can result in a 35% performance gain, seen in many environments over the years. © 2014 IBM Corporation

AUTO_LLOG – Syntax onconfig.std value 0
By default this is 0, disabled. AUTO_LLOG 1, dbspace_name, max_size 1 - Add Logical logs when needed to improve performance. dbspace_name - dbspace name in which to add logical log files. This dbspace must have the default page size for the operating system. max_size = Optional. Default is KB (2 GB). The maximum size in KB of all logical log files, including any logical log files that are not stored in the dbspace dbspace_name. If max_size is not specified, the AUTO_TUNE_SERVER_SIZE configuration parameter setting affects the maximum size. Takes effect: After you edit the onconfig file and restart the database server. Reset the value dynamically in your onconfig file by running the onmode -wf command Reset the value in memory only by running the onmode -wm command. © 2014 IBM Corporation

AUTO_LLOG – Syntax Notes
When the maximum size of the logical log files is reached, logical log files are no longer added to improve performance. However, if the DYNAMIC_LOGS configuration parameter is enabled, logical logs are added to prevent transaction blocking. The settings of the DYNAMIC_LOGS and the AUTO_LLOG configuration parameters do not interact. If the value of the max_size field is larger than the size of the specified dbspace, enable automatic expansion of storage spaces. In the past, before SAN's and disk arrays and such, it made sense to have logs on different disks, with different dbspaces housing each for performance reasons to make sure the contention of logical logs filling with transactions did not suffer contention resulting from the previous log backed being backed up after it was filled. That only applies if the logs are on direct attached single disk drives. Most systems now attach to SAN devices and the like which have many disks stripped so AUTO_LLOG and its mechanics work perfectly within a dbspace. © 2014 IBM Corporation

Warnings on Insufficient Logical Logs
In the message log file: Thru the onstat –m command line utility: You may see unusual numbers of logical logs have been added. Thru onstat –g ckp utility: Look for a “Trigger” of “Llog” with “Block Time” > 0.0 Onstat –g ckp only reports information on the last 20 checkpoints ……… so what you are looking for may not be here ….. message log file (onstat –m) should have it if it exists. Of course, this is what a dba would look for when trying to trace this and it is provided here as a convenience. The realty is that the server will be looking here as well behind the scenes, and much quicker than can be gathered by the dba. © 2014 IBM Corporation

Bufferpool Added Capabilities
Dynamic Buffer pools have been requested for a while by our users. We will therefore add the ability to dynamically extend a buffer pool and do so automatically, which will allow us to start with relatively small pools and increase the size of the pools only as needed. This will improve performance in many cases by getting the memory resources necessary to the database server it needs, if available, to increase application transaction performance much sooner, in near real-time, rather than after the fact, which has been the norm. It is impossible for even a highly trained dba to know when his/her system is going to require additional memory. Workloads are ever changing and in growth systems in particular, the reaction times and change windows for this are non-existent. Dynamic on the fly change is required. There are many items to monitor to determine when this dynamic allocation of resources, in this case, memory buffers, should take place. Buffer pool additions are prior to xC3, not dynamic. They are located in the static resident segment and require downtime to increase; this does not work in today’s marketplace. So, in 12.10xC3, they will now be dynamic, and there will be increased performance resulting from this, as you will have the memory you need to keep up with growing workloads and databases. © 2014 IBM Corporation

BUFFERPOOL Configuration Parameter Expansion
Three goals: Use a new memory format method of allocating pool memory: Specifying buffer pool size in bytes (Kb, Mb, Gb) units. Marking the pool as extendable. Maintain the legacy format method of BUFFERPOOL for current user compatibility: Specifying buffer pool size and its limits in pagesize units. Some BUFFERPOOL arguments are now incompatible with others. Must follow rules to properly set this parameter: Order of precedence for all BUFFERPOOL definitions: All definitions shall have the same format (all legacy or all memory) or the server will not boot. If an argument’s value is not defined, we use the value from the default BUFFERPOOL setting. If the default BUFFERPOOL setting is not defined, use the internal definition for the format being used. If unable to determine format (ex. BUFFERPOOL size=2k), use internal format based on any other BUFFERPOOL definitions present. If not, use internal legacy. So there will be two format in which BUFFERPOOL’s can be declared in the configuration file. The legacy format we have used for years and the new format, where the bufferpool size is simply expressed in MB, GB, etc, instead of numbers of KB which is legacy. The legacy method states all sizes in pages of KB sizes page units, for example, 1 1 GB buffer pool is expressed as in BUFFERPOOL; in the new format it is expressed as 1gb. © 2014 IBM Corporation

Bufferpool Added Capabilities
Detection We want to add more memory/buffers to a buffer pool only when the working set doesn’t fit. To determine when to extend the buffer pool, we keep track of the cache hit ratio of each buffer pool by sampling the cache hit ratio once a minute for 5 minutes. If the average cache hit ratio is less than a configured threshold, we extend the buffer pool. Extension In order to support buffer pool extension, we need to move buffer pools into their own shared memory segments. Shared Memory in Informix can now have 5 possible segments Resident Virtual Message Bufferpool *** NEW **** Extended Only expands when the total working set request for memory from all active sessions exceed the available memory. Read cache is the primary cache monitored here. © 2014 IBM Corporation

Bufferpools Bufferpool Segments Bufferpool Extending for Legacy Format
Buffer pools will no longer be allocated within the resident segment. We will add a new IDS memory segment class that will contain only buffer pool memory structures and adhere to the RESIDENT flag. The buffer pool class segment will have virtually no overhead (bitmap or TTree) for maintaining the segment. This will allow for maximum memory usage. Bufferpool Extending for Legacy Format The BUFFERPOOL configuration parameter’s legacy “buffers” format uses these attributes: buffers – starting number of buffers next_buffers – is the number of buffers to add when the buffer pool is extended and will be doubled every 4th extension *. max_extends – maximum number of extensions the buffer pool is allowed * (See the onstat –g buf slide example further below.) © 2014 IBM Corporation

Bufferpools (cont’d) Bufferpool Extending for Memory Format
The BUFFERPOOL configuration parameter’s new “Memory” format uses these attributes: memory – target amount of memory that a buffer pool can grow to. Values will be rounded up to 32mb alignments. start_memory – starting amount of memory Both memory and start_memory values can be expressed in a variety of units. For example: BUFFERPOOL size=4k,start_memory=32mb,memory=54gb BUFFERPOOL size=8k,start_memory=48000,memory=204mb © 2014 IBM Corporation

BUFFERPOOL Configuration Parameter Expansion
A BUFFERPOOL definition can NOT include aspects of both formats; either it’s expressed in legacy format or memory format. A BUFFERPOOL definition with mixed formats is rejected and the database server will not start. The charts on the next several slides describe each argument to the BUFFERPOOL parameter: both = works with legacy & memory format. legacy & memory are mutually exclusive. © 2014 IBM Corporation

BUFFERPOOL Argument Compatibility Chart
Format Dynamic Values size both no Required. Default,2k,4k,…. lrus Optional Proposed change internal default from numcpuvps to 4 * physical processors on the box. lru_min_dirty Yes, auto Internal default = 50 lru_max_dirty Internal default = 60 extendable 0=FALSE 1=TRUE Determines if bufferpool is extendable or not. Internal default: legacy = 0, memory = 1 cache_hit_ratio Yes Min=0, Max=100, default = 90 Only applies if bufferpool is extendable What you can and can’t do in these slides of mixing the old and legacy formats for the BUFFERPOOL arguments. Some settings can be as in “both” formats, some cannot. Setting extendable is the key setting here, as it will allow the bufferpool to expand as needed. both = works with legacy & memory format. legacy & memory are mutually exclusive. © 2014 IBM Corporation

BUFFERPOOL Argument Compatibility Chart (cont’d)
Format Dynamic Values buffers legacy No # of buffers Initial # of buffers in 1st segment. Min=MIN_BUFFERS(1000) Max=MAX_BUFFERS(based on virtual memory) Internal default=1000 next_buffers Yes, won’t impact existing segments or growth projections, only next segment Requires extendable=1, otherwise, ignored. Specifies amount of growth the next bufferpool segment will be. max_extends Optional Maximum # of segments the bufferpool can expand to Min=0 Max=MAX_BUFFSEGS (platform specific… 64bit=24, 64bit NT=8, 32bit=16) Internal Default=8 both = works with legacy & memory format. legacy & memory are mutually exclusive. © 2014 IBM Corporation

BUFFERPOOL Argument Compatibility Chart (cont’d)
Format Dynamic Values start_memory memory No Optional for new format determines size of initial bufferpool segment. Min=32Mb Max=Amount of free memory auto=IDS determines starting memory Internal default=Auto Yes Required for new format Determine max size bufferpool can grow to Max=? auto=IDS determines maximum amount of memory to use both = works with legacy & memory format. legacy & memory are mutually exclusive. © 2014 IBM Corporation

BUFFERPOOL examples BUFFERPOOL size=2k,extendable=1,buffers=1000,next_buffers=2000,max_extends=8 Create a 2k pagesize buffer pool. The buffer pool is extendable up to 8 times. The 1st 3 extensions will be of 2000 buffers, Next 4 will be of 4000 buffers and The 8th extension will be of 8000 buffers. BUFFERPOOL size=10k,start_memory=auto,memory=4gb Create a 10k pagesize buffer pool. The buffer pool will be extendable and have a max size of 4gb. IDS will determine the initial # of buffers and the extension sizes. Note: when using legacy format, it is possible to ask for N buffers and get slightly more or fewer depending on memory alignment. We allocate extra memory to allow for alignment and then consume all of the memory allocated. Examples and explanations of settings and their consequences within the database configuration file, as defined by value of the environment variable $ONCONFIG, for setting up bufferpools preconfigured. Self explanatory. © 2014 IBM Corporation

onstat –p – Monitor your cache hit ratio
This read cache ratio in particular is traditionally monitored by many dba’s and we will be looking at this particular number to determine whether we should dynamically add more bufferpools to increase the performance throughput of the database server. Taken in the Open Admin Tool. Below are the column header descriptions for the output on the slide above. Output Description: dskreads The number of actual reads from disk pagreads The number of pages read bufreads Is the number of reads from shared memory %cached The percent of reads cached in the buffer pool. If bufreads exceeds the maximum integer (or long) value, its internal representation becomes a negative number, but the value appears as 0.0. dskwrits The actual number of physical writes to disk. This number includes the writes for the physical and logical logs reported in onstat -l . pagwrits The number of pages written bufwrits The number of writes to shared memory %cached The percent of writes cached in the buffer pool. The next portion of the -p display tabulates the number of times different ISAM calls were executed. The calls occur at the lowest level of operation and do not necessarily correspond one-to-one with SQL statement execution. A single query might generate multiple ISAM calls. These statistics are gathered across the database server and cannot be used to monitor activity on a single database unless only one database is active or only one database exists: isamtot The total number of calls open Increments when a tblspace is opened start Increments the pointer within an index read Increments when the read function is called write Increments with each write call rewrite Increments when an update occurs delete Increments when a row is deleted commit Increments each time that an iscommit() call is made. No one-to-one correspondence exists between this value and the number of explicit COMMIT WORK statements that are executed. rollbk Increments when a transaction is rolled back. The next portion of the onstat -p command output displays information about generic pages. The Generic Page Manager provides an API for Informix to manage nonstandard pages in the database server buffer pool. The following table describes the Generic Page Manager fields in the onstat -p command output. gp_read The number of generic page reads gp_write The number of generic page writes gp_rewrt The number of generic page updates gp_del The number of generic page deletes gp_alloc The number of generic page allocations gp_free The number of generic pages freed and returned to tblspaces gp_curs The number of cursors used against generic pages The next portion of the onstat -p command output displays the number of times that a resource was requested when none was available: ovlock Number of times that sessions attempted to exceed the maximum number of locks ovuserthread The number of times that a user attempted to exceed the maximum number of user threads ovbuff The number of times that the database server did not find a free shared-memory buffer. When no buffers are free, the database server writes a dirty buffer to disk and then tries to find a free buffer. usercpu Is the total user CPU time that all user threads use, expressed in seconds. This entry is updated every 15 seconds. syscpu The total system CPU time that all user threads use, expressed in seconds. This entry is updated every 15 seconds. numckpts The number of checkpoints since the boot time flushes The number of times that the buffer pool was flushed to the disk. The next portion of the onstat -p command output contains miscellaneous information, as follows: bufwaits Increments each time that a user thread must wait for a buffer lokwaits Increments each time that a user thread must wait for a lock lockreqs Increments each time that a lock is requested deadlks Increments each time that a potential deadlock is detected and prevented dltouts Increments each time that the distributed deadlock time-out value is exceeded while a user thread is waiting for a lock ckpwaits Is the number of checkpoint waits compress Increments each time that a data page is compressed seqscans Increments for each sequential scan * The last portion of the onstat -p command output contains the following information: ixda-RA The count of read-aheads that go from index leaves to data pages idx-RA The count of read-aheads that traverse index leaves da-RA The count of data-path-only scans logrec-RA The log records that the database server read ahead RA-pgsused The number of pages used that the database server read ahead lchwaits Stores the number of times that a thread was required to wait for a shared-memory latch. Many latch waits typically results from a high volume of processing activity in which the database server is logging most of the transactions. So if the %cached for reads is less that 95% then this is usually a good bet that you are doing too much disk i/o and not enough reading from memory, which does affect performance and this is usually because there are not enough memory buffers to cache all of the data. © 2014 IBM Corporation

onstat –g seg – Memory Segments in Use
As before, the ‘class’ column indicates the type of memory segment, the ‘B’ within the column indicating the new ‘Buffer pool’ segment, and as before: ‘R’ indicating resident and ‘V’ indicating virtual segments. Screen shot from the Open Admin Tool showing the new bufferpool allocation in onstat –g seg. id The ID of the shared memory segment key The shared memory key associated with the shared memory segment ID addr The address of the shared memory segment size The size of the shared memory segment in bytes ovhd The size of the shared memory segment control information (overhead) in bytes class The class of the shared memory segment (R is for Resident, V is for Virtual, VX is for Virtual Extended, and M is for Message.) blkused The number of blocks of used memory blkfree The number of blocks of free memory If LOW_MEMORY_RESERVE is specified in the configuration file $ONCONFIG, also seen here (not shown above) will be the following: Virtual segment low memory reserve (bytes) The size of reserved memory for use when critical activities are needed and the server has limited free memory, specified in bytes (You specify reserved memory in the LOW_MEMORY_RESERVE configuration parameter.) Low memory reserve used 0 times and used maximum block size 0 bytes) The number times that the server has used the reserved memory and the maximum memory needed © 2014 IBM Corporation

onstat –g buf – Monitor Buffer Pools – Memory Format
Produces output per bufferpool in use. Will report either format in use. onstat –g buf produces output on a per buffer pool size basis. Output description Buffer pool page size Number of bytes in a page in the buffer pool being profiled dskreads Number of disk read operations performed to bring pages into this buffer pool. Each read operation reads one or more pages. pagreads Number of pages read from disk to this buffer pool bufreads Number of times a memory image for a page was read from this buffer pool %cached Percentage of page reads for this buffer pool that were satisfied by a cached page image (rather than having to perform a disk read). Computed as (bufreads - dskreads) / bufreads x 100. Higher percentages indicate better caching performance. dskwrits Number of disk write operations performed to write changed pages from this buffer pool back to disk. Each write operation writes one or more pages. pagwrits Number of pages written to disk from this buffer pool bufwrits Number of times a memory image of a page was written to in this buffer pool Percentage of page writes for this buffer pool that were satisfied by a cached page image (rather than having to perform a disk write). Computed as (bufwrits - dskwrits) / bufwrits x 100. bufwrits_sinceckpt Number of times a memory image of a page was written to in this buffer pool since the last checkpoint bufwaits Number of times a thread had to wait for a lock on a buffer in this buffer pool. Higher numbers indicate more contention among multiple threads for mutually incompatible locks on the same pages. ovbuff Number of times a changed buffer from this buffer pool was written to disk specifically to create a free buffer to read another requested page. If the ovbuff value is high, it may indicate that the buffer pool is not large enough to hold the working set of pages needed by the applications using this buffer pool, which may lead to performance degradation from I/O thrashing. flushes Number of times the server performed a mass flush of all dirty buffers in the buffer pool. This can occur for a variety of reasons, including as part of checkpoint processing or if the buffer pool is running out of clean buffers despite normal LRU cleaning activity. Fg Writes Number of changed buffers from this buffer pool that were written to disk by a non-I/O flusher thread that was accessing the buffer. This number is a superset of ovbuff. In addition to the writes to service page faults counted by ovbuff, this value also includes foreground writes that are done by certain operations to maintain the consistency of database logs and reserved pages in order to guarantee correctness of crash recovery in special cases. This number should always be zero; if it is not, this is usually sign there are not enough buffers in the instance resident memory. Typically, this number, if > 0, is the same as ovbuff’s number. See ovbuff above. LRU Writes Number of changed buffers from this buffer pool that were written to disk by an LRU cleaner thread. LRU cleaners are activated if the buffer pool exceeds the value specified in the lru_max_dirty field of the BUFFERPOOL configuration parameter or if foreground writes occur due to buffer pool overflows. Avg. LRU Time Average amount of time taken by an LRU cleaner thread to clean a single LRU chain. Chunk Writes Number of changed buffers that were written to disk by a chunk cleaning operation. Chunk cleaning writes out all changed buffers of a given chunk that are in the buffer pool. This is done in a variety of special circumstances that need to clean a large number of buffers quickly, such as checkpoint processing and fast recovery. Fast Cache Stats Statistics for the fast cache, which is a type of cache that reduces the time needed for accessing the bufferpool gets The number of times the server looked for a buffer in the fast cache hits The number of times that the server found the buffer it was searching for in the fast cache %hits The percentage of hits, which is hits*100/gets puts The number of times that the server inserted buffers inserted into the fast cache © 2014 IBM Corporation

Onstat –g buf – Legacy Format
Really poor read performance here (89.69%) – ovbuff and Fg Writes have the same number (24630), indicative that you were out of buffers in the bufferpool, in this case, 24,630 times; solution: add buffers to the bufferpool. In addition to the Read Cache triggering additional BUFFERS, Fg_Writes > 0 will also automatically trigger additional BUFFERS. This is the traditional legacy way of monitoring buffer pools within the resident segment and this does not show the new memory format if the pre 12.10xC3 behavior is still active. Image above shows a instance badly tuned from write cache. From the ovbuff and Fg_Writes (foreground writes, direct calls to a disk for a page, should always be 0) sections we have been running out of memory and %cached for writes bears this out with a terrible write cache ratio of 85.6%. Waits on buffers (bufwaits) are high. %cache for reads is at 89.69%; here we have been running out of memory. Output description Buffer pool page size Number of bytes in a page in the buffer pool being profiled dskreads Number of disk read operations performed to bring pages into this buffer pool. Each read operation reads one or more pages. pagreads Number of pages read from disk to this buffer pool bufreads Number of times a memory image for a page was read from this buffer pool %cached Percentage of page reads for this buffer pool that were satisfied by a cached page image (rather than having to perform a disk read). Computed as (bufreads - dskreads) / bufreads x 100. Higher percentages indicate better caching performance. dskwrits Number of disk write operations performed to write changed pages from this buffer pool back to disk. Each write operation writes one or more pages. pagwrits Number of pages written to disk from this buffer pool bufwrits Number of times a memory image of a page was written to in this buffer pool Percentage of page writes for this buffer pool that were satisfied by a cached page image (rather than having to perform a disk write). Computed as (bufwrits - dskwrits) / bufwrits x 100. bufwrits_sinceckpt Number of times a memory image of a page was written to in this buffer pool since the last checkpoint bufwaits Number of times a thread had to wait for a lock on a buffer in this buffer pool. Higher numbers indicate more contention among multiple threads for mutually incompatible locks on the same pages. ovbuff Number of times a changed buffer from this buffer pool was written to disk specifically to create a free buffer to read another requested page. If the ovbuff value is high, it may indicate that the buffer pool is not large enough to hold the working set of pages needed by the applications using this buffer pool, which may lead to performance degradation from I/O thrashing. This should always be 0. If > 0 then you need more buffers. flushes Number of times the server performed a mass flush of all dirty buffers in the buffer pool. This can occur for a variety of reasons, including as part of checkpoint processing or if the buffer pool is running out of clean buffers despite normal LRU cleaning activity. Fg Writes Number of changed buffers from this buffer pool that were written to disk by a non-I/O flusher thread that was accessing the buffer. This number is a superset of ovbuff. In addition to the writes to service page faults counted by ovbuff, this value also includes foreground writes that are done by certain operations to maintain the consistency of database logs and reserved pages in order to guarantee correctness of crash recovery in special cases. This number should always be 0; if it is not, this is usually sign there are not enough buffers in the instance resident memory. Typically, this number, if > 0, is the same as ovbuff’s number. See ovbuff above. LRU Writes Number of changed buffers from this buffer pool that were written to disk by an LRU cleaner thread. LRU cleaners are activated if the buffer pool exceeds the value specified in the lru_max_dirty field of the BUFFERPOOL configuration parameter or if foreground writes occur due to buffer pool overflows. Avg. LRU Time Average amount of time taken by an LRU cleaner thread to clean a single LRU chain. Chunk Writes Number of changed buffers that were written to disk by a chunk cleaning operation. Chunk cleaning writes out all changed buffers of a given chunk that are in the buffer pool. This is done in a variety of special circumstances that need to clean a large number of buffers quickly, such as checkpoint processing and fast recovery. Total Mem Total amount of memory allocated to the bufferpool # extends # of times bufferpool has been extended max extends Maximum # of times a bufferpool can be extended next buffers # of buffers the next extend will allocate cache hit ratio Cache hit ratio threshold for extending bufferpool last Time of last extend id Segment id within bufferpool segments segment address Address of segment (internal) size Size of segment # buffs # of buffers in segment Fast Cache Stats Statistics for the fast cache, which is a type of cache that reduces the time needed for accessing the bufferpool gets The number of times the server looked for a buffer in the fast cache hits The number of times that the server found the buffer it was searching for in the fast cache %hits The percentage of hits, which is hits*100/gets puts The number of times that the server inserted buffers inserted into the fast cache © 2014 IBM Corporation

VPCLASS This configuration parameter is now expanded to use with cpu class to make the cpu class extendable. VPCLASS class,aff=range,noage,num=N,autotune,max=N autotune is optional. When set, IDS will determine when to add another VP of class. max is the maximum number of CPU VPs we can extend to. When using autotune, affinity will not be used for any cpu vps added as part of automatic tuning. This is again for performance reasons, adding an additional cpu vp dynamically thru autotune allows more resource availability to make those transactions complete quicker due to less likelihood of there being not enough cpu to complete a given operation quickly. Manual intervention is likely to be not as quick in many such cases. Max should be the total number of physical cpu vps on the machine. © 2014 IBM Corporation

CPU VP’s Detection Configuration Expansion
CPU VPs are added whenever the average ready Q count across all the CPU VPs for the last minute exceeds 4. If the average depth of the ready Q is >= 5, we add another CPU VP up to a maximum of the number of physical processors on the box. Monitoring the ready queue (onstat –g rea, large number of threads sitting waiting), and the global usage of the CPU virtual processors (onstat –g glo, the eff column always at 100). Configuration A new argument to the VPCLASS configuration parameter format has been added: autotune. This argument will be processed only for the CPU and AIO vp classes. For more info see VPCLASS below. Expansion Internal procedure to add a cpu vp. See Appendix A for Event Alarm relevant to this. © 2014 IBM Corporation

onstat –g rea – Threads Ready to Run
onstat –g rea IBM Informix Dynamic Server Version FC3 -- On-Line -- Up 2 days 23:53: Kbytes Ready threads: tid tcb rstcb prty status vp-class name 6 536a ready 3cpu main_loop() 28 60cfe8 40a124 4 ready 1cpu onmode_mon a20 409dc4 2 ready 3cpu sqlexec Ready threads: Status is ready here, which means it always ready to run. A quick way to tell if the system cpus are processing as they should is to simply run a slight variant of this monitor. Run onstat –gr rea. The -r has a default repeat option of 5 seconds; which is to say that the same command will repeat the current ready queue every 5 seconds. If the same sqlexec threads with the same thread id’s (tid) are there every five seconds, it is quite possible that there are not enough of them (they should all be running at 100% - see onstat –g glo) or some other cause (lots of extents on the query tables are also a possibility here, which can be caused by I/O wait of traversing all of the extents if there is a lot of them, backing up into CPU wait as a result.). Column headers for the above: tid Thread ID tcb Thread control block access rstcb RSAM thread control block access prty Thread priority status Thread status (this is always “ready”) vp-class Virtual processor class name Thread name © 2014 IBM Corporation

onstat –g cpu Column header output: tid The ID of the thread
name The name of the thread vp The ID of the virtual processor in which the thread is running Last Run The timestamp when the thread last ran CPU Time The time taken until now by the thread #scheds The number of times the thread was scheduled to run status The status of the thread. Possible status values are: cond wait IO Idle join wait mutex wait ready sleeping terminated running yield © 2014 IBM Corporation

onstat –g glo – Monitor your CPU usage
The following table explains each column in the global information section of the example output above. Column name Description sessions The number of running sessions threads The total number of threads vps The total number of virtual processors lngspins The number of times a thread had to spin more than 10,000 times to acquire a latch on a resource time The number of seconds over which the statistics were gathered. Statistics start when the server starts or the statistics are reset by running the onstat –z command. sched calls The total number of scheduled calls. thread switches The total number of switches from one thread to another. yield Statistics on thread yields, which occur when a thread can no longer continue its task until some condition occurs class The type of virtual processor. The following are the class types of virtual processors that can appear here, as well as the class column further below: adm – Administrative Virtual Processor adt – Auditing Virtual Processor aio – Disk I/O Virtual Processor bts – Basic Text Search Virtual Processor cpu – CPU Virtual Processor csm – Communications Support Module Virtual Processor dwavp – Data Warehousing Virtual Processor encrypt – Encryption Virtual Processor idsxmlvp – XML Virtual Processor jvp – Java Virtual Processor lio – Logical Log Virtual Processor mq – MQ Messaging Transaction Virtual Processor msc – Miscellaneous Virtual Processor pio – Physical Log Virtual Processor shm – Shared Memory Connection Virtual Processor soc – Socket Network Connection Virtual Processor tli – Transport Layer Interface Connection Virtual Processor wsvp – Web Feature Service Interface Virtual Processor classname - Runs user-defined routines in a thread-safe manner so that if the routine fails, the database server is unaffected. Specified with the VPCLASS configuration parameter. You must specify classname. vps The number of instances of the class of virtual processor. usercpu The total user time, in seconds, that the class of virtual processor spent running on the CPU. syscpu The total system time, in seconds, the class of virtual processor spent running on the CPU. total The total CPU time for the virtual processor class, as the sum of the user time plus the system time. vp The virtual processor number. pid The Process ID of the oninit process. class The type of virtual processor. usercpu The total user time, in seconds, that the virtual processor spent running on the CPU. syscpu The total system time, in seconds, that the virtual processor spent running on the CPU. total The total CPU time for the virtual processor, as the sum of the user time plus the system time. thread The total time the threads ran on the virtual processor. eff Efficiency. The ratio of the total CPU time to the total time the threads ran on the virtual processor. The Eff column measure processor efficiency usage, if all CPU class processors are at 100% usage then it may be advisable to add another. If one CPU is < 100%, then is probably ok as it means it has reserve capacity to handle new workload immediately. If multiple cpus have eff < 100% and by a substantial amount under 100%, then too many CPU’s may have been allocated. © 2014 IBM Corporation

AIO VP with AUTOTUNE Use a VPCLASS configuration parameter entry for the AIO virtual processor class to specify an exact number of AIO virtual processors or to enable the database server to add AIO virtual processors as needed. Again this being done for performance, it is sometimes hard for a DBA to know when he might need an AIO VP and where to look to get this information. AIO VP’s are used on operating systems that do not support KAIO, the database server uses the AIO class of virtual processors to perform database I/O that is not related to physical or logical logging. The database server uses the CPU class to perform KAIO for database I/O when KAIO is available on a platform. If the database server implements KAIO, a KAIO thread performs all I/O to raw disk space, including I/O to the physical and logical logs. KAIO – Kernel Asynchronous I/O. © 2014 IBM Corporation

AUTO_TUNE_SERVER_SIZE
Configuration parameter used to buffer pool size when automatic tuning is not possible. When BUFFERPOOL:memory is set to auto, we set memory using AUTO_TUNE_SERVER_SIZE. Sets internal structures to assume that’s how much memory will get used. Therefore, it is possible to exceed these amounts. AUTO_TUNE_SERVER_SIZE [OFF | SMALL | MEDIUM | LARGE | XLARGE] OFF use 10% available memory SMALL use 10% available memory MEDIUM use 20% available memory LARGE use 33% available memory XLARGE use 50% available memory © 2014 IBM Corporation

This is set only when you install the product and request, as part of the installation process, that a server instance be created: It sets the sizes of memory and storage spaces to allocate based on the number of expected concurrent users provided by the user during the install. SMALL = users MEDIUM = users LARGE = users XLARGE = more than 1000 users The setting affects the following: The size of the buffer pool. The maximum size of logical log files (from 200 MB up to 2 GB) before the server stops automatically adding logical logs to improve performance The initial size (from 50 MB to 500MB) of the following created storage spaces, which are created automatically during installation: An extendable plogspace for the physical log A dbspace for the logical log Dbspaces for databases and tables A temporary dbspace An sbspace A temporary sbspace © 2013 IBM Corporation

This parameter can be used even if you did not create a server during installation, or if you change its value after you initialize the server for the first time. In this case, the new value affects the size of only the following properties: The size of the buffer pool, if the BUFFERPOOL configuration parameter setting includes the memory='auto' option. The maximum size of all logical log files before the server stops automatically adding logical logs to improve performance. Auto-instance-creation with the physical log, logical log and smart large object spaces, and 2 tempspaces all in their own separate dbspaces, along with the separately created rootdbs, will lead to less down time, fewer configuration steps, and increased ‘out-of-the-box’ experiences for all users, from the very first initial install. At this point, all they have to do is create databases. But where? © 2013 IBM Corporation

AUTOLOCATE Configuration Parameter
This controls the dbspace location of user defined databases, tables, and indexes and the automatic fragmentation of tables. If 0 (default) this is disabled: New databases still default created in rootdbs New table and indexes still default created in the same dbspace as the database Set from 1 to 32, enables automatic location and fragmentation: Indicates how many round-robin fragments to initially allocate to a table. Stores new databases, tables, and indexes in server defined optimal dbspaces: By default, all dbspaces are available. Users can control the list of available dbspaces. Fragments new tables by round-robin, where the number of fragments is equal to the value of the AUTOLOCATE configuration parameter. Adds more table fragments as the table grows. “In” dbspace_name clause overrides © 2013 IBM Corporation

AUTOLOCATE Configuration Parameter
If enabled, you can use the autolocate database arguments with the admin() or task() function to Manage the list of dbspaces for automatic location and fragmentation: The list of available dbspaces is in the sysautolocate system catalog table. Disable automatic location and fragmentation for the specified database. You can use the AUTOLOCATE environment option of the SET ENVIRONMENT statement of SQL to enable or disable the value of the AUTOLOCATE configuration parameter for a session. Users can still override automatic server based allocation of location by using the “in dbspace” clause of the create database, table and index clauses © 2013 IBM Corporation

AUTOLOCATE It is anticipated that this will lead to fewer outages caused by having user defined databases and tables and indexes being stored in the rootdbs and causing that space to fill up completely (and stop the instance) or extend/expand unnecessarily beyond its original size. This has been a long standing user issue and seen in many places. This is a good feature. Stands alone as a feature. In combination with the implementation of the storage pool it will be very difficult, if configured and administered properly initially, to run out of space. Best practice. With the storage pool feature implemented. © 2013 IBM Corporation

SQL Admin API / onmode –wf/wm
Turn off autolocation for the stores database: execute function task(“autolocate database off”, “stores”); Turn on autolocation for the stores databases: execute function task(“autolocate database add”, “stores”, “datadbs1”); execute function task(“autolocate database add”, “stores”, “datadbs1,datadbs2,datadbs3,datadbs4”); execute function task(“autolocate database anywhere”, “stores”); Remove datadbs2 from the list of available dbspaces for the stores database: execute function task("autolocate database remove", “stores", "datadbs1"); onmode –wf/wm © 2013 IBM Corporation

Other Server Improvements
Improve performance with private memory caches Default mode for VP_MEMORY_CACHE_KB is STATIC To change it to DYNAMIC make changes to ONCONFIG VP_MEMORY_CACHE_KB 800, DYNAMIC Best Practice Tip: If using DYNAMIC, memory caches grow and uses more memory Consider setting SHMTOTAL to a specific value instead of 0 Complex Query processing improvements Use View folding mechanisms Avoid materialized views or large temporary tables More efficient when query processing through 3rd party BI tools Private memory caches can improve performance for memory that is allocated by threads in the Informix® server. Private memory caches have no impact on the memory that is allocated to and used by buffer pools or shared memory communication. When you set the value of the VP_MEMORY_CACHE_KB configuration parameter to a number other than 0, a private memory cache is created for each CPU virtual processor. By default, size of all private memory caches combined is limited to the specified number of KB. If you want the size of each private memory cache to increase and decrease automatically, as needed, include a comma and the word DYNAMIC after the size, for example, VP_MEMORY_CACHE_KB 1000,DYNAMIC. Although the maximum initial size of all private memory caches combined cannot exceed 40 percent of the value of the SHMTOTAL configuration parameter, with DYNAMIC mode set, the size of the caches can expand beyond the initial limit. The total size of the caches cannot exceed the value of the SHMTOTAL configuration parameter. Dynamic memory caches on busy systems can grow quickly and use a large amount of available memory. Therefore, if you set the mode to DYNAMIC, set the SHMTOTAL configuration parameter to a specific limit instead of the default value of 0, which does not limit the amount of memory. Complex Query processing improvements 3rd Party tools like Cognos / Business objects / Tableau formulate complex queries to get to the business intelligence / analytics out of the data. Often these queries are huge practically incomprehensible and take too much processing time. They involve building materialized views or large temporary tables. With view folding mechanism and avoiding materialized views and large temporary tables it is now possible to get more efficiency when processing queries from 3rd party BI tools. © 2014 IBM Corporation

INFORMIXCONTIME & INFORMIXCONRETRY
INFORMIXCONTIME & INFORMIXCONRETRY will now support remote servers INFORMIXCONTIME Indicates the number of seconds before reporting a connection failure INFORMIXCONRETRY Indicates the number of connection attempts to retry before reporting a connection failure Precedence Session level through set environment Client level through clients user environment Server level through configuration parameters Can also be set with onmode –wf/wm 47 © 2014 IBM Corporation 47

GLS Improvements With GLS 6.00xC4, specify the display widths that DB-Access and other character-based Informix applications use for characters in the Unicode Private Use Area (PUA) ranges: First, set the new IFX_PUA_DISPLAY_MAPPING environment variable Second, create a mapping file: $INFORMIXDIR/gls/etc/pua.map. List each character in the file followed by the character representation display width. Valid display widths in the file are: 1 (halfwidth character representation, default) or 2 (fullwidth character representation). If there is not a display width for a character in the file, the default is halfwidth. © 2014 IBM Corporation

Automatic removal of in-place alters
Removing outstanding in-place alter operations improves performance Remove outstanding in-place alter operations for tables or fragments with: IBM OpenAdmin Tool (OAT) for Informix Update_ipa or fragment update_ipa argument of the admin() or task() SQL administration command. Previously, a dummy UPDATE was done on tables to remove outstanding in-place alter operations. Speed up operations faster by including the parallel option with the table update_ipa or fragment update_ipa argument of the admin() or task() SQL administration command. A prerequisite for reverting to an earlier version of Informix. An example showing removal of pending in-place alter operations on a table that is named auto in parallel: EXECUTE FUNCTION task("table update_ipa parallel","auto"); © 2014 IBM Corporation

Storage Optimization Improvements
Parallelized Storage Optimization Compress, Uncompress, Repack of table and index data in parallel Parallel operations reduces maintenance windows Examples: EXECUTE FUNCTION task("table compress rows parallel","dental"); EXECUTE FUNCTION task("fragment uncompress rows parallel"," "); EXECUTE FUNCTION task("table update_ipa parallel","auto"); EXECUTE FUNCTION task("index compress repack shrink parallel","ind5", "customer", "jayson"); Compress data rows in table dental in parallel Uncompress data rows in a fragment in parallel Perform dummy updates to table auto which has in-place alters in parallel. Compress / Repack / Shrink the index named ind5 on customer table owned by user “jayson” in parallel 50 © 2015 IBM Corporation 50

Limit size of extendable storage spaces
Use SQL Administration API functions to set the limits Arguments are space name / max size / new create size / new extend size Useful for multi-tenant Informix instances EXECUTE FUNCTION task("modify space sp_sizes", "dbspace3", "60000", "10000", "100000"); dbspace3 is modified with new chunk size of KB of space when expanding the storage pool entry for a storage space, a new extend size of KB and a maximum total size of KB. EXECUTE FUNCTION task("modify space sp_sizes", "dbspace8", "20", "1.5"); dbspace8 is modified with new chunk size of 20 % of storage space, an extend size of 1.5 % of the storage space and a maximum size of undefined. The example defines the minimum create size to 60 MB, the minimum extend size to 10 MB, and the maximum size to 100 MB for a dbspace that is named dbspace3: The second example defines the minimum create size to 20 percent and the minimum extend size 1.5 percent for a dbspace that is named dbspace8: 51 © 2015 IBM Corporation 51

Primary Storage Manager
© 2014 IBM Corporation

Primary Storage Manager (PSM) Faster and more functional than ISM
Ontape ease of use with Onbar functionality; No daemons, no configuration files. 46 files in ISM vs. 2 files in PSM. 25 MB to 1 MB. Transfer buffer size is 2GB vs. ISM’s 62 KB. OnBar is our fastest solution for backup and recovery because it supports parallel backup and restore. OnBar has a simple interface, however it depends upon an additional component called the Storage Manager that manages the actual devices. The Storage Manager can be very Complex and Expensive. Enters ISM Introduced in IDS 7.3x And XPS 8.2x. As a free SM to use. It was licensed and rebranded from an old version of Legato Networker. ISM is a limited version of networker, it supports only 4 devices and only local connections. It has been stated by our customers that it is difficult to understand & configure and performance is restricted due the multiple layers of software for features they are not allowed to use. XBSA = X/Open Backup Services API – API that manages the exchange between a calling utility and PSM: - Control data (verify that the calling utility and XBSA are compatible and to ensure that objects are restored to the proper instance of database server, and to keep track of the history of the backup objects) - Backup/Restore data (dbspaces, blobspaces, sbspaces, and logical log files) ISM – Informix Storage Manager – IBM licensed version of Legato Networker which ships with Onbar Problems with ISM are: - poor performance, limitation to 4 storage devices, difficulty of configuration and management, - requirement to run background daemons, cannot easily clone an instance from an ISM backup No client side like ISM or TSM. No separate daemon. OnPSM is a simple C-ISAM program that interfaces with the catalogs and files directly Primary Storage Manager (PSM) Faster and more functional than ISM Easy way to configure OnBar to take backups and restores Easy way for embedded solutions to take parallel backups using OnBar Easy way for Ontape customers to migrate to OnBar Replaces ISM © 2014 IBM Corporation

A brief history of Informix backup utilities ontape For years, the only backup/restore (B/R) available Operated serially Supported a single device (tape or disk) for backing up the instance and logical logs In Informix 11, intelligent file naming for instance backups to disk to prevent overwriting a previous backup Fixed with the introduction of backups to directories and the IFX_ONTAPE_FILE_PREFIX parameter ontape worked well when database sizes were small but as they grew, Informix implemented a two-tier solution that provided more options More sophisticated backup devices Disk, tape, jukeboxes The ability to backup multiple dbspaces in parallel © 2014 IBM Corporation

The first tier – ON-Bar Informix’s implementation of the XBSA API defined by the X/Open organization An open interface between database servers and tape management systems to support B/R operations Contains a number of services and components to support B/R tasks including communication, read/write tasks, metadata creation/searching etc. ON-Bar is the pipe that moves data to / from the instance and the backup software Is used to connect to backup solutions like Tivoli Storage Manager The second tier – the Informix Storage Manager (ISM) A limited use and functionality backup solution included with the engine A licensed port of a (now very old) B/R product For customers who needed something between ontape and a full-fledged tape management system ISM was originally built by Legato and licensed by Informix and then IBM. © 2014 IBM Corporation

The ISM had a number of limits and disadvantages Only 4 “simple” backup devices (tape or disk) directly connected to the physical server Difficult to configure and administer Code line was difficult to maintain since it was third-party and very, very old IBM had to pay a license fee to use this ancient, difficult to maintain code With the rapid decrease in disk-based storage, most small-to-mid sized customers are now backing up to disk No longer need ISM with its problems Need a faster, parallelized solution to get instance data to disk Solution – The Primary Storage Manager (PSM)! © 2014 IBM Corporation

The PSM Replaces the ISM Has better integration with the ON-Bar API Is simple to configure Only 1 parameter at its most basic well maybe 2 Is simple to administer using the onpsm utility Provides a parallelizable backup/recovery stream to a disk device Provides many of the features of a more full-featured tape management system Ability to define storage pools, including devices, sizes and priorities Expire and prune backups Using the onsmsync utility Some export capabilities to perform a remote restore Object and meta data verification, locking and unlocking Supports optional backup compression and/or encryption Through the BACKUP_FILTER parameter ISM does not ship with 12.1. © 2014 IBM Corporation

Components of the PSM The XBSA library Single shared library, called dynamically by ON-Bar operations Default location Linux/Unix/Mac: $INFORMIXDIR/lib/libbsapsm.<sharedlib suffix> Windows: %INFORMIXDIR\bin\libbsapsm.dll The onpsm utility CLI-based front end tool to administer objects in the storage manager The PSM catalogs C-ISAM tables stored in $INFORMIXDIR/etc/psm Repository of information for all backups, devices, pools, etc. © 2014 IBM Corporation

Components of the PSM (cont) The ixbar.<servernum> file (aka ON-Bar emergency boot file) Contains information to execute a cold restore What was backed up and when The onsmsync utility Used to synch information between the sysutils database, the ixbar file and the PSM catalog repository © 2014 IBM Corporation

There is very little to configure for the PSM to work Required Create one (or more) directories where instance and logical backups will be created Can have different locations for logical logs and instance backups Fixed sizes can be assigned, as shown later, to prevent overflowing the file system Configure the PSM_DEVICE parameter with one fully pathed directory Default location for instance and logical backups Can be over-ridden with additional configuration Optional, though highly recommended Configure the BAR_BSALIB_PATH parameter to point to the XBSA library Execute backups!!! © 2014 IBM Corporation

There is very little to configure for the PSM to work (cont.) It is possible to execute a backup without any configuration at all Backups would be created in $INFORMIXDIR/backups Probably not the best location Would want to specify one or more locations via onpsm operations Full syntax discussed later From a minimalist perspective, there is one optional, though highly recommended initial configuration setting Configure the BAR_BSALIB_PATH parameter to point to the XBSA library © 2014 IBM Corporation

Additional configuration parameters to consider PSM_ACT_LOG – log for PSM activities Not much is recorded here PSM_DEBUG_LOG – debug file location PSM_DEBUG – used to set the level of debug messages 0 (off) to 9 (high) PSM_CATALOG_PATH – location of the PSM catalog repository if not in the default location of $INFORMIXDIR/etc/psm PSM_DBS_POOL – the name of the storage pool for instance backups Default name: DBSPOOL Unique names are strongly encouraged if multiple instances are active on a server PSM_LOG_POOL – the name of the storage pool for instance backups Default name: LOGPOOL © 2014 IBM Corporation

Additional configuration parameters to consider (cont.) BAR_MAX_BACKUP – Controls the number of parallel data streams from the instance to the PSM Regardless of this value, only one output file for each space is created Illustrated later BAR_XFER_BUF_SIZE – Can be used to set a custom data transfer buffer size LTAPEDEV – can not be set to /dev/null if backing up with ON-Bar Some restore operations, including specific moments in time, may require logical log restore operations © 2014 IBM Corporation

Three interesting quirks to the PSM First, critical instance files are backed up as part of an instance backup operation The ixbar.n, $ONCONFIG, sqlhosts, oncfg files Stored in a separate directory structure, shown later Second, the PSM organizes data differently As administrators, used to dealing with backup “levels” 1 2 While the PSM supports these levels, the PSM organizes the backup output and meta data by “generation” and “descendants” Some administrative operations affect entire generations Such as export or expire Third, the PSM uses a single name for all output objects, consists of Instance name Descendant version Part number Example: inst_2.2.1 © 2014 IBM Corporation

In a newly created and populated instance, suppose these actions occur: A L0 backup is created The PSM considers this the beginning of a new generation A L2 backup is created The PSM considers this a “first descendant” Is stored in a different location than the previous L0 backup A L1 backup is created The PSM considers this as a “first descendant” as well Is stored in the same location as the previous L2 backup The L2 descendant is NOT deleted In order to support point-in-time restore The PSM considers this a “second descendant”, linked to the previous L1 Is stored in a different location than the L1 and earlier L2 backup It is a different descendant class than the earlier L2 L0 = Level-0 backup L1 = Level-1 backup L2 = Level-2 backup © 2014 IBM Corporation

Let’s see this in action with the inst_2 instance The DBSPOOL is configured to /opt/IBM/informix/backups First backup (L0) occurs at 5:13 pm Directory automatically created for the inst_2 instance objects Directories created for the instance spaces. The “2” directory contains logical log backups. Labeled “2” because it’s DBSERVERNUM of instance © 2014 IBM Corporation

Looking at the directory for the rootdbs A single directory for new backup generations is created Populated with the L0 backup of the space For new generations Backup of the dbspace. Notice the file name. © 2014 IBM Corporation

Look in more detail at what happens in the data_space_1 directory Output created by L0 backup in the new generation subdirectory for this space. Notice the output file name is identical to that in the rootdbs directory structure. © 2014 IBM Corporation

Look in more detail at what happens in the data_space_1 directory (cont.) Data changes occur in the dbspace L2 backup executed at 5:22 pm Output created is created in the FIRST descendant directory. Notice the file name. Nothing in the SECOND descendant directory © 2014 IBM Corporation

Look in more detail at what happens in the data_space_1 directory (cont.) More data changes occur in the dbspace L1 backup executed at 5:26 pm Additional output created is created in the FIRST descendant directory. Notice the file name. It’s the second version of a first descendant. Still nothing in the SECOND descendant directory © 2014 IBM Corporation

Look in more detail at what happens in the data_space_1 directory (cont.) Still more data changes occur in the dbspace L2 backup executed at 5:28 pm Nothing new added to the FIRST descendant directory Finally, output in the SECOND descendant directory. Notice the file name. © 2014 IBM Corporation

Conceptually this makes sense To restore to very beginning: Only need the content of the new generation (aka 0) directories. To restore to the first L2 backup (and changes in the data_space_1 space): Need the new generation AND The first descendant with the L2 backup. To restore to the L1 backup Plus the second version of the first descendant with the L1 backup: Supersedes the L2 backup. To restore to the last L2 backup Need the new generation, The second version of the first descendant with the L1 backup AND The second descendant with the L2 backup: Because the L2 backup contains changes occurring after the L1 backup. © 2014 IBM Corporation

Let’s look at some of the other PSM components (cont.) If ixbar doesn’t exist when the first ON-Bar / PSM backup is executed, it is automatically created as part of the operation No ixbar file It is created and the backup continues © 2014 IBM Corporation

The onpsm utility is used to manage backup devices and the files created by backup operations Can add and delete “devices” or output locations Can check the consistency of catalog tables Can also delete the catalog tables Can unlock the catalog tables if a B/R operation terminates abnormally Can list or delete backup files created by backup operations Not recommended as the primary method of deleting backups as shown later Main syntax tree onpsm -C – check consistency of (or delete) the PSM catalog tables -D – manage backup devices and pools -L – manage locks on the PSM catalog tables -O – list (or delete) backup files created by a backup operation © 2014 IBM Corporation

onpsm - device management -D Parameters for adding a device or backup location Only one device in a pool can be classified with a HIGH priority If multiple devices are configured with LOW, the actual device used for a backup operation will be randomly selected For tape sizes, 0 (zero) can be used to indicate “unlimited” or write until the device returns a “media full” error. Other sizes can be entered using the B, K, M, G, T, or P suffix (for bytes, kilobytes, megabytes, gigabytes, terabytes, or petabytes) add full_path -p [ HIGH | LOW ] – the priority for the location -g pool_name -t [ FILE | TAPE ] -s size – optional, maximum output size if using a tape -b block_size – optional, the block size for the tape device © 2014 IBM Corporation

onpsm - device management -D Use list to list configured devices (cont.) With the –u (lowercase) file_name parameter, can export the device configuration to a named flat file With the –l (lowercase) file_name parameter, an exported list of devices can be imported into the PSM for use © 2014 IBM Corporation

onpsm - device management -D To delete a device from a pool del device_path –g pool_name [ -y ] To modify device information update device_path -p [ HIGH | LOW ] – optional, if needed to change device prioritization -s size – optional, if needed to change tape device size With the B, K, M, G, T, or P suffix © 2014 IBM Corporation

onpsm – backup object management -O (“oh”, uppercase) One or more backup objects can be deleted from disk using the del parameter This does NOT refresh information in the ixbar file, it still thinks the object is there For example, in this list object 85 is the rootdbs for a third generation L0 backup Is also the last generation created Notice the file name – inst_2.3.1 © 2014 IBM Corporation

onpsm – backup object management -O (“oh”, uppercase) The only way, in this case, to successfully restore the system is to use point-in-time syntax to execute a second (or first) generation restore Take away from the onpsm –O del discussion Should only be used for objects that have already been deleted (or expired) from the ixbar file but somehow didn’t get properly removed from disk Use the “expiration” functionality in onsmsync to properly remove “old” backups © 2014 IBM Corporation

The onsmsync utility is used to perform several operations Synchronize: The sysutils database The ixbar file The PSM catalog tables Expire old backups Will delete the backup files (on disk) Prune the ixbar and catalog / sysutils tables as needed Can recreate a missing or damaged ixbar file so a restore can occur Can “export” a backup so it can be used to restore on a different server Can “import” a backup so it can be used in a restore operation © 2014 IBM Corporation

Administration Features - Storage Manager Deduplication
De-duplication as an alternative to incremental backups Eliminates redundant data in backups Less storage requirements Decrease load on network if storage devices are not on the same machine where the backup is being taken Usable with storage managers Enabled with IFX_BAR_USE_DEDUP Can not use DEDUP and incremental backups at the same time. Support of Tivoli and other storage vendors DEDUPLICATION From documentation: --- De-duplication eliminates redundant data in backups. To enable Informix support for de-duplication, set the IFX_BAR_USE_DEDUP environment variable in the Informix environment and restart the database server. Update the TSM client systems option file. It is not required to set this variable to any value at all, just set it. SUPPORT FOR DE-DUPLICATION TECHNOLOGY IN THE STORAGE MANAGER De-Duplication is a technology used by storage managers that allows them to remove parts of a backup by referencing them to an older backup. It is basically a support for incremental archives in the storage manager. In order for de-duplication to work correctly, the backup objects need to send the data in a predictable way, meaning that the pages must be sent in the same order every time, otherwise the de-duplicator cannot locate similar blocks. Informix Dynamic Server already does this, however our backup algorithm has an oddity that was intended as a mechanism to support incremental backups. The oddity is that our incremental archives depend on a 4-byte timestamp in each page, during the incremental backup, the timestamp of each page is compared against the timestamp of the L 0 backup and if it was modified after the L 0 then is sent to the backup device. However, the timestamp has become too small, modern systems can wrap the 4-byte in a few days, and to deal with this, during the L 0 archive we will update the pages that are about to become out of range with a new timestamp. Once this situation happens for the first time, it will happen every time we take an archive and the de-duplicator will be unable to work SOLUTION: Avoid the update of the pages going to the archive. L1 in a restored system will not be used anyways, and the main system can have already an scenario in which timestamps are out of range. This change requires to set the env variable IFX_BAR_USE_DEDUP at the moment of starting IDS. Support for incremental archives in IDS is still supported. © 2014 IBM Corporation

Multi-Tenancy in Informix 12
April 28, 2017 96

Agenda Multi-tenancy within Informix

Multi Tenancy – Defined
Tenancy refers to the ability of an Informix server to support a set of users in a client organization needing to access the same data and system resources while denying non-administrative users on the same server from accessing those same resources without permission. Multi tenancy is multiple organizations supported on the same server needing to access only their own segregated data and resources while denying other users access to their data and resources. In tenancy, a dedicated database is created and assigned storage and processing resources for that database based on defined service-level agreements with the client organization. It is possible to have services for multiple companies (tenants) that run efficiently within a single Informix instance.

Multi-Tenancy in Informix
A single instance of the software runs on a server, serving multiple client-organizations (tenants). With a multi-tenant architecture, software is designed to virtually partition its data and configuration, and each client organization works with a customized virtual application. Multiple tenant databases within a single instance of Informix Assign resources (cpu / storage) specific to tenant databases Segregate data specific to each tenant Informix instance will share certain resources across all tenants (pio / lio etc) Capability to associate VP’s specific to tenant databases Differentiation from virtualization In a multitenancy environment, multiple customers share the same application, running on the same operating system, on the same hardware, with the same data-storage mechanism. The distinction between the customers is achieved during application design, thus customers do not share or see each other's data. Compare this with virtualization where components are abstracted enabling each customer application to appear to run on a separate virtual machine. 99

Advantages of a Multi-tenant environment
Shared costs License cost could be shared across all the tenants Several tenants are allocated only a portion of the software and hardware resources Ideally suited for SAAS / PAAS environment Cost effective when compared with virtualization Ideal for service providers Secure and isolated data for each tenant Informix treats each tenant database as an independent unit from the application perspective Storage spaces are specific to each tenant Typically database objects are the same in all tenant databases. Makes for easier implementation in a cloud environment Capability to alter schema (if required) just for tenant(s)

Multi-Tenant Architecture - Features
1 tenant = 1 database Segregated and dedicated storage for tenants Storage provisioning can be used to allocate dbspaces on demand for a multi-tenant system Dbspaces allocated as a small size, but set to auto-expand New dbspace maximum size can now be set to prevent expansion above a certain size. Tenants could share CPU resources using CPU VPs or have user defined VPs per tenant Tenants, however will share VP resources for non user defined VPs (aio, pio, lio, net, etc) Capability to specify session limits on… Locks Manage tenants with a graphical view (OAT) A tenant can see only information relevant to a that specific tenant Storage information for that tenant CPU utilization for that tenant 101

Multi-Tenant Storage Storage provisioning can be used to allocate dbspaces on demand for a multi-tenant system Dbspaces allocated as a small size, but set to auto-expand New dbspace maximum size can now be set to prevent expansion above a certain size.

Multi-Tenancy The illustration shows a possible configuration for two clients in an Informix server instance. Each client has a database and users who are allowed to access the tenant database. Each tenant database has dedicated storage spaces. Both tenant databases share the default temporary sbspace. Tenant A has a tenant virtual processor class with two virtual processors, while Tenant B has a virtual process class with one virtual processor. 103

Creating a Tenant execute function task(“create dbspace from storagepool”, “tenant1_1”, “10m”); execute function task(“create dbspace from storagepool”, “tenant1_2”, “10m”); execute function task(“create sbspace from storagepool”, “tenant1_3”, “10m”); execute function task(“tenant create”, “tenant1”, ‘{ “DBSPACE”:”tenant1_1,tenant1_2”, “SBSPACE”,”tenant1_3”, “VPCLASS”:”VPCLASS tenant1,num=1”, “SESSION_LIMIT_LOCKS”:”2000” }’); Created in sysadmin:tenant table, see Appendix B for details.

Creating a Tenant execute function task(“tenant create”, “tenant1”, ‘{
“DBSPACE”:”dbslist”, “BLOBSPACE”:,”dbslist”, “SBSPACE”:,”dbslist”, “VPCLASS”:”vpclass definition”, “DBSPACETEMP”:”dbslist”, “SBSPACETEMP”:”sblist”, “SESSION_LIMIT_LOCKS”:”value”, “LOGMODE”:”db log mode definition”, “CASE”:”case”, “LOCALE”:”locale definition”, “CATALOGS”:”dbslist” }’);

Multi-Tenant – Limiting Database Creation
DBCREATE_PERMISSION Specifies the users who can create databases (by default, any user can) Example DBCREATE_PERMISSION informix Typically in multi-tenant environments, only user id Informix can have permission to create databases. If this is not restricted it could cause problems like resource utilization (cpu / storage) could go out of control. 106

Multi-Tenancy Management
Use SQL Administration API command to manage tenant databases EXECUTE FUNCTION task('tenant create', 'companyA', '{dbspace:"companyA_dbs1, companyA_dbs2, companyA_dbs3", sbspace:"companyA_sbs", vpclass:"tvp_A,num=6", dbspacetemp:"companyA_tdbs", session_limit_locks:"1000", logmode:"ansi", locale:"fr_ca "'}); EXECUTE FUNCTION task('tenant update', 'companyA', '{dbspace:"companyA_dbs4, companyA_dbs5", sbspace:"companyA_sbs3", vpclass:"tvp_B"}’); The tenant database gains two dbspaces and a sbspace. EXECUTE FUNCTION task('tenant drop', 'companyA'); Cannot just “drop database database_name ” a tenant database. You must ‘tenant drop’ a database created as tenant. Note: The options for the tenant database are treated as a JSON document. Must have DBA privileges or “tenant” privilege to perform tenant operations. The tenant options like dbspace, subspace, vpclass etc are treated as a JSON doc. Return to Agenda 107

The vpclass option for task/admin() tenant update (1)
vpclass_name is limited to 8 characters. A maximum of 200 tenant virtual processor classes can be created. If the vpclass_name is unique, You create a new tenant virtual processor class. If the vpclass_name exists, The tenant database shares the class with other tenant databases. If the vpclass property has an existing value, You change the virtual processor class for the tenant database. If you do not include the num=vps property, one virtual processor is started. IDS Admin Ref for xC4 – Chap. 22, Page 108

The vpclass option for task/admin() tenant update (2)
From a couple of slides earlier: EXECUTE FUNCTION task('tenant update', 'companyA', '{dbspace:"companyA_dbs4, companyA_dbs5", sbspace:"companyA_sbs3", vpclass:"tvp_B"}’); This much is certain: The instance gains two dbspaces and a sbspace. From the previous slide: If the vpclass_name is unique, You create a new tenant virtual processor class. If the vpclass_name exists, The tenant database shares the class with other tenant databases. If the vpclass property has an existing value, You change the virtual processor class for the tenant database. If you do not include the num=vps property, One virtual processor is started.

Multi Tenancy - Session Level Resource Limitations
Session Level Resource Control Limits To improve performance and restrict the tenant database size. Prevents any session owned by non-administrative users from using so many resources that other sessions cannot continue processing transactions. To prevent performance issues. Be useful in embedded environments. Two ways to do this: Via a configuration parameter SQL Admin API task and admin function calls Tenant properties in the tenant definition created when you run the admin() or task() SQL Admin API command with the tenant create or tenant update arguments. Tenant properties have precedence over related configuration parameters.

Multi Tenancy - Session Level Lock Limits (1)
The SESSION_LIMIT_LOCKS configuration parameter specifies the maximum number of non-Admin user locks available for a session. Default is none If not in onconfig file, Values 500 – Not dynamic Administrators can set SESSION_LIMIT_LOCKS to reduce the risk of ordinary users in concurrent sessions depleting the lock resources of the database server. EXECUTE FUNCTION task('tenant create', 'company_A', {dbspace:"company_A_dbs1, company_A_dbs2,company_A_dbs3", sbspace:"company_A_sbs", vpclass:"tvp_A,num=6", dbspacetemp:"company_A_tdbs", session_limit_locks:“2000", session_limit_memory:"110MB", session_limit_tempspace:“35MB", session_limit_logspace:"35MB", session_limit_txn_time:"180", tenant_limit_space:“1TB", tenant_limit_memory:"1GB", tenant_limit_connections:"1000", logmode:"ansi", locale:“en_us ", case:"insensitive",}‘ ); SESSION_LIMIT_LOCKS replaces an undocumented configuration parameter in releases earlier than xC4. In repeatable read isolation level, because each row in the active set requires a lock, be careful about setting too low a limit for locks on the server. Similarly, setting too small a lock limit can interfere with Enterprise Replication tasks or with cdr commands issued by non-DBSA users.

Multi Tenancy - Session Level Lock Limits (2)
You can override the session_limit_locks property for a session by setting the IFX_SESSION_LIMIT_LOCKS environment option to a lower value than the session_limit_locks property value Cannot be less than 500 locks Limits do not apply to administrative users such as DBSA or Informix SET ENVIRONMENT IFX_SESSION_LIMIT_LOCKS '5000'; If the IFX_SESSION_LIMIT_LOCKS session environment option and the SESSION_LIMIT_LOCKS configuration parameter are set to different values, The session environment option takes precedence over the configuration parameter for operations during the current sessions by users who are not administrator, but only until the current session ends If neither are set, the default limit for every user is 2,147,483,647 locks Tenancy setting have precedence over environment setting if tenancy setting is greater than the environment setting To avoid user override, DBA’s can set in sysdbopen. SESSION_LIMIT_LOCKS replaces an undocumented configuration parameter in releases earlier than xC4. In repeatable read isolation level, because each row in the active set requires a lock, be careful about setting too low a limit for locks on the server. Similarly, setting too small a lock limit can interfere with Enterprise Replication tasks or with cdr commands issued by non-DBSA users.

Multi Tenancy - Session Level Memory Limits
Memory size threshold: Set the session_limit_memory to terminate sessions exceeding a specified maximum amount of session shared memory for non-administrative users Off (0) by default in $ONCONFIG and not dynamic The value is in KB, MB or GB and between KB KB Limit does not apply to a user who holds informix or DBSA privileges If the property is not set in the create tenant or update tenant statements for the task or admin functions, the SESSION_LIMIT_MEMORY configuration parameter sets it. If the configuration parameter is also not set, a session can only allocate up to KB. EXECUTE FUNCTION task('tenant create', 'company_A', {dbspace:"company_A_dbs1, company_A_dbs2,company_A_dbs3", sbspace:"company_A_sbs", vpclass:"tvp_A,num=6", dbspacetemp:"company_A_tdbs", session_limit_locks:“2000", session_limit_memory:"110MB", session_limit_tempspace:“35MB", session_limit_logspace:"35MB", session_limit_txn_time:"180", tenant_limit_space:“1TB", tenant_limit_memory:"1GB", tenant_limit_connections:"1000", logmode:"ansi", locale:“en_us ", case:"insensitive",}‘ );

Multi Tenancy - Session Level Temp Space Limits
Set the session_limit_tempspace to the maximum amount of session usable temporary storage for non-admin users, terminate a session exceeding that limit, and write an error out to the message log file. Default is 0 (off) in $ONCONFIG and not dynamic Values are between KB – KB EXECUTE FUNCTION task('tenant create', 'company_A', {dbspace:"company_A_dbs1, company_A_dbs2,company_A_dbs3", sbspace:"company_A_sbs", vpclass:"tvp_A,num=6", dbspacetemp:"company_A_tdbs", session_limit_locks:“2000", session_limit_memory:"110MB", session_limit_tempspace:“35MB“, session_limit_logspace:"35MB", session_limit_txn_time:"180", tenant_limit_space:“1TB", tenant_limit_memory:"1GB", tenant_limit_connections:"1000", logmode:"ansi", locale:“en_us ", case:"insensitive",}‘ );

Multi Tenancy – Session Transaction Log Space Limits
Transaction log space size threshold: Set the session_limit_logsize to the maximum size of a session transaction. Non-Admin user transactions are rolled back if the log size threshold is reached. Default value if 0 (off) and not dynamic Range of values from 5120 KB – KB if set Writes an error in the database server message log. EXECUTE FUNCTION task('tenant create', 'company_A', {dbspace:"company_A_dbs1, company_A_dbs2,company_A_dbs3", sbspace:"company_A_sbs", vpclass:"tvp_A,num=6", dbspacetemp:"company_A_tdbs", session_limit_locks:“2000", session_limit_memory:"110MB", session_limit_tempspace:“35MB“, session_limit_logspace:"35MB", session_limit_txn_time:"180", tenant_limit_space:“1TB", tenant_limit_memory:"1GB", tenant_limit_connections:"1000", logmode:"ansi", locale:“en_us ", case:"insensitive",}‘ );

Multi Tenancy - Session Level Transaction Time Limits
Sets the session_limit_txn_time to the maximum number of seconds that a non-administrative transaction can run. Default 0 (off) and not dynamic Values are between 60 – seconds Prevents individual session transactions from monopolizing the logical log, terminates a transaction over the time limit, writes an error in the message log. EXECUTE FUNCTION task('tenant create', 'company_A', {dbspace:"company_A_dbs1, company_A_dbs2,company_A_dbs3", sbspace:"company_A_sbs", vpclass:"tvp_A,num=6", dbspacetemp:"company_A_tdbs", session_limit_locks:“2000", session_limit_memory:"110MB", session_limit_tempspace:“35MB“, session_limit_logspace:"35MB", session_limit_txn_time:"180", tenant_limit_space:“1TB", tenant_limit_memory:"1GB", tenant_limit_connections:"1000", logmode:"ansi", locale:“en_us ", case:"insensitive",}‘ );

Ceiling Caps on Tenant Permanent Storage
You can limit the total amount of all permanent storage space types for a tenant database by setting the tenant_limit_space property. Default 0 (off) and not dynamic Values between KB and KB When the limit is reached, subsequent operations that require additional disk space are rejected. EXECUTE FUNCTION task('tenant create', 'company_A', {dbspace:"company_A_dbs1, company_A_dbs2,company_A_dbs3", sbspace:"company_A_sbs", vpclass:"tvp_A,num=6", dbspacetemp:"company_A_tdbs", session_limit_locks:“2000", session_limit_memory:"110MB", session_limit_tempspace:“35MB“, session_limit_logspace:"35MB", session_limit_txn_time:"180", tenant_limit_space:“1TB", tenant_limit_memory:"1GB", tenant_limit_connections:"1000", logmode:"ansi", locale:“en_us ", case:"insensitive",}‘ );

Limit Shared Memory for Tenant Databases
Limit shared memory amounts to all tenant database connected sessions. If the limit is exceeded, the session using the most shared memory is terminated. Two ways to enable this: Set the TENANT_LIMIT_MEMORY configuration parameter 0 (off) default Not dynamic Range of values in KB, MB, GB, or TB from KB to KB Include the tenant_limit_memory parameter with the tenant create or tenant update argument to the task or admin SQL administration API command. EXECUTE FUNCTION task('tenant create', 'company_A', {dbspace:"company_A_dbs1, company_A_dbs2,company_A_dbs3", sbspace:"company_A_sbs", vpclass:"tvp_A,num=6", dbspacetemp:"company_A_tdbs", session_limit_locks:“2000", session_limit_memory:"110MB", session_limit_tempspace:“35MB", session_limit_logspace:"35MB", session_limit_txn_time:"180", tenant_limit_space:“1TB", tenant_limit_memory:"1GB", tenant_limit_connections:"1000", logmode:"ansi", locale:“en_us ", case:"insensitive",}‘ ); This tenant database will have the following attributes: Three dedicated dbspaces A dedicated sbspace Six tenant virtual processors A dedicated temporary dbspace A limit of 2000 locks per session A memory allocation limit of 110 MB per session A 35 MB limit for temporary table space per session A 35 MB limit for log space per session A 180 second limit on transaction times A limit of 1 TB on the total amount of storage space the tenant database can use A limit of 1 GB on the total amount of shared memory for all sessions that are connected to the tenant database A limit of 1000 connections ANSI logging mode English locale Case insensitivity Temporary smart large objects are stored in the sbspace that is specified by the database server's SBSPACETEMP configuration parameter. No blobspaces

Limit the Number of Client Connections to a Tenant Database
When the limit is reached, subsequent connection requests to the tenant database are rejected. Two ways to enable this: Set the TENANT_LIMIT_CONNECTIONS configuration parameter 0 (Off) by default Not dynamic Maximum number of connections is Include the tenant_limit_connections parameter with the tenant create or tenant update argument to the task or admin SQL Admin API commands: EXECUTE FUNCTION task('tenant create', 'company_A', {dbspace:"company_A_dbs1, company_A_dbs2,company_A_dbs3", sbspace:"company_A_sbs", vpclass:"tvp_A,num=6", dbspacetemp:"company_A_tdbs", session_limit_locks:“2000", session_limit_memory:"110MB", session_limit_tempspace:“35MB", session_limit_logspace:"35MB", session_limit_txn_time:"180", tenant_limit_space:“1TB", tenant_limit_memory:"1GB", tenant_limit_connections:"1000", logmode:"ansi", locale:“en_us ", case:"insensitive",}‘ ); This tenant database will have the following attributes: Three dedicated dbspaces A dedicated sbspace Six tenant virtual processors A dedicated temporary dbspace A limit of 2000 locks per session A memory allocation limit of 110 MB per session A 35 MB limit for temporary table space per session A 35 MB limit for log space per session A 180 second limit on transaction times A limit of 1 TB on the total amount of storage space the tenant database can use A limit of 1 GB on the total amount of shared memory for all sessions that are connected to the tenant database A limit of 1000 connections ANSI logging mode English locale Case insensitivity Temporary smart large objects are stored in the sbspace that is specified by the database server's SBSPACETEMP configuration parameter. No blobspaces

Tenant Private Memory Caches on Virtual Processors
You can now improve processing performance by creating private memory caches for all tenant virtual processors. Set the VP_MEMORY_CACHE_KB configuration parameter to a nonzero value to create private memory caches for all CPU and tenant virtual processors. Default is 0 (off) Values are the total size in KB of all private memory caches, optionally followed by a comma and the mode of the caches: 800 to 40% of the SHMTOTAL configuration parameter value. There presently is not an SQL Admin API Command for this.

VP_MEMORY_CACHE_KB (1)
Modes: STATIC Default. The specified size is the maximum combined size of all private memory caches. DYNAMIC The specified size is the initial size of all private memory caches. The cache size changes dynamically but cannot exceed the value of the SHMTOTAL configuration parameter. Takes Effect After you edit your onconfig file and restart the database server. When you reset the value dynamically in your configuration file via the onmode -wf command. When you reset the value in memory via the onmode -wm command.

Private memory caches can improve performance for Informix server memory allocated by threads but have no impact on memory allocated to and used by buffer pools or shared memory communication. If > 0, a private memory cache is created for each CPU and Tenant VP. Size of all private memory caches combined is default limited to the specified number of KB. Each private memory cache size increases and decreases dynamically Include a comma and the word DYNAMIC after the size: VP_MEMORY_CACHE_KB 1000,DYNAMIC. Initial size total of VP private caches cannot exceed 40% of SHMTOTAL in dynamic mode Cache sizes can expand beyond the initial limit. The total size of the caches cannot exceed the value of the SHMTOTAL configuration parameter. If you reset the VP_MEMORY_CACHE_KB configuration parameter to 0, the memory caches are emptied and disabled.

Dynamic memory caches on busy systems can grow quickly and use a large amount of available memory. If you set the mode to DYNAMIC, set the SHMTOTAL configuration parameter to a specific limit instead of the default value of 0, which does not limit the amount of memory. To be safe, no more than 80% of the total physical memory on your database machine. Please do not forget this. You do have to work for a living. Make sure you take into account memory needs of all apps running on your database machine before setting this. We call Informix Dynamic Server for a reason. Work with the system administration people to be certain; you may not “see” everything running on your machine.

Other Server Improvements
Limit size of extendable storage spaces Use SQL Administration API functions to set the limits Arguments are space name / max size / new create size / new extend size Useful for multi-tenant Informix instances EXECUTE FUNCTION task("modify space sp_sizes", "dbspace3", "60000", "10000", "100000"); dbspace3 is modified with new chunk size of KB of space when expanding the storage pool entry for a storage space, a new extend size of KB and a maximum total size of KB. EXECUTE FUNCTION task("modify space sp_sizes", "dbspace8", "20", "1.5"); dbspace8 is modified with new chunk size of 20 % of storage space, an extend size of 1.5 % of the storage space and a maximum size of undefined. The example defines the minimum create size to 60 MB, the minimum extend size to 10 MB, and the maximum size to 100 MB for a dbspace that is named dbspace3: The second example defines the minimum create size to 20 percent and the minimum extend size 1.5 percent for a dbspace that is named dbspace8: 125

OAT Enhancements Monitor the tenant databases of a multi-tenant Informix instance Tenant Information can be found in Space Administration

Questions

REST Definition Roy Fielding is the first person to coin the term REST (for Representational State Transfer ) in his Ph.D dissertation to describe a design pattern for implementing Networked Systems 129 © 2015 IBM Corporation 129

REST Architecture Distributed communication architecture
Widely popular in cloud environments REST An architectural style for web based communication Permits clients to communicate with servers in a unique manner Represents resources (databases in this case) as URI’s Architecture uses HTTP protocol A set of operations (GET/POST/PUT/DELETE) permit manipulation of resources RESTful architectures are stateless Server does not maintain any client context between transactions Transaction must contain all information necessary to satisfy the particular request. Makes RESTful architectures more reliable and also helps to expand their scalability. In most cases the client is always a browser or a web enabled application. The server which provides the data typically could be a database server. By using the HTTP protocols the database is now easily web enabled and can serve data to the client . All objects of a database are now accessible via an URI. The representation from the server back to the client could be in the form of HTML / XML / JSON / JPEG etc… : “Applied to web services Web service APIs that adhere to the architectural constraints are called RESTful. HTTP based RESTful APIs are defined with these aspects: base URL such as an Internet media type for the data. This is often JSON but can be any other valid Internet media type (e.g. XML, Atom, microformats, images, etc.) standard HTTP methods (e.g., GET, PUT, POST, or DELETE) hypertext links to reference state hypertext links to reference related resources ………. The PUT and DELETE methods are referred to as idempotent, meaning that the operation will produce the same result no matter how many times it is repeated. The GET method is a safe method (or nullipotent), meaning that calling it produces no side-effects. In other words, retrieving or accessing a record doesn't change it. Unlike SOAP-based web services, there is no "official" standard for RESTful web APIs. This is because REST is an architectural style, unlike SOAP, which is a protocol. Even though REST is not a standard, a RESTful implementation such as the Web can use standards like HTTP, URL, XML, etc. “ The strength of REST REST is an architectural style, not a protocol or an implementation. REST has some core principles, but in the end, it's an abstraction, not a specific implementation. (Source: 130 © 2015 IBM Corporation 130

Access Informix from REST API Clients
Directly connect applications or devices that communicate through the REST API to Informix No client side drivers needed, freedom from client dependency Web based applications can connect seamlessly to the database using HTTP protocol Create connections by configuring the wire listener for the REST API Use MongoDB and SQL queries against JSON and BSON document collections, traditional relational tables, and time series data The REST API uses MongoDB syntax and returns JSON documents Widely popular in Cloud / IOT architectures Simplify web application development in Cloud environments A subset of the HTTP protocol (GET / POST / DELETE / PUT) supported POST method maps to mongo db insert or create command GET method maps to mongo db query command PUT method maps to mongo db update command DELETE method maps to mongo db delete command 131 © 2015 IBM Corporation 131

Access Informix from REST API Clients (contd)
The REST API provides an alternative method for accessing JSON collections in Informix® and provides driverless access to your data. With the REST API, you can use MongoDB and SQL queries against JSON and BSON document collections, traditional relational tables, and time series data. The REST API uses MongoDB syntax and returns JSON documents. The jsonListener.jar file is the executable file that includes the wire listener configuration file, jsonListener.properties, which defines the operational characteristics for theMongoDB API and REST API. To use REST, you must alter the jsonListener.properties file. 132 © 2015 IBM Corporation 132

Wire Listener & REST (1) The wire listener is a mid-tier gateway server that enables communication between MongoDB applications and the Informix® database server. The wire listener is provided as an executable JAR file that is named $INFORMIXDIR/bin/jsonListener.jar. The JAR file provides access to the MongoDB API and REST API. You can connect to a JSON collection by using the REST API. When a client is connected to the wire listener by using the REST API, each database is registered; session events such as create or drop a database. If a REST request refers to a database that exists but is not registered, the database is registered and a redirect to the root of the database is returned. To modify the jsonListener.properties file you have to shutdown the listener, modify the file and then restart it. The primary advantage to it is that it eliminates software client drivers necessary to access a database. The URL box on a browser is used instead. 133 © 2015 IBM Corporation 133

Wire Listener & REST (2) The JSONListener.properties file has an optional parameter called listener.type It specifies the type of wire listener to start: The default is mongo which connects the wire listener to the MongoDB API listener.type=mongo To connect to a REST API, connect to the wire listener, connect the wire listener to the REST API using the following parameter value which must be specified to use the REST API: listener.type=rest There are some new REST related optional parameters for the JSONListener.properties file which may be necessary for use. 134 © 2015 IBM Corporation 134

Multiple wire listeners configuration (1)
You can run multiple wire listeners at the same time to access both Mongo and REST data, by creating a properties file for each: Create each properties file in the $INFORMIXDIR/etc directory using the $INFORMIXDIR/etc/jsonListener-example.properties file as a template. Customize each properties file and assign a unique name: The url parameter must be specified, either in each individual properties file or in the file that is referenced by the include parameter. Optional: Specify the include parameter to reference another properties file. The path can be relative or absolute. If you have multiple properties files, you can avoid duplicating parameter settings in the multiple properties files by specifying a subset of shared parameters in a single properties file, and the unique parameters in the individual properties files. Start the wire listeners. 135 © 2015 IBM Corporation

Multiple wire listeners configuration (2) - Example
The same url, authentication.enable, and security.sql.passthrough parameters are used to run two separate wire listeners: Create a properties file named shared.properties that includes the following parameters :url=jdbc:informix-sqli://localhost:9090/sysmaster: INFORMIXSERVER=lo_informix1210; authentication.enable=true security.sql.passthrough=true Create a properties file for use with the MongoDB API that is named mongo.properties, with the parameter setting include=shared.properties included: include=shared.properties listener.type=mongo listener.port=27017 Create a properties file for use with the REST API that is named rest.properties, with the parameter setting include=shared.properties included: include=shared.properties listener.type=rest listener.port=8080 136 © 2015 IBM Corporation

Multiple wire listeners configuration (3) - Example
Start the wire listeners by using the command line: java -jar jsonListener.jar -start -config json.properties -config rest.properties 137 © 2015 IBM Corporation

HTTP: POST The POST method maps to the MongoDB insert or create command. Method Path Description POST / Create a database /databaseName Create a collection databaseName – database name /databasename/collectionName Create a document collectionName – collection name 138 © 2015 IBM Corporation

HTTP: POST – Create a database
With the locale specified. Request: Specify the POST method: POST / Data: Specify database name mydb and an English UTF-8 locale: {name:"mydb",locale:"en_us.utf8"} Response: The following response indicates that the operation was successful: Response does not contain any data. 139 © 2015 IBM Corporation

HTTP: POST – Collection Creation
Creates a collection in the mydb database. Request: Specify the POST method and the database name as mydb: POST /mydb Data: Specify the collection name as bar: {name:“bar”} Response: The following response indicates that the operation was successful: {"msg":"created collection mydb.bar","ok":true} 140 © 2015 IBM Corporation

HTTP: POST – Relational Table Creation
This example creates a relational table in an existing database. Request: Specify the POST method and stores_mydb as the database: POST /stores_mydb Data: Specify the table attributes: { name: "rel", columns: [{name:"id",type:"int",primaryKey:true,}, {name:"name",type:"varchar(255)"}, {name:"age",type:"int",notNull:false}]} Response: The following response indicates that the operation was successful: {msg: "created collection stores_mydb.rel" ok: true} 141 © 2015 IBM Corporation

HTTP: POST – Insert a Single Document
Inserts a document into an existing collection. Request: Specify the POST method, mydb database, and people collection: POST /mydb/people Data: Specify John Doe age 31: {firstName:"John",lastName:"Doe",age:31} Response: Because the _id field was not included in the document, the automatically generated _id is included in the response. Here is a successful response: {"id":{"$oid":"537cf433559aeb93c9ab66cd"},"ok":true} 142 © 2015 IBM Corporation

HTTP: POST – Insert Multiple Documents
This example inserts multiple documents into a collection. Request: Specify the POST method, mydb database, and people collection: POST /mydb/people Data: Specify John Doe age 31 and Jane Doe age 31: [{firstName:"John",lastName:"Doe",age:31}, {firstName:"Jane",lastName:"Doe",age:31}] Response: Here is a successful response: {ok: true} 143 © 2015 IBM Corporation

HTTP: GET The GET method maps to the MongoDB query command. Method
Path Description GET / List all databases /databaseName List all collections in a database databaseName – database name /databasename/collectionName? queryParameters Query a collection collectionName – collection name queryParameters - The query parameters. The supported Informix queryParameters are: batchSize, query, fields, and sort. These map to the equivalent MongoDB batchSize, query, fields, and sort parameters. 144 © 2015 IBM Corporation

HTTP: GET – List All Databases on the Server
Specify the GET method and forward slash (/): GET / Data: None. Response: Here is a successful response: [ "mydb" , "test“] 145 © 2015 IBM Corporation

HTTP: GET – List All Collections in a Database
Request: Specify the GET method and mydb database: GET /mydb Data: None. Response: Here is a successful response: ["bar"] 146 © 2015 IBM Corporation

HTTP: GET – Show Sorted Data from a Collection
This example sorts the query results in ascending order by age. Request: Specify the GET method, mydb database, people collection, and query with the sort parameter. The sort parameter specifies ascending order (age:1), and filters id (_id:0) and last name (lastName:0) from the response GET /mydb/people?sort={age:1}&fields={_id:0,lastName:0} Data: None. Response: The first names are displayed in ascending order with the _id and lastName filtered from the response: [{"firstName":"Sherry","age":31}, {"firstName":"John","age":31}, {"firstName":"Bob","age":47}, {"firstName":"Larry","age":49}] 147 © 2015 IBM Corporation

HTTP: PUT The PUT method maps to the MongoDB update command. Method
Path Description PUT /databasename/collectionName?queryParameters Update a document databaseName – database name collectionName – collection name queryParameters - The supported Informix queryParameters are query, upsert, and multiupdate. These map to the equivalent MongoDB query, insert, and multi query parameters, respectively.- 148 © 2015 IBM Corporation

HTTP: PUT – Document Update in a Collection
Update the value for Larry in an existing collection, from age 49 to 25: [{"_id":{"$oid":"536d20f1559a60e677d7ed1b"},"firstName":"Larry" ,"lastName":"Doe","age":49},{"_id":{"$oid":"536d20f1559a60e677d7ed1c"} ,"firstName":"Bob","lastName":"Doe","age":47}] Request: Specify the PUT method and query the name Larry: PUT /?query={name:"Larry"} Data: Specify the MongoDB $set operator with age 25: {"$set":{age:25}} Response: Here is a successful response: {"n":1,"ok":true} 149 © 2015 IBM Corporation

HTTP: DELETE The DELETE method maps to the MongoDB delete command.
Path Description DELETE / Delete all databases /databaseName Delete a database databaseName – database name /databasename/collectionName Delete a collection collectionName – collection name /databasename/collectionName?queryParameter Delete a document queryParameter - The query parameter. The supported Informix queryParameter is query. This maps to the equivalent MongoDB query parameter. 150 © 2015 IBM Corporation

HTTP: DELETE (1) – Database Deletion
Delete a database called mydb. Request: Specify the DELETE method and the mydb database: DELETE /mydb Data: None. Response: Here is a successful response: {msg: "dropped database"ns: "mydb"ok: true} 151 © 2015 IBM Corporation

HTTP: DELETE (2) – Collection deletion
This example deletes a collection from a database. Request: Specify the DELETE method, mydb database, and bar collection: DELETE /mydb/bar Data: None. Response: Here is a successful response: {"msg":"dropped collection""ns":"mydb.bar""ok":true} 152 © 2015 IBM Corporation

Text/Byte Compression
Compression of in-partition data stored in Partition Blobs. Partition Blobs are simple large objects (mostly text and byte data types) in which the data is stored outside the row, but in same partition of the same dbspace as the row data. So even though the blob data is stored on a separate page from the row data, the partition blob page still belongs to the same partition. © 2014 IBM Corporation

Auto Compression - Overview
Create a compression dictionary for the data rows of a table automatically when it has the required numbers of rows. Benefits of automatic compression: Informix compresses data rows as the data is loaded. Informix sets compression as a property of the table, so any new fragments added also get compressed automatically. You use current SQL admin API commands to set a table or fragment for auto compression. You have an SQL interface to create a compressed table. © 2014 IBM Corporation

Auto Compression Details
New data will be loaded as compressed. A minimum of 2000 rows are needed for the compression dictionary to be created. You can start compression with the same SQL admin API commands that you currently use. You can also compress a table, using SQL syntax when you create the table. © 2014 IBM Corporation

Auto Compression How To
Using Admin API Execute function task(“table compress”, “my_table”, “my_database”); Execute function task(“fragment compress”, “my_fragid”); Even if there are not enough rows to sample and create a compression dictionary, the SQL admin commands will succeed indicating “Auto compression is set”. © 2014 IBM Corporation

Auto Compression - How to
Using SQL Create table my_table on (my_col ….) compressed; After the table is created, as data is loaded into the table, a compression dictionary will be created when 2000 rows are inserted. © 2014 IBM Corporation

Auto Compression When a table load uses light append, a dictionary will be automatically created: The rows that loaded before the dictionary was created will be compressed. During a normal insert, the new rows inserted will be compressed: Data rows already in the table (before the compress) will not be compressed. Auto compression is not supported for indexes and blobs. © 2014 IBM Corporation

Index Compression Restrictions
Normal detached BTree indexes only Native types No functional indexes Only the leaf level is compressed Can be done as part of index creation or to existing indexes while online Must contain at least 2000 unique key values Full HDR support Log the uncompressed key Uncompress not available at this time for indexes © 2014 IBM Corporation

Index Compression Benefits
More index data fits into the bufferpool Reduced I/O Unlike Oracle, SQL Server, and even DB2, Informix index compression supports composite indexes with either “prefix” and “suffix” keys Prefix – unique value is last value of composite Ex) Red, Honda, Civic, Oregon, VIN # Suffix – unique value is first value of composite (key only scan) Ex) VIN #, Red, Honda, Civic, Oregon © 2014 IBM Corporation

Indexing Varchars vs Indexing Compressed Chars
Varchars are used to save space, in both the data row and the index, by storing only the relevant data and not padding with white space We did an experiment to compare key-only scan performance of indexed varchars against indexed compressed chars © 2014 IBM Corporation

Varchars vs Compressed Chars
create table people ( first_name varchar(64), middle_name varchar(64), last_name varchar(64), address varchar(255), city varchar(64), state char(2), sex varchar(6), ssn char(11), eye_color varchar(24), ancestry varchar(64)); © 2014 IBM Corporation

Varchars vs Compressed Chars
create unique index ssn_people on people(ssn); create index ix1_people on people(last_name, first_name, middle_name, ssn); Key only scan select count(*) from people where last_name in ('SMITH', 'JOHNSON', 'WILLIAMS', 'BROWN', 'JONES') AND first_name in ('JAMES', 'JOHN', 'ROBERT', 'MARY', 'PATRICIA', 'LINDA'); © 2014 IBM Corporation

Query Results (Several Million Rows)
Varchar Uncompressed Char % better Compressed Char Unbuffered 100s 5.38s 1758% 1.9s 183% Buffered 6.4s 0.77s 731% 0% # of index nodes 514748 634095 © 2014 IBM Corporation

Why Is Performance Better?
Varchars require IDS to read the data row even if it appears that a key only scan could be performed Have to compare white space White space stored in row White space removed from index key © 2014 IBM Corporation

Varchar index (oncheck –pT)

Char index (oncheck –pT)

Char index compressed (oncheck –pT)

Results So to look at the earlier slide again:
In both cases, the queries were faster on the compressed character column. Results were not so good on the uncompressed varchar column, due to the byte by byte comparison earlier mentioned. Index usage: there were almost 120,000 more index nodes on the character (634095) than there were on the varchar (514748). Here a compressed character column index, loses almost 78% of index nodes ( vs uncompressed char ); and it is faster performance wise. © 2014 IBM Corporation

Index Compression (1) Compress an existing index:
execute function task(“index compress”, “j”, “testdb”); Repack and shrink an existing index: execute function task(“index repack shrink”, “j”, “testdb”); Estimate Compression on an index: execute function task(“index estimate_compression”,“176_576”, “stores_demo”, “ifxjson”); You can estimate compression only for a detached B-tree index on a fragmented or non-fragmented table. © 2014 IBM Corporation

Index Compression (2) Pushes all nodes forward on the same page as they are found and compresses them, leaving free space at the end. Repack consolidates partially filled index pages and fills them, leaving empty pages behind; repack if you intend to shrink. Shrink reduces an index overall size by returning empty pages to its dbspace; we will not shrink beyond the index first extent size. © 2014 IBM Corporation

Some index repack and shrink results, before and after.
An Index before compression, repack and shrink, pages, oncheck –Pt output. After: 2778 Total Pages Allocated, 86.43 % reduction 15.57 % of index Space remains. Before: After: © 2014 IBM Corporation

Minus, Plus & Intersect Support - Functionality
New Extensions to the existing UNION/UNION ALL SET operation. Same rules of UNION also applies e.g. Both query block should have exact same number of columns. Comparable data types. Projection clause will not have BYTE or TEXT. © 2014 IBM Corporation

Minus, Plus & Intersect Support - Functionality
Order by should be at the end. Precedence will be from left to right, unless they are grouped using parentheses. There are existing restrictions for UNION and they will apply to these operators too. © 2014 IBM Corporation

Intersect and Minus/Except
Intersect returns only those rows which are returned by both queries and results will be distinct. Minus/Except will returns distinct rows from the left query which are not found in the right side query. The All keyword is not available for the intersect and minus/except operations. NULL friendly, means when comparing NULL to NULL they are considered equal. © 2014 IBM Corporation

Minus, Plus & Intersect Support - Examples
create table t1 (col1 int); insert into t1 values (1); insert into t1 values (2); insert into t1 values (3); insert into t1 values (4); insert into t1 values (NULL); create table t2 (col1 int); insert into t2 values (1); insert into t2 values (3); insert into t2 values (4); insert into t2 values (NULL); © 2014 IBM Corporation

Minus, Plus & Intersect Support - Examples (cont’d)
Select * from t1 intersect select * from t2; Returns NULL, 1,3 and 4 Select * from t1 minus/except select * from t2; Returns 2 © 2014 IBM Corporation

Minus, Plus & Intersect Support - Rewrite
Select col1, col2 from t1 intersect select col1, col2 from t2; Rewrite: Select distinct * from (select col1, col2 from t1) t11 where exist (select col1, col2 from (select * from t2) t22 where t11.col1 = t22.col1 and t11.col2 = t22.col2 ); © 2014 IBM Corporation

Minus, Plus & Intersect Support - Minus/Except Rewrite
select col1, col2 from t1 minus select col1, col2 from t2; select distinct * from (select col1, col2 from t1) t11 where not exists (select * from (select col1, col2 from t2) t22 where t11.col1 = t22.col1 and t11.col2 and t11.col2= t22.col2); © 2014 IBM Corporation

Select ….. INTO TABLE You can now create persistent tables in IDS that can be accessed across sessions using the result set of a SELECT statement. Combines the functionality of CREATE TABLE and INSERT into a single statement. Supports all the Storage options of CREATE TABLE on IDS. You can now change the EXTENT SIZE and NEXT SIZE of an existing table without having to unload the table first. Allows you to create new schema from existing tables and populate it with only a subset of data for testing purposes. Supported in a Flexible Grid environment. Functionality supported via two different syntax enhancements: SELECT col1, col2 FROM sourcetab INTO newtab; CREATE TABLE newtab AS SELECT col1, col2 FROM sourcetab; © 2014 IBM Corporation

4GB Stored Procedure Language
The maximum length of SQL statements and SPL routines is now 4 GB The actual limit may depend on the available memory on the system. Valid for Client SDK 3.70.xC5, JDBC 3.70.xC5 or higher. Transient extended types, collection types are limited to 65535 sysadmin task command length is limited to 2020 (slot table has 28 bytes required). Perhaps writing SQL or SPL possibly so large should be avoided for programmer maintenance issues ….. © 2014 IBM Corporation

CASE Statement in SPL CASE statement without else clause
CASE mod(r,314) WHEN THEN select tabid into r from informix.systables where tabid = 1; WHEN 313 THEN LET r = r + 4; WHEN THEN LET r = r + 3; END CASE; CASE statement with else clause CASE r WHEN 317 THEN select tabid into r from informix.systables where tabid = 1; WHEN 312 THEN LET r = r + 4; ELSE LET r = 1; © 2014 IBM Corporation

New Mathematical Functions – Oracle Compatibility
SIGN(num) Returns the sign of the input numeric value (returns integer): -1 for n < 0, 0 for n = 0, and 1 for n > 0 COSH(num) Returns hyperbolic cosine (returns float) SINH(num) Returns hyperbolic sine (returns float) TANH(num) Returns hyperbolic tangent (returns float) ACOSH(num) Returns arc hyperbolic cosine (returns float) ASINH(num) Returns arc hyperbolic sine (returns float) ATANH(num) Returns arc hyperbolic tangent (returns float) LN(num) Alias for existing LOGN(num) function (returns float) Fully compliant with the same math functions of the competition: Usable where any executable function is currently used provided the input(s) are an expected valid value(s). © 2014 IBM Corporation

Multiple distinct Operators
Example Legal syntax Applied to each predicate Illegal syntax Nested operators select avg(unique col1), count (distinct col2) from my_tab; select distinct avg(distinct col1) from my_tab; © 2014 IBM Corporation

SQL Enhancements – nulls first / last
Example Default null sort order with the asc keyword Modified null sort order with the asc nulls last keywords © 2014 IBM Corporation

SQL Enhancements – nulls first / last
Example Default null sort order with the desc keyword Modified null sort order with the desc nulls first keywords © 2014 IBM Corporation

Virtual Table Interface – in Clause
The in clause can be used with the virtual table interface. VTI tables are commonly used with Timeseries. select …. where meter_id in (5,10,30) Previously had to rewrite this type of a query using a series of or’s in the where clause. select ….. where meter_id = 5 or meter_id = 10 …. SQL Commonly used sql statements will use the in clause. Example: select * from table where id in (5,10,30,50,90). This has not been supported through vti so this would have to be rewritten as. Select * from table where id =5 or id=10 or ….. This feature now adds support for the in clause with VTI. © 2014 IBM Corporation 197

Application Development LATERAL Correlated Subquery of Derived Tables - Description
Error Example (-999) select * from t1, ( select t2.a as t2_a from t2 where t2.a = t1.a) LATERAL Example LATERAL ( select t2.a as t2_a from t2 where t2.a = t1.a) IDS has not previously allowed correlation variable references from derived table to outer table list. The first example will give a -999 error , since t1.a column is out of the scope of the derived table. Correlated references can be used in nested table expressions or as arguments to table functions. The basic rule that applies for both these cases is that the correlated reference must be from a table-reference at a higher level in the hierarchy of sub-queries. This hierarchy includes the table-references that have already been resolved in the left-to-right processing of the FROM clause. For nested table expressions, the LATERAL keyword must appear before the full select. © 2014 IBM Corporation

App. Development - LATERAL Correlated Subquery of Derived Tables - Why
Compatibility with OData OData = Open Data Protocol Web protocol for data reads and writes Compatibility with other DBMS DB2 SQL Server Oracle PostgreSQL “The Open Data Protocol (OData) is a Web protocol for querying and updating data that provides a way to unlock your data and free it from silos that exist in applications today. OData does this by applying and building upon Web technologies such as HTTP, Atom Publishing Protocol (AtomPub) and JSON to provide access to information from a variety of applications, services, and stores. The protocol emerged from experiences implementing AtomPub clients and servers in a variety of products over the past several years. OData is being used to expose and access information from a variety of sources including, but not limited to, relational databases, file systems, content management systems and traditional Web sites.” Comments: This feature request was initiated by a customer who wanted to be able to include a Microsoft OData layer in their application stack and that layer generates SQL queries using LATERAL clauses. © 2014 IBM Corporation

Administration Features - SERIAL In-Place ALTER
In-place ALTER for SERIAL types Increased table availability Data converted as needed Less space required Reduces logging Supported conversion types Small-to-large or large-to-large serial to serial8 serial to bigserial serial8 to bigserial bigserial to serial8 Not large-to-small Alter SERIAL IN-PLACE ALTER In-place alter is faster than standard alter since it benefits from the following: Increased table availability because it locks the table for only the time it takes to update the table definition and rebuild indexes (if any) on the altered columns. Does not convert the data rows at the time of the ALTER or make a copy of the data Requires less space than the slow alter Reduced logging Comment: This was a feature requested by a customer who found that their SERIAL columns were running out of values. © 2014 IBM Corporation

NOVALIDATE Data Migration enhancement
Eliminate time consuming constraint checking on foreign key constraints ALTER TABLE catalog ADD CONSTRAINT (FOREIGN KEY (stock_num, manu_code) REFERENCES stock ON DELETE CASCADE CONSTRAINT ab NOVALIDATE); SET CONSTRAINTS cons_child_x1 ENABLED NOVALIDATE; When migrating data with data loads and unloads, time can be saved by not having to perform the foreign key constraint checking. Since the data is coming from a known state, these can be skipped. 201 © 2014 IBM Corporation 201

New SQL - Quarter and Limit
Both Informix and IWA support the new SQL functions QUARTER and LIMIT SQL syntax. Enhanced support, performance and integration with BI tools and time-based queries retrieving the quarter of a date or a datetime Enhanced DB portability, integration and performance when using BI tools and applications that use this specific LIMIT syntax for top/bottom N queries and preview data/results Similar results to the usage of SELECT FIRST or LAST Database compatibility with other DB products LIMIT is not presently documented in the xC3 manuals. © 2014 IBM Corporation

New SQL - Quarter and Limit - Examples
SELECT QUARTER("orders"."ship_date") AS "qr_ship_date_ok", SUM("orders"."sales") AS "sum_sales_qk“ FROM "informix"."orders" "orders" INNER JOIN "informix"."returns" "returns" ON ("orders"."order_id" = "returns"."order_id") INNER JOIN "informix"."users" "users" ON ("orders"."region" = "users"."region") GROUP BY 1 Limit SELECT "order"."o_custid" AS "o_custid", "order"."o_date" AS "o_date", "order"."o_orderid" AS "o_orderid", "order"."o_priority" AS “o_priority", "order"."o_shipping" AS "o_shipping", "order"."o_status" AS "o_status" FROM "dwa"."order" "order" LIMIT 10000 © 2014 IBM Corporation

SQL Enhancements – Windows XML Enhancement
The XML4C and XSLT4C libraries have been ported to Windows 64 enabling XML functionality on that port. © 2014 IBM Corporation

Informix Warehouse Accelerator

Informix Warehouse Accelerator - Cluster Environment
IWA now has the ability to create an accelerator server across multiple physical servers as a cluster An MPP implementation Enables the creation of significantly larger accelerators and marts No longer limited by the amount of memory a single server can support Informix Warehouse Accelerator Previously IWA was limited to a single node and 3 TB maximum of physical memory for most hardware architectures. This has now changed. IWA now clusters and can be expanded with multiple physical nodes, primarily based on a cheap 64bit Intel based architecture. This allows performance previously reserved for very large scale mainframes costing multiple millions of dollars to be brought down to customers with much smaller budgets. This is ideal for Informix customers who have large scale data warehouse needs. SQL Results © 2014 IBM Corporation

SQL Queries (from apps) TCP/IP Informix Warehouse Accelerator Informix SQL (via DRDA) Query Processor Query Router Results Compressed DB partition Data Warehouse Bulk Loader Informix: Routes SQL queries to accelerator User need not change SQL or apps. Can always run query in Informix, e.g., if too short an est. execution time Informix Warehouse Accelerator: Connects to Informix via TCP/IP & DRDA Analyzes, compresses, and loads Copy of (portion of) warehouse Processes routed SQL query and returns answer to Informix © 2014 IBM Corporation

Partition Update: Example This figure illustrates the case of reloading only table partition p4 (in dbspace dbs4) of the fact table, from the Informix table into the data mart, since p4 was the only fact table’s partition containing recently added or changed data since the previous data mart’s refresh. The new functions ifx_dropPartMart and followed by ifx_loadPartMart (the syntax here in the speaker notes is correct, the graphic says dropPartMart and loadPartMart), allow reloading a fact table’s partition in the Accelerator’s data mart with the new snapshot of data set from that partition in Informix. © 2014 IBM Corporation

Refreshing Data on IWA – Multiple Options
Methods Frequency of Deployment Considerations Complete Load Nightly or a small number of times per day For initial load and for periodic complete refresh of mart Automatic or manual Partition Update Hourly, nightly or on- demand For partitioned FACT table For non-partitioned or partitioned dimension tables Trickle Feed Continuous For Inserts (Append) to FACT table and I/U/D for dimension tables Complete reload periodically following Trickle Feed We have looked at a complete load previously. There is little new there. The next several slides will talk about new features in Automatic or Manual partition Update, with a focus briefly on the new refresh mart function. We will then look at Trickle Feed, or continuous feed of updates to the primary data source from the primary data source. © 2014 IBM Corporation

Partition Update: Refresh data in a data mart at partition level Provides efficient refresh of data mart data Enables fast refresh of fact tables which are usually very large Dimension table (usually very small) full load refresh is very fast Refresh can be done on The entire table or The partition (or subset) that contains data updates A data mart in the Accelerator is created based on the definition of a subset of columns within a group of related tables in your Informix database, which are relevant or used in the complex queries and table joins of a given query workload that you want to accelerate. After this data mart definition is deployed into the Accelerator, the data mart needs to be loaded (full refresh) with the corresponding data from the source Informix database into the Accelerator, before it can become active and therefore ready to use for query acceleration. After this first full load of the Accelerator’s data mart, subsequent periodic refresh of the data mart are required – for instance, once a day - to keep the Accelerator’s data mart updated as its corresponding data in the source tables in the Informix database changes with new and modified rows. A partial load of the data mart was not available. Therefore, whenever you wanted to update the data in a data mart with the most recent information in your database, the only way to proceed was to drop, deploy and fully load the same data mart again. If there was enough memory available, it was possible to do this new refresh as a separate data mart deployment and load, to avoid disruption of query acceleration on the data mart’s queries in the process. Depending on the size of the Informix data set to refresh, and other factors like the amount of workers configured in the Accelerator, a full load of a very large data mart could take a significant amount of time. Now, a very large partitioned table in Informix that is used in your data mart could be refreshed by either reloading the entire table, or just a subset of partitions that are known to contain updated data. With this new feature, you could process a faster refresh of a very large data mart if, for the largest Informix table it uses – which is your data mart’s fact table — you do a partial update by focusing on re-loading only those table partitions (or fragments) from your large Informix table containing any of its new or changed data, rather than reloading the entire Informix table’s data into your data mart. The refresh of the smaller Informix tables used by your data mart – as dimension tables — should be done as a full load, which is normally very fast. You can refresh the partition data by reloading the partition, removing (dropping) the partition, or adding the partition. Notice that, in order to take advantage of this feature at all, the large Informix table used as your data mart’s fact table needs to be partitioned/fragmented, which is common to find in large Informix databases. However, if the partitioning strategy of the table is such that the new or most recently changed data could be in any of the table’s partitions, or in all of them – which happens with round-robin fragmentation — you will not be able to fully exploit the benefits of refreshing your data mart’s fact table at a partition level. Informix table partitions based on time expressions, lists or intervals, which limit the placement of the most recent data in just a few table partitions, are useful for partial data mart updates and improving the times for refresh of your Accelerator’s data marts. Another way of doing a partial refresh of a data mart for roll-out and roll-in of time-cyclic data, is by using a combination of ifx_dropPartMart() and detach old fragment from the fact table into a stand-alone table in Informix –to delete partition-based old data from a data mart— and then attach a stand-alone table containing the new data as a fragment of the fact table in Informix and ifx_loadPartMart() – to add data from a table’s new fragment into the data mart. Example: Time-cyclic-based roll-out and roll-in of data, at partition-level, on fact table salesfact: Detach a partition containing the old data from fact table salesfact and then attach a partition containing the new data. First, drop the partition p1 from your data mart salesmart in the accelerator SLSACC. Then, detach the old partition p1 from the dmuser.salesfact table to a standalone table salesfact_p1 EXECUTE FUNCTION ifx_dropPartMart('SLSACC', 'salesmart', 'dmuser', 'salesfact', 'p1'); ALTER FRAGMENT ON TABLE salesfact DETACH p1 salesfact_p1; Now, attach a table p4 that contains recent data, as a new partition p4 into the Informix table dmuser.salesfact and then load this fragment into your data mart salesmart in the accelerator SLSACC. CREATE TABLE p4 (sk1_cust INT, sk2_store INT, id INT, val DECIMAL(9,2), CHECK (id = 4)); ALTER FRAGMENT ON TABLE salesfact ATTACH p4 AS (id = 4) AFTER p3; EXECUTE FUNCTION ifx_loadPartMart('SLSACC', 'salesmart', 'dmuser', 'salesfact', 'p4'); © 2014 IBM Corporation

Informix Warehouse Accelerator – Trickle Feed
The source database can now be configured to push new data into the marts every N seconds without any further administrative intervention Inserts only, deleted / modified fact data is not pushed into the mart See next slide for more details Is controlled by the ifx_setupTrickleFeed() function Available in the sysadmin database In an H/A cluster, this function can only be executed on the cluster primary Syntax Where accel_name – the name of the accelerator which supports the mart mart_name – the name of the data mart buffer_time – the number of seconds between refreshes 1 – (2,400 hours) – refresh both the fact and dimension tables -1 – (2,400 hours) – refresh just the fact table To use this feature, MAX_PDQPRIORITY in the source instance must be set >= 50 for the initial data load and all subsequent loads ifx_setupTrickleFeed(‘accel_name’, ‘mart_name’, buffer_time ) This is an automatic procedure, once initially setup. execute procedure ifx_setupTrickleFeed (--'--accelerator_name--'--,--'--data_mart_name--‘, buffertime); Time is measured in seconds. Obviously, accelerator_name and data_mart_name must exist and be up and running to successfully execute this. After you load a data mart, run the ifx_setupTrickleFeed() function to enable trickle feed. On HA clusters, this should be run on the primary server. As data is inserted into the database, the data in the specified data mart is refreshed at the frequency specified by the buffertime parameter. Between refreshes, the new data inserted into the fact table is stored in the $INFORMIXDIR/tmp/trickleFeed directory on the database server. A small value of the buffertime parameter, such as 60, has the following advantages: The data in the data mart is current. Loading the data has a minimal impact on accelerated queries. A small amount of disk space is needed to store the updates on the database server between refreshes. Obviously, here you need to be careful of the amount of space allocated to and the existence of $INFORMIXDIR/tmp/tricklefeed, depending on the amount of trickling over. ifx_setupTrickleFeed() function The ifx_setupTrickleFeed() function enables trickle feed, which is the continuous refreshing of data in the fact and dimension tables of the data mart as the data changes in the database. Syntax >>-ifx_setupTrickleFeed--(--'--accelerator_name--'--,--'--data_mart_name--'--> >--,--buffertime--) >< accelerator_name The name of the accelerator that contains the data mart. data_mart_name The name of the data mart. buffertime An integer that represents the time interval in seconds between refreshes and whether dimension tables are refreshed. For best results, set to at least 10 seconds. Possible values: -1 through = The absolute value is used as the number of seconds between fact table refreshes. 1 through = The number of seconds between refreshes of the fact and dimension tables. Usage After you load a data mart, run the ifx_setupTrickleFeed() function to enable trickle feed. For a high-availability cluster, run the ifx_setupTrickleFeed() function on the primary server. As data is inserted into the database, the data in the specified data mart is refreshed at the frequency specified by the buffertime parameter. Between refreshes, the new data that is inserted into the fact tables is stored in the $INFORMIXDIR/tmp/trickleFeed directory on the database server. A small value of the buffertime parameter, such as 60, has the following advantages: Choose the value of the buffertime parameter carefully to suit your business needs. Set the buffertime parameter to a negative number when you want to refresh insertions into the fact tables but ignore changes to the dimension tables. The ifx_setupTrickleFeed() function adds the ifx_TrickleFeed_load_ID task to the Scheduler, where ID is a unique number. Every data mart for which you enable trickle feed has a trickle feed task in the Scheduler. You can determine if trickle feed is enabled for a data mart by querying the ph_task table in the sysadmin database. The tk_name column contains the name of the task and the tk_description column contains the description, which lists the data mart and accelerator names. If a task description includes the relevant data mart name, trickle feed is running for that data mart. Important: You must periodically disable trickle feed and reload the data mart. The ifx_TrickleFeed_load_ID task in the Scheduler stops running after refreshes. Trickle feed captures the data that is inserted into the fact tables and all types of data changes to the dimension tables. Use other methods to make any other changes to the data or the data mart. The ifx_setupTrickleFeed() function does not make the following changes: Update or delete data in the fact tables. Typically, changes to existing data is not allowed in warehouses. If data is updated, disable trickle feed and run the ifx_refreshMart() function. Attach or detach a partition in the data mart when a fragment is attached to or detached from a fact or dimension table. You must disable trickle feed and run the ifx_refreshMart() function or reload the data mart. Update the data mart definition. If a table is added or dropped from the database server, you must disable trickle feed, drop the data mart, update the data mart definition, and then re-create the data mart. Do not do any of the following tasks while trickle feed is enabled: Refresh a partition by running the ifx_loadPartMart() function. Drop a partition by running the ifx_dropPartMart() function. Load the data mart by running the ifx_loadMart() function. Drop the data mart by running the ifx_dropMart() function. The ifx_dropMart() function disables trickle feed. To disable trickle feed, run the ifx_removeTrickleFeed() function. Return value The ifx_setupTrickleFeed() function returns the text string "The operation was completed successfully." or an error message. Examples Example 1: Start trickle feed The following statement starts refreshing data every 60 seconds for a data mart named Datamart1 in an accelerator named MyAccelerator: EXECUTE FUNCTION ifx_setupTrickleFeed('MyAccelerator', 'Datamart1', 60); Example 2: Determine if trickle feed is running The following SQL statement returns every trickle feed task name and description: SELECT * FROM sysadmin::ph_task WHERE tk_name MATCHES 'ifx_TrickleFeed_load*'; The results of this query show that trickle feed is running for the data mart named Datamart1 in an accelerator named MyAccelerator: ... tk_id 84 tk_name ifx_TrickleFeed_load_16 tk_description trickle feed loader for data mart © 2014 IBM Corporation

New SQL Support for query acceleration Several SQL functions and expressions now support query acceleration. Example: COUNT (DISTINCT CASE …) Multiple DISTINCT with aggregates, such as COUNT (DISTINCT…), in a query LEN / LENGTH SUBSTR / SUBSTRING TODAY TRIM YEAR UNITS UNION, UNION ALL RANK, DENSE_RANK, DENSERANK, CUME_DIST, PERCENT_RANK, NTILE, ROW_NUMBER, ROWNUMBER, FIRST, LAST, RATIO_TO_REPORT, RATIOTOREPORT New SQL Support for query acceleration Informix SQL functions and expressions that are now supported for extreme query acceleration through the use of Informix Warehouse Accelerator, such as: COUNT (DISTINCT CASE …) Multiple DISTINCT with aggregates, such as COUNT (DISTINCT…), in a query LEN / LENGTH SUBSTR / SUBSTRING TODAY TRIM YEAR UNITS OLAP functions also are accelerated by IWA if it is possible to do so. The next section discusses these functions in detail. UNION, UNION ALL, RANK, DENSE_RANK, DENSERANK, CUME_DIST, PERCENT_RANK, NTILE, ROW_NUMBER, ROWNUMBER, FIRST, LAST, RATIO_TO_REPORT, RATIOTOREPORT UNION & UNION ALL Support: The UNION or UNION ALL set operators merge the results of two SELECT statements. If either SELECT statement is eligible for query acceleration, it is sent to Informix® Warehouse Accelerator for processing. For example in this query, SELECT COUNT(*) FROM sales UNION ALL SELECT COUNT(*) FROM systables; The query block SELECT COUNT(*) FROM sales is sent to Informix Warehouse Accelerator for acceleration. SELECT COUNT(*) FROM systables does not qualify for acceleration. In the example below, both SELECT query blocks are sent to Informix Warehouse Accelerator for processing. Each query block is sent separately. SELECT COUNT(*) FROM sales UNION ALL SELECT COUNT(*) FROM sales_returns; © 2014 IBM Corporation

Additional platform support with Solaris Intel/AMD Accelerator supported on Linux Intel/AMD 64-bit. Informix Server for the accelerator supported on the 64-bit Aix, HP IA, Linux Intel x86, Solaris Sparc and Solaris x86. Adds more cross platform integration. Additional platform support with Solaris Intel/AMD It is possible to use the Informix Warehouse Accelerator, which runs on a computer with Linux Intel/AMD 64-bit, with an Informix database server running on a different computer with Solaris on Intel/AMD 64-bit. This new platform integration support allows existing Informix databases systems on Solaris Intel 64-bit to be leveraged and benefit from the accelerator’s breakthrough technology without moving the database to another platform. © 2014 IBM Corporation

Using the accelerator with secondary servers: Example The figure shows a possible configuration scenario for selectively using Informix Warehouse Accelerator within an Informix 4-node HA cluster with mixed OLTP/Warehouse workloads. A proper reroute of clients to servers within the groups can be done by a Connection Manager. In this case, the RSS is used to create and load/refresh different data marts into the Accelerator, and it can also accelerate queries for its clients using the Accelerator’s technology; while the HDR is set to access and use the Accelerator just for the benefit of achieving extreme query acceleration for its users, so it transparently offloads matching heavy queries to the Accelerator, but do not deploy or maintain Accelerator’s data marts. Finally, the Primary and SDS are fully dedicated to OLTP workloads, not using the Accelerator since their clients most probably do not need to. © 2014 IBM Corporation

You can use IWA’s In-Memory Analytics to Speed Up queries on…
Synonyms to local or remote tables NoSQL Data ex: JSON collections SQL Data from local Informix tables External Tables ASCII/binary files in file system or network pipes Views to Informix tables or NoSQL data Sensor data TimeSeries data, time-stamped meter data © 2014 IBM Corporation 216

Views & Synonyms IWA can now use views when defining a datamart Create a view of multiple tables Local or remote by using a synonym pointing to a remote Informix table Views could map to a subset of another Informix table or also to NoSQL data Use views to accelerate self-joins Benefits Ability to combine and accelerate queries on a local server with remote tables, no need to make all tables local Allows to accelerate data in JSON collections by using views Warehouse Accelerator RTC 24952: Data marts in IWA can now be created and defined using views and synonyms. Data Marts 217217 © 2014 IBM Corporation 217

IWA Data Mart Views - NoSQL query support & usage (1)
Example: Accelerating NoSQL data (in a JSON collection) using IWA’s view support From MongoDB shell: Create two collections (JSON): comments and users From Informix: Create a view on each JSON collection (comments, users) Deploy an IWA data mart by probing a join between them: Run the accelerated query. $ dbaccess demo_database - > create view vcomments(uid,pid,comment) as select bson_value_int(data,'uid'), bson_value_int(data,'pid'), bson_value_varchar(data,'comment') from comments; > create view vusers(uid,name) as select bson_value_varchar(data,'name') from users; $ mongo demo_database MongoDB shell version: 2.4.9 connecting to: demo_database mongos> db.comments.insert( [ { uid:12345, pid:444, comment:"first" }, { uid:12345, pid:888, comment:"second" }, { uid:99999, pid:444, comment:"third" } ] ) mongos> db.users.insert( [ { uid:12345, name:"john" }, { uid:99999, name:"mia" } mongos> exit set environment use_dwa 'probe cleanup'; set environment use_dwa 'probe start'; select {+ avoid_execute} * from vcomments c, vusers u where c.uid=u.uid; set environment use_dwa 'probe stop'; execute procedure ifx_probe2mart('demo_database','noSQL_mart'); execute function ifx_createmart('demo_dwa','noSQL_mart'); execute function ifx_loadmart('demo_dwa','noSQL_mart','NONE'); © 2014 IBM Corporation

IWA Data Mart Views - NoSQL query support & usage (2)
Example: Accelerating NoSQL data (in a JSON collection) using IWA’s view support From Informix: Run the accelerated query on the JSON collection data: set environment use_dwa 'accelerate on'; select c.uid,name,comment from vcomments c, vusers u where c.uid=u.uid and pid=444; uid name john comment first uid name mia comment third © 2014 IBM Corporation

IWA data mart view support to accelerate SQL-NoSQL query (1) – SQL/NoSQL joins
From Informix: We have an Informix table sqldoc1: We also have a JSON collection doc1: In Informix using its JSON compatibility or its MongoDB driver: Create a view for the NoSQL data: Run the accelerated query on the JSON collection data: create view viewdoc1 (c1,fname,lname,age,street) as SELECT c1, bson_value_lvarchar(data,"fname"), bson_value_lvarchar(data,"lname"), bson_value_double(data,"age"), bson_value_lvarchar(data,"address.street") FROM doc1; > select * from viewdoc1; c fname John lname Miller age street Informix ave c fname Scott lname Lashley age street Blazer ave 2 row(s) retrieved. create table sqldoc1 ( name varchar(10), value integer ); insert into sqldoc1 values ("John", 1); insert into sqldoc1 values ("Scott", 2); create table doc1 ( c1 serial, data BSON ); Insert into doc1(c1,data) values (0, '{ fname:"John", lname:"Miller", age:21, address: { street:"Informix ave" } }'::JSON ); (0, '{ fname:"Scott", lname:"Lashley", age:21.50, address: { street:"Blazer ave" }, job:"Ref" }'::JSON ); © 2014 IBM Corporation

IWA data mart view support to accelerate SQL- NoSQL query (2) – Query Acceleration
Probe the query we want to accelerate Use workload analysis to find datamart for a query joining SQL table with NoSQL data (JSON collection) Deploy datamart proposed: Run the query with acceleration. set environment use_dwa '3'; Environment set. select a.value,a.name,b.* from sqldoc1 a, viewdoc1 b where a.value=b.c1 ; value 1 name John c fname John lname Miller age street Informix ave value 2 name Scott c fname Scott lname Lashley age street Blazer ave Online.log: 01:22:11 SQDWA: select a.value,a.name,b.* from sqldoc1 a, viewdoc1 b where a.value=b.c 01:22:11 SQDWA: Identified 1 candidate AQTs for matching 01:22:11 SQDWA: matched: aqt46221ac7-8eae-4b9a-9275-bd00dcca357c 01:22:11 SQDWA: matching successful (17 msec) aqt46221ac7-8eae-4b9a-9275-bd00dcca357c 01:22:11 SQDWA: offloading successful (3036 msec) select a.value,a.name,b.* from sqldoc1 a, viewdoc1 b where a.value=b.c1 ... <mart name="dm_sqlnosql1"> <table name="sqldoc1" schema="root" isFactTable="true"> <column name="name"/> <column name="value"/> </table> <table name="viewdoc1" schema="root" isFactTable="false"> <column name="age"/> <column name="c1"/> <column name="fname"/> <column name="lname"/> <column name="street"/> <reference referenceType="LEFTOUTER" isRuntimeJoin="true" parentCardinality="n" dependentCardinality="n" dependentTableSchema="root" dependentTableName="sqldoc1" parentTableSchema="root" parentTableName="viewdoc1"> <parentColumn name="c1"/> <dependentColumn name="value"/> </reference> </mart> </dwa:martModel> © 2014 IBM Corporation

Views do not maintain information about number of “rows” in the system catalogs. If the Fact table is a view and the Server is not able to identify it as a fact table Force the optimizer directive using the FACT directive. If the view is chosen improperly as the probe or build; you can use the /probe and /build options of the join-method optimizer directives. {+ FACT ... /probe … /build} Warehouse Accelerator Since views do not maintain system catalog statistical information, if the SQL being run is not able to identify the view as a fact table, then it should be forced as the fact table. This can be done using the FACT optimizer directive. Also, with joins, if the view is not uses appropriately as the build or probe, this too can be forced. This all comes back to the fact that views don’t have statistical data. Data Marts © 2014 IBM Corporation 222

ntile(n) reports nth group by ranking
New OLAP Functions Ranking functions RANK, DENSE_RANK, DENSERANK PERCENT_RANK, CUME_DIST, NTILE Numbering functions ROW_NUMBER, ROWNUMBER Aggregate functions SUM, COUNT, AVG, MIN, MAX STDEV, VARIANCE, RANGE FIRST_VALUE/LAST_VALUE functions FIRST_VALUE, LAST_VALUE LEAD/LAG functions LEAD, LAG RATIO_TO_REPORT function RATIO_TO_REPORT, RATIOTOREPORT All of these are accelerated by IWA Subset of functions support by Informix. All of these are accelerated by the Informix Warehouse Accelerator. RANK, DENSE RANK and DENSERANK: Order by clause is required Window clause is not allowed Duplicate value handling is different between rank() and dense_rank() same rank given to all duplicates next rank used “skips” ranks already covered by duplicates in rank(), but uses next rank for dense_rank() PERCENT_RANK, CUME_DIST, NTILE percent_rank() and cume_dist() reports ranking information as a percentage ntile(n) reports nth group by ranking Numbering functions: ROW_NUMBER, ROWNUMBER ROW_NUMBER and ROWNUMBER are synonyms Provides sequential row number (regardless of duplicates) to result set Aggregate functions: SUM, COUNT, AVG, MIN, MAX, STDEV, VARIANCE, RANGE All currently supported aggregate functions SUM, COUNT, MIN, MAX, AVG, STDEV, RANGE, VARIANCE DISTINCT is supported, however DISTINCT is mutually exclusive with ORDER BY or window clause FIRST_VALUE/LAST_VALUE functions: FIRST_VALUE, LAST_VALUE Gives FIRST/LAST value of current partition NULL handling RESPECT NULLS (default) IGNORE NULLS LEAD/LAG functions: LEAD, LAG Reports value for nth preceding (LAG) or following (LEAD) row n defaults to 1 if not specified Default value can be specified for boundary cases; if not specified, then NULL is used RATIO_TO_REPORT function: RATIO_TO_REPORT, RATIOTOREPORT Reports a percentage of current value to sum of all values in current partition RATIOTOREPORT is synonym © 2014 IBM Corporation

Example - Ranking functions
Order by clause is required Window clause is not allowed Duplicate value handling is different between rank() and dense_rank() rank() - assigns the next relative ordinal to the next result set value dense_rank() - assigns the next absolute ordinal to the next result set value (alias is denserank()) Examples to follow on the slides following. © 2014 IBM Corporation

Rank Example select emp_num, sales,
rank() over (order by sales) as rnk, dense_rank() over (order by sales) as dense_rnk from sales; emp_num sales rnk dense_rnk , , , , , , In the sample data above, there are 6 rows of data reported. 6 unique employee numbers (emp_num), 4 unique sales numbers of 6 reported. The rank function is used to “rank” the domain of the over clause, in ascending order as shown above, while maintaining an internal counter of the total number rows processed. When the next unique value in the sales column is attained, the total counter of rows processed at that point will be applied as the rank, and to all duplicate sales values to follow. It skips all new total counters in the interlude while duplicates are processed. This number is reported in the rnk output column above. The dense_rank function, is used to “rank” the domain of the over clause as well, in ascending order as shown above, while maintaining an internal counter of the total number rows processed. When the next unique value in the sales column is attained, the total count of unique valued rows processed at that point will be applied as the rank. It does not skip any total count processed at any point. This number is reported in the dense_rnk output column above. © 2014 IBM Corporation

Top-N Rank Use a derived table to obtain top-n rank select *
from (select name, salary rank() over (order by salary desc) as rnk from employee) where rnk <= 5; Derived tables can be used to obtain a rank. © 2014 IBM Corporation

Rank in Sports Stats – All Time Baseball Home Runs
Player Home Runs …. 11 Albert Pujols 581 12 Harmon Killebrew 573 13 Rafael Palmeiro 569 14 Reggie Jackson 563 15 Manny Ramirez 555 16 Mike Schmidt 548 17 Mickey Mantle 536 18 Jimmie Foxx 534 19 David Ortiz 528 20 Willie McCovey 521 Frank Thomas Ted Williams 23 Ernie Banks 512 Eddie Mathews Current Home Run Hitters list All Time in Major League Baseball (partial list) Recently, the player David Ortiz moved past Willie McCovey and others into 19th place all time. If he hits his 535th home run, he will rise into 18th place alone, possibly this year He has announced retirement, end of this year. Everyone on this partial list, other than Pujols and Ortiz is retired or deceased. The current occupant of 18st place, Jimmie Foxx, will then be in 19th place. © 2014 IBM Corporation

Rank in Sports Stats – All Time Women’s Soccer Goals
Name Goals 1 Abby Wambach 184 2 Christine Sinclair 164 3 Mia Hamm 158 4 Kristine Lilly 130 5 Birgiz Prinz 128 6 Julie Fleeting 116 7 Patricia Panico 110 8 Elisabetta Vignotte 107 9 Sun Wen 106 10 Carolina Morace 105 Michelle Akers 12 Han Duan 101 Portia Modice 14 Marta 100 Tiffeny Milbrett Sinclair, Panico and Marta are the only active players on this list. Source is wikipedia: © 2014 IBM Corporation

SQL Enhancements – Grid Queries
In a Flexible Grid environment, it is possible for a cluster of instances to contain the same dbspaces and tables but no replicated data Common in retail and or distribution environments where stores / warehouses have different SKUs (products) and inventory levels Currently, to gather data from multiple instances requires executing multiple SQL operations into a temporary table or joining the results Informix introduces the grid query operation which, when executed from an administrative node, will gather data from the requested nodes with a single SQL statement © 2014 IBM Corporation

Grid queries can be executed against the entire cluster or against a subset of the instances These subsets are defined as “regions” but have nothing to do with geography The cdr define region command is used to define regions Syntax region_name must be unique across all clusters Will be used by SQL statements and cdr commands so have to be able to resolve to the correct region list_of_instances is a whitespace separated list of server group names cdr define region [--grid | -g ]=grid_name region_name [list_of_instances] © 2014 IBM Corporation

Example Instances can be members of more than one region if needed Regions can contain a subset of instances of another region The great_west region can contain g_inst_1, g_inst_2, g_inst_3, g_inst_4 The pac_coast region can contain g_inst_1, g_inst_3 cdr define region –g=my_grid region_1 g_inst_1 g_inst_2; cdr define region -–grid=my_grid region_2 g_inst_3 g_inst_4; © 2014 IBM Corporation

After defining any regions that are needed, next step to executing grid queries is to verify the consistency of the tables across the grid Ensures the table(s) schemas are identical across the grid All the columns and data types match so a consistent response to the query can occur Prevents table alters from occurring unless executed through a grid operation Connect to the grid, execute alter, disconnect from the grid © 2014 IBM Corporation

Examples Notice long and short versions of the commands are used Add all ds2 database tables Delete the customer table from the gridtables list Add the my_tab_1 and my_tab_2 tables to the gridtables list cdr change gridtable –g=my_grid -D=ds2 -a –all; cdr change gridtable –g=my_grid -D=ds2 –d customer; cdr change gridtable –-grid=my_grid --database=ds2 --add my_tab_1 my_tab_2 © 2014 IBM Corporation

With any regions defined, and tables added to the gridtable list, can now execute grid queries Restrictions Only query operations are supported Queries must “simple”, can not contain subqueries, joins, unions or the intersect, minus, or except qualifiers The grid query itself *can* be a subquery nested inside another query though Queries can only be executed against “normal” tables, not against views, synonyms or external tables Exception is sysmaster tables which are allowed Queries can only be executed against data types that are supported by distributed operations Excludes TimeSeries and any extended types that can’t be used in distributed operations © 2014 IBM Corporation

New syntax to the select operation – the grid clause Syntax at the statement level The optional all keyword determines whether or not all matching rows are returned ( a union all qualifier) from all instances or just the unique values (a basic union qualifier) Default is union or unique values select column_names from table_names grid [all] [‘region_name’ | ‘grid_name’] where © 2014 IBM Corporation

New syntax to the select operation – at the session level Rather than modifying existing statements, the scope and union qualifier can be set at the session level With the set environment select_grid operation Use the default union operator on the grid queries Use the union all operator on the grid queries set environment select_grid [‘grid_name’ | ‘region_name’] set environment select_grid_all [‘grid_name’ | ‘region_name’] © 2014 IBM Corporation

New syntax to the select operation – at the session level (cont.) Once set, all queries will execute as grid queries Only one default setting can be active at a time To change from one setting to the other re-execute the set environment command These settings can be changed for any individual statement(s) by including a grid clause within the statement © 2014 IBM Corporation

To turn off grid queries Use the option that matches the command enabling grid queries Individual statement(s) can still execute as a grid query by including the grid clause in the statement(s) set environment select_grid default; set environment select_grid_all default; © 2014 IBM Corporation

But what happens if a node with the grid / region is not available when the grid query executes? It depends, can either Abort the query and return an error Execute the query with the available nodes Can query later to see how many and which nodes didn’t participate Set at a session level with the set environment ifx_node_skip operation © 2014 IBM Corporation

Syntax Where default | off – the default setting. Query is aborted and an error returned on – rows are returned from the nodes available at the time the query was executed When the environment is set to on, any skipped nodes are captured and can be returned to the calling session Important – this information is gathered on a statement-by-statement basis and only lasts until the next grid query is executed in that session set environment ifx_node_skip [ default | off | on ] © 2014 IBM Corporation

To determine how many nodes, if any, were skipped and their names Returns an integer value with the number of skipped nodes Returns an lvarchar with the name of one of the skipped nodes To get all the node names, you will need to execute this the same number of times as returned by ifx_gridquery_skipped_nodes_count() All names aren’t returned at once to facilitate easier parsing of the result set The function operates like a fetch against a cursor execute function ifx_gridquery_skipped_nodes_count() execute function ifx_gridquery_skipped_nodes() © 2014 IBM Corporation

Example set environment select_grid my_region; set environment ifx_node_skip on; select from ; {results returned} execute function ifx_gridquery_skipped_nodes_count(); 3 execute function ifx_gridquery_skipped_nodes(); g_inst_2 g_inst_19 g_inst_38 © 2014 IBM Corporation

In the event a schema change needs to be made The change must be made through a grid operation as explained earlier A metadata flag will be set indicating an alter operation is in-flight Prevents any grid queries from executing As tables are being altered and acknowledged as completed, a cdr remaster gridtable operation is automatically executed to re-verify consistency across the cluster When the cdr remaster gridtable operation returns 0 (zero), the metadata flag is removed and grid queries can be resumed The cdr remaster gridtable operation can be invoked by an administrator to check the status of the alter operation Can know when to turn on grid query applications © 2014 IBM Corporation

Shard data across Enterprise Replication servers
Horizontally partition (shard) data across multiple database servers. Distribute performance horizontally across hardware Fast scalability with the cloud and adding more database server nodes Create a shard cluster using hash based sharding or expression based sharding Insert East Coast West Coast This example shows a sharded collection being created that would place the shard-key value of NV to g_shard_server_B. cdr define shardCollection collection_1 db_1:joe.clients) --type=delete --key=state --strategy=expression –-versionCol=version g_shard_server_A "IN ('WA','OR','ID')" g_shard_server_B "IN ('CA','NV','UT','AZ')" g_shard_server_C "IN ('TX','OK','NM','AR','LA')" g_shard_server_D REMAINDER Midwest Southern Region 248 © 2014 IBM Corporation 248

Connection Manager SLA’s aware of Secondary backlog
More control of the Connection Manager’s (CM) SLA policy. Scenarios: If a secondary is a specified number of logical log pages behind the primary, the SLA can be instructed not to use it Until it has caught up When a secondary is taken down for hours for maintenance on the primary. When the secondary is brought back they are out of sync with the primary. Modify the CM configuration file to specify how many logical log pages behind a secondary can be to still be considered Connection Manager ? Primary Secondary The above example shows the CM policy, and we’ve added that the secondary call fall 500 pages behind. Once it hits this threshold, the secondary will not be chosen by the connection manager for a new connection SLA mySLA DBSERVERS=(HDR,RSS,SDS) POLICY=SECAPPLYBACKLOG:500 © 2014 IBM Corporation 249

Improved memory pool allocation for ER
Aids in efficient ER transaction throughput Better memory pool allocation handling for ER New ONCONFIG parameter CDR_MEM 0 - The traditional method of memory allocation Use this setting when resource allocation is more important than performance 1 - Prevents multiple threads from simultaneously accessing a memory pool In large-scale ER environments significant performance improvements occur, because memory allocation is done by multiple threads that work in parallel 2 - Improves performance at the cost of increased memory usage Memory allocation requests are increased to the closest fixed-block size, so that free memory blocks can be found faster. Memory pools are not associated with specific CPU virtual processors, so memory can be freed directly to the memory pool. 250 © 2015 IBM Corporation 250

Informix Extensibility - Spatial

Geographic Transform The previous spatial data type transformation behavior was as follows: GPS Data is WGS-1984, most countries use a different EOGS that is more accurate for their part of the world (USA tends to use NAD-1983 in their State-Plane PROJCS) The previous Spatial Data Type can only transform a geometry value from one spatial reference system to another spatial reference system if both spatial reference systems share the same Geographic Coordinate System (GEOGCS) Previous transform behavior, when transforming between 2 spatial reference systems they had to share the same geographic coordinate system. This slide shows that behavior. Wikipedia: The World Geodetic System (WGS) is a standard for use in cartography, geodesy, and navigation. It comprises a standard coordinate system for the Earth, a standard spheroidal reference surface (the datum or reference ellipsoid) for raw altitude data, and a gravitational equipotential surface (the geoid) that defines the nominal sea level. The latest revision is WGS 84 (aka WGS 1984, EPSG:4326), established in 1984 and last revised in 2004.[1] Earlier schemes included WGS 72, WGS 66, and WGS 60. WGS 84 is the reference coordinate system used by the Global Positioning System. he coordinate origin of WGS 84 is meant to be located at the Earth's center of mass; the error is believed to be less than 2 cm.[2] The WGS 84 meridian of zero longitude is the IERS Reference Meridian,[3] 5.31 arc seconds or metres (336.3 ft) east of the Greenwich meridian at the latitude of the Royal Observatory.[4][5] The WGS 84 datum surface is an oblate spheroid (ellipsoid) with major (equatorial) radius a = m at the equator and flattening f = 1/ [6] The polar semi-minor axis b then equals a times (1−f), or m.[6] Currently, WGS 84 uses the EGM96 (Earth Gravitational Model 1996) geoid, revised in This geoid defines the nominal sea level surface by means of a spherical harmonics series of degree 360 (which provides about 100 km horizontal resolution).[7] The deviations of the EGM96 geoid from the WGS 84 reference ellipsoid range from about −105 m to about +85 m.[8] EGM96 differs from the original WGS 84 geoid, referred to as EGM84. NAD (North American Datum) – The North American Datum (NAD) is the datum now used to define the geodetic network in North America. A datum is a formal description of the shape of the earth along with an "anchor" point for the coordinate system. In surveying, cartography, and land-use planning, two North American Datums are in use: the North American Datum of 1927 (NAD27) and the North American Datum of 1983 (NAD83). Both are geodetic reference systems based on slightly different assumptions and measurements. NAD North American Datum of 1927 As more data were gathered discrepancies appeared so the datum was recomputed in 1927 using the same spheroid and origin as its predecessor. The North American Datum of 1927 (NAD27) was based on surveys of the entire continent from a common reference point that was chosen in 1901 because it was as near the center of the contiguous United States as could be calculated: It was based on a triangulation station at the junction of the transcontinental triangulation arc of 1899 on the 39th parallel north and the triangulation arc along the 98th meridian west that was near the geographic center of the contiguous United States. The datum declares a point on Meades Ranch to be 39°13′26.686″ north latitude, 98°32′30.506″ west longitude (NGS data). NAD27 is oriented by declaring the azimuth from Meades Ranch to Waldo to be 255°28′14.52″ from north. The latitude and longitude of every other point in North America is then based on its distance and direction from Meades Ranch: If a point was X meters in azimuth Y degrees from Meades Ranch, measured on the Clarke Ellipsoid of 1866, then its latitude and longitude on that ellipsoid were defined and could be calculated. Ellipsoid Semimajor axis† Semiminor axis† Inverse flattening†† Clarke ,378,206.4 m ,356,583.8 m †By definition. ††Calculated. These are the defining dimensions for NAD27, but Clarke actually defined his 1866 spheroid as a = 20,926,062 British feet, b = 20,855,121 British feet. The conversion to meters uses Clarke's 1865 inch-meter ratio of (The length of a foot or meter at the time could not practically be benchmarked to better than about 0.02 mm.) Most USGS topographic maps were published in NAD27 and many major projects by the US Army Corps of Engineers and other agencies were defined in NAD27, so the datum remains important despite more refined datums being available. NAD North American Datum of 1983 Because Earth deviates significantly from a perfect ellipsoid, the ellipsoid that best approximates its shape varies region by region across the world. Clarke 1866, and North American Datum of 1927 with it, were surveyed to best suit North America as a whole. Likewise, historically most regions of the world used ellipsoids measured locally to best suit the vagaries of earth's shape in their respective locales. While ensuring the most accuracy locally, this practice makes integrating and disseminating information across regions troublesome. As satellite geodesy and remote sensing technology reached high precision and were made available for civilian applications, it became feasible to acquire information referred to a single global ellipsoid. This is because satellites naturally deal with Earth as a monolithic body. Therefore the GRS 80 ellipsoid was developed for best approximating the earth as a whole, and it became the foundation for the North American Datum of Though generally GRS 80 and its close relative WGS 84 are not the best fit for any given region, a need for the closest fit largely evaporates when a global survey is combined with computers, databases, and software able to compensate for local conditions. NAD 1927 – vs NAD A point having a given latitude and longitude in NAD27 may be displaced on the order of many tens of meters from another point having the identical latitude and longitude in NAD83, so it is important to specify the datum along with the coordinates. The North American Datum of 1927 is defined by the latitude and longitude of an initial point (Meades Ranch in Kansas), the direction of a line between this point and a specified second point, and two dimensions that define the spheroid. The North American Datum of 1983 is based on a newer defined spheroid (GRS 80); it is an Earth-centered (or "geocentric") datum having no initial point or initial direction. North American Datum 1983 and WGS84 The initial definition of NAD83(1986) was intended to match GRS80 and WGS84, and many older publications indicate no difference. Subsequent more accurate measurements found a difference typically on the order of a meter over much of the US. Each datum has undergone refinements with more accurate and later measurements. One fundamental difference is that NAD83 is defined to remain essentially constant over time for points on the North American Plate, whereas WGS84 is defined with respect to the average of stations all over the world. Thus there is a change over time as to the difference between the systems. For much of the US the relative rate is on the order of 1 to 2 cm per year. Latest version of North American Datum The US National Spatial Reference System NAD 83(2011/MA11/PA11) epoch , is a refinement of the NAD83 datum using data from a network of very accurate GPS receivers at Continuously Operating Reference Stations (CORS). GEOGCS PROJCS2 PROJCS1 © 2014 IBM Corporation

Geographic Transform Transform between two spatial reference systems
The two spatial systems do not have to use the same GEOGCS Will transform between two GEOGCS’s. ST_Transform function GEOGCS1 PROJCS2 PROJCS1 GEOGCS2 GEOGTRANS We now support conversion between different GEOCS’s, so to transform from two spatial systems, they don’t have to share the same GEOCS. The ST_Transform() function The ST_Transform() function transforms an ST_Geometry into the specified spatial reference system. The following transformations are valid: Between two UNKNOWN coordinate systems (that is, the srtext column in the spatial_references table for both SRIDs contains UNKNOWN) Between a projected coordinate system and an unprojected coordinate system Between two projected coordinate systems Between two coordinate systems that have different false origins or system units The geographical coordinate systems of the source and target spatial reference systems do not need to be the same. A spatial reference system in one geographical coordinate system can be transformed into a spatial reference system in a different geographical coordinate system if the transform is supported by the ESRI libraries. Syntax ST_Transform(g ST_Geometry, SRID integer) Return type ST_Geometry Example: Change the false origin of a spatial reference system Suppose you create a spatial_references table entry suitable for Australia with the SE_CreateSrid() function: EXECUTE FUNCTION SE_CreateSrid (110, -45, 156, -10, "Australia: lat/lon coords"); (expression) 1002 Now load all of your data for Australia: CREATE TABLE aus_locns (name varchar(128), locn ST_Point); INSERT INTO aus_locns VALUES ("Adelaide", '1002 point( )'); INSERT INTO aus_locns VALUES ("Brisbane", '1002 point( )'); INSERT INTO aus_locns VALUES ("Canberra", '1002 point( )'); INSERT INTO aus_locns VALUES ("Melbourne", '1002 point( )'); INSERT INTO aus_locns VALUES ("Perth", '1002 point( )'); INSERT INTO aus_locns VALUES ("Sydney", '1002 point( )'); After you load all of your data for the Australian mainland, you realize you must include data for a few outlying islands, such as Norfolk Island and the Cocos Islands. However, the false origin and scale factor that you chose for SRID 1002 does not work for these islands as well: INSERT INTO aus_locns VALUES ("Norfolk Is.", '1002 point( )'); (USE19) - Coordinates out of bounds in ST_PointIn. INSERT INTO aus_locns VALUES ("Cocos Is.", '1002 point( )'); The solution is to create a spatial_references table entry with a false origin and scale factor that accommodates both the old data and new data, and then update the old data: EXECUTE FUNCTION SE_CreateSrid (95, -55, 170, -10, "Australia + outer islands: lat/lon coords"); (expression) 1003 INSERT INTO aus_locns VALUES ("Norfolk Is.", '1003 point( )'); INSERT INTO aus_locns VALUES ("Cocos Is.", '1003 point( )'); UPDATE aus_locns SET locn = ST_Transform(locn, 1003)::ST_Point WHERE ST_Srid(locn) = 1002; Example: Project data dynamically In a typical application, spatial data is stored in unprojected latitude and longitude format. Then, when you draw a map, you retrieve the data in a particular projection. First, create a spatial_references table entry that is suitable for your unprojected data. For this example, use the 1983 North American datum. Because this datum is a well-known, standard datum you can use the SE_CreateSrtext() function to create the srtext field: INSERT INTO spatial_references (srid, description, falsex, falsey, xyunits, falsez, zunits, falsem, munits, srtext) VALUES (1004, "Unprojected lat/lon, NAD 83 datum", -180, -90, , 0, 1000, 0, 1000, SE_CreateSrtext(4269)); Now create a table and load your data: CREATE TABLE airports (id char(4), name varchar(128), locn ST_Point); INSERT INTO airports VALUES( 'BTM', 'Bert Mooney', '1004 point( )'); INSERT INTO airports VALUES( 'BZN', 'Gallatin Field', '1004 point( )'); INSERT INTO airports VALUES( 'COD', 'Yellowstone Regional', '1004 point( )'); INSERT INTO airports VALUES( 'JAC', 'Jackson Hole', '1004 point( )'); INSERT INTO airports VALUES( 'IDA', 'Fanning Field', '1004 point( )'); Create one or more spatial_references table entries for any projections that you need. Be sure that the underlying geographic coordinate system (in this case, NAD 83) is the same: INSERT INTO spatial_references (srid, description, falsex, falsey, xyunits, falsez, zunits, falsem, munits, srtext) VALUES (1005, "UTM zone 12N, NAD 83 datum", , , 1000, 0, 1000, 0, 1000, SE_CreateSrtext(26912)); Transform the data to a projected coordinate system on an as needed basis: SELECT id, ST_Transform(locn, 1005) as utm FROM airports; id BTM utm 1005 POINT ( ) id BZN utm 1005 POINT ( ) id COD utm 1005 POINT ( ) id JAC utm 1005 POINT ( ) id IDA utm 1005 POINT ( ) Example: Compare geometries that have different SRIDs You can use the ST_Transform() function to transform a geometry when you compare two geometries that have different SRIDs: SELECT * FROM tab1 a, tab2 b WHERE ST_Intersects(a.shape, ST_Transform(b.shape, ST_SRID(a.shape))); Example: Transform between geographic spatial reference systems The following statements create a table and insert data for the geographic spatial reference system 4326: CREATE TABLE geogcs_to_geogs_xform (pid smallint, geom ST_Geometry) ; INSERT INTO geogcs_to_geogs_xform VALUES (5, ST_GeomFromText ('point ( )', 4326)) ; INSERT INTO geogcs_to_geogs_xform VALUES (6, ST_GeomFromText ('point z ( )', 4326)) ; INSERT INTO geogcs_to_geogs_xform VALUES (7, ST_GeomFromText ('point m ( )', 4326)) ; INSERT INTO geogcs_to_geogs_xform VALUES (8, ST_GeomFromText ('point zm ( )', 4326)) ; The following query transforms the rows from the geographic spatial reference system 4326 to the geographic spatial reference system 4269: SELECT pid, ST_Transform (geom, 4269) FROM geogcs_to_geogs_xform; pid 5 (expression) 4269 POINT ( ) pid 6 (expression) 4269 POINT Z ( ) pid 7 (expression) 4269 POINT M ( ) pid 8 (expression) 4269 POINT ZM ( ) 4 row(s) retrieved. Example: Transform between projected spatial reference systems This example transforms data between projected spatial reference systems that are in different geographic coordinate systems. The following statements create a table and insert data for the projected spatial reference system 2153: CREATE TABLE projcs_to_projcs_xform (pid smallint, geom ST_Geometry) ; INSERT INTO projcs_to_projcs_xform VALUES (11, ST_GeomFromText ('point( )', 2153)) ; INSERT INTO projcs_to_projcs_xform VALUES (12, ST_GeomFromText ('multipoint( , , , , )', 2153)) ; INSERT INTO projcs_to_projcs_xform VALUES (13, ST_GeomFromText ('linestring( , )', 2153)) ; INSERT INTO projcs_to_projcs_xform VALUES (14, ST_GeomFromText ('linestring( , , , , )', 2153)) ; INSERT INTO projcs_to_projcs_xform VALUES (15, ST_GeomFromText ('multilinestring(( , , , , ),( , , , , ))', 2153)) ; INSERT INTO projcs_to_projcs_xform VALUES (16, ST_GeomFromText ('polygon(( , , , , ))', 2153)) ; INSERT INTO projcs_to_projcs_xform VALUES (17, ST_GeomFromText ('multipolygon((( , , , , )),(( , , , , )))', 2153)) ; The following query transforms the rows from the projected spatial reference system 2153 to the projected spatial reference system 32611: SELECT pid, ST_Transform (geom, 32611) FROM projcs_to_projcs_xform; pid 11 (expression) POINT ( ) pid 12 (expression) MULTIPOINT ( , , , , ) pid 13 (expression) LINESTRING ( , ) pid 14 (expression) LINESTRING ( , , , , ) pid 15 (expression) MULTILINESTRING (( , , , , ),( , , , , )) pid 16 (expression) POLYGON (( , , , , )) pid 17 (expression) MULTIPOLYGON ((( , , , , )),(( , , , , ))) 7 row(s) retrieved. © 2014 IBM Corporation

ST_Transform() Transforms in the following cases between:
Two UNKNOWN coordinate systems The srtext column in the spatial_references table for both SRIDs contains UNKNOWN) A projected coordinate system and an un-projected coordinate system Two projected coordinate systems Two coordinate systems that have different false origins or system units The GEOGCS of the source and target spatial reference systems do not need to be the same. A spatial reference system in one GEOGCS can be transformed into a spatial reference system in a different GEOGCS if the transform is supported by the ESRI libraries. The ST_Transform() function The ST_Transform() function transforms an ST_Geometry into the specified spatial reference system. The following transformations are valid: Between two UNKNOWN coordinate systems (that is, the srtext column in the spatial_references table for both SRIDs contains UNKNOWN) Between a projected coordinate system and an unprojected coordinate system Between two projected coordinate systems Between two coordinate systems that have different false origins or system units The geographical coordinate systems of the source and target spatial reference systems do not need to be the same. A spatial reference system in one geographical coordinate system can be transformed into a spatial reference system in a different geographical coordinate system if the transform is supported by the ESRI libraries. Syntax ST_Transform(g ST_Geometry, SRID integer) Return type ST_Geometry Example: Change the false origin of a spatial reference system Suppose you create a spatial_references table entry suitable for Australia with the SE_CreateSrid() function: EXECUTE FUNCTION SE_CreateSrid (110, -45, 156, -10, "Australia: lat/lon coords"); (expression) 1002 Now load all of your data for Australia: CREATE TABLE aus_locns (name varchar(128), locn ST_Point); INSERT INTO aus_locns VALUES ("Adelaide", '1002 point( )'); INSERT INTO aus_locns VALUES ("Brisbane", '1002 point( )'); INSERT INTO aus_locns VALUES ("Canberra", '1002 point( )'); INSERT INTO aus_locns VALUES ("Melbourne", '1002 point( )'); INSERT INTO aus_locns VALUES ("Perth", '1002 point( )'); INSERT INTO aus_locns VALUES ("Sydney", '1002 point( )'); After you load all of your data for the Australian mainland, you realize you must include data for a few outlying islands, such as Norfolk Island and the Cocos Islands. However, the false origin and scale factor that you chose for SRID 1002 does not work for these islands as well: INSERT INTO aus_locns VALUES ("Norfolk Is.", '1002 point( )'); (USE19) - Coordinates out of bounds in ST_PointIn. INSERT INTO aus_locns VALUES ("Cocos Is.", '1002 point( )'); The solution is to create a spatial_references table entry with a false origin and scale factor that accommodates both the old data and new data, and then update the old data: EXECUTE FUNCTION SE_CreateSrid (95, -55, 170, -10, "Australia + outer islands: lat/lon coords"); (expression) 1003 INSERT INTO aus_locns VALUES ("Norfolk Is.", '1003 point( )'); INSERT INTO aus_locns VALUES ("Cocos Is.", '1003 point( )'); UPDATE aus_locns SET locn = ST_Transform(locn, 1003)::ST_Point WHERE ST_Srid(locn) = 1002; Example: Project data dynamically In a typical application, spatial data is stored in unprojected latitude and longitude format. Then, when you draw a map, you retrieve the data in a particular projection. First, create a spatial_references table entry that is suitable for your unprojected data. For this example, use the 1983 North American datum. Because this datum is a well-known, standard datum you can use the SE_CreateSrtext() function to create the srtext field: INSERT INTO spatial_references (srid, description, falsex, falsey, xyunits, falsez, zunits, falsem, munits, srtext) VALUES (1004, "Unprojected lat/lon, NAD 83 datum", -180, -90, , 0, 1000, 0, 1000, SE_CreateSrtext(4269)); Now create a table and load your data: CREATE TABLE airports (id char(4), name varchar(128), locn ST_Point); INSERT INTO airports VALUES( 'BTM', 'Bert Mooney', '1004 point( )'); INSERT INTO airports VALUES( 'BZN', 'Gallatin Field', '1004 point( )'); INSERT INTO airports VALUES( 'COD', 'Yellowstone Regional', '1004 point( )'); INSERT INTO airports VALUES( 'JAC', 'Jackson Hole', '1004 point( )'); INSERT INTO airports VALUES( 'IDA', 'Fanning Field', '1004 point( )'); Create one or more spatial_references table entries for any projections that you need. Be sure that the underlying geographic coordinate system (in this case, NAD 83) is the same: INSERT INTO spatial_references (srid, description, falsex, falsey, xyunits, falsez, zunits, falsem, munits, srtext) VALUES (1005, "UTM zone 12N, NAD 83 datum", , , 1000, 0, 1000, 0, 1000, SE_CreateSrtext(26912)); Transform the data to a projected coordinate system on an as needed basis: SELECT id, ST_Transform(locn, 1005) as utm FROM airports; id BTM utm 1005 POINT ( ) id BZN utm 1005 POINT ( ) id COD utm 1005 POINT ( ) id JAC utm 1005 POINT ( ) id IDA utm 1005 POINT ( ) Example: Compare geometries that have different SRIDs You can use the ST_Transform() function to transform a geometry when you compare two geometries that have different SRIDs: SELECT * FROM tab1 a, tab2 b WHERE ST_Intersects(a.shape, ST_Transform(b.shape, ST_SRID(a.shape))); Example: Transform between geographic spatial reference systems The following statements create a table and insert data for the geographic spatial reference system 4326: CREATE TABLE geogcs_to_geogs_xform (pid smallint, geom ST_Geometry) ; INSERT INTO geogcs_to_geogs_xform VALUES (5, ST_GeomFromText ('point ( )', 4326)) ; INSERT INTO geogcs_to_geogs_xform VALUES (6, ST_GeomFromText ('point z ( )', 4326)) ; INSERT INTO geogcs_to_geogs_xform VALUES (7, ST_GeomFromText ('point m ( )', 4326)) ; INSERT INTO geogcs_to_geogs_xform VALUES (8, ST_GeomFromText ('point zm ( )', 4326)) ; The following query transforms the rows from the geographic spatial reference system 4326 to the geographic spatial reference system 4269: SELECT pid, ST_Transform (geom, 4269) FROM geogcs_to_geogs_xform; pid 5 (expression) 4269 POINT ( ) pid 6 (expression) 4269 POINT Z ( ) pid 7 (expression) 4269 POINT M ( ) pid 8 (expression) 4269 POINT ZM ( ) 4 row(s) retrieved. Example: Transform between projected spatial reference systems This example transforms data between projected spatial reference systems that are in different geographic coordinate systems. The following statements create a table and insert data for the projected spatial reference system 2153: CREATE TABLE projcs_to_projcs_xform (pid smallint, geom ST_Geometry) ; INSERT INTO projcs_to_projcs_xform VALUES (11, ST_GeomFromText ('point( )', 2153)) ; INSERT INTO projcs_to_projcs_xform VALUES (12, ST_GeomFromText ('multipoint( , , , , )', 2153)) ; INSERT INTO projcs_to_projcs_xform VALUES (13, ST_GeomFromText ('linestring( , )', 2153)) ; INSERT INTO projcs_to_projcs_xform VALUES (14, ST_GeomFromText ('linestring( , , , , )', 2153)) ; INSERT INTO projcs_to_projcs_xform VALUES (15, ST_GeomFromText ('multilinestring(( , , , , ),( , , , , ))', 2153)) ; INSERT INTO projcs_to_projcs_xform VALUES (16, ST_GeomFromText ('polygon(( , , , , ))', 2153)) ; INSERT INTO projcs_to_projcs_xform VALUES (17, ST_GeomFromText ('multipolygon((( , , , , )),(( , , , , )))', 2153)) ; The following query transforms the rows from the projected spatial reference system 2153 to the projected spatial reference system 32611: SELECT pid, ST_Transform (geom, 32611) FROM projcs_to_projcs_xform; pid 11 (expression) POINT ( ) pid 12 (expression) MULTIPOINT ( , , , , ) pid 13 (expression) LINESTRING ( , ) pid 14 (expression) LINESTRING ( , , , , ) pid 15 (expression) MULTILINESTRING (( , , , , ),( , , , , )) pid 16 (expression) POLYGON (( , , , , )) pid 17 (expression) MULTIPOLYGON ((( , , , , )),(( , , , , ))) 7 row(s) retrieved. © 2014 IBM Corporation

Great Circle Calculations
Support of SQL/MM/OGS features linear distance with GEOGCS based spatial references Great Circle calculations work with angular coordinates (ie latitude/longitude) in a GEOGCS and calculates the linear distance across the ellipsoid. Example GEOGCS: GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984",SPHEROID["WGS_1984", , ]],PRIMEM["Greenwich",0],UNIT["Degree", ]] For customers using the previous Geodetic product in their applications, this provides a way to reuse great circle calculations as part of the IDS provided Spatial feature. The basic note here, is that with the loss of the geodetic blade we lost certain capabilities. This brings back the great circle calculations which are critical for geospatial. © 2014 IBM Corporation

Geospatial General Enhancements
The false origin and scale values for the system are derived from the Projected Bounds of the underlying Coordinate System. For example, EPSG 2193 (NZGD 2000 New Zealand Transverse Mercator) derives the falsex, falsey and xyunits values based on the extent of the projection system: falsex = falsey = xyunits = The z and m coordinates for all predefined spatial reference systems have the following values in the following columns: falsez = zunits = falsem = munits = © 2014 IBM Corporation

Geospatial General Enhancements
SDE 10.2 libraries (ESRI) Added predefined Spatial Reference Systems: Authority Authority ID’s (SRID) Description EPSG Projection systems defined by the EPSG 46008 GEOGCS NAD 1927 4759 GEOGCS NAD 1983_NSRS2007 4322 GEOGCS WGS 1972 4326 GEOGCS WGS 1984 4760 GEOGCS WGS 1966 WGS 1984 UTM Zones ESRI Various world projection systems 54034 EPSG - European Petroleum Survey Group (now defunct, but their work lives on, this is current still). The International Association of Oil & Gas Producers (OGP) absorbed this group in 2005. SRID Spatial Reference ID, is a unique value used to unambiguously identify projected, unprojected, and local spatial coordinate system definitions. These coordinate systems form the heart of all GIS applications. Virtually all major spatial vendors have created their own SRID implementation or refer to those of an authority, such as the European Petroleum Survey Group (EPSG). (NOTE: As of 2005 the EPSG SRID values are now maintained by the International Association of Oil & Gas Producers (OGP) Surveying & Positioning Committee). SRIDs are the primary key for the Open Geospatial Consortium (OGC) spatial_ref_sys metadata table for the Simple Features for SQL Specification, Versions 1.1 and 1.2, which is defined as follows: CREATE TABLE SPATIAL_REF_SYS ( SRID INTEGER NOT NULL PRIMARY KEY, AUTH_NAME CHARACTER VARYING(256), AUTH_SRID INTEGER, SRTEXT CHARACTER VARYING(2048) ) In spatially enabled databases (such as IBM DB2, IBM Informix, Microsoft SQL Server, MySQL, Oracle, Teradata, PostgreSQL and SQL Anywhere), SRIDs are used to uniquely identify the coordinate systems used to define columns of spatial data or individual spatial objects in a spatial column (depending on the spatial implementation). SRIDs are typically associated with a well known text (WKT) string definition of the coordinate system (SRTEXT, above). From the Well Known Text Wikipedia page, “A WKT string for a spatial reference system describes the datum, geoid, coordinate system, and map projection of the spatial objects”. Here are two common coordinate systems with their EPSG SRID value followed by their well known text: This is also a key column in the spatial_references table in Appendix A. GEOGCS - Geographic Coordinate Systems SDE – Spatial Database Engine ESRI - ESRI is an international supplier of Geographic Information System software, web GIS and geodatabase management applications, located in Redwood City California. WGS – wikipedia: - The World Geodetic System (WGS) is a standard for use in cartography, geodesy, and navigation. It comprises a standard coordinate system for the Earth, a standard spheroidal reference surface (the datum or reference ellipsoid) for raw altitude data, and a gravitational equipotential surface (the geoid) that defines the nominal sea level. The latest revision is WGS 84 (aka WGS 1984, EPSG:4326), established in 1984 and last revised in 2004.[1] Earlier schemes included WGS 72, WGS 66, and WGS 60. WGS 84 is the reference coordinate system used by the Global Positioning System. UTM – wikipedia: - The Universal Transverse Mercator (UTM) projected coordinate system uses a 2-dimensional Cartesian coordinate system to give locations on the surface of the Earth. It is a horizontal position representation, i.e. it is used to identify locations on the Earth independently of vertical position, but differs from the traditional method of latitude and longitude in several respects. The UTM system is not a single map projection. The system instead divides the Earth into sixty zones, each a six-degree band of longitude, and uses a secant transverse Mercator projection in each zone. NAD - The North American Datum (NAD) is the datum now used to define the geodetic network in North America. A datum is a formal description of the shape of the earth along with an "anchor" point for the coordinate system. In surveying, cartography, and land-use planning, two North American Datums are in use: the North American Datum of 1927 (NAD27) and the North American Datum of 1983 (NAD83). Both are geodetic reference systems based on slightly different assumptions and measurements, used in geographically locating a single point or points on the earth. NAD83 has as its basis the idea that the North American plate does not move on the earth on a rough line east of California, Nevada, Oregon, Washington State and Alaska, making for reliable locations that do not move. West of the line is another matter and calls for a different standard since the earth moves there; thus was born NAD-1983_NSRS2007 NAD 1983_NSRS2007 – NAD 1983 (see previous note) National Spatial Reference System adopted an alternative realization of NAD 83 called NAD 83(NSRS2007) for the distribution of coordinates at 67,693 passive geodetic control monuments. This realization approximates the more rigorously defined NAD 83(CORS96), but can never be equivalent to it. NAD 83(NSRS2007) was created by adjusting GPS data collected during various campaign-style geodetic surveys performed between mid-1980 and The NAD 83(CORS96) positional coordinates for 685 CORS were held fixed (predominantly at the epoch for the stable North American plate, but in Alaska and in Western Continental United States. Derived NAD 83(NSRS2007) positional coordinates should be consistent with corresponding NAD 83(CORS96) positional coordinates to within the accuracy of the GPS data used in the adjustment. In California, the NAD 83 epoch values for the California CORS (CGPS) were obtained through Scripps’ Sector utility and are available through the CSRC website at: We are using the current 10.2 ESRI libraries, and now support ~ 1000 spatial reference systems, making working with spatial/geospatial data much easier. © 2014 IBM Corporation

spatial_references table
The spatial_references table stores data about each map projection that you use to store the spherical geometry of the Earth, for example, your data might use the Mercator projection. The table contains a unique spatial reference ID (SRID) for each spatial reference system. All spatial reference systems that you use in your database must have a record in this table. All geometries in a spatial column must use the same spatial reference system. The Informix spatial functions use the parameters of a spatial reference system to translate and scale each floating point coordinate of the geometry into 54-bit positive integers before storage. When retrieved, the coordinates are restored to their external floating point format. Appendix C has a schema of the table. © 2014 IBM Corporation

Spatial Reference - Creation samples
Insert a spatial reference system into the spatial_references table: INSERT INTO spatial_references (srid, description, auth_name, auth_srid, falsex, falsey, xyunits, falsez, zunits, falsem, munits, srtext) VALUES (1, NULL, NULL, NULL, 0, 0, 100, 0, 1, 0, 1, 'UNKNOWN'); In this example, the spatial reference system has an SRID value of 1, a false X, Y of (0,0), and its system units are 100. The Z coordinate and measure offsets are 0, while the Z coordinate and measure units are 1. Create a spatial reference system with the SE_CreateSRID() function For the hazardous sites and sensitive areas example, the coordinates are in a local countywide XY coordinate system. The X and Y coordinates range from Create a new spatial reference system with the SRID of 5: EXECUTE FUNCTION SE_CreateSRID(0, 0, , , "Springfield county XY coord system"); (expression) 5 The Database Extensions User’s Guide for Informix 12.10xC3 has more information. © 2014 IBM Corporation

Links, Helpful Information and Resources
Informix CTP wiki Informix Developer Edition free download: Informix Virtual Appliance MongoDB site MongoDB download: MongoDB drivers John Miller’s Blog John Miller’s NoSQL Chat With The Labs session Keshava Murthy’s Blog Informix NoSQL Whitepaper ibmnosql.com © 2014 IBM Corporation

Appendix A – New Event Alarms

Appendix A – New Event Alarms
Class Id Internal Name Event ID Description 24 ALRMU_85_AUTOTUNE_DAEMON_FAIL 24013 Auto Tune daemon has insufficient resources to perform tuning operations. Online log: Performance Advisory User action: Restart the server with more resources Class Id Internal Name Event ID Description 24 ALRMU_85_AUTOTUNE_ADD_CPUVP_FAIL 24014 Auto tuning failed to start a CPU VP Online log: Performance Advisory User action: Restart the server with more resources © 2014 IBM Corporation

Appendix A – New Event Alarms (cont’d)
Class Id Internal Name Event ID Description 24 ALRMU_24_ADD_BP_FAIL 24015 Auto tuning was unable to extend the buffer pool due to insufficient resources. Online log: Performance Advisory User action: Restart the server with more resources Class Id Internal Name Event ID Description 85 ALRMU_85_OVER_LLOG 85001 Auto tuning failed to add another logical log, because adding another log would exceed the maximum log space as defined by configuration parameter AUTO_LLOG. Online log: Performance Advisory User action: Increase the maximum amount of log space by changing AUTO_LLOG configuration parameter. © 2014 IBM Corporation

Class Id Internal Name Event ID Description 85 ALRMU_85_OVER_BPOOL 85002 Auto tuning failed to extend a bufferpool, because the buffer pool would exceed the maximum amount of memory or extensions as defined by configuration parameter BUFFERPOOL. Online log: Performance Advisory User action: Increase the maximum amount defined by the configuration parameter BUFFERPOOL. © 2014 IBM Corporation

Class Id Internal Name Event ID Description 85 ALRMU_85_OVER_CPU 85003 Auto tuning failed to add a CPU VP because another CPU VP would exceed the maximum number specified in the configuration parameter VPCLASS or exceed the number of processors on the computer. User action: If there are more processors available on the computer, increase the maximum number of CPU VPs allowed by changing the VPCLASS configuration parameter. © 2014 IBM Corporation

Other Enterprise Replication (ER) improvements
ER now supports Hertz and Compressed timeseries New Event alarm for blocked replicate transactions New event alarm appears If ER transactions are blocked when a replicated table is in alter mode 268 © 2015 IBM Corporation

Appendix B – Sysadmin Tenant Catalog Table

Appendix A – Sysadmin:tenant Table
Column Type Description tenant_id int The unique ID of the tenant database. tenant_dbsname varchar(128) The name of the tenant database. tenant_resources bson The properties of the tenant database and the state of the tenant. Cast this column to JSON “:JSON” to view the information. tenant_last_updated datetime year to second The time stamp of the last configuration change to the tenant database. tenant_comment lvarchar(2048) Comments about the tenant database.

Appendix C – Spatial References Table
Scott Pickett WW Informix Technical Sales For questions about this presentation, to: © 2014 IBM Corporation

Appendix C - spatial_references table
Column name Type Example value Description srid INTEGER NOT NULL 12345 Primary key: the unique key for the record that describes a particular spatial reference system description VARCHAR(64) WGS 1984 A text description of the spatial reference system auth_name VARCHAR(255) EPSG The name of the standard or standards body cited for the reference system auth_srid INTEGER 4326 The ID of the spatial reference system as defined by the authority cited in auth_name column. falsex FLOAT NOT NULL -180 The external floating point X coordinates are converted to integers for storage in the database by subtracting the falsex values. falsey -90 The external floating point Y coordinates are converted to integers for storage in the database by subtracting the falsey values. xyunits Before the external floating point X and Y coordinates are inserted into the database, the coordinates are scaled by the value in xyunits. The scaling process adds a half unit and truncates the remainder. falsez -1000 The external floating point Z coordinates are converted to integers for storage in the database by subtracting the falsez values. zunits 1000 A factor that is used to scale the Z-coordinate falsem The external floating point M coordinates are converted to integers for storage in the database by subtracting the falsem values. munits A factor that is used to scale the measure values srtext CHAR(2048) GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984", SPHEROID["WGS_1984", , ]], PRIMEM["Greenwich",0], UNIT["Degree", ]] © 2014 IBM Corporation

Simply Powerful – Informix xC1 thru xC4

Similar presentations

Presentation on theme: "Simply Powerful – Informix xC1 thru xC4"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Simply Powerful – Informix xC1 thru xC4

Similar presentations

Presentation on theme: "Simply Powerful – Informix xC1 thru xC4"— Presentation transcript:

Similar presentations

About project

Feedback