Presentation is loading. Please wait.

Presentation is loading. Please wait.

Abstract Linux continues to gain momentum as a cost-effective operating system [OS]. Many companies are obtaining significant ROI & savings by migrating.

Similar presentations


Presentation on theme: "Abstract Linux continues to gain momentum as a cost-effective operating system [OS]. Many companies are obtaining significant ROI & savings by migrating."— Presentation transcript:

1

2 Abstract Linux continues to gain momentum as a cost-effective operating system [OS]. Many companies are obtaining significant ROI & savings by migrating to this platform from other proprietary flavors of Unix. In today's business environment, Informix DBA's must be proficient with the Linux toolset. If you are a DBA in a Linux shop, then it is "vital" that you know how to use the OS effectively.

3 Items discussed o How to manage server software o View & configure system resources o Analyze server performance o Handle server log files o Miscellaneous tips, tricks & treats

4 Flavors of Linux o Debian – debian.org o Fedora – fedoraproject.org [Redhat community project] o Mandrake or Mandriva - www2.mandriva.com/en o Redhat – redhat.com o SuSE – novell.com/linux o Ubuntu - ubuntu.com [derivative of Debian]

5 Managing server software o rpm – Redhat Package Manager & now stands for RPM Package Manager [used by many Linux distributions]. o up2date – pulls rpm's & associated dependencies o yum - Yellowdog Updater, Modified [uses a xml repository] o apt – Advanced Packaging Tool [Debian & variants and modified to work with rpm via apt-rpm] Package extension o.src.rpm – download source & compile the rpm o.noarch – no dependency on architecture of the system

6 Managing server software o RPM system comprises of a local database in /var/lib/rpm, rpm executable & rpm package files. Local database houses metadata information about installed packages including package prerequisites & file attributes. Therefore, removing a package becomes relatively simple. o Install package: rpm -ihv i – install; h – hash marks; v – verbose output o Identify rpm package to install: rpm -qf q – query; f – file; filename - /usr/bin/gedit [executable] or libc.iso [shared object], for example

7 Managing server software o View contents of package & peek at destination where files will be extracted: rpm -qlp q – query; l – list; p – view information directly from package o Upgrade an existing package: rpm -Uhv or -- upgrade option [original package removal & new one installed] -F or -- freshen will not install package if the package doesn't already exist o Remove a package: rpm -e or – erase avoid dependency check -- nodeps option [not recommended]

8 View & configure system resources o View system information uname -a cat /proc/version The Linux /proc virtual filesystem acts as an interface for viewing and configuring kernel parameters. The /proc directory is a hierarchy of files and subdirectories that contain the current settings of kernel values [appropriately named because this sends information to other system processes & virtual because its files dont actually reside on disk.

9 View & configure system resources Descriptions of Virtual Files in the /proc Directory File Name Contains Information Regarding ____________________________________________________ /proc/cpuinfo CPU and system architecture /proc/meminfo Free and used memory for both physical RAM and swap /proc/net Directory containing network information /proc/mounts All mounted filesystems /proc/diskstats Disk I/O statistics for each disk

10 View & configure system resources Descriptions of Virtual Files in the /proc Directory File Name Contains Information Regarding ____________________________________________________ /proc/devices PCI devices /proc/filesystems Filesystems compiled into the kernel /proc/sys Contains subdirectories and files pertaining to kernel variables [some can be configured with sysctl command] /proc/cmdline Parameters passed to the kernel at boot time /proc/version Version of the operating systems

11 View & configure system resources o Some of /proc/ may need root access to review o man proc will show documentation on /proc virtual filesystem o top & free extract some information from /proc virtual files Mem Total – Memory Physical RAM Swap Total – Memory Virtual o View per-process memory consumption cat /proc/ /maps ; cat /proc/ /status o Viewing kernel parameters grep. /proc/sys/kernel/* [some may need root access to review]

12 View & configure system resources o Viewing kernel parameters - sysctl -a -w to dynamically modify kernel parameters o For making changes persistent across reboots, add parameters to /etc/sysctl.conf or do a "sysctl -w" & then issue a "sysctl -p" [entries in this file are instantiated as current values used by the linux kernel] It helps to keep a backup copy of kernel parameters before change – in case one needs to revert back + maintain an audit trail of all kernel changes [use diff command to display differences in file versions] cp /etc/sysctl.conf /etc/sysctl.conf

13 View & configure system resources o Network settings sysctl -a | grep -i net o The /etc/sysctl.conf file is read from top to bottom – last entry is set if multiple lines configure the same kernel parameter echo >> /proc/sys/fs/file-max or sysctl -w fs.file-max=65536 [remember to add an entry to /etc/sysctl.conf to make the change persistent across reboots]

14 View & configure system resources o Not all virtual files in the /proc/sys directory can be modified. One quick way to determine whether a /proc/sys file can be altered is to check the permissions – any file that shows the writeable permission can be changed. o Shared memory segments – messages, queues, semaphores A semaphore is a construct which signifies whether a resource is busy [or not] Inter Process Communication Status: ipcs -a - - displays list of currently allocated shared components

15 View & configure system resources o Linux imposes a limit on the overall # of files simultaneously open on a server. Those that host database applications tend to have many simultaneously open files. If the default value for the maximum # of open files is too low, one will most likely have to increase it for preventing errors related to "running out of file handles". cat /proc/sys/fs/file-nr - - this output will show the current allocated, free & maximum file handle #'s respectively.

16 View & configure system resources o A logon shell often imposes default maximum limits on various resources that a process can use such as the number of open files, processes per user, amount of memory allocated, and so on. These shell limits are defined by the ulimit command. Each resource has soft & hard limit settings. The soft limit setting establishes the default resource limit when a user logs on to system. If the user process exceeds resource soft limit setting, an error will be obtained. List of system resource soft & hard limits - ulimit -a ; ulimit -aH

17 View & configure system resources o A user can modify the hard limit down but cannot modify the hard limit up. This means if one sets a hard limit to a lower value, one cannot reset it to its original value. Only the root user can modify a hard limit to a higher value. o The default values for soft and hard limits on the server are established by adding entries into /etc/security/limits.conf file. o Database processes often tend to consume more resources than what default shell limits allow.

18 Analyze server performance OS monitoring can assist in providing answers to questions like - o Details of resources an application is using o Applications doing work or waiting for other resources o Server bottlenecks possibly affecting database performance o Potential resources used by database engine which likely can't be monitored with bundled-in database commands & tools

19 Analyze server performance ToolPerformance Monitoring Purpose _______________________________________________ df Reports on free disk space du Displays disk usage free Displays free and used memory iostat Displays disk I/O statistics mpstat Reports CPU statistics netstat Reports on network statistics

20 Analyze server performance ToolPerformance Monitoring Purpose _______________________________________________ ps Identifies highest CPU- and memory-consuming sessions sar Displays CPU, memory, disk I/O, and network usage, both current and historical top Identifies sessions consuming the most resources vmstat Monitors processes, CPU, memory, or disk I/O bottlenecks watch Periodically runs another command {io|mp|net|vm}stat

21 Analyze server performance Identifying System Bottlenecks Problem: The application users are reporting that the database seems slow. One wants to determine whether there are any system resource bottlenecks on the database server. Solution: The vmstat (virtual memory statistics) tool is intended to help in quickly identifying bottlenecks on a server. It displays real-time performance information about processes, memory, paging, disk I/O, and CPU usage. Some general techniques one can use for interpreting the output of vmstat are in the following slide.

22 Analyze server performance o If b (processes sleeping) is consistently > 0, then one may not have enough CPU processing power. Recipe 1 aids in identifying items consuming the most CPU. o If so/si (memory swapped out to/in from disk) are consistently > 0, one may have a memory bottleneck. Recipe 3 aids in identifying items consuming the most memory. o If wa (time waiting for I/O) column is high, this is usually an indication that the storage subsystem is overloaded. Recipes 4 & 5 aid in identifying the sources of I/O contention. Note: The Linux vmstat command does not count itself as a currently running process.

23 Analyze server performance By default, only 1 line of server statistics is displayed when running vmstat (without supplying any options) - average statistics calculated from the last time the system was rebooted. One can also redirect vmstat output to a file - this is useful for analyzing historical performance over a period of time. Another useful way to use vmstat is with watch which is used to execute another program on a periodic basis: For example, watch -n -d vmstat When running vmstat in watch -d (differences) mode, one will visually see changes on screen as they alter from snapshot to snapshot. To exit from watch, press Ctrl+C.

24 Analyze server performance Recipe 1 - Identifying CPU-Intensive Processes Problem: Identify database sessions consuming the most CPU on the server [if running a SQL query, then display related SQL also] Solution A: The ps command identifies session process IDs consuming most CPU. The command below displays the top 10 CPU-consuming statements and the associated process ids - ps -e -o pcpu,pid,user,tty,args | grep -i informix | sort -n -k 1 -r | head -10 Solution B: Use top command – with batch mode [-b option] to run until ending (with a Ctrl+C) or a specified # of iterations. Note: These may be tuned/terminated as needed

25 Analyze server performance Recipe 2 – Analyzing Historical CPU Load Problem: View CPU load over past several days Solution: Use sar with -u option to report on CPU statistics – by default, it will report on current day activities. The -f option reports on the previous day and files that sar uses to report on statistics for different days of the month are located in the /var/log/sa directory where ## is two-digit day of month. For example, sar -u -f /var/log/sar10 [here ## is 10th day of month] Note: Since reports are on events in past, determine what system activity was taking place that caused any unusual spikes.

26 Analyze server performance Recipe 3 - Identifying Memory-Intensive Processes Problem: Identify database sessions consuming the most memory [if running a SQL query, then display related SQL also] Solution A: The ps command identifies session process IDs consuming most memory. The command below displays the top 10 memory-consuming statements & the associated process ids - ps -e -o pmem,pid,user,tty,args | grep -i informix | sort -n -k 1 -r | head -10 Solution B: Use top command – with batch mode [-b option] to run until ending (with a Ctrl+C) or a specified # of iterations. Note: These may be tuned/terminated as needed

27 Analyze server performance Recipe 4 - Identifying I/O-Intensive Processes Problem: Identify database sessions consuming the most I/O [if running a SQL query, then display related SQL also] Solution: The iostat & top commands identify devices with disk bottlenecks. For example, iostat -xd When trying to determine whether device I/O is a bottleneck, the general guidelines below will aid in examining iostat output: o Devices with abnormally high blocks read/written per second o If any device is near 100% utilization, thats a strong indicator of an I/O bottleneck Note: These may be tuned/terminated as needed

28 Analyze server performance Recipe 5 – Analyzing Historical I/O Load Problem: View I/O load over past several days Solution: Use sar with -d option to report on I/O statistics – by default, it will report on current day activities. The -f option reports on the previous day and files that sar uses to report on statistics for different days of the month are located in the /var/log/sa directory where ## is two-digit day of month. For example, sar -d -f /var/log/sar10 [here ## is 10th day of month] Note: Since reports are on events in past, determine what system activity was taking place that caused any unusual spikes.

29 Analyze server performance Recipe 6 - Identifying Network-Intensive Processes Problem: Identify database sessions with the most network traffic [if running a SQL query, then display related SQL also] Solution: The netstat & top commands provide reports on network traffic. For example, netstat -ptc will display the process id & tcp connections. Note: These may be tuned/terminated as needed

30 Handle server log files o Use the logwatch utility to monitor server log files logwatch --print Parameter file - /etc/logd/logwatch.conf o Use the logrotate utility to perform tasks such as renaming, compressing & removing old logfiles logrotate -f -s -f – force logrotate to do a rotation

31 Miscellaneous tips, tricks & treats DBA's often tar & untar binaries from one server to another The command below assists one in not incurring double storage with storing a local & remote copy of tar archive - tar cvf - { } | ssh { } "cd ; tar xvf - " The tar command piped to ssh ensures that symbolic links get copied over as symbolic links Scheduling: o use crontab command for jobs which need to be run often o use at command to run a job once at a specified time in future

32 Miscellaneous tips, tricks & treats Temporarily adding swap space to a server [done by root user]: If short on swap space, one can temporarily add a swap file on the server [commands will add around 1GB of swap space] - # dd if=/dev/zero of=tempswap bs=1k count= # chmod 600 tempswap; mkswap tempswap; swapon tempswap Verify that the swap space was added - swapon -s To remove this temporary swap file - swapoff tempswap; rm tempswap After disabling the swap file, swap space in /proc/meminfo should return to its original value.

33 Miscellaneous tips, tricks & treats o Menu-driven setup utility on SuSE – yast [Yet Another Simple Tool] & yast2 o Nmon – Swiss army-knife tool for monitoring performance o Data Studio – o Determining ip-address & domain for a servername: hostname -i – ip-address hostname -d – domain [on other OS like AIX, a separate command like "nslookup " determines this information]

34 Miscellaneous tips, tricks & treats o Removing in-memory structures When a database crashes unexpectedly, it may happen that for some reason semaphores and shared memory don't get released. If other databases are running on the server, one cannot reboot the system to release the shared memory objects. One can manually remove these orphaned memory structures. ipcs -sm - - view a list of structures to be removed; choose with care ipcrm -m ipcrm -q ipcrm -s

35 Miscellaneous tips, tricks & treats o Linux user has default shell of bash – if korn shell is preferred, then chsh /bin/ksh o man command provides online help - " --help" o du -sk *. | sort -nr | head -10 provides a report of the top 5 sub- directories consuming space in a particular directory with a comparison to the total $ alias tn [acronym for top #] = du -sk *. | sort -nr | head -10 or function tnf [short for top # function] { du -sk *. |sort -nr | head -10 } $ tnf 10

36 Linux meets INFORMation on unIX's needs!

37 Questions ?!?

38


Download ppt "Abstract Linux continues to gain momentum as a cost-effective operating system [OS]. Many companies are obtaining significant ROI & savings by migrating."

Similar presentations


Ads by Google