1. ITEC400 Week Five Professor Robert D’Andrea

2. Agenda
Administrative Issues This Week’s Expected Outcomes Review for midterm exam Moving around in UNIX Break-Out Problems Upcoming Deadlines Hands-on Information Lab Assistance, Questions, and Answers

3. Administrative Announcements
The midterm exam will be administered the seventh week of the term from June 19 through 24, Exam: True/False 60 questions (1 point) Multiple Choice 40 questions (1 point) Essay 5 questions (10 points) Midterm points: 150

4. This Week’s Expected Outcome
Upon successful completion of this module, the student will be able to:
Create makefile scripts for software programs.
Interfaces
Use pattern rules in make files.
Create an effective PowerPoint presentation.
Create make files with multiple targets.
Install software packages on a server.

5. Unix Operating System
Free Perl and CGI scripts Perl is dual licensed under both the Artistic License and the GNU General Public Licenses. Distributions are available for most operating systems.

6. Unix Operating System
Why do we love Perl? The Perl Programming Language currently has 21,000 Comprehensive Perl Archive (CPAN) modules. The CPAN has over 18,000 open source modules available for download. Perl is a highly capable, feature-rich programming language with over 30 years of development. Parts of the Perl language are still actively being pursued. Perl has over 300 local community groups. Perl 6 is a sister language, part of the Perl family.

7. UNIX Operating System
SCSI interface SCSI (pronounced SKUH-zee and sometimes colloquially known as "scuzzy"), the Small Computer System Interface, is a set of ANSI standard electronic interfaces that allow personal computers to communicate with peripheral hardware such as disk drives, tape drives, CD-ROM drives, printers, and scanners faster and more flexibly than previous interfaces. Developed at Apple Computer and still used in the Macintosh, the present set of SCSIs are parallel interfaces. SCSI ports continue to be built into many personal computers today and are supported by all major operating systems.

8. UNIX Operating System Linux device naming
This section provides information to help you understand naming patterns for devices on Linux.
Special device files are used by kernel drivers to control a device. There can be more than one special device file that maps to the same physical device. For example, in a multipath environment with four paths to the same device, four different device files will map to the same physical device.
The device files are located in the /dev directory and are addressed by a major and minor number pair. Fiber- channel attached devices are managed as SCSI disk devices by the sd driver. Thus, each of the attached storage unit LUNs has a special device file in the Linux directory /dev.

9. UNIX Operating System
SCSI disk devices have special device files that start with an ‘sd’ prefix, and have the following naming format: /dev/sd[a-z][a-z][1-15] Names without a trailing digit refer to the whole disk, while names with a trailing digit refer to a partition of that whole disk. By convention, SCSI disks have a maximum of 16 minor numbers mapped to a single disk. Thus, for each whole disk, there is a maximum of 15 partitions per disk because one minor number is used to describe the entire disk (for example /dev/sda), and the other 15 minor numbers are used to refer to partitions for that disk (for example /dev/sda1, /dev/sda2, etc).

10. UNIX Operating System
ls -l /dev/sda* brw-rw root disk 8, 0 May 24 08:09 /dev/sda brw-rw root disk 8, 1 May 24 08:09 /dev/sda1 brw-rw root disk 8, 10 May 24 08:09 /dev/sda10 brw-rw root disk 8, 11 May 24 08:09 /dev/sda11 brw-rw root disk 8, 12 May 24 08:09 /dev/sda12 brw-rw root disk 8, 13 May 24 08:09 /dev/sda13 brw-rw root disk 8, 14 May 24 08:09 /dev/sda14 brw-rw root disk 8, 15 May 24 08:09 /dev/sda15 brw-rw root disk 8, 2 May 24 08:09 /dev/sda2

11. UNIX Operating System
ls -l /dev brw-rw root disk 8, 0 Jan 28 00:33 sda brw-rw root disk 8, 1 Jan 28 00:33 sda1 brw-rw root disk 8, 2 Jan 28 00:33 sda2 crw-rw-rw- 1 root tty 5, 0 Feb 4 09:59 tty crw--w root tty 4, 0 Jan 28 00:33 tty0 crw root root 4, 1 Jan 28 00:35 tty1 crw--w root tty 4, 10 Jan 28 00:33 tty10 crw--w root tty 4, 11 Jan 28 00:33 tty11 crw--w root tty 4, 12 Jan 28 00:33 tty12 crw--w root tty 4, 13 Jan 28 00:33 tty13

12. UNIX Operating System
Major and minor numbers When accessing a device file, the major number selects which device driver is being called to perform the input/output operation. The minor number is used by the kernel to determine exactly which device is being referred to. Depending on how your driver is written, you can either get a direct pointer to your device from the kernel, or you can use the minor number yourself as an index into a local array of devices. Either way, the kernel itself knows almost nothing about minor numbers beyond the fact that they refer to devices implemented by your driver.

13. UNIX Operating System
Major and minor numbers $ ls -l /dev/hda brw-rw root disk 8, 0 mars 9 07:56 /dev/sda Major device number Minor device number

14. UNIX Operating System
TTY and PTS TTY originally meant, teletypewriter, but now it means terminal on Unix-like systems. It also means any serial port on a UNIX/Linux system. PTS means, pseudo terminal slave. The difference between the tty and pts is the type of connection to the computer. TTY ports are direct connections to the computer like the keyboard/mouse or a serial connection to the device. PTS connections are SSH or telnet type connections. All of these connections can be connected to a shell program which will allow you to issue commands to the computer.

15. UNIX Operating System
TTY and PTS drwxr-xr-x 2 root root 0 Jan pts crw-rw-rw- 1 root tty 5, 0 Jan 22 10:14 tty

16. UNIX Operating System
Creating partitions One big problem arises when splitting off lots of separate partitions for new administrators, is settling on the appropriate partition size. The appropriate partition size of various partitions can vary considerably from system to system. A workstation is likely to need a small /var partition (100MB), but a mail or news server may need a /var partition that is gigabytes is size. Guessing the wrong size isn’t fatal, but is annoying. It is time consuming to resize your partitions so that sub- directories on one partition can be stored on other partitions.

17. UNIX Operating System Partition recommendations for new Linux admins
Root (/) required, and swap is a good idea. Set root some where between 5-25GB
/boot can be helpful on HDD with 8GB
As you gain more experience, you may break off other directories into their own partitions on subsequent installations or when upgrading disk hardware.
Keep the partitions simple!
Linux traditionally uses a tool for disk partitioning called fdisk (fixed disk).The fdisk command name is followed by the name of the disk device you want to partition, fdisk /dev/hda to partition the primary master PATA disk.

18. UNIX Operating System
PATA Parallel ATA is an Integrated Development Environment (IDE) standard for connecting storage devices like hard drives and optical devices to the motherboard. PATA generally refers to the types of cables and connections that follow this standard. An IDE interface is an interface for mass storage devices, in which the controller is integrated into the disk or CD-ROM drive.

19. UNIX Operating System

20. UNIX Operating System

21. UNIX Operating System
Preparing a partition for use After you have created a partition, you must prepare it for use. Preparing it for use is called, “making a file system” or “formatting a partition.” This involves writing low level data structures to disk. Linux can read and modify these data structures to access and store files in the newly created partition.

22. UNIX Operating System
Preparing a partition for use Low-level formatting is the process of marking out cylinders and tracks for a blank hard disk, and then dividing tracks into multiple sectors. This process is often called the “real” formatting since it creates physical format which defines where the data is saved. If users perform low-level formatting when data have been installed, all existing files will be erased, and it is almost impossible to recover them. Therefore, some users make such a format to avoid privacy leakage. Nevertheless, performing low-level formatting will bring great influence on hard disk, thus shortening hard disk service time. Therefore, it is not suggested.

23. UNIX Operating System
Preparing a partition for use High level formatting creates a file system. Hard Disk Drive (HDD) are low-level formatted at the factory and should not need to be low-level formatted.

24. UNIX Operating System
Preparing a partition for use After a low-level formatting has been completed, users need to make high-level formatting which makes it possible to save data and should be done on a partition. It is the process of writing a file system, cluster size, partition label, and so on for a newly created partition or volume. And we can also say high- level formatting just clears data on hard disk, generates boot information, initializes FAT, and labels logical bad sectors when the partition has existed. This process does no harm to hard disk in general situations, so we suggest taking such a format to fix a logically damaged partition or device, for example, Windows asks to format a SD card.

25. UNIX Operating System

26. UNIX Operating System

27. UNIX Operating System
Common file system types Linux supports both native file systems and those intended for other operating systems. Ext2fs (The Second Extended File System) Good for small /boot partition (2TB) Non-journaling Reliable file system throughout 1990s. Ext3fs (The Third Extended File System) Journaling Better reliability, extension of ext2fs. Works well with very large disks (2TB)

28. UNIX Operating System Journaling file system
A journaling file system keeps track of the changes that will be made in a journal (usually a circular log in a dedicated area of the file system) before committing them to the main file system. In the event of a system crash or power failure, such file systems are quicker to bring back online and less likely to become corrupted.
Updating file systems to reflect changes to files and directories usually requires many separate write operations. This makes it possible for an interruption (like a power failure or system crash) between writes to leave data structures in an invalid intermediate state.

29. UNIX Operating System Journaling file system
Depending on the actual implementation, a journaling file system may only keep track of stored metadata, resulting in improved performance at the expense of increased possibility for data corruption. Alternatively, a journaling file system may track both stored data and related metadata, while some implementations allow selectable behavior in this regard.

30. UNIX Operating System
Common file system types Ext4fs (The Fourth Extended File System) Works well with very large disks (over 16TB) Journaling Improved performance ReiserFS Better reliability Works well handling large numbers of small files.

31. UNIX Operating System
Installation Software Package Methods Package manager - What are the tasks of a package manager? - The version of UNIX dictates which package manager will be used. - Many Linux versions utilize the Debian Package Manager. Manual - The utilization of makefiles or shell scripts to automate the install as much as possible. - On occasion, the installation of software is done by sending the source code to the site. - What steps are required to install a new software version? - How is a “configure” script utilized?

32. UNIX Operating System
Installation Software Package Methods Package manager - Linux uses yum and rpm as there package manager. - Software has to be set or identified to use rpm. Otherwise, the software is configured manually.

33. UNIX Operating System
Test_build.sh script The test_build.sh script is used to execute the make command which in turn will call the makefile. test_build.sh -> make -> makefile

34. UNIX Operating System

35. UNIX Operating System
Make command
The make utility is a software engineering tool for managing and maintaining computer programs. Make provides most help when the program consists of many component files. As the number of files increase in a project, so to does the compile time, and complexity of compilation. The likelihood of an error occurring with human(s) manageability is high.

36. UNIX Operating System Makefile
Make goes through the makefile (descriptor) file first starting with the target it is going to create. Make looks at each of the target’s dependencies to see if they are listed as targets. It follows the chain of dependencies until it reaches the end of the chain and then begins backing out and executing the commands found in each target's rule. Actually every file in the chain may not need to be compiled. Make looks at the time stamp for each file in the chain and compiles from the point that is required to bring every file in the chain up to date. If any file is missing it is updated if possible.

37. UNIX Operating System
Makefile components Comments within a makefile (#) makefile continuation lines (\) The rule tells make both what and how to make a file Dependency Line is a line with a colon (:) test_build.sh contains the make command which calls the makefile. The test_build.sh script is located in the /tmp directory.

38. UNIX Operating System
Key Points about the makefile
Makefile recompiles all source files for an entire project.
Makefile recompiles only those files that have different time stamps.
Bottom up control flow.
Rules are shell commands.

39. UNIX Operating System
Key Points about the makefile The file names used in this example have been abbreviated for simplicity. C++ compilations end in .cpp suffix. The .c suffix extension causes the C compiler to be invoked. The .o suffix represents a object file. The .h suffix represents a header file. The g++ is a compiler, and not a preprocessor. The g++ builds object code from your C++ program source. The –c option, compiles source to object code. The –o option, compiles object code.

40. UNIX Operating System
Other software products that perform similar bundling operations
- ANT developed by Apache Jakarta
- ANT2 supplied from the Free Software Foundation which is part of Red Hat.
What is a dependency between source files?
main.cpp  employee.cpp  address.cpp
What are the two parts of a dependency rule?
1) What files a file is dependent on?
2) Rule that tells how to recompile the file
What are the differences between a makefile and a shell script?
The rules in a makefile are executed based on dependency, and not sequential order. Scripts execute in sequential order.

41. UNIX Operating System

42. UNIX Operating System
Hash initialization my %colors = (“black”, 0xff0000, “orange”, 0x00ff00, “blue”, 0x0000ff); my %colors; $colors{black] = 0xff0000; $colors{orange} = 0x00ff00; $colors{blue} = 0x0000ff; For better reading and understanding use the => operator.

43. UNIX Operating System
Hash initialization Comma => arrow The comma-arrow (=>) operator is used to initialize the % hash member variable. The left side of the comma-arrow operator is expected to be a simple string and therefore it is not necessary to be quoted.

44. UNIX Operating System
Hash initialization my %colors = ( black => 0xff0000, orange => 0x00ff00, blue => 0x0000ff, );

45. UNIX Operating System
Perl Hash table is an unordered set of scalars, accessed by some string value that is associated with each scalar. Keys are 12, 19, and 30. Values are gray, black, and red. %colormap=( “12” => “gray”, “19” => “black”, “30” => “red”,); %colormap = (“12”, “gray”, “19”, “black”, “30”, “red”); %members = (John => "father", Paul => "son", Marie => "daughter");

46. UNIX Operating System Keys & Values
%colormap = (“12”, “gray”, “19”, “black”, “30”, “red”);
@codes = keys %colormap; # codes array contains just codes
@colors = values %colormap; # colors array contains the colors
# Printing the values out of a hash table (hash_prt.pl)
while (($key, $value) = each(%colormap)) {
print $key, “, “, $value, “\n"; }
Executing the loop prints:
30, red
19, black
12, grey

47. UNIX Operating System

48. UNIX Operating System

49. UNIX Operating System

50. UNIX Operating System
Awk Script # Author: Bob D'Andrea # Script name: awksrc.sh # Command line awk -f awksrc.sh test.txt # Command line: awk -f awksrc.sh /^$/ { print "This is a blank line." } /[0-9]+/ { print "That is an integer." } /[A-Za-z]+/ { print "This is a string." }

51. UNIX Operating System
Without options, print the desired fields in any order. awk ‘{ print $1, $2, $3, $4, $5 }’ names The –F option changes the field separator on the command line. The \t is an Escape Sequence for a horizontal tab. awk –F”\t” ‘{ print $1 }’ names

52. UNIX Operating System names File Content
Bob Dee /19/1947 male
Bo Happy /01/ male
Jane Smith /23/ female
Razi Jake /26/ male
The file named “names” contains the above information. There are tabs in two places in the above data. One tab is after the name and after the birth date.

53. UNIX Operating System
Fork() and exec() The fork syscal is a technique used to “divide” the parent process into two identical parts. After the parent executes the fork() syscal, the created child process is an exact copy of the parent except for the return value. The two processes can continue to run in the same binary, but most generally, the child process switches to running another binary executable by issuing the exec() syscal.

54. UNIX Operating System
Fork() and exec() The new child of the parent process has a new PID. The fork() function returns the child’s PID to the parent, while it returns 0 to the child process, in order to allow the two identical processes to know one from the other.

55. UNIX Operating System
Fork() and exec() The parent can continue to execute or wait for the child process to complete. After the child discovers that it is a child, and it replaces itself completely with another program so that the code and address space of the original program are lost. The child calls exec(), all data in the original program is lost , and is replaced with a running copy of the new program.

56. UNIX Operating System
Fork() and exec() If the parent process decides to wait for the child process to die, then the parent will receive the exit code of the program that the child executed. To prevent a child from becoming a zombie, the parent will call the wait() function on its children either periodically or when receiving the SIGCHLD signal, which indicates a child process has terminated.

57. UNIX Operating System
Fork() and exec()

58. UNIX Operating System
Fork() and exec()

59. UNIX Operating System
PID (Process ID) When a parent process creates a child process, the child inherits specific attributes from the parent. The child inherits the parent’s terminal type, home directory, stdin, stdout, and stderr.

60. UNIX Operating System

61. UNIX Operating System
Column Description Output from ps –f command UID User ID that this process belongs to (the person running it). PID Process ID. PPID Parent process ID (the ID of the process that started it).

62. UNIX Operating System
Options which can be used with ps command -a Shows information about all users -x Shows information about processes without terminals. -u Shows additional information like -f option. -e Display extended information.

63. UNIX Operating System Signals
Signals are a form of process communication. Because they can come from another process, the kernel or the process itself, they might be better thought of as events that occur as a program runs. A crude example might be the bell most of us remember from our early days in school; when the bell rang, we reacted by switching from playful children to industrious students.

64. UNIX Operating System Signals
The kill –l command will display the signals available on your system. The kill –l <signal name> displays the signal code.
HUP (1) ALRM (14)
INT (2) TERM (15)
QUIT (3) CHLD (17)
KILL (9) STOP (19)
TRAP (5) CONT (18)
IOT (6) BUS (7)
FPE (8)

65. UNIX Operating System
Signals For each signal there is a default action, almost all of which terminate the process. For most signals, a program may specify another action—this is called catching or handling the signal—or may specify that no action occurs, which is called ignoring the signal. The signal SIGKILL cannot be caught or ignored; it always terminates a process.

66. UNIX Operating System
kill Command Used to send a signal to another process. The kill command is used on Linux and other UNIX-like operating systems to terminate processes without having to log out or reboot (i.e., restart) the computer. Format: kill [signal or option] PID(s) kill -l kill –l TERM kill –l KILL

67. UNIX Operating System
kill Command Signals

68. UNIX Operating System
Demonstrate test_build.sh pid_ppid.sh

69. Moving Around in UNIX
umask umask –S diff make (GNU make utility to maintain groups of programs ) fork() and exec() tty (name of terminal) w (who is logged in and what they are doing) touch (create a file and update timestamp)

70. Break-out problems
touch command What function does the makefile provide a project? What are the two parts of a dependency rule? What language is used for writing makefile rules? What two methods are used for installing software packages?

71. Hands-On-Information
Lab Assignment 5-1, Makefile_Scripting_1, due June 18, Lab Assignment 6-1, Prel_Scripting_3, due June 25, 2017.

72. After class assistance
Questions? Comments? Concerns? After each Franklin Live session, I will remain on the session to provide assistance unless otherwise indicated.

73. Lab Assistance available by phone and/or email