Day 1 Introduction to Unix

What is Unix? Unix is a computer operating system What does an O/S do? –Manage hardware and software resources, e.g. processor, memory, disk space. –Provide a consistent way for applications and users to interact with hardware without having to know all its details.

Why does a computer need an O/S? Not all computers have an O/S. E.g., a microwave computer does not have an O/S. Microwave computer: –Has one set of simple tasks to perform, –Very simple input and output (a keypad and an LCD screen), –Simple, never-changing hardware to control.

Brief History of Unix Developed at Bell Labs in 1969 by Ken Thompson, Dennis Ritchie and others. First developed for a small computer with limited resources (small memory and disk). - Thus, Unix programs had to be small. Many small programs, each did one small thing. But, programs can be put together in various combinations to get desired functionality.

Comparisons Unix vs. Linux –Linux is a Unix-like O/S. –“The goal of Linux was to develop a free Unix system that could be run on both a personal level and on large network servers.” –Most popular vendor of Linux is RedHat. –Many kinds of Unix, e.g. Solaris from Sun, HP-UX from Hewlett-Packard, AIX from IBM

Comparisons (cont.) Unix vs. Windows (Traditionally) –Windows designed by Microsoft to be “user-friendly.” –Programs are easy to learn and use; each program (e.g. word processor) combines many tasks. –Unix designed for researchers and developers. Programs can be easily combined to perform a multitude of various tasks.

Files and directories All information on Unix is stored in files that are organized into directories. A file is simply data associated with a name (the filename): –e.g., text file (.txt extension) contains plain text, Directories provide a means of organizing files into meaningful groups: –When you log in, you are in your home directory, –Can create a directory for your term papers, a directory for your digital images, a directory for your web pages, etc.

Command Shell The shell is a “command interpreter”. - It allows a user to interact with Unix O/S. User enters a command using the shell, O/S executes the command, shell shows output. - Shell Prompt: where you type-in a command. - term: window where prompt appears. Various kinds of shells, e.g., csh, tcsh, bash.

Command A command consists of a command-name, followed by 0 or more options (flags), followed by 0 or more operands, e.g., % ls -l *.txt. Command-name is a name of a Unix program, e.g., ls. Options/Flags modify the way the command works, e.g., -l. Operands are names of a file the command will operate on, e.g., *.txt.

Listing files in a directory (ls) % ls –Lists files in a directory (except “dotfiles”). % ls -a –Lists all files in the directory, including “dotfiles”. % ls -l – shows more information about files (e.g. owner, file size, file modify-time). % ls -F – shows type of files (such as whether it is a regular file or directory). Can combine flags, e.g., % ls -alF

Working with Directories pwd –print working directory –shows where you are; your current directory. mkdir directory –Create a new directory, e.g., –% mkdir foome cd directory –Change working directory, e.g., –% cd foome % pwd % mkdir fooyou % cd fooyou % pwd

Directory Specifications The current working directory, “.”. The parent directory, “..”. Home directory, “~”. % cd ~ % cd foome/fooyou % pwd % cd..

Relative vs. Full pathnames Full name implies a full path to the file/directory on the file system. Relative name implies name of file/directory relative to current directory. % cd /tmp/foome/fooyou/ or % cd /tmp % cd foome % cd fooyou

Editing files: Emacs text editor emacs filename & –Start emacs editor, –“&” runs emacs in the background (we discuss this later). –% cd ~/foome % emacs paper.txt & Used to create, view, and edit files. If no file exists, a file is created with the name entered Important Command: C-x C-x –Save changes

Displaying files (cat, more) cat file –displays contents of a file on the screen, e.g., –% cat ~/foome/paper.txt more file –Displays contents of a file on the screen once screen at a time, e.g., –% more ~/foome/paper.txt –Use spacebar to step forward through screens. –Use b to step back through screens.

Copying and Moving files cp sourceFile copyFile –% cd ~ % cd foome % cp paper.txt bar.txt % ls mv fromFile toFile –% mv bar.txt bar2.txt –% ls

Removing files or directories rm file –% cd ~ % cd foome % rm bar.txt rmdir directory –% rmdir fooyou

Documentation There are hundreds of Unix commands, and most commands have several options. Be able to find and use commands out as you need them rather than memorizing all of them (learn how to learn). - Online resources are readily available. - Unix manual (man) pages is an excellent reference manual. - man command - Used to search man-pages.

Example: % man ls % man ls Describes what ls does and the options it takes, Describes each of the options, Gives examples of usages of ls, Also gives other commands similar to ls, e.g., (chmod, cp, setfacl, etc. …). Best way to learn is by doing and exploring!

whatis command –Summarizes what command does, –It actually gives the NAME field of the relevant man page. Example: –% whatis ls –% whatis cd –% whatis whatis

Having fun with commands % date % time % cal % echo % wc % history % man man % whoami % touch

Day 2 Combining commands Environment variables Customizing Unix

Combining commands - Command Redirection Heart of UNIX is being able to combine commands to create new commands. Before, input to command was specified with the operand, and the output is printed to the screen. Pipe ( | ) operation redirects the output of one command to the input of another command.

Examples % ls –alF | more –Shows output of ls a page at a time % ls –alF | sort –r | more –Shows output of ls, in reverse alphabetical order, a page at a time % ls | wc –l –Gives number of files in the current directory % ls –s | sort –nr –Lists files by the largest first % ls –s | sort –nr | head -5 –List top five largest files

Examples contd. Create the file, stuff.txt, and add the lines One Two One Two Three One Try –% cat stuff.txt –% cat stuff.txt | uniq –% cat stuff.txt | uniq | wc -l

File redirection > –Write output of command to a file, –E.g., ls > myListFile.txt < –Read input from a file instead of an operand, –E.g., sort < stuff.txt >> –Append output of command to a file, –E.g., echo Four >> stuff.txt << marker –Allows you to input a number of lines to a command until you reach marker

More Examples % cat > numbers.txt ^D % cat >> numbers.txt 4 5 ^D % cat numbers.txt > numbers2.txt % cat numbers.txt >> numbers2.txt

More examples % ls –s | sort –nr > moreStuff.txt % cat moreStuff2.txt % history | sort > mycommands.txt % cat mycommand.txt | wc –c % cat mycommand.txt | sort | uniq % history | sort | uniq | wc –l > mc2.txt % echo simple test > mycommand.txt % echo simple test >> mycommand.txt

Wildcard (*) “*” matches all files except those whose names begin with. (period), or any number of characters within a filename. Examples: –*.f Matches all filenames with a.f extension e.g. h.f and verylongfilename.f –a* Matches all filenames that begin with “a”, e.g., a and anotherfilename.c

Wildcard(?) “?” matches a single character. Examples: –?.f Matches all one-character filenames with a “.f” e.g., a.f would match, but ab.f would not match. –?his?ile.txt Matches both ThisFile.txt and thisfile.txt, but does not match ThisWillNotMatch.txt or hisFile.txt. A “?” requires that a character be present in the space reserved by the “?”. A * does not have this restriction.

Examples % touch a.txt % touch b.txt % touch c.txt % touch k.txt % touch owl.a % touch owl.d % touch owl.abc % ls *.txt % ls -l a* b* k* % ls owl.?

grep grep pattern filenames –Seaches filenames for pattern –grep, without flags, finds patterns in a case- sensitive manner Most helpful grep flags – -c List a count of matching lines only – -i Ignore the case of letters in the pattern – -l List only the names of files that contain pattern – -n Include line numbers

Examples % cat > grepExample.txt alpha beta gamma delta ^D % grep beta grepExample.txt % grep Beta grepExample.txt % grep –i Beta grepExample.txt % grep –i –n Beta grepExample.txt % grep –l beta grepExample.txt

More Examples % grep ‘b’ grepExample.txt % grep ‘e’ grepExample.txt | sort % grep –l ‘e’ *.txt | sort % grep –n ‘e’ *.txt | sort > grepResult.txt

find find path pattern –Finds files containing pattern in the directory hierarchy beginning with path. –Like grep –l pattern, but more powerful –In general, find is used to search for files, and grep is used to search within files. Most helpful find flags – -atime n True if file was accessed n days ago (use –n, +n, and n to indicate n days or less, n days or more, or exactly n days) – -type t True if file is of type t (t=d  directory, t=f  file) – -name pattern True if filename that matches pattern – -print Print names of the files that matched

Examples % find. -print –List of all files and directories in and below the current directory % find. -name “*.txt” –print –List all.txt files in and below the current directory % find. –atime -7 –name “*.txt” –print –List all.txt files in and below the current directory that have been accessed in the last 7 days. % find ~ –type d –print | more –List of directories in the home directory

alias name definition –Creates a shortcut (name) for a command or series of commands (definition) –E.g., % alias ls ‘ls –alF’ % ls alias name –Gives the definition for the shortcut, if there is one. –E.g., % alias ls % alias mv unalias name –Removes an alias –E.g., % unalias ls % ls Customizing your Unix account: alias

set prompt = ‘something’ –sets your prompt to something E.g., set prompt = ‘my new prompt> ’ Customizing your prompt

Useful codes you can use in your prompt –%c Current working directory –%m Machine name –%t Current time –%d Current day –\n Prints a new line E.g., set prompt = ‘\n[%d] on [%m::%c] at [%t]\n->’ Customizing your prompt

Customizing your Unix account: Dotfiles Files that begin with “.” (period) Use ls -a to list them Special files –The current working directory, “.” –The parent directory, “..” Other dotfiles describe your session initialization procedures and store user customizations.

The.cshrc file File does two main things –Environment Setup Sets up UNIX environment variables, command search path, etc. –Shell Setup Sets up aliases, command prompt, etc. The file is executed when the user logs in (global effect), or starts an xterm (local effect), or performs the command source ~/.cshrc (local effect)

Creating a file of personal customizations Add the following line to ~/.cshrc, so that your customizations are setup when you log in: source ~/.myAliases Add your customizations to ~/.myAliases. If you make changes to ~/.myAliases, to get the changes to take effect you can either log out and log back in, or just do source ~/.myAliases in an xterm window.

Useful Customizations for ~/.myAliases alias rm ‘rm –i’ alias mv ‘mv –i’ alias cp ‘cp –i’ (prevents accidentally removing a file, moving a file onto another file, or copying a file onto another file). set prompt = ‘\n[%m::%c] at [%t]\n->’ Remember, you need to: source ~/.myAliases for the changes to take effect (or log out and log back in again).

Unix environment The working environment is a list of variables that are accessible to Unix programs. The O/S sets up the environment, with the aid of the “dotfiles”, when a user logs in. Programs can reference variables in the environment to help them function. % env or % printenv –Prints out a user’s environment.

Descriptions of some environment variables HOME: user’s home directory USER: user’s username PATH: defines which directories for Unix to search to find a command the user has specified at the prompt SHELL: gives the type of shell (command interpreter) To print a particular environment variable, you can also printenv –E.g., printenv HOME

Day 3 Ownership and Permissions Job Control

Why does UNIX need file permissions? UNIX is a multi-user operating system, thus, it allows many different users to take advantage of the computer’s resources simultaneously. Question: Because UNIX was designed to support many users, how to determine who sees what file? Answer: use file permissions.

Users and Groups Unix uses a three-part system to determine file access: there’s what the file owner can do, what a group is allowed to do, and what everyone else is allowed to do. Every user on the system has a username. Unix also supports creating groups of users. A user can belong to any number of groups.

ls -a ls -a returns, for each file: –A ten-character-long string of letters and hyphens (permissions), –The number of hard links to the directory (do not worry about this for now), –The username of the file owner, –The name of the group of users that can access the file, –The size of the file, in bytes, –Date and time of the last modification for the file, –And finally, the filename.

Reading Permissions First letter (d or -): –Indicates this is a directory Next three letters: –Indicates what the user who owns the file can do –Read: ability to see the contents of the file –Write: ability to change the contents of the file or delete the file –Execute: indicates whether the file is also a command that can be run on the system (later, we will show how all commands in Unix are stored in files that can be created, modified, and deleted like any other file) Second set of three letters: –Indicate permissions for group Third set of three letters: –Indicate permissions for everyone else Directories treat the execute permission differently. If a directory does not have execute permissions, that user (or group, or other users) cannot cd into the directory.

Changing Permissions (chmod) chmod [ugoa][+-][rwx] filename/directory –Change permissions –Useful flag: -R - changes permissions on directories recursively Example: –% mkdir ~/songs % chmod go-rwx ~/songs The ~/songs directory will be accessible by only the owner Example: –% touch publicReadOnlyFile.txt % chmod go-wx publicReadOnlyFile.txt % chmod a+r publicReadOnlyFile.txt

Brief Introduction to Groups groups username –Groups to which the username belongs to By default, when you create a file, the file’s group is the first group that is listed.

Program vs. Process Program is a file that can be executed. Process is a program running on the computer: –That is, each time you run a program, a new process is created. –For each program, you can have one or more processes. –A process is initiated by the machine.

Looking at Processes (top) top –Shows the processes currently running on the machine. An interactive program does not simply produce output and exit, but produces output at regular intervals until your press q or C-c.

top (cont.) First line shows the current time, the amount of time since the computer was started or rebooted, the number of users logged in, and 3 load averages. The load averages show how much work the computer has been doing for the last minute, 5 minutes and 15 minutes.

top (cont.) Second lines shows the total number of processes, the processes that are waiting to run, the processes that are currently running, zombie processes, and processes that have been stopped by a user. Zombie process: Programmers sometimes make mistakes, and sometimes a process exits (“dies”), but is not fully removed from the system. Zombies are rare, and rebooting the machine gets rid of them.

top (cont.) Third and fourth lines show the memory usage of the system, i.e., the amount of memory available and the amount being used. The third line represents physical memory and the fourth line is for virtual memory (we don’t go into memory usage in this course).

top (cont.) List of running processes Top displays only those processes using the greatest % of the CPU. Each line shows –process’s ID (PID) –process’s owner (USER) –% of CPU being used by the process (%CPU) –% of physical memory being used by the process (%MEM) –command being run (COMMAND) If your computer is running slowly, %CPU and %MEM are good indicators of which processes of taking up the most computation.

Jobs A job is a process that you (the user) have started, as opposed to a process the machine has started. jobs –Shows the jobs that you have started in that particular xterm. You can –move jobs to the background (bg) –move jobs to the foreground (fg) –kill background jobs (kill) –kill foreground jobs (C-c) –suspend foreground jobs (C-z) –unsuspend jobs (bg, fg)

Foreground vs. Background Foreground process is a process currently accepting input from the xterm. Background process is a process currently running but is not accepting input from the xterm (but may accept input through other means). Suspended process is one that is not running (and thus, cannot accept input from any means), but can be restarted. Killed process a process that is not running and cannot be restarted.

Example % xterm & % emacs % C-z % jobs % fg %1 % C-z % bg %1 % jobs % kill %1 % emacs & % jobs

Example % xterm & % emacs & % xterm % C-z % jobs % fg %2 % C-z % bg %2 % fg %1 % C-z % jobs % kill %1 % jobs % kill %2 % jobs

Day 4 File System Organization Creating our own commands

Unix File Organization Unix is organized in directories and files. Commands are actually executable files. Organization is hierarchical, that is, the entire file system is a rooted tree, with the root being / (the root directory) % cd / % ls

Unix File Organization: Main root directories /bin: most frequently used standard utilities. /usr: system files, and the games directory. /usr/bin: less frequently used standard utilities, possibly some non-standard ones. /dev: files associated with devices - each terminal, printer, disk, and the main memory has a filename here. /etc: miscellaneous system files, including /etc/passwd which contains users’ login information. /tmp: holds temporary files generated by many programs. /home: user home directories.

Unix File Organization (cont.) Some Other root directories (in case you were curious) /lib: library files which programs can dynamically load when they are running. /lost+found: files which the O/S has created when recovering from a failure, and for which the O/S cannot determine the proper location in the file system (this directory is usually empty). /mnt: files used to mount external media (e.g. hard disks). /sys: files used in system configuration.

where command –Find occurrences of command in the user’s path Recall the user’s path (% printenv PATH) defines which directories for Unix to search to find a command the user has specified at the prompt Example: –% where ls –% where cd –% where man Can actually specify a UNIX command using its full path, e.g., % /bin/ls. - However, because /bin is in our PATH environment variable, we only need to specify the name, ls.

System Admin (root) On Unix, there is a special account, the root account. If you are able to log in as root (have root access), you have access to any file and can perform any command on the machine. Person with root access is usually the person responsible for maintaining the machine (the machine’s system administrator). Most of directories in the / directory are owned by root and modifiable only by root.

What we will be doing for the rest of the lecture We will write our own commands and store them as files in our own home directory. We will modify our PATH environment variable to reference our new command, so that we do not have to specify the path to our command, thus, we can call them by name. This is very similar to how regular Unix commands, e.g., ls, cd, more, work.

Introduction to UNIX shell scripts A shell script allows you to create a new Unix command out of regular shell commands. Why not just use the alias command? Shell scripts are usually for more complex combinations of Unix commands and can be made to be more powerful.

Example % mkdir ~/bin % cd ~/bin % emacs myPrintNothing & Add the following lines #!/bin/sh cd ~; pwd; SOMETHING=nothing; echo $SOMETHING; pwd; echo $SOMETHING; Then do % chmod u+x./myPrintNothing Then do %./myPrintNothing or % ~/bin/myPrintNothing

Example (cont.) In the previous slide, we had to specify the path to our new command. But, suppose we just want to execute our new command without specifying the path? For Unix to find the command, it has to be in our PATH environment variable. To see our current PATH: % printenv PATH The O/S looks for the command in the order of the directories specified in the PATH variable. As a note, the #!/bin/sh line tells the O/S that the file is a shell script.

Editing the PATH variable % emacs ~/.myAliases Add the line: set path = ( $path $home/bin ) $path references the original path $home references your home directory % cd ~ % source ~/.myAliases % myPrintNothing

Another Example % cd ~/bin % emacs myAdd & add the following lines: #!/bin/sh SUM=0 while read THINGY do SUM=‘expr $SUM + $THINGY’ done echo $SUM % chmod u+x./myAdd The program adds a list of integers fed to it and prints their sum

Another example (cont.) Example of how to run the script on the previous slide. % ~ % myAdd Ctrl-D 26

What is the point? Unix commands are files themselves. They are files which can be executed. You can write your own commands, and place them in the ~/bin/ directory. We just used simple shell scripts for our commands, but commands can be written in other languages, including Java, C, Perl, etc. Regular Unix commands (e.g. ls, cd, more) are files owned by root. They are usually written in C, compiled to create a binary, and the binary is put in /bin or /usr/bin, which are directories in a user’s path.

Day 5 Apache Server

On the World Wide Web Microsoft Internet Explorer and Netscape browsers: –Are Web (HTTP) clients, –Allow a user to request a Web page, and display the requested page. Apache Server (“a patchy Web server”): –Is a Web (HTTP) server, –Freely available (freeware) and open source, –Waits in a loop for requests. When it gets a client request, it replies with the appropriate Web page.

Installing the Apache web server The Apache web server is freely available from Download the Unix source, (e.g., httpd tar.gz). Make the directory ~/myserver, and copy the tar.gz file into it. % tar xvfz http tar.gz tar –Used to encapsulate several files into one.tar file, or un-encapsulate a.tar file into its original files.

% cd httpd The file INSTALL contains the instructions on how to install and run Apache. %./configure –prefix=PREFIX (For the PREFIX, you should specify the full path to your myserver directory.) % make % make install Do not run the server yet (i.e., do you run % PREFIX/bin/apachectl start yet). Installing the Apache web server

The server’s main directories htdocs: contains web pages bin: contains apache’s commands including apachectl conf: contains Apache’s configuration files log: contains Apache’s log files manual: contains manual web pages for Apache (also at

Apache’s main configuration file Apache’s main configurations are contained in PREFIX/conf/httpd.conf When Apache starts, it binds and listens for client requests on a particular “port” on the computer. The default port it listens on is port 80. We need to change this port, because a program must be run as root to be able to port 80. Only root can bind to ports below We need to thus change Apache’s port to greater than 1024.

% emacs httpd.conf Look for the line “Listen 80” (line 218). This line specifies Apache’s port Change it to be “Listen 8080”. Apache’s main configuration file contd.

apachectl A script that can be used to start and stop the web server. To start the server: % PREFIX/bin/apachectl start To stop the server: % PREFIX/bin/apachectl stop Documentation is available at: 2.0/programs/apachectl.html You should now start the server. If you put the PREFIX/bin directory in your PATH environment variable, then you do not have to specify the path to the apachectl command.

Viewing web pages on your server Start Netscape. To test your webserver, go to the webpage :8080/ :8080/ The webpage you are seeing is located at PREFIX/htdocs/index.html.en You can also create your own webpages in the htdocs directory.

Apache’s log files PREFIX/logs/access_log shows the IP addresses of clients who have been accessing your web server, and which web pages they were looking at. PREFIX/logs/error_log logs every time your start or stop the server; also logs errors if the server does not work properly so that you can determine what went wrong.