1. Shell Programming

2. The UNIX Shell  The UNIX shell listens to what you type and executes commands at your request. User command: lpr file UNIX Shell UNIX Kernel Command Library ls, lpr, mv, rm, telnet, netscape,... result or status Printers Files Memory results (on screen)

3. Popular Shells  sh Bourne shell (the original shell)  csh C-shell (pronounced as “sea shell”)  tcsh Like csh with more functions  bash “Bourne again” shell  ksh Korn shell  zsh Z-shell

4. More Utilities (2)  echo Display command line input to screen $ echo Hi, I am Bill, the President of the US! Hi, I am Bill, the President of the US!  date Print the date and time $ date Wed Feb 3 12:13:07 HKT 1999

5. More Utilities (3)  head Display first few lines of file $ head -2 letter3 Ms. Lewinski: It is getting late. Please order some pizza and stop  tail Display last few lines of file $ tail -2 letter3 Thanks! Bill  grep Find a pattern in a file $ grep ”some pizza” letter3 It is getting late. Please order some pizza and stop

6. More Utilities (4)  sort Sort the lines in lexical order $ sort letter3 Bill It is getting late. Please order some pizza and stop Ms. Lewinski: Thanks! by my office. We'll tidy up a few more things before calling it a night. $ sort -r letter3 calling it a night. by my office. We'll tidy up a few more things before Thanks! Ms. Lewinski: It is getting late. Please order some pizza and stop Bill

7. Input/Output Redirection (2)  Using the “>” character after a command to redirect output to a named file: $ sort names > names.sort $ uniq names.sort Bill Clinton Bill Gates Monica Lewinski  This will create a file test: $ cat > test type line 1 type line 2 $ cat test type line 1 type line 2 uniq display sort names names.sort

8. Input/Output Redirection (3)  Using the “>>” character after a command to redirect output to APPEND to a named file:  Typing to the end of a file $ cat >> test type line 3 type line 4 $ cat test type line 1 type line 2 type line 3 type line 4  Append file1 to file2 $ cat file1 >> file2

9. Input/Output Redirection (4)  Using the “<” character after a command to redirect input from a named file: $ uniq < names.sort This is the same as: $ uniq names.sort  Using input and output redirection together: $ sort names.sort uniq display sort names names.sort

10. Pipes (1)  The standard output of a program can be “piped” into the standard input of another program: $ sort names | uniq Bill Clinton Bill Gates Monica Lewinski uniq display sort names

11. Pipes (2)  Several pipes can be connected: $ sort names | uniq | grep "Bill" Bill Clinton Bill Gates  Pipes and I/O redirection can be used together: $ sort -r names | uniq >names.rev $ cat names.rev Monica Lewinski Bill Gates Bill Clinton

12. Directory Shorthands  “.” is the directory itself  “..” is the parent directory  In most shells “~” means your home directory)  ~user means user’s home directory, so: $ more ~jbond/.plan  looks at the file.plan in /home/jbond, which is jbond ’s home directory.

13. Special Directories  “ / ” (pronounced “slash” and also called “the “root”) is the ancestor of all files in the file system  /bin and /usr/bin contain UNIX utilities (e.g., cat )  /dev contains special files which describe “devices” such as terminals and printers  /etc has administrative programs like password files  /tmp is for temporary files; periodically deleted  Every directory has at least two entries: “.” is the directory itself, and “..” is the directory’s parent

14. Security and Access Permissions (1)  There are three types of users:  The owner of the file (user)  The group of the file (group)  Anyone else (other)  There are three types of permission (independent of each other):  Read permission  Write permission  Execute permission

15.  Use ls -l to see file permissions -rw-r--r-- 1 jbond cs 154 Feb 4 15:00 letter3  There are four sets of items in the permissions: -rw-r--r--  The type is: “ - ” regular files, “ d ” directories, “ l ” symbolic links.  The next nine characters indicate if the file is readable, writable, or executable for the file owner, the file group, or other users, respectively. Security and Access Permissions (2) PermissionsUser Group Byte size Last modification Name #links usergroupother type

16. Changing Permissions (1)  The chmod command is used to modify permissions.  chmod can only be used by the owner of a file/dir (or the administrator root).  The arguments are: chmod [ugoa] [+-=] [rwxdd] [file/dir] In other words:  Optionally, one of the characters: u (user/owner), g (group), o (other), or a (all).  Optionally, one of the characters: + (add permission), - (remove permission), or = (set permission).  Any combination of the characters r (read), w (write), or x (execute).

17. Changing Permissions (2)  Another way to change permission is to use numbers representing the permissions. The arguments are: chmod n u n g n o [file/dir] n u - user’s permission code n g - group’s permission code n o - other’s permission code -rwxrwxrwx Code table (3 bits) --- 0 r-- 4 --x 1 r-x5 -w- 2 rw-6 -wx 3 rwx7

18. Permission Example  To let everybody read or write the file letter1 $ chmod a+rw letter1 (chmod 666 letter1) $ ls -l letter1 -rw-rw-rw- 1 jbond cs 154 Feb 4 15:00 letter1  To allow user to execute file letter1 $ chmod u+x letter1(chmod 766 letter1) $ ls -l letter1 -rwxrw-rw- 1 jbond cs 154 Feb 4 15:00 letter1*  To not let “other” to read or write file letter1 $ chmod o-rw letter1(chmod 760 letter1) $ ls -l letter1 -rwxrw---- 1 jbond cs 154 Feb 4 15:00 letter1*  To let “group” only read the file letter1 $ chmod g=r letter1(chmod 740 letter1) $ ls -l letter1 -rwxr----- 1 jbond cs 154 Feb 4 15:00 letter1*

19. Alias  The C Shell has the alias command, which allows you to create command shortcuts. $ alias dir "ls -F" $ alias rm "rm -i" $ alias + "chmod u+x *" $ alias - "chmod u-x *" $ alias 111 "cd ~horner/111" $ pwd /bin $ 111 $ pwd /homes/horner/111  If you put the alias commands in your.cshrc file, you can use them every time you login.

20. Combining Commands (2)  Commands can be grouped together using parentheses  There are two main reasons to group commands:  To create a “single command” out of a group of commands (especially useful before a pipe): $ (cat letter1; head -2 names) | sort >list  To run a set of commands in their own subshell (especially when trying to limit the effect of a cd command): $ (cd secret; ls | wc -l); ls | wc -l 3 25 This line has the effect of counting the files in secret, and then counting the files in the current directory.

21. Shells  A shell can be used in one of two ways:  A command interpreter, used interactively  A programming language, to write shell scripts (your own custom commands)

22. Shell Scripts  A shell script is just a file containing shell commands, but with a few extras:  The first line of a shell script should be a comment of the following form: #!/bin/sh for a Bourne shell script. Bourne shell scripts are the most common, since C Shell scripts have buggy features.  A shell script must be readable and executable. chmod +rx scriptname  As with any command, a shell script has to be “in your path” to be executed. –If “.” is not in your PATH, you must specify “./scriptname ” instead of just “ scriptname ”

23. Shell Script Example  Here is a “hello world” shell script: $ ls -l -rwxr-xr-x 1 horner 48 Feb 19 11:50 hello* $ cat hello #!/bin/sh # comment lines start with the # character echo " Hello world " $ hello Hello world $  The echo command functions like a print command in shell scripts.

24. Shell Variables (1)  The user variable name can be any sequence of letters, digits, and the underscore character, but the first character must be a letter.  To assign a value to a variable: number=25 name= " Bill Clinton "  There cannot be any space before or after the “=“  Internally, all values are stored as strings.

25. Shell Variables (2)  To use a variable, precede the name with a “$”: $ cat test1 #!/bin/sh number=25 name= " Bill Clinton " echo " $number $name " echo ' $number $name ' $ test1 25 Bill Clinton $number $name $  Note the difference of single and double quotes in this example.

26. User Input (1)  Use the read command to get and store input from the user. $ cat test2 #!/bin/sh echo "Enter name: " read name echo "How many friends do you have? " read number echo "$name has $number friends!” $ test2 Enter name: Bill Clinton How many friends do you have? too many Bill Clinton has too many friends!

27. User Input (2)  read reads one line of input from the keyboard and assigns it to one or more user-supplied variables. $ cat test3 #!/bin/sh echo "Enter name and how many friends:" read name number echo "$name has $number friends!" $ test3 Enter name and how many friends: Bill Clinton 63 Bill has Clinton 63 friends! $ test3 Enter name and how many friends: BillC 63 BillC has 63 friends! $ test3 Enter name and how many friends: Bill Bill has friends!  Leftover input words are all assigned to the last variable.

28. Special Symbol : $ (1)  Use a backslash before $ if you really want to print the dollar sign: $ cat test4 #!/bin/sh echo "Enter amount: " read cost echo "The total is: \$$cost" $ test4 Enter amount: 18.50 The total is $18.50

29. Special Symbol : $ (2)  You can also use single quotes for printing dollar signs.  Single quotes turn off the special meaning of all enclosed dollar signs: $ cat test5 #!/bin/sh echo "Enter amount: " read cost echo 'The total is: $' "$cost" $ test5 Enter amount: 18.50 The total is $ 18.50

30.  Shell programming is not good at numerical computation, it is good at text processing.  However, the expr command allows simple integer calculations.  Here is an interactive Bourne shell example: $ i=1 $ expr $i + 1 2  To assign the result of an expr command to another shell variable, surround it with backquotes: $ i=1 $ i=`expr $i + 1` $ echo " $i " 2 Simple calculation using expr (1)

31.  The * character normally means “all the files in the current directory”, so you need a “\” to use it for multiplication: $ i=2 $ i=`expr $i \* 3` $ echo $i 6  expr also allows you to group expressions, but the “(“ and “)” characters also need to be preceded by backslashes: $ i=2 $ echo `expr 5 + \( $i \* 3 \)` 11 Simple calculation using expr (2)

32. $ cat test6 #!/bin/sh echo "Enter height of rectangle: " read height echo "Enter width of rectangle: " read width area=`expr $height \* $width` echo "The area of the rectangle is $area" $ test6 Enter height of rectangle: 10 Enter width of rectangle: 5 The area of the ractangle is 50 $ test6 Enter height of rectangle: 10.1 Enter width of rectangle: 5.1 expr: non-numeric argument Does not work for floats! Simple calculation using expr (3) - Example

33. Backquotes – Command Substitution  A command or pipeline surrounded by backquotes causes the shell to:  Run the command/pipeline  Substitute the output of the command/pipeline for everything inside the quotes  You can use backquotes anywhere: $ whoami clinton $ cat test7 #!/bin/sh user=`whoami` numusers=`who | wc -l` echo "Hi $user! There are $numusers users logged on." $ test7 Hi clinton! There are 6 users logged on.

34. Backquote Example (1) $ cat big #!/bin/sh dir=`pwd` big=`ls -l | sort +4 | tail -1 | cut -c55-70` size=`ls -l | sort +4 | tail -1 | cut -c33-40` echo "The biggest file in $dir is $big." echo "$big is $size bytes." $ big The biggest file in /homes/horner/111 is letter1. letter1 is 155 bytes.  sort +4 sorts in increasing order based on the fourth field (fields start from zero, and are separated by whitespace).  cut -cx-y cuts characters from position x to position y (the first position is 1). 1 2 3 4 5 6 123456789012345678901234567890123456789012345678901234567890123 -r--r--r-- 1 horner cs 155 Feb 12 16:00 letter1 field4 field0

35. Backquote Example (2 ) $ wc -w letter1 7 letter1 $ cat big1 #!/bin/sh dir=`pwd` big=`ls -l | sort +4 | tail -1 | cut -c55-70` nline=`wc -l $big | cut -c6-8` nword=`wc -w $big | cut -c6-8 ` nchar=`wc -c $big | cut -c6-8 ` echo "The biggest file in $dir is $big." echo "$big has $nline lines, $nword words, $nchar characters." $ big1 The biggest file in /homes/horner/111 is letter1. letter1 has 7 lines, 29 words, 155 characters.  wc -w counts the number of words in the file (“words” are separated by whitespace).  wc -c counts the number of characters in the file.

36. Control Flow  The shell allows several control flow statements:  if  while  for