Shell Basics CS465 - Unix

Shell Basics Shells provide: –Command interpretation –Multiple commands on a single line –Expansion of wildcard filenames –Redirection of input and output –Pipelines –Background execution –Script-programming facility

Running Multiple Commands ; Causes commands to run in sequence. $ cmd1 ; cmd2 ( )Causes sequenced commands to be grouped $ (cmd1 ; cmd2) && Runs the next command only if the current command succeeds. $ cmd1 && cmd2 ||Runs the next command only if the current command fails. $ cmd1 || cmd2

Sequence Example Commands can be sequenced (run in sequence) using the semicolon (;): $ ls ; who ; date ; pwd file1 file2 file3 newfile small000 pts/1 Jun 4 14:41 Tue Jun 4 14:42:51 MDT 2003 /export/home/small000 $ If any one command fails, the others will still execute. NOTE: To make the command line easier to read, separate commands from the semicolon (;) with blanks.

Grouped Commands Example Commands can be grouped together using parentheses –To create a “single command” out of a group of commands (especially useful before a pipe): $ (cat letter1; head -2 names) | sort

&& Example To run the second command only if the first command succeeds (i.e. short-circuit “and” operator): BAD: $ cd /somedir; rm -r * GOOD: $ cd /somedir && rm -r *

|| Example To run the second command only if the first command fails (i.e. short-circuit “or” operator): Example: $ ls file1 || cp fileX file1

Shell Environment Your environment defines such characteristics as your user identity, where you are working on the system, and what commands you are running. Your working environment is defined by both environment variables and shell variables. –Your default login shell uses environment variables and passes them to all processes and subshells that you create. –Shell variables are valid only for your current shell and are not passed to subshells.

Environment Variables HOME Name of your login directory. Set by login program. HOST Name of host machine running your shell program. LOGNAME Your login name. Set by login program. MAIL Pathname of the file used by the mail system to detect the arrival of new mail. Set by login program.

Environment Variables PATH Directory list and order that your system uses to search for, find, and execute commands. Set by login scripts. SHELL Your default shell. Set using the shell specified in your entry in the /etc/passwd file. TERM Type of terminal you are using. Set by your login script. TZ Your current time zone. Set by the system login script.

Environment Variables To see the value of a defined variable, place a $ in front of the variable name: $ echo $HOME /home/small000 $ echo $TZ MDT $ Note the difference if you leave off the "$": $ echo HOME HOME

Metacharacters Metacharacters are used to create patterns to match filenames and command names. –For example, we often wish to apply a command to a selection of files or directories –The shell wildcard (often known as globbing) feature allows this. Each shell program recognizes a set of special characters called metacharacters: ; & ( ) | \ ^ ~ % { } $ # ´ “ * ? [ ] !

Wildcard Metacharacters ? matches any character * matches any number of characters [abc] matches any one of the characters in the brackets [a-z] matches any one character in the range of characters in the brackets [!a-z] matches any character except the range of characters in the brackets

Wildcard Examples $ ls abc?def –matches abcXdef, abc.def, abc3def but not abcdef $ ls abc* –matches abc., abcd, abcdef, abc.another.part $ ls *.* –matches any name with a “.” in it $ ls * –matches any name (except names beginning with “.”) $ ls [a-z]* –matches any name beginning with a lower-case letter

Wildcard Exercise Assume we have the following files under the current directory: a.outc22cap Doc.1Doc.2Doc.3 one.ctwo.cthree.c What would match for: ls *.cls ?2? ls D*ls [a-z]*[!0-9] ls ???ls [23]* ls *ls.*

Avoiding Wildcard Expansion Sometimes you may NOT want wildcard characters expanded. The Unix glob utility translates wildcard characters on the command line. To turn off ALL wildcard expansion: $ set noglob To turn ALL wildcard expansion back on: $ set glob

Avoiding Wildcard Expansion Sometimes you need to selectively enter a wildcard character on a command line, and not have it expanded. Method 1: Double quote the argument containing the character: $ ls “abc*def” abc*def ls the file whose name is abc*def (if the system allows such a name)

Avoiding Wildcard Expansion Method 2: “Escape” the character $ ls abc\*def –again lists the file whose name is abc*def Can use these methods in combination $ echo “Show asterisks *” and \* Show asterisks * and * $ –* inside the double quotes is not expanded –* after the backslash is not expanded

Double Quotes vs Single Quotes Within double quotes, the reserved characters: dollar sign ($) grave accent (`) backslash (\) keep their special meanings All other characters are taken displayed literally.

Double vs Single Quote Example Displaying a variable name without it being interpreted by the shell: $ echo “Value of $USER is “ $USER Value of small000 is small000 $ $echo 'Value of $USER is ' $USER Value of $USER is small000 $

Grave Accent/Backquotes Within backquotes (graves), command substitution is forced: Example: $ echo I am user whoami I am user whoami $ echo I am user `whoami` I am user small000 $

Summary of Quoting Metacharacters Single quotes –Contents are used verbatim No wildcard expansion No variable interpolation No command execution Double quotes –Only turns off wildcard expansion Backquotes –Forces execution of command in quotes –Output from command replaces quoted string

Redirection of I/O Every program run under Unix is provided with three standard I/O streams: stdin, stdout and stderr command stdin (0) stdout (1) stderr (2) By default: - the standard input (stdin) is the keyboard - the standard output (stdout) and standard error (stderr) are the display screen Your shell allows you to divert any or all of these streams.

Output Redirection Use the “>” character to redirect the standard output of a command to a file: $ ls > myfiles $ cat myfiles file1 file2 file3 $ $ echo hello $USER > greeting $ cat greeting hello small000 $ echo hi there > greeting $ cat greeting hi there $

Output Redirection Use the “>>” character after a command to redirect output to APPEND to a file: $ cat greeting hi there $ echo “How are you?” >> greeting $ cat greeting hi there How are you? $

Output Redirection $ ls x* ls: x*: No such file or directory $ ls x* 1>xfiles 2>errs $ cat xfiles $ cat errs ls: x*: No such file or directory $ echo roses are red 1> rhyme 2> errs $ echo violets are blue 1>> rhyme 2>> errs $ cat errs $ cat rhyme roses are red violets are blue $ Can separate stdout (1) and stderr (2):

Output Redirection $ cat test type line 1 type line 2 $ $ cat > test type line 1 type line 2  to end your input $ Redirecting cat’s output can create a file (no input file name given, so gets input from stdin):

Output Redirection  Typing to append to the end of a file $ cat >> test type line 3 type line 4 $ cat test type line 1 type line 2 type line 3 type line 4  Appending file1 to file2 $ cat file1 >> file2

Overwriting Files? By default, the use of ‘>’ will overwrite any existing file, without warning. Use “noclobber” to prevent this: $ set –o noclobber(+o to revoke) Gives error if you try to overwrite an existing file. Can still use >| to force overwriting of an existing file. $ ls file1 file2 $ set -o noclobber $ cat file1 > file2 ksh: cannot create file2: File exists $ cat file >| file2 $

Input Redirection Use the “<” character to redirect input from a file to a command: $ sort < names $ mail user < letter.txt Can use input and output redirection together: $ sort names.sort

Filters Most UNIX utilities (commands) are filters A filter is a program which: –reads input (text) from standard input only –writes output and errors to standard output only Filters can be combined to work together using pipes A pipe takes the stdout of one command and uses it as the stdin of another command

Pipes The standard output of one program is “piped” to the standard input of another: $ who | sort Several pipes can be connected: $ ls | sort | more No limit on number of processes that can be chained by pipes

Pipes Pipes and I/O redirection can be used together: $ ls | sort > files.sorted $ sort < names | more

Why Pipes? Could chain commands together $ ls > temp; sort < temp; rm temp Works, but requires temporary intermediate file, which must be manually removed Solution – Use a pipe: $ ls | sort –No intermediate file, processes run in parallel

Multiple Outputs What if you want the output of a command to be saved in a file AND displayed on the screen? Use the tee command Example: $ ls –la | tee filelist Displays directory listing to the screen, AND saves it to the file filelist.

Summary of Shell Command Interpretation The shell reads a line from a script file or from a user. Then: 1.Meta-characters are "handled.“ 2.Executable is found (via PATH). 3.Arguments are passed to the program. 4.File redirection is setup. 5.The program is executed.