1. CSc 352 Shell Scripts Saumya Debray Dept. of Computer Science
The University of Arizona, Tucson

2. What is a shell script?
A shell script is a list of commands to be run by a shell
basically a program, but using shell commands instead of C or Java statements
Why?
automate repetitious tasks
e.g.: testing a program on a large set of test inputs
package up commonly executed command sequences
create our own commands

3. Creating and executing a shell script
Create a file, say “foo”, containing the commands to be executed by the script
Execute the script:
invoke the appropriate shell with foo as argument, e.g.:
bash foo
csh foo
chmod a+x foo
./foo

4. A simple example a shell script to print “hello world!”
the “echo” command, as executed from the command line
contents of the shell script file “hello_world.script”
this script behaves the same for a variety of different shells
with appropriate permissions, the script can be executed directly

5. Syntax issues across shells
Different shells have slightly different syntax rules
the same script may not work on all shells:
script contents
different behaviors

6. Which shell runs my script?
my current shell is csh
the script executes as expected when csh is invoked explicitly
if not invoked explicitly, the default shell /bin/sh is used

7. Solution: specify shell in the script
#!pathToShell on first line specifies which shell to use
executes as expected when invoked in csh
executes as expected when invoked in bash

8. Specifying the shell interpreter
#!/bin/csh
‘#!’ at the beginning of a file indicates that it is a script
The rest of the first line specifies (absolute) path to the interpreter
What if the interpreter is at a different location on a different system?
use /usr/bin/env to improve portability:
#!/usr/bin/env csh

9. Variables Variables different than in C or Java:
don’t have to be declared in advance
they are untyped: the same variable can hold an integer value or a string
Syntax for using variables (bash):
Defining the value of a variable X:
X=value
Using the variable X:
$X or ${X}

10. Example variable definitions variable uses
undefined variable evaluates to empty string

11. Guarding against uninitialized variables
-u : “exit if using an unitialized variable”
-u :
works if we specify /bin/bash
but not with /usr/bin/env

12. Guarding against uninitialized variables

13. Bash scripts: whitespace
without whitespace  the command works as expected
with whitespace 
weird error message
variable value not set as expected

14. Different kinds of quotes
back quotes: evaluates to the output of the enclosed commands
single quotes: variables not expanded
double quotes: variables inside the quotes are expanded

15. A simple scripting problem
Iterate over the files in a given directory:
for each file, express its size in human-readable form (i.e., using KB and MB where appropriate):
give 2 decimal places for MB sizes; 1 decimal place for KB sizes
functionality similar to using “ls –lh”
Requires:
list files in a directory [valid directory?  if-then-else]
iterate over these files
obtain the size of each file
perform arithmetic [KB/MB decision  if-then-else]

16. The test command Used to guide control flow in conditionals and loops
Syntax:
test operand1 operator operand2
test operand
often abbreviated to:
[ operand1 operator operand2 ]
[ operand ]
see “man test” for a list of operators
Returns a nonzero status if the test is true; zero o/w

17. Example: test

18. Example: if-then-else
if-then-else command

19. Example: if-then-else
need a ‘;’ between the test expression and ‘then’

20. Example: if-then-else

21. Alternative formulation of test

22. For Loops Syntax 1: for X in word ; do cmds done
Convenient for iterating over lists
Syntax 2:
for (( e1; e2; e3 )) ; do
cmds
done
Convenient for iteration involving arithmetic expressions

23. For Loops: Example 1

24. For Loops: Example 2

25. Iterating over lists: pitfalls!
print out the first couple of lines of a file listing

26. Iterating over lists use quotes to prevent splitting at whitespace
Note: the loop iterates only once – not once per file!

27. Arithmetic
arithmetic evaluation doesn’t happen automatically; must be specified explicitly
get file size (see “man stat” for other file properties)

28. Arithmetic
integer arithmetic only!
arithmetic evaluation:
$(( expr ))

29. get bc to do the floating point math for us
Arithmetic
get bc to do the floating point math for us

30. Command-line arguments
Arguments at known positions, e.g., 1st, 2nd, ..., can be accessed using $1, $2, …
$0 = the name of the script file
Total no. of arguments: $#
However, we can’t get to the ith argument this way
$i  value of variable i

31. Example

32. Iterating over the argument list

33. Getting at the entire list of arguments

34. Iterating over the argument list
Placement of quotes matters:
all of the arguments are seen as a single word

35. Iterating over the argument list
Placement of quotes matters:

36. Arguments with whitespace
is similar to $*, but each argument is a quoted string
no interpretation or expansion
allows us to properly access arguments that may contain whitespace
needs to be quoted “…”

37. Arguments with whitespace

38. Debugging

39. prints shell input lines as they are read
Debugging: -v
prints shell input lines as they are read

40. Debugging: -v
these are the lines from the script that are being executed
Note that variable values etc. are not being shown

41. Debugging: -x
prints out command traces before they are executed

42. Debugging: -x
variable values, command arguments are printed out

43. Debugging: summary
Debugging can be done by observing what commends get executed
“set –v”
prints out shell commands as they are read in
does not print out runtime variable values
“set –x”
prints out a trace of the command before it is actually executed
prints out variable and argument values
Debugging can be turned off using “set +v”, “set +x”
useful for localizing which parts of the script are observed

44. String manipulation shell scripts often involve string manipulation
e.g., to pattern match on file names, create new files based on existing files, etc.
bash gives extensive facilities for string matching and manipulation
see “man expr”

45. A sample problem
Write a bash script to convert Unix make files to “mymake” make files
unix make
mymake
a : b.o c.o d.o
gcc *.o –o a
b.o : b.c b.h
gcc –Wall –c b.c
@target a : b.o c.o d.o
@cmd gcc *.o –o a
@target b.o : b.c b.h
@cmd gcc –Wall –c b.c 

46. Attempt 1 Ignore definitions Assume Issues:
assume make file does not have stuff like “CC = gcc”
Assume
user has added whitespace where necessary
e.g.: on either side of ‘ : ’ in rules
all dependencies have been made explicit
Issues:
need to process a line at a time
lines may contain whitespace
need to check whether a line starts with a ‘\t’

47. Attempt 1

48. Attempt 1 IFS = “Internal Field Separator”
used by bash to split the input into words
the new definition of IFS means that the input is now split at newlines
identify lines that begin with a tab

49. Attempt 1

50. Attempt 2 relax assumption about whitespace around : Issue:
want:
input: “a.o: a.c b.h c.h”
output: a.o : a.c b.h c.h”
Issue:
replace ‘:’ by ‘ : ’
bash offers two replacement operators:
${str/substr/replacement}
replaces the first occurrence of substr in str with replacement
${str//substr/replacement}
replaces all occurrences of substr in str with replacement

51. Attempt 2

52. Attempt 2

53. Only the first ‘:’ had whitespace inserted
Attempt 2
Only the first ‘:’ had whitespace inserted

54. “replace all occurrences”
Attempt 2
“replace all occurrences”

55. Attempt 2

56. Attempt 3 Relax assumption about definitions Issues:
allow “CC=gcc”, “CFLAGS= –Wall –g –c”
assume no whitespace around ‘=‘
Issues:
identify definitions, i.e., lines that contain a ‘=‘
extract LHS, RHS of the definition
replace every occurrence of LHS by RHS

57. Attempt 3 if the line contains a ‘=‘ … figure out where it occurs
LHS = substring from position 0 to position of ‘=‘
RHS = substring after position of ‘=‘ to end
substitute RHS for every occurrence of LHS

58. Attempt 3

59. Attempt 3
Oops!

60. Attempt 3

61. Attempt 3