1. Introduction to Advanced UNIX March 9 2010 Kevin Keay

2. Outline Overview Overview Resources Resources The basic form of a typical C-shell script The basic form of a typical C-shell script Introduction to C-Shell Programming Introduction to C-Shell Programming Introduction to C-Shell Programming Introduction to C-Shell Programming Advanced UNIX materials: The following are PDF documents that may be used for reference but won’t be discussed in detail during this short course: Advanced UNIX materials: The following are PDF documents that may be used for reference but won’t be discussed in detail during this short course: Part 1 : Scripts Scripts Part 2 : A summary of useful software A summary of useful softwareA summary of useful software Part 3 : Reanalysis products Reanalysis productsReanalysis products Lab session (informal) 2-3 PM UNIX Lab Lab session (informal) 2-3 PM UNIX Lab See: http://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/Advanced_UNIX_Lab_Session_2010.pdf See: http://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/Advanced_UNIX_Lab_Session_2010.pdfhttp://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/Advanced_UNIX_Lab_Session_2010.pdf

3. Overview This is a practical course that is designed to give you the basic skills to write UNIX C-shell scripts This is a practical course that is designed to give you the basic skills to write UNIX C-shell scripts You can only truly understand the concepts by putting them into practice! You can only truly understand the concepts by putting them into practice!

4. Overview (2) We will look at the basic form of a C-shell script We will look at the basic form of a C-shell script There will be a quick tour through Introduction to C-Shell Programming There will be a quick tour through Introduction to C-Shell Programming There are some extra advanced UNIX materials (Parts 1-3) that you may need to reference. Some aspects will be looked at during the lab session but they are intended for personal reference There are some extra advanced UNIX materials (Parts 1-3) that you may need to reference. Some aspects will be looked at during the lab session but they are intended for personal reference Part 1 covers C-shell scripts and some additional concepts Part 1 covers C-shell scripts and some additional concepts Part 2 is a summary of some useful software Part 2 is a summary of some useful software Part 3 focuses on downloading and decoding reanalysis data (NetCDF and GRIB) Part 3 focuses on downloading and decoding reanalysis data (NetCDF and GRIB) Finally, there is a short Lab session Finally, there is a short Lab session

5. Resources A very useful and concise reference book covering UNIX and the C shell is: A very useful and concise reference book covering UNIX and the C shell is: UNIX in a Nutshell, O’Reilly and Associates (Engineering Library: 005.43 GILL). A useful online resource from the University of Surrey, UNIX Tutorial for Beginners, is available at: http://www.ee.surrey.ac.uk/Teaching/Unix/ A useful online resource from the University of Surrey, UNIX Tutorial for Beginners, is available at: http://www.ee.surrey.ac.uk/Teaching/Unix/http://www.ee.surrey.ac.uk/Teaching/Unix/ For PDF files of the handouts see: http://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/ For PDF files of the handouts see: http://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/ http://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/ All UNIX commands should have a manual (man) page on the machine that you are using e.g. man awk. However sometimes the pages are not installed or are in an unexpected location. It may be more convenient to use a search engine like Google to find information on a command. All UNIX commands should have a manual (man) page on the machine that you are using e.g. man awk. However sometimes the pages are not installed or are in an unexpected location. It may be more convenient to use a search engine like Google to find information on a command.

6. The basic form of a typical C-shell script The first line of a C-shell script (text file) is: The first line of a C-shell script (text file) is: #!/bin/csh –f The –f option tells the script not to source (read) the user’s.cshrc file; this is faster and also makes the script more portable. All other lines starting with # are comments All other lines starting with # are comments Commands may be continued onto subsequent lines with \ Commands may be continued onto subsequent lines with \ Multiple commands can be placed on a single line with ; Multiple commands can be placed on a single line with ; Spaces around operators and commands are required but there are no other restrictions Spaces around operators and commands are required but there are no other restrictions

7. The basic form of a typical C-shell script (2) For neatness, end the script with exit (not essential) For neatness, end the script with exit (not essential) Shell variables start with a $ (this is only omitted with using set or @ - see below) Shell variables start with a $ (this is only omitted with using set or @ - see below) The shell variable $#argv contains the number of arguments (items) typed on the command line. The items are referred to as shell variables $1, $2, $3, …, $#argv The shell variable $#argv contains the number of arguments (items) typed on the command line. The items are referred to as shell variables $1, $2, $3, …, $#argv

8. The basic form of a typical C-shell script (3) For example: For example: myscript jja -4.56 yes would give: $#argv = 3 $1 = jja $2 = -4.56 $3= yes

9. The basic form of a typical C-shell script (4) Shell variables are either string (text) or numeric Shell variables are either string (text) or numeric A string variable is set (defined) by: A string variable is set (defined) by: set var = value e.g. set x = Fred Note: Variables and values are case-sensitive i.e. x is not the same as X Variable names can be several characters e.g. set Case2 = 3 Variable names can be several characters e.g. set Case2 = 3 To print (display) the value of a variable: To print (display) the value of a variable: echo $var e.g. echo $x Fred

10. The basic form of a typical C-shell script (5) A numeric variable is set (defined) by: A numeric variable is set (defined) by: @ var = integer e.g. @ k = -7 Simple calculations may be performed Simple calculations may be performed e.g. @ j = ($k + 1)[j would be – 7 + 1 = -6] @ k = ($k + 1) would change k from -7 to -6 Could also use C notation: @ k ++ @ j = (2 * $k - 5) [k= -7 would give j= -19] Note: Put spaces around operators like *, + etc. Floating point operations are not normally used in scripts but can be done with the command bc Floating point operations are not normally used in scripts but can be done with the command bc e.g. echo "2.1 + 6.3" | bc would print 8.4 on the screen set y = `echo "2.1 + 6.3" | bc` would save the answer in variable y

11. The basic form of a typical C-shell script (6) A shell variable may be given the output of a command (or user program) as in the bc example. In general: A shell variable may be given the output of a command (or user program) as in the bc example. In general: set var = `command` e.g. set ff = `ls *.dat` Remember: ls is used to list files. If the files are 5.dat 12.dat 13.dat then echo $ff would display: 5.dat 12.dat 13.dat 5.dat 12.dat 13.dat

12. The basic form of a typical C-shell script (7) To extract the individual elements of variable ff we can use $#ff to find the number of items (similar to $#argv) To extract the individual elements of variable ff we can use $#ff to find the number of items (similar to $#argv) e.g. echo $#ff would display 3 echo $ff[1] would display 5.dat echo $ff[2] would display 12.dat echo $ff[3] would display 13.dat

13. The basic form of a typical C-shell script (8) A while loop is used to set up an iterative or repetitive procedure: A while loop is used to set up an iterative or repetitive procedure: while (condition is true) commands end end e.g.@ k = 1 # Initialise k while ($k <= 3) # Process loop while k <= 3 echo $k # Display k echo $k # Display k @ k = ($k + 1) # Increment k by 1 @ k = ($k + 1) # Increment k by 1end

14. The basic form of a typical C-shell script (9) A foreach loop is used to set up an iterative or repetitive procedure involving files: A foreach loop is used to set up an iterative or repetitive procedure involving files: foreach var (files) commands end end e.g.foreach f ([A-Z]*) # f is a file starting with a capital letter echo $f # Print filename echo $f # Print filename mv $f $f.cap # Rename file to have extension.cap mv $f $f.cap # Rename file to have extension.capend

15. The basic form of a typical C-shell script (10) An if-else-endif structure is used to control script branching: An if-else-endif structure is used to control script branching: if (condition 1 is true) commands else if (condition 2 is true) commandselsecommands endif endif Note: use ‘else if` not elseif Note: use ‘else if` not elseif

16. The basic form of a typical C-shell script (11) e.g. e.g. if ($x == 6 || $s == sea) then do something else if ($x > 10) then do something else else go here if neither of the above conditions is true endif Simpler or more complex forms are possible Simpler or more complex forms are possible e.g. if ($y == 2) ls –l *.dat

17. The basic form of a typical C-shell script (12) The while or if (condition) uses C notation for logical operations: The while or if (condition) uses C notation for logical operations: ||or && and == equal >= greater than or equal <= less than or equal != not equal != not equal >greater than <less than

18. The basic form of a typical C-shell script (13) An example An example #!/bin/csh –f if ($#argv != 1) then echo “Usage: myscript name” exit # If incorrect usage end the script else # If only 1 argument, branch here set n = ($1) # Put n equal to $1 endif echo “Your name is “$n # You could do other things here exit

19. The basic form of a typical C-shell script (14) Another example Another example #!/bin/csh –f set ff = `ls *.dat` # ff contains filenames *.dat @ nf = $#ff # nf equals the no. of files (items in ff) @ j = 1 # Set counter j to 1 while ($j <= $nf) # Loop while j <= nf set f = $ff[$j] # Set f to be the jth file set f = $ff[$j] # Set f to be the jth file ls –l $f # List the details of f ls –l $f # List the details of f # We could use the file f as an argument to a command or program myprog $f myprog $f @j = ($j + 1) # Increment counter j by 1 @j = ($j + 1) # Increment counter j by 1 end # Go here when j > nf echo “We are finished!” # A message to the screen exit

20. Introduction to C-shell programming Refer to the PDF guide: Refer to the PDF guide: http://www.earthsci.unimelb.edu.au/~kevin /UNIX_Course/Intro_to_C- Shell_Programming_2008.pdf http://www.earthsci.unimelb.edu.au/~kevin /UNIX_Course/Intro_to_C- Shell_Programming_2008.pdf

21. Advanced UNIX materials The following are PDF documents that may be used for reference but won’t be discussed in detail during this short course: The following are PDF documents that may be used for reference but won’t be discussed in detail during this short course: Part 1 : Scripts Scripts Part 2 : A summary of useful software A summary of useful softwareA summary of useful software Part 3 : Reanalysis products Reanalysis productsReanalysis products

22. Lab session Lab session (informal) 2-3 PM UNIX Lab Lab session (informal) 2-3 PM UNIX Lab See: http://www.earthsci.unimelb.edu.au/~kevin/UNIX _Course/Advanced_UNIX_Lab_Session_2010.pdf See: http://www.earthsci.unimelb.edu.au/~kevin/UNIX _Course/Advanced_UNIX_Lab_Session_2010.pdf http://www.earthsci.unimelb.edu.au/~kevin/UNIX _Course/Advanced_UNIX_Lab_Session_2010.pdf http://www.earthsci.unimelb.edu.au/~kevin/UNIX _Course/Advanced_UNIX_Lab_Session_2010.pdf