1. sed

2. Class Issues vSphere Issues – root only until lab 3

3. sed sed = stream editor Allows you make changes to text files on the fly on a line by line basis Basic syntax is: sed ‘s/pattern/replacement/’ filename Works on files, default output is to the screen

4. Basic Examples Replace: –sed ‘s/1/ONE/g’ file.txt (would replace the first 1 with ONE in file.txt) Swap: – sed 'y/abc/ABC' file.txt (replace a with A, b with B, c with C) Show line numbers then lines: –sed ‘=‘ file.txt Lowercase: sed 's/[A-Z]/\l&/‘ file.txt

5. Special Replacements \L - turn the replacement to lowercase until a \U or \E is found, \l - turn the next character to lowercase \U - turn the replacement to uppercase until a \L or \E is found \u - turn the next character to uppercase, \E - stop case conversion started by \L or \U.

6. Remember the g? First Character Lower Case: –sed 's/[A-Z]/\l&/' addresses.txt All Characters Lower Case: –sed 's/[A-Z]/\l&/g' addresses.txt 5th Character Lower Case: –sed 's/[A-Z]/\l&/5' addresses.txt

7. Useful sed command line -f means read sed commands from file –sed –f mysed.txt

8. Gotchas Don’t do: –sed 's/[A-Z]/\l&/[3-5]' addresses.txt > addresses.txt Create a new file and mv it back.

9. awk

10. Programming language for data streams; usually files or subsets of files Developed at Bell Labs in the ’70’s by: Alfred Aho, Peter Weinberger, and Brian Kernighan Like sed but does the following as well: –specify particular fields of a matching line to manipulate and/or output –use variables to store values temporarily –perform calculations on those variables.

11. How to run? Usually: awk ‘ ’ files to parse Example (a cat replacement): awk ‘{print $0}’ *

12. Fields in awk By default lines are delimited by white space: space characters or tabs $0 – whole line $1 – is first field in line $2 – is 2 nd field in line NF – number of fields on line No error reported if field doesn’t exist, simply outputs a blank line

13. Basic Pattern Matching /pattern/ { do something } Example: /A/ { print “This line has an A”} No pattern means do the work on every line

14. Multiple patterns Just list them: awk ‘ /A/ { print “this line has an A”} /B/ { print “this line has a B”} { print “this gets printed always” } ‘ my_file Don’t forget the closing ‘

15. Forming Patterns Can use most of the extended regular expressions, except {} Can do the relational expressions from other languages:, ==, !=, =

16. Comparisons Numeric example: ls -l | awk '$5 >= 512 {print $0}' String example: ls -l | awk '$6 == “Sep” {print $0}'

17. Variables Variables are allowed: ls -l | awk '$5 >= 512 {size += $5}‘ Variables don’t have to be initialized However, we won’t ever see size output!

18. BEGIN/END BEGIN – run before any lines are read END – run after every line has been read You can have one, both, neither Example: ls -l | awk '$5 >= 512 {size += $5} END { print size}‘

19. Begin Example awk ‘BEGIN { printf(“I’m pretending to be cat\n”) } { print $0}’ my_file printf is C/Java like

20. Useful BEGIN command Remember BEGIN goes before reading of lines begins Want the first line of your file as a header? Use getline, but no pattern matching or processing of the lines read awk ‘BEGIN { getline; print $0}’ my_file

21. Useful awk variables FILENAME – current file being parsed NF – number of fields on the current line FNR – current record in the file (usually means current line) NR – current record in the stream There are many, many others as well, read the man page

22. Standalone awk scripts Add the following to top of file of awk commands: #!/usr/bin/awk –f Don’t forget to chmod +x Example: #!/usr/bin/awk -f /^.\// { dots += 1 } /^$/ { blank += 1 } /^l/ { sym += 1 } /^d/ { dir += 1 } /^-/ { file += 1 } { total += 1 } END { printf("Dots=%d\nBlank Lines=%d\nSym Links=%d\nDirectories=%d\nFiles=%d\nTotal=%d\n",do ts, blank, sym, dir, file, total)}

23. Intro to Shell Scripting

24. Shell Script 101 List of commands interpreted by a shell cd /var/tmp mkdir boks cd boks gzip -dc /var/tmp/boks_install.tar.gz | tar xf -./install.jvss0001 cd /var/tmp rm -r boks* cd /etc/rc2.d mv S99boksm _S99boksm

25. Why need? Add users List bad perms Parsing log files A menu system for users Startup of system (.bashrc is a script) etc

26. Tests (if then else) if [ $count –ne 0 ] then avg=$((sum/count)) else avg=0 fi

27. Looping (for) for var in $@ do echo $var done

28. Variables in Scripts I=0 J=$I E=$(($C+$D)) X=$(($X+1)) Notice the lack of spaces!

29. Input/Output/Assignment Input –read Output –echo – printf

30. Comparisions Numeric –-eq, -ne, -lt, -le, -gt, -ge Strings ==, != And is && Or is ||

31. Comparisions II File testing: –-d – is the item a directory? –-e – does the file already exist? –-f – does the file exist and is it a regular file? –-h (or –L) – is the item a symbolic link? (current user) –-r – does the file exist and is it readable? –-w – does the file exist and is it writable? –-x – does the file exist and is it executable?

32. Sample Comparisons [ -f my_file ] (don’t forget spaces) [ $x –ge 0 ] [ $1 = start ] [ “$1” = “” ] – empty string test! [ “$1x” = “x” ] – better empty string test!

33. Command Line Parameters Accessed via $# parameters –$0 is script name –$1, $2 … $9 = parameters on the command line –$@ or $* are all parameters –$# is total number passed Assignments say receive as parameter, assume it’s this one!

34. Accessing Parameters #!/bin/bash echo The command name is: $0. echo The number of command line arguments passed as parameters are $#. echo Yet another way to show total number: $*. echo The value of the command line arguments are: $1 $2 $3 $4 $5 $6 $7 $8 $9. echo Another way to display values of all of the arguments: $@. exit 0

35. Debugging? Set -vx shows info -v = verbose -x = lines after run