Download presentation
Presentation is loading. Please wait.
1
CSC 352– Unix Programming, Spring 2015 April 28 A few final commands
2
Based on student assn 3 solutions These are some bash shell mechanisms and Unix utilities used by students in assignment 3 that can be very useful. I had forgotten about some of these. Some may be buried in or missing from the textbook. I always learn or re-learn when teaching Unix.
3
bc – the “bench calculator” It reads standard input, not command line. Following POSIX bc operators behave exactly like their C counterparts: + - * / += -= *= /= ++ -- == != = ( ) [ ] { } GNU bc has others, but they are not portable.
4
Some bc examples [:-) ~] echo '1.1 + 20 * -2.0' | bc -38.9 [:-) ~] bc << EOF 1.1 + 20 * -2.0 EOF -38.9 Latter is a “here document” for supplying literal text to standard input. Note that 0 means false and non-0 means true in a bc expression, just like C & C++, but opposite of shell exit status.
![Some bc examples [:-) ~] echo * -2.0 | bc [:-) ~] bc << EOF * -2.0 EOF Latter is a here document for supplying literal text to standard input.](http://images.slideplayer.com/25/8132884/slides/slide_4.jpg "Note that 0 means false and non-0 means true in a bc expression, just like C & C++, but opposite of shell exit status..")
5
Some pitfalls in = assignment [:-) ~] expression='1.1 + 20 * -2.0' [:-) ~] echo $expression 1.1 + 20 bignum.py... FILES... -2.0 – Glob matching captures the * [:-) ~] expression='1.1 + 20 \* -2.0' [:-) ~] echo $expression 1.1 + 20 \* -2. – The backslash is now part of the string.
![Some pitfalls in = assignment [:-) ~] expression= * -2.0 [:-) ~] echo $expression bignum.py...](http://images.slideplayer.com/25/8132884/slides/slide_5.jpg "FILES – Glob matching captures the * [:-) ~] expression= \* -2.0 [:-) ~] echo $expression \* -2. – The backslash is now part of the string..")
6
More pitfalls in = assignment expression="1.1 + 20 * -2.0” glob matches expression="1.1 + 20 \* -2.0” captures \ [:-) ~] expression=1.1 + 20 \* -2.0 -bash: +: command not found [:-) ~] set –f # SHUT OFF GLOBBING [:-) ~] ls * ls: cannot access *: No such file or directory [:-) ~] expression='1.1 + 20 * -2.0' [:-) ~] echo $expression 1.1 + 20 * -2.0 [:-) ~] echo $expression | bc -38.9
![More pitfalls in = assignment expression= * -2.0 glob matches expression= \* -2.0 captures \ [:-) ~] expression= \* bash: +: command not found [:-) ~] set –f # SHUT OFF GLOBBING [:-) ~] ls * ls: cannot access *: No such file or directory [:-) ~] expression= * -2.0 [:-) ~] echo $expression * -2.0 [:-) ~] echo $expression | bc -38.9](http://images.slideplayer.com/25/8132884/slides/slide_6.jpg "More pitfalls in = assignment expression= * -2.0 glob matches expression= \* -2.0 captures \ [:-) ~] expression= \* bash: +: command not found [:-) ~] set –f # SHUT OFF GLOBBING [:-) ~] ls * ls: cannot access *: No such file or directory [:-) ~] expression= * -2.0 [:-) ~] echo $expression * -2.0 [:-) ~] echo $expression | bc -38.9")
7
“No spaces” works without -f [:-) ~] expression='1.1+20*-2.0' [:-) ~] echo $expression 1.1+20*-2.0 [:-) ~] echo $expression | bc -38.9
![No spaces works without -f [:-) ~] expression= *-2.0 [:-) ~] echo $expression *-2.0 [:-) ~] echo $expression | bc -38.9](http://images.slideplayer.com/25/8132884/slides/slide_7.jpg "No spaces works without -f [:-) ~] expression= *-2.0 [:-) ~] echo $expression *-2.0 [:-) ~] echo $expression | bc -38.9")
8
[[ EXPRESSION ]] [[ expression ]] Return a status of 0 or 1 depending on the evaluation of the conditional expression expression. Expressions are composed of the primaries described below under CONDITIONAL EXPRESSIONS. Word splitting and pathname expansion are not performed on the words between the [[ and ]]; tilde expansion, parameter and variable expansion, arithmetic expansion, command substitution, process substitution, and quote removal are performed. Conditional operators such as -f must be unquoted to be recognized as primaries.
![[[ EXPRESSION ]] [[ expression ]] Return a status of 0 or 1 depending on the evaluation of the conditional expression expression.](http://images.slideplayer.com/25/8132884/slides/slide_8.jpg "Expressions are composed of the primaries described below under CONDITIONAL EXPRESSIONS. Word splitting and pathname expansion are not performed on the words between the [[ and ]]; tilde expansion, parameter and variable expansion, arithmetic expansion, command substitution, process substitution, and quote removal are performed. Conditional operators such as -f must be unquoted to be recognized as primaries..")
9
[[ … ]] examples http://www.gnu.org/software/bash/manual/b ashref.html#Conditional-Constructs http://www.gnu.org/software/bash/manual/b ashref.html#Conditional-Constructs [:-) ~] [[ a -lt 4 && a -gt 2 ]] [:-) ~] echo $? 0 You can use logical connectives && and || inside the test brackets.
![[[ … ]] examples ashref.html#Conditional-Constructs ashref.html#Conditional-Constructs [:-) ~] [[ a -lt 4 && a -gt 2 ]] [:-) ~] echo $.](http://images.slideplayer.com/25/8132884/slides/slide_9.jpg "0 You can use logical connectives && and || inside the test brackets..")
10
[[ … ]] regular expression matching [:-) ~] number='^(([0-9]+(\.[0-9]*)?)|([0-9]*\.[0-9]+))$’ The \. Is necessary to match literal “.” --. by itself matches any char. [:-) ~] [[ 1 =~ $number ]] ; echo $? 0 [:-) ~] [[ 1. =~ $number ]] ; echo $? 0 [:-) ~] [[ 1.1 =~ $number ]] ; echo $? 0 [:-) ~] [[.1 =~ $number ]] ; echo $? 0 [:-) ~] [[ 1f =~ $number ]] ; echo $? 1
![[[ … ]] regular expression matching [:-) ~] number= ^(([0-9]+(\.[0-9]*) )|([0-9]*\.[0-9]+))$’ The \.](http://images.slideplayer.com/25/8132884/slides/slide_10.jpg "Is necessary to match literal by itself matches any char. [:-) ~] [[ 1 =~ $number ]] ; echo $. 0 [:-) ~] [[ 1. =~ $number ]] ; echo $. 0 [:-) ~] [[ 1.1 =~ $number ]] ; echo $. 0 [:-) ~] [[.1 =~ $number ]] ; echo $. 0 [:-) ~] [[ 1f =~ $number ]] ; echo $. 1.")
11
egrep (extended grep) patterns pattern *0 or more occurrences of previous character or (expression) pattern +1 or more occurrences of previous character or (expression) pattern ?0 or 1 occurrence of previous character or (expression) pattern.any single character pattern [pqr]any single character from set p, q or r pattern [a-zA-Z] any single character from range a-z or A-Z pattern [^pqr] any single character *not* from set p, q or r pattern ( EXPR )is for subexpression grouping pattern P1 | P2is for pattern P1 or P2 pattern ^the start of the string being searched for a match pattern $the end of the string being searched for a match pattern \escapes the next character so it is treated as a regular char These are the most useful regular expression patterns available to grep, sed, [[ $string =~ $pattern ]] tests, and for searching in emacs and vi. The shell uses so-called “glob-style matching” for strings (*.java), which differ from regular expressions (.*\.java) used by grep, sed, emacs and vi.
![egrep (extended grep) patterns pattern *0 or more occurrences of previous character or (expression) pattern +1 or more occurrences of previous character or (expression) pattern 0 or 1 occurrence of previous character or (expression) pattern.any single character pattern [pqr]any single character from set p, q or r pattern [a-zA-Z] any single character from range a-z or A-Z pattern [^pqr] any single character *not* from set p, q or r pattern ( EXPR )is for subexpression grouping pattern P1 | P2is for pattern P1 or P2 pattern ^the start of the string being searched for a match pattern $the end of the string being searched for a match pattern \escapes the next character so it is treated as a regular char These are the most useful regular expression patterns available to grep, sed, [[ $string =~ $pattern ]] tests, and for searching in emacs and vi.](http://images.slideplayer.com/25/8132884/slides/slide_11.jpg "The shell uses so-called glob-style matching for strings (*.java), which differ from regular expressions (.*\.java) used by grep, sed, emacs and vi..")
12
Regular expression examples p.1 [:-) ~/unix] cat re1.txt abcdefg 1234567 123.456 123abc !@#$%^&*() ABCDEFGHIJKLM AaAaBbVv [:-) ~/unix] egrep '[0-9]+' re1.txt 1234567 123.456 123abc
![Regular expression examples p.1 [:-) ~/unix] cat re1.txt abcdefg abc ABCDEFGHIJKLM AaAaBbVv [:-) ~/unix] egrep [0-9]+ re1.txt abc](http://images.slideplayer.com/25/8132884/slides/slide_12.jpg "Regular expression examples p.1 [:-) ~/unix] cat re1.txt abcdefg abc ABCDEFGHIJKLM AaAaBbVv [:-) ~/unix] egrep [0-9]+ re1.txt abc")
13
Regular expression examples p.2 [:-) ~/unix] egrep '^[0-9]+$' re1.txt 1234567 [:-) ~/unix] egrep '^[0-9]*$' re1.txt 1234567 [:-) ~/unix] egrep '[0-9]*' re1.txt abcdefg 1234567 123.456 123abc !@#$%^&*() ABCDEFGHIJKLM AaAaBbVv
![Regular expression examples p.2 [:-) ~/unix] egrep ^[0-9]+$ re1.txt [:-) ~/unix] egrep ^[0-9]*$ re1.txt [:-) ~/unix] egrep [0-9]* re1.txt abcdefg abc ABCDEFGHIJKLM AaAaBbVv](http://images.slideplayer.com/25/8132884/slides/slide_13.jpg "Regular expression examples p.2 [:-) ~/unix] egrep ^[0-9]+$ re1.txt [:-) ~/unix] egrep ^[0-9]*$ re1.txt [:-) ~/unix] egrep [0-9]* re1.txt abcdefg abc ABCDEFGHIJKLM AaAaBbVv")
14
Regular expression examples p.3 [:-) ~/unix] egrep '[0-9]+.?[0-9]*' re1.txt 1234567 123.456 123abc [:-) ~/unix] egrep '^[0-9]+.?[0-9]*$' re1.txt 1234567 123.456 [:-) ~/unix] egrep '^[0-9]+[^0-9][0-9]*$' re1.txt 123.456 [:-) ~/unix] egrep '^[0-9]+[^0-9]?[0-9]*$' re1.txt 1234567 123.456 [:-) ~/unix] egrep '^[0-9]+[^0-9]?[A-Fabcdef]+$' re1.txt 123abc
![Regular expression examples p.3 [:-) ~/unix] egrep [0-9]+. [0-9]* re1.txt abc [:-) ~/unix] egrep ^[0-9]+. [0-9]*$ re1.txt [:-) ~/unix] egrep ^[0-9]+[^0-9][0-9]*$ re1.txt [:-) ~/unix] egrep ^[0-9]+[^0-9] [0-9]*$ re1.txt [:-) ~/unix] egrep ^[0-9]+[^0-9] [A-Fabcdef]+$ re1.txt 123abc](http://images.slideplayer.com/25/8132884/slides/slide_14.jpg "Regular expression examples p.3 [:-) ~/unix] egrep [0-9]+. [0-9]* re1.txt abc [:-) ~/unix] egrep ^[0-9]+. [0-9]*$ re1.txt [:-) ~/unix] egrep ^[0-9]+[^0-9][0-9]*$ re1.txt [:-) ~/unix] egrep ^[0-9]+[^0-9] [0-9]*$ re1.txt [:-) ~/unix] egrep ^[0-9]+[^0-9] [A-Fabcdef]+$ re1.txt 123abc")
15
Regular expression examples p.4 [:-) ~/unix] egrep '(([0-9]+)|([A-Za-z]+))$' re1.txt abcdefg 1234567 123.456 123abc ABCDEFGHIJKLM AaAaBbVv [:-) ~/unix] egrep '[^A-Za-z0-9]+$' re1.txt !@#$%^&*()
![Regular expression examples p.4 [:-) ~/unix] egrep (([0-9]+)|([A-Za-z]+))$ re1.txt abcdefg abc ABCDEFGHIJKLM AaAaBbVv [:-) ~/unix] egrep [^A-Za-z0-9]+$ re1.txt](http://images.slideplayer.com/25/8132884/slides/slide_15.jpg "Regular expression examples p.4 [:-) ~/unix] egrep (([0-9]+)|([A-Za-z]+))$ re1.txt abcdefg abc ABCDEFGHIJKLM AaAaBbVv [:-) ~/unix] egrep [^A-Za-z0-9]+$ re1.txt")
16
Regular expression examples p.5 [:-) ~/unix] egrep '[^A-Za-z0-9]?$' re1.txt abcdefg 1234567 123.456 123abc !@#$%^&*() ABCDEFGHIJKLM AaAaBbVv [:-) ~/unix] egrep '[^A-Za-z0-9]*$' re1.txt abcdefg 1234567 123.456 123abc !@#$%^&*() ABCDEFGHIJKLM AaAaBbVv
![Regular expression examples p.5 [:-) ~/unix] egrep [^A-Za-z0-9] $ re1.txt abcdefg abc ABCDEFGHIJKLM AaAaBbVv [:-) ~/unix] egrep [^A-Za-z0-9]*$ re1.txt abcdefg abc ABCDEFGHIJKLM AaAaBbVv](http://images.slideplayer.com/25/8132884/slides/slide_16.jpg "Regular expression examples p.5 [:-) ~/unix] egrep [^A-Za-z0-9] $ re1.txt abcdefg abc ABCDEFGHIJKLM AaAaBbVv [:-) ~/unix] egrep [^A-Za-z0-9]*$ re1.txt abcdefg abc ABCDEFGHIJKLM AaAaBbVv")
17
sed writes to standard output, sed –r uses extended Res p.1 [:-) ~/unix] egrep '[^A-Za-z0-9]+$' re1.txt !@#$%^&*() [:-) ~/unix] sed -e 's/[^A-Za-z0-9]+$/AllFixedUp/g' re1.txt abcdefg 1234567 123.456 123abc !@#$%^&*() ABCDEFGHIJKLM AaAaBbVv
![sed writes to standard output, sed –r uses extended Res p.1 [:-) ~/unix] egrep [^A-Za-z0-9]+$ re1.txt [:-) ~/unix] sed -e s/[^A-Za-z0-9]+$/AllFixedUp/g re1.txt abcdefg abc ABCDEFGHIJKLM AaAaBbVv](http://images.slideplayer.com/25/8132884/slides/slide_17.jpg "sed writes to standard output, sed –r uses extended Res p.1 [:-) ~/unix] egrep [^A-Za-z0-9]+$ re1.txt [:-) ~/unix] sed -e s/[^A-Za-z0-9]+$/AllFixedUp/g re1.txt abcdefg abc ABCDEFGHIJKLM AaAaBbVv")
18
sed writes to standard output, sed –r uses extended Res p.1 [:-) ~/unix] sed -r -e 's/[^A-Za-z0-9]+$/AllFixedUp/g' re1.txt abcdefg 1234567 123.456 123abc AllFixedUp ABCDEFGHIJKLM AaAaBbVv sed does NOT modify the source file. It sends to stdout but you could: – sed -r -e 's/[^A-Za-z0-9]+$/AllFixedUp/g' re1.txt > tmpfile.txt – mv tmpfile.txt re1.txt
![sed writes to standard output, sed –r uses extended Res p.1 [:-) ~/unix] sed -r -e s/[^A-Za-z0-9]+$/AllFixedUp/g re1.txt abcdefg abc AllFixedUp ABCDEFGHIJKLM AaAaBbVv sed does NOT modify the source file.](http://images.slideplayer.com/25/8132884/slides/slide_18.jpg "It sends to stdout but you could: – sed -r -e s/[^A-Za-z0-9]+$/AllFixedUp/g re1.txt > tmpfile.txt – mv tmpfile.txt re1.txt.")
19
[[ =~ ]] pattern matching [:-) ~/unix] pattern='^[0-9]+.?[a-z]*$' [:-) ~/unix] echo $pattern ^[0-9]+.?[a-z]*$ [:-) ~/unix] for line in `cat re1.txt` > do > if [[ $line =~ $pattern ]] # [[ ! $line =~ $pattern ]] for non-matching lines > then > echo "MATCH $line" > fi > done MATCH 1234567 MATCH 123abc
![[[ =~ ]] pattern matching [:-) ~/unix] pattern= ^[0-9]+. [a-z]*$ [:-) ~/unix] echo $pattern ^[0-9]+. [a-z]*$ [:-) ~/unix] for line in `cat re1.txt` > do > if [[ $line =~ $pattern ]] # [[ .](http://images.slideplayer.com/25/8132884/slides/slide_19.jpg "$line =~ $pattern ]] for non-matching lines > then > echo MATCH $line > fi > done MATCH MATCH 123abc.")
Similar presentations
![1 Operating Systems Lecture 3 Shell Scripts. 2 Brief review of unix1.txt n Glob Construct (metacharacters) and other special characters F ?, *, [] F Ex.](/24/6946865/big_thumb.jpg "1 Operating Systems Lecture 3 Shell Scripts. 2 Brief review of unix1.txt n Glob Construct (metacharacters) and other special characters F ?, *, [] F Ex.>")
