I/O Redirection & Regular Expressions CS 2204 Class meeting 4 *Notes by Doug Bowman and other members of the CS faculty at Virginia Tech. Copyright

(C) Doug Bowman, Virginia Tech, Redirecting stdout Instead of writing to the terminal, you can tell a program to print its output to another file using the > operator >> operator is used to append to a file Examples: man ls > ls_help.txt Echo $PWD > current_directory cat file1 >> file2

(C) Doug Bowman, Virginia Tech, Redirecting stdin Instead of reading from the terminal, you can tell a program to read from another file using the < operator Examples: Mail < message interactive_program < command_list

(C) Doug Bowman, Virginia Tech, Pipes and filters Pipe: a way to send the output of one command to the input of another Filter: a program that takes input and transforms it in some way wc - gives a count of words/lines/chars grep - searches for lines with a given string more sort - sorts lines alphabetically or numerically

(C) Doug Bowman, Virginia Tech, Examples of filtering ls -la | more cat file | wc man ksh | grep “history” ls -l | grep “gifford” | wc who | sort > current_users

(C) Doug Bowman, Virginia Tech, What is a regular expression (RE)? A pattern Defines a set of strings of characters Any string in the set is said to be “matched” by the RE (the RE matches the string)

(C) Doug Bowman, Virginia Tech, Why REs? Pattern matching is a useful tool in many real- world situations: Search for a file on a filesystem Find and replace strings in a file Extract particular data elements from a database REs are an important part of formal languages - one of the basic CS theory disciplines

(C) Doug Bowman, Virginia Tech, UNIX programs that use REs grep (search within files) egrep ( grep but with extended RE’s) vi/emacs (text editors) ex (line editor) sed (stream editor) awk (pattern scanning language) perl (scripting language)

(C) Doug Bowman, Virginia Tech, Characters vs. metacharacters In patterns, characters can be any character except a newline Metacharacters are special characters that have a special meaning To use metacharacters as regular characters in a pattern, quote them with the ‘\’ character

(C) Doug Bowman, Virginia Tech, Basic vs. Extended RE’s In basic regular expressions the metacharacters ?, +, {, |, (, and ) have no special meaning ( grep). To make them special, instead use the backslashed versions \?, \+, \{, \|, \(, and \) For extended regular expressions these are interpreted as special Grep –E  egrep

(C) Doug Bowman, Virginia Tech, Using egrep egrep pattern filename(s) To be safe, put quotes around your pattern Examples: egrep “abc” file.txt (print lines containing “abc”) egrep -i “abc” file.txt (same, but ignore case) egrep -v “abc” file.txt (print lines not containing “abc”) egrep -n “abc” file.txt (include line numbers)

(C) Doug Bowman, Virginia Tech, Metacharacters 1 Period (.): matches any single character “a.c” matches abc, adc, a&c, a;c, … “u..x” matches unix, uvax, u3(x, … Asterisk (*) matches zero or more occurrences of the previous RE Not the same as wildcards in the shell! “ab*c” matches ac, abc, abbc, abbbc, … “.*” matches any string

(C) Doug Bowman, Virginia Tech, Metacharacters 2 Plus (+): matches one or more occurrences of the preceding RE “ab+c” matches abc, abbc, but not ac Question mark (?): matches zero or one occurrences of the preceding RE “ab?c” matches ac, abc but not abbc Logical or (|): matches RE before or RE after bar “abc|def” matches abc or def

(C) Doug Bowman, Virginia Tech, Metacharacters 3 Caret (^): means beginning of line “^D” matches all strings starting with D Dollar sign ($) means end of line “d$” matches all strings ending with d Backslash (\): used to quote other metacharacters “file\.txt” matches file.txt but not fileatxt

(C) Doug Bowman, Virginia Tech, Metacharacters 4 Square brackets ([ ]) indicate a set/range of characters Any characters in the set will match ^ before the set means “not” - between characters indicates a range Examples: “[fF]un” matches fun, Fun “b[aeiou]g” matches bag, beg, big, bog, bug “[A-Z].*” matches any string beginning with a captial letter “[^abc].*” matches any string not starting with a, b, or c

(C) Doug Bowman, Virginia Tech, Metacharacters 5 Parentheses ( ): used to group REs when using other metacharacters “a(bc)*” matches a, abc, abcbc, abcbcbc, … “(foot|base)ball” matches football, baseball Braces ({ }): specify the number of repetitions of an RE “[a-z]{3}” matches three lowercase letters “m.{2,4}” matches strings starting with m between three and five letters

(C) Doug Bowman, Virginia Tech, What do these mean? egrep “^B.*s$” file egrep “ [0-9]{3} ” file egrep “num(ber)? [0-9]+” file egrep “word” file | wc -l egrep “[A-Z].*\?” file What about if they were grep?

(C) Doug Bowman, Virginia Tech, Practice Construct egrep commands that find in file : Lines beginning with a word of at least 10 characters Lines containing a student ID number in standard 3- part form Number of lines with 2 consecutive capitalized words Number of lines not ending in an alphabetic character Lines containing a word beginning with a vowel at the end of a sentence