1. Lecture 5  Regular Expressions;  grep; CSE4251 The Unix Programming Environment

2. Regular Expressions (R.E.) What’s R.E. – a set of symbols/rules to describe a string pattern e.g., strings a, aa, aaa, aaaa,.... can be described simply as a+ – facilitate search/replace/delete... Unix commands/utilities that support regular expressions: – grep(fgrep, egrep) - search a file for a string or regular expression – sed - stream editor – awk (nawk) - pattern scanning and processing language

3. Regular Expression Symbols.any single character *0 or more instances of previous character e.g., a* matches <>, a, aa, aaa, aaaa,... ?0 or 1 instance of previous character + 1 or more instances of previous character ^beginning of a line e.g., ^A matches A only if A is at the beginning of a line $end of a line e.g., A$ matches A only if A is at the end of a line \Turn off the special meaning

4. […] matches any of the enclosed characters – [abc] matches a single a b or c – [a-z] matches any of abcdef…xyz [^…] matches any thing not included – [^A-Za-z] matches a single character as long as it is not a letter. Regular Expression Symbols

5. \<beginning of word e.g., \<abc matches “abcd” but not “dabc” \>end of word e.g., abc\> matches “dabc” but not “abcd” \(…\) stores the pattern inside \( and \) e.g., \(abc\)def matches “abcdef” and stores abc in \1. So \(abc\)def\1 matches “abcdefabc”. Can store up to 9 matches e.g., \(ab\)c\(de\)f\1\2 will match abcdefabde Regular Expression Symbols

6. X{n} The preceding item X is matched exactly n times. X{n,} The preceding item X is matched n or more times. X{n,m} The preceding item X is matched at least n times, but not more than m times. Repeat

7. can put a subpattern inside (), and apply +, *, and ? to the entire subpattern. e.g.: a(bc)*d matches “ad”, "abcbcd”, “abcbcbcbcbcbcd”,... Subpattern

8. More examples x[abc]?xmatches "xax" or "xx“ [abc]* matches "aaaaa" or "acbca" 0*10 matches "010" or "0000010"or "10“ [Dd][Aa][Vv][Ee] – Matches "Dave" or "dave" or "dAVE", – Does not match "ave" or "da"

9. Review: metacharacters for filename abbreviation (lecture 2) Metacharacters: * ? [ ] ~ * matches anything $ ls *.doc # list all files ending with.doc ? matches single character $ ls ?.doc # list a.doc, b.doc, but not ab.doc […] matches any of the enclosed charactors $ ls [ab].doc # only list a.doc or b.doc $ ls *[cx].doc # list files ending with c.doc or x.doc ~ is a shortcut of your home directory $ cd ~ # change to home directory 9

10. Difference Interpreted by different programs – filename expansion is done by the shell. – regular expressions are used by commands (programs). Be careful when specifying R.E. on the command line – Good idea to always quote R.E. with special chars (‘’or “”)on the command line – Example: $ grep ‘[a-z]*’ somefile.txt

11. grep - search for a string grep [-bchilnsvw] PATTERN [filename...] – Read files or standard /redirected input – Search for specified pattern in each line – Send results to the standard output Examples: $ grep ‘^X11’ *- search all files for lines starting with the string “X11” $ grep -v text file - print lines that do not match “text ” Exit status – 0 – pattern found; 1 - not found

12. grep - options Some useful options -c count number of lines -i ignore case -l list only the files with matching lines -L list files that dose not match -v display lines that do not match -n print line numbers -r recursively search the sub-directories

13. Regular Expressions for grep \Xturn off any special meaning of character X ^beginning of line $end of line.any single character [...]any of characters in range.… [^....]any single character not in range.… r*zero or more occurrences of r

14. grep with pipes we can use pipes when a file is expected e.g., $ ls –l | grep a*

15. gamefile northwestNWCharles Main 3.0.98334 westernWESharon Gray5.3.97523 southwestSWLewis Dalsass2.7.8218 southernSOSuan Chin5.1.95415 southeastSEPatricia Heme4.0.7417 easternEATB Savage4.4.84520 northeastNEAM Main Jr.5.1.94313 northNOMargot Webber4.5.8959 centralCTAnn Stephens5.7.94513

16. gamefile northwestNWCharles Main 3.0.98334 westernWESharon Gray5.3.97523 southwestSWLewis Dalsass2.7.8218 southernSOSuan Chin5.1.95415 southeastSEPatricia Heme4.0.7417 easternEATB Savage4.4.84520 northeastNEAM Main Jr.5.1.94313 northNOMargot Webber4.5.8959 centralCTAnn Stephens5.7.94513 $ grep NW gamefile northwest NW Charles Main 3.0.98 3 34

17. gamefile northwestNWCharles Main 3.0.98334 westernWESharon Gray5.3.97523 southwestSWLewis Dalsass2.7.8218 southernSOSuan Chin5.1.95415 southeastSEPatricia Heme4.0.7417 easternEATB Savage4.4.84520 northeastNEAM Main Jr.5.1.94313 northNOMargot Webber4.5.8959 centralCTAnn Stephens5.7.94513 $ grep '^n' gamefile northwest NW Charles Main 3.0.98 3 34 northeast NE AM Main Jr. 5.1.94 3 13 north NO Margot Webber 4.5.89 5 9

18. gamefile northwestNWCharles Main 3.0.98334 westernWESharon Gray5.3.97523 southwestSWLewis Dalsass2.7.8218 southernSOSuan Chin5.1.95415 southeastSEPatricia Heme4.0.7417 easternEATB Savage4.4.84520 northeastNEAM Main Jr.5.1.94313 northNOMargot Webber4.5.8959 centralCTAnn Stephens5.7.94513 $ grep '4$' gamefile northwest NW Charles Main 3.0.98 3 34

19. gamefile northwestNWCharles Main 3.0.98334 westernWESharon Gray5.3.97523 southwestSWLewis Dalsass2.7.8218 southernSOSuan Chin5.1.95415 southeastSEPatricia Heme4.0.7417 easternEATB Savage4.4.84520 northeastNEAM Main Jr.5.1.94313 northNOMargot Webber4.5.8959 centralCTAnn Stephens5.7.94513 $ grep TB Savage gamefile grep: Savage: No such file or directory gamefile:eastern EA TB Savage 4.4.84 5 20

20. gamefile northwestNWCharles Main 3.0.98334 westernWESharon Gray5.3.97523 southwestSWLewis Dalsass2.7.8218 southernSOSuan Chin5.1.95415 southeastSEPatricia Heme4.0.7417 easternEATB Savage4.4.84520 northeastNEAM Main Jr.5.1.94313 northNOMargot Webber4.5.8959 centralCTAnn Stephens5.7.94513 $ grep -l 'SE' * gamefile

21. northwestNWCharles Main 3.0.98334 westernWESharon Gray5.3.97523 southwestSWLewis Dalsass2.7.8218 southernSOSuan Chin5.1.95415 southeastSEPatricia Heme4.0.7417 easternEATB Savage4.4.84520 northeastNEAM Main Jr.5.1.94313 northNOMargot Webber4.5.8959 centralCTAnn Stephens5.7.94513 $ grep '5\..' gamefile western WE Sharon Gray 5.3.97 5 23 southern SO Suan Chin 5.1.95 4 15 northeast NE AM Main Jr. 5.1.94 3 13 central CT Ann Stephens 5.7.94 5 13

22. gamefile northwestNWCharles Main 3.0.98334 westernWESharon Gray5.3.97523 southwestSWLewis Dalsass2.7.8218 southernSOSuan Chin5.1.95415 southeastSEPatricia Heme4.0.7417 easternEATB Savage4.4.84520 northeastNEAM Main Jr.5.1.94313 northNOMargot Webber4.5.8959 centralCTAnn Stephens5.7.94513 $ grep '\<north' gamefile northwest NW Charles Main 3.0.98 3 34 northeast NE AM Main Jr. 5.1.94 3 13 north NO Margot Webber 4.5.89 5 9

23. gamefile northwestNWCharles Main 3.0.98334 westernWESharon Gray5.3.97523 southwestSWLewis Dalsass2.7.8218 southernSOSuan Chin5.1.95415 southeastSEPatricia Heme4.0.7417 easternEATB Savage4.4.84520 northeastNEAM Main Jr.5.1.94313 northNOMargot Webber4.5.8959 centralCTAnn Stephens5.7.94513 $ grep '\ ' gamefile north NO Margot Webber 4.5.89 5 9

24. gamefile northwestNWCharles Main 3.0.98334 westernWESharon Gray5.3.97523 southwestSWLewis Dalsass2.7.8218 southernSOSuan Chin5.1.95415 southeastSEPatricia Heme4.0.7417 easternEATB Savage4.4.84520 northeastNEAM Main Jr.5.1.94313 northNOMargot Webber4.5.8959 centralCTAnn Stephens5.7.94513 $ grep -v "Suan Chin" gamefile northwest NW Charles Main 3.0.98 3 34 western WE Sharon Gray 5.3.97 5 23 southwest SW Lewis Dalsass 2.7.8 2 18 southeast SE Patricia Heme 4.0.7 4 17 eastern EA TB Savage 4.4.84 5 20 northeast NE AM Main Jr. 5.1.94 3 13 north NO Margot Webber 4.5.89 5 9 central CT Ann Stephens 5.7.94 5 13

25. gamefile northwestNWCharles Main 3.0.98334 westernWESharon Gray5.3.97523 southwestSWLewis Dalsass2.7.8218 southernSOSuan Chin5.1.95415 southeastSEPatricia Heme4.0.7417 easternEATB Savage4.4.84520 northeastNEAM Main Jr.5.1.94313 northNOMargot Webber4.5.8959 centralCTAnn Stephens5.7.94513 $ grep -c 'west' gamefile 3

26. gamefile northwestNWCharles Main 3.0.98334 westernWESharon Gray5.3.97523 southwestSWLewis Dalsass2.7.8218 southernSOSuan Chin5.1.95415 southeastSEPatricia Heme4.0.7417 easternEATB Savage4.4.84520 northeastNEAM Main Jr.5.1.94313 northNOMargot Webber4.5.8959 centralCTAnn Stephens5.7.94513 $ grep -i "$LOGNAME" /etc/passwd zhengm:x:503:504::/home/zhengm:/bin/bash