1. Lecture 9  sed

2. sed  sed is a stream-oriented editor the input (file/std input) flows through the program sed and is directed the standard output the input (file/std input) flows through the program sed and is directed the standard output Used primarily for non interactive operations Used primarily for non interactive operations  sed [-n] –f script_file file or sed [-n] `command` file sed executes the given command or script file that contain commands on each line of the input (file) sed executes the given command or script file that contain commands on each line of the input (file) -n: turn off default printing -n: turn off default printing

3. sed Commands  p: print line -n prevents lines from being printed twice -n prevents lines from being printed twice  d: delete line  s/old/new/: substitute old with new  s/old/new/g: substitute all occurrences of old with new  !: negates a command  Full list of commands can be found on page 129

4. sed Examples  sed p file.txt  sed –n p file.txt  sed d file.txt  sed \!d file.txt  p and d seem a bit worthless, don’t they? They purpose will become more clear when we discuss addresses.

5. sed: Substitution  The strongest feature of sed  Syntax is s/expression/string/flag expression is a regular expression expression is a regular expression string is a string string is a string  sed ‘s/|/:/’ data.txt substitute the character ‘|’ with the character ‘:’ substitute the character ‘|’ with the character ‘:’  sed ‘s/|/:/g’ data.txt

6. Some Useful Substitution Flags  g: global (replace all matches on the line).  p: print the line if a successful match sed ‘s/old/new/g’ file.txt sed ‘s/old/new/g’ file.txt sed ‘s/old/new/gp’ file.txt sed ‘s/old/new/gp’ file.txt sed –n ‘s/old/new/gp’ file.txt sed –n ‘s/old/new/gp’ file.txt

7. Regular Expressions for sed  The usual suspects ^, $,., *, [ ], [^ ], \( \), \ ^, $,., *, [ ], [^ ], \( \), \  A new operator &: the string which matches the expression &: the string which matches the expression can be used in the substitution stringcan be used in the substitution string s/hello/**&**/g replaces all occurrences of hello with **hello**s/hello/**&**/g replaces all occurrences of hello with **hello**

8. sed Addressing  So far, we have been applying sed commands to every line makes p and d not very useful makes p and d not very useful  With addressing, we can apply commands to some, but not all lines  sed can use 0 addresses (all lines) 0 addresses (all lines) 1 address (a single line) 1 address (a single line) 2 addresses (a range of lines) 2 addresses (a range of lines)  Address can be line numbers of context (defined by regular expressions)

9. Line Number Addressing Examples %sed –n ‘3,4p’ foo.txt Since sed prints each line anyway, if we only want lines 3 & 4 (instead of all lines with lines 3 & 4 duplicated) we use the –n %sed –n ‘$p’ foo.txt For each line, if that line is the last line, print %sed –n ‘3,$p’ foo.txt For each line, if that line is the third through last line, print

10. Context Addressing Examples  Use patterns/regular expressions rather than explicitly specifying line numbers %sed –n ‘/^From: /p’ $HOME/mbox retrieve all the sender lines from the mailbox file, i.e., for each line, if that line starts with ‘From’, print it. Note that the / / mark the beginning and end of the pattern to match retrieve all the sender lines from the mailbox file, i.e., for each line, if that line starts with ‘From’, print it. Note that the / / mark the beginning and end of the pattern to match %ls –l | sed –n ‘/^.....w/p’ For each line, if the sixth character is a W, print For each line, if the sixth character is a W, print

11. Context Ranges  sed ‘/hello/,/there/d’ file.txt delete all lines that occur between a line that matches hello and a line that matches there. The hello and there lines are also removed. delete all lines that occur between a line that matches hello and a line that matches there. The hello and there lines are also removed.  Multiple contexts are possible two contexts specified by a single range two contexts specified by a single range

12. sed Addressing  Using a ! after the address means all lines which do not match the address sed ‘1\!d’ test.txt sed ‘1\!d’ test.txt

13. Example file northwestNWCharles Main 3.0.98334 westernWESharon Gray5.3.97523 southwestSWLewis Dalsass2.7.8218 southernSOSuan Chin5.1.95415 southeastSEPatricia Heme4.0.7417 easternEATB Savage4.4.84520 northeastNEAM Main Jr.5.1.94313 northNOMargot Webber4.5.8959 centralCTAnn Stephens5.7.94513 sed ‘/north/p’ filesed –n ‘s/west/north/g’ file sed ‘3,$d’ filesed ‘s/\(Mar\)got/\1ianne/p’ file sed ‘s/west/north/g’ filesed ‘/west/,/east/s/$/**VACA**/’ file sed '/north/a\ hello,word' datafile hello,word' datafile

14. sed: Using files  Tedious to type in commands at the prompt, especially if commands are repetitive  Can put commands in a file and sed can use them  sed –f cmds.sed data.txt file with commands

15. sed scripts  Series of commands can be put in a file and use the ‘-f’ option.  Can also create an sed script: s/vi/emacs/g /[Ww]indows/d p

16. Another Example  sed script to remove all HTML tags from a file: s/ ]*>//g p

17. Reading Assignment  Chapter 5