Introduction to sed

Sed : a “S tream ED itor ” What is Sed ?  A “non-interactive” text editor that is called from the unix command line.  Input text flows through the program, is modified, and is directed to standard output.

Sed : a “S tream ED itor ” An Example: The following sentence is input to the sed program: echo "Instrumental in ruining entire operation for a Midwest chain operation." | sed 's/ruining/running/‘ Instrumental in running entire operation for a Midwest chain operation.

sed: string replacement  Line-oriented tool for pattern matching and replacement (stream editor)  Not really a programming language (cf. awk)  E.g., apply same change to lots of source files  Filter, i.e., does not modify input file

sed description  pattern a text  add to output  address s /regex/replacement/  address d  delete line  delete lines 1-10: sed -e '1,10d‘  delete comments: sed -e '/^#/d‘  print only matching: sed -n -e '/regexp/p  convert Unix to DOS: sed -e 's/$/\r/' myunix.txt > mydos.txt

 Eliminate the tedium of routine editing tasks! (find, replace, delete, append, insert) … but your word processor can already do that right? Wrong. Sed is extremely powerful AND comes with every Unix system in the world! Why Use Sed?

Sed is designed to be especially useful in three cases: To edit files too large for comfortable interactive editing; To edit any size file when the sequence of editing commands is too complicated to be comfortably typed in interactive mode. To perform multiple `global' editing functions efficiently in one pass through the input. Why Use Sed?

How Sed Works (cont…) echo “amy enjoys hiking and ben enjoys skiing” | sed –e ‘s/skiing/hiking/g; s/hiking/biking/g’ 1 ) Sed read in the line “amy enjoys hiking and ben enjoys skiing” and executed the first ‘substitute’ command. The resulting line – in the pattern space: “amy enjoys hiking and ben enjoys hiking” 2) Then the second substitute command is executed on the line in the pattern space, and the result is : “amy enjoys biking and ben enjoys biking” 3) And the result is written to standard out.

Invoking Sed Commands $ sed [-e script] [-f script-file] [-n] [files...] -ean "in-line" script, i.e. a script to sed execute given on the command line. Multiple command line scripts can be given, each with an -e option. -nby default, sed writes each line to stdout when it reaches the end of the script (being whatever on the line) this option prevents that. i.e. no output unless there is a command to order SED specifically to do it -fread scripts from specified file, several -f options can appear files are the files to read, if a "-" appears, read from stdin,if no files are given, read also from stdin

Invoking Sed Commands Different Ways to Invoke Sed: sed –e 'command;command;command' input_file see results sed –e 'command;command;command' input_file > output_file save results.... | sed –e 'command;command;command' |.... use in a pipeline sed -f sedcommands input_file > output_file commands are in file somewhere else

Invoking Sed (some notes) sed commands are usually on one line if we want more (multi-line commands), then we must end the first line with an `\' if a command is one line only, it can be separated by a `;‘ if it is a multi-line, then it must contain all of its line (except the first) by themselves on command line, what follows a `-e' is like a whole line in a sed script

Regular Expressions Sed uses regular expressions to match patterns in the input text, and then perform operations on those patterns. ^matches the beginning of the line $ matches the end of the line.Matches any single character \Escapes any metacharacter that follows, including itself. (character)*Match arbitrarily many occurences of (character) (character)?Match 0 or 1 instance of (character) (character)+Match 1 or more instances of (character) [abcdef]Match any character enclosed in [ ] (in this instance, a b c d e or f) [^abcdef]Match any character NOT enclosed in [ ] (character)\{m,n\}Match m-n repetitions of (character) (character)\{m,\}Match m or more repetitions of (character) (character)\{,n\}Match n or less (possibly 0) repetitions of (character) (character)\{n\}Match exactly n repetitions of (character) \{n,m\} range of occurrences, n and m are integers \(expression\)Group operator. expression1|expression2 Matches expression1 or expression 2. () groups regular expressions

Regular Expressions (character classes) The following character classes are short-hand for matching special characters. [:alnum:]Printable characters (includes white space) [:alpha:]Alphabetic characters [:blank:]Space and tab characters [:cntrl:]Control characters [:digit:]Numeric characters [:graph:]Printable and visible (non-space) characters [:lower:]Lowercase characters [:print:]Alphanumeric characters [:punct:]Punctuation characters [:space:]Whitespace characters [:upper:]Uppercase characters [:xdigit:]Hexadecimal digits

Regular Expressions (cont…) /^M.*/ /..*/ /^$/ ab|cd a(b*|c*)d [[:space:][:alnum:]] Line begins with capital M, 0 or more chars follow At least 1 character long (/.+/ means the same thing) The empty line Either ‘ab’ or ‘cd’ matches any string beginning with a letter a, followed by either zeroor more of the letter b, or zero or more of the letter c, followed by the letter d. Matches any character that is either a white space character or alphanumeric. Note: Sed always tries to find the longest matching pattern in the input. How would you match a tag in an HTML document ?

Line Addresses Each line read is counted, and one can use this information to absolutely select which lines commands should be applied to. 1 first line 2 second line... $ last line i,j from i-th to j-th line, inclusive. j can be $ Examples : sed ’53!d’ prints through line number 52 sed –n ‘4,9p’ prints only lines 4 through 9

Context Addresses The second kind of addresses are context, or Regular Expression, addresses. Commands will be executed on all pattern spaces matched by that RE. Examples: sed ‘/^$/d’ will delete all empty lines sed ‘/./,$!d’ will delete all leading blank lines at the top of file Some Rules:  commands may take 0, 1 or 2 addresses  if no address is given, a command is applied to all pattern spaces  if 1 address is given, then it is applied to all pattern spaces that match that address  if 2 addresses are given, then it is applied to all formed pattern spaces between the pattern space that matched the first address, and the next pattern space matched by the second address.  If pattern spaces are all the time single lines, this can be said like, if 2 addrs are given, then the command will be executed on all lines between first addr and second (inclusive)

Sed Commands We will go over the only some basic sed commands. a append c change lines d delete lines i insert p print lines s substitute