1. Department of Computer Science and Engineering
Unix and Shell Programming
Department of Computer Science and Engineering

2. Unit-3
User Communication, Electronic mail, Remote access, and File Transfer. Vi Editor – Local, Global and Range commands and Text manipulation in vi. Editor, and Over view of ex Editor. Atoms and
Operators. grep – family and operations and searching for file contents. Overview of sed and awk

3. Communicating with Others
who
talk
write
mesg
finger

4. who Used to display names of users logged into system Format:
who [options]
who or who am i or whoami
Options:
-H displays head above user information
-i displays # of minutes user was idle
-T displays message reception status

5. talk / write
talk allows user to conduct a two-way text-based conversation. To initiate talk you type: talk phobos_user_id (use who -T to determine if user is receiving messages)
write is used to send a message to another user. To send a message, type: write phobos_user_id (Mainly used to broadcast a message but can be used to chat - user who -T to check mesg status first)

6. mesg
Used to “turn on” or “turn-off” reception of messages from other users.
To turn on message reception status, type: mesg y <ENTER>
To turn-off message reception status, type: mesg n <ENTER>

7. finger Used to display user names and related information format:
finger [options] [user-list]
Options:
-l displays detailed information of all users
-m matches names specified from user-list
-q displays short report of users logged on
-s displays a short report for each user incl.name

8. finger Interesting point:
finger utility will display contents in file “.plan” and “.project” in the user’s home directory.
These files are useful to provide additional information about the user
eg. finger msaul (In phobos)
finger jankul (In phobos)
Note: you need to set the file permissons to allow group & others to read file!

9. UNIX Utilities: ftp
FTP (File Transfer Protocol) allows you to transfer files to and from a remote network site (machine).
ftp [-dgintv] [hostname]
> ftp einstein.franklin.edu
Uses put and get commands to send and receive files.
Type bin for setting binary transfer mode
Type prompt, to turn off prompting
Type ls, or dir for a directory listing
Demonstrate?
This exists on your Personal Computer

10. UNIX Utilities: telnet
Telnet is a user interface to a remote system using the TELNET protocol.
telnet [-8ELcdr] [hostname]
> telnet einstein.franklin.edu
Demonstrate?
This exists on your personal computer

11. UNIX Utilities: e-mail
Two most popular mail programs are elm and pine.
Note: Most Unix systems still have the “more basic” package called mail
The elm mail program was created in 1986 by programmer Dave Taylor
Elm has a handy users guide
elm
q or Q x or X
similar to vi commands
Demonstrate the program

12. pine
The pine program has a menu-driven interface and is quite simple to use. One advantage of pine over elm is that it is very simple to include attachments.
> pine
Follow the prompts in the menu
The elm mail program was created in 1986 by programmer Dave Taylor
Elm has a handy users guide
elm
q or Q x or X
similar to vi commands
Demonstrate the program

13. elm
The elm program is the mail utility that is most often used. It is more difficult to use, but more powerful. It uses vi to compose messages.
> elm
The elm mail program was created in 1986 by programmer Dave Taylor
Elm has a handy users guide
elm
q or Q x or X
similar to vi commands
Demonstrate the program

14. What is Vi Editor? The visual editor on the Linux.
The vi editor comes with every version of Linux or Unix.
Using vi is similar to using other editors in that you can see your file on the screen ..
The vi editor is the most popular editor in linux. The current version is really "vim", but to invoke it simply type "vi".
Before vi the primary editor used on Unix was the line editor
User was able to see/edit only one line of the text at a time
The vi editor is not a text formatter (like MS Word, Word Perfect, etc.)
you cannot set margins
center headings
Etc…
 Page 14

15. History of Vi Editor
Although other stories exist, the true one tells that vi was originally written by Bill Joy in 1976.
Who is Bill Joy you ask?
He co-founded Sun Microsystems in 1982 and served as chief scientist until 2003.

16. Characteristics of vi The vi editor is:
A very powerful
It is hard to learn, specially for windows users
- Move from point to point in the file, and make changes.
- Available on all UNIX systems

17. Starting vi Type vi <filename> at the shell prompt
After pressing enter the command prompt disappears and you see tilde(~) characters on all the lines
These tilde characters indicate that the line is blank

18. Vi modes There are three modes in vi Command mode
Accessed by typing Esc
In this mode all the keys pressed by the user are interpret to the editor command.
Input mode
Accessed by typing “i”
This mode permits insertion of new text, editing of existing text or replacement of existing text.
Ex mode( Last Line Mode)
-The bottom line of the vi screen is called ex mode.
When you start vi by default it is in command mode
-You exit the input mode by pressing the Esc key to get back
to the command mode.

19. VI EDITING MODES : Return Command Mode Esc Input Mode Insert (i, I)
Append (a, A),
Open (o, O)
Change (c),
Replace (r, R) :
Return
Last-Line
Mode

20. Common Vi Commands Moving Cursor Position
You can move around only when you are in the command mode
Arrow keys usually works(but may not)
The standard keys for moving cursor are:
h - for left
l - for right
j - for down
k - for up
w - to move one word forward
b - to move one word backward
$ - takes you to the end of line
<enter> takes the cursor the beginning of next line
 Page 20

21. Conti..
- - (minus) moves the cursor to the first character in the current line
H - takes the cursor to the beginning of the current screen Home position)
L - moves to the Lower last line
M - moves to the middle line on the current screen

22. Conti.. Control-d scrolls the screen down (half screen)
Control-u scrolls the screen up (half screen)
Control-f scrolls the screen forward (full screen)
Control-b scrolls the screen backward (full screen).

23. Editing text x - deletes the current character
d - is the delete command but pressing only d will not delete anything you need to press a second key
dw - deletes to end of word
dd - deletes the current line
d0 - deletes to beginning of line
There are many more keys to be used with delete command
The vi commands can be used followed by a number such as
n<command key(s)>
For example dd deletes a line 5dd will delete five lines.
 Page 23

24. Copy, cut and paste in vi yy - (yank) copy current line to buffer
nyy - Where n is number of lines
p - Paste the yanked lines from buffer to the line below
P - Paste the yanked lines from buffer to the line above

25. Undo and repeat command
u - undo the changes made by editing commands
(dot or period) repeats the last edit command

26. How to exit from vi (command mode)
ZZ is for save and Exit (Note this command is uppercase)
:q <enter> is to exit, if you have not made any changes to the file

27. Regular Expression
A pattern of special characters used to match strings in a search
Typically made up from special characters called metacharacters
Regular expressions are used thoughout UNIX:
Editors: ed, ex, vi
Utilities: grep, egrep, sed, and awk

28. Metacharacters any non-metacharacter matches itself RE Metacharacter.
Any one character, except new line
[a-z]
Any one of the enclosed characters (e.g. a-z) *
Zero or more of preceding character
? or \?
Zero or one of the preceding characters
+ or \+
One or more of the preceding characters

29. The grep Utility “grep” command: searches for text in file(s)
Examples:
% grep root mail.log
% grep r..t mail.log
% grep ro*t mail.log
% grep ‘ro*t’ mail.log
% grep ‘r[a-z]*t’ mail.log

30. more Metacharacters RE Metacharacter Matches… ^ beginning of line $
end of line
\char
Escape the meaning of char following it
[^]
One character not in the set
\<
Beginning of word anchor
\>
End of word anchor
( ) or \( \)
Tags matched characters to be used later (max = 9)
| or \|
Or grouping
x\{m\}
Repetition of character x, m times (x,m = integer)
x\{m,\}
Repetition of character x, at least m times
x\{m,n\}
Repetition of character x between m and m times

31. An operator combines regular expression atoms.
An atom specifies what text is to be matched and where it is to be found.
An operator combines regular expression atoms.

32. An atom specifies what text is to be matched and where
it is to be found.

33. Single-Character Atom
A single character matches itself

34. matches any single character except for a new
Dot Atom
matches any single character except for a new
line character (\n)

35. Class Atom matches only single character that can be any of
the characters defined in a set:
Example: [ABC] matches either A, B, or C.
Notes:
1) A range of characters is indicated by a dash, e.g. [A-Q]
2) Can specify characters to be excluded from the set, e.g.
[^0-9] matches any character other than a number.

36. Example: Classes

37. short-hand classes [:alnum:] [:alpha:] [:upper:] [:lower:] [:digit:]
[:space:]

38. Anchors Anchors tell where the next character in the pattern must
be located in the text data.

39. Back References: \n used to retrieve saved text in one of nine buffers
can refer to the text in a saved buffer by using a back reference:
ex.: \1 \2 \3 ...\9
more details on this later

40. Operators

41. Sequence Operator
In a sequence operator, if a series of atoms are shown in
a regular expression, there is no operator between them.

42. Alternation Operator: | or \|
operator (| or \| ) is used to define one
or more alternatives
Note: depends on version of “grep”

43. Repetition Operator: \{…\}
The repetition operator specifies that the atom or expression immediately before the repetition may be
repeated.

44. Basic Repetition Forms

45. Short Form Repetition Operators: * + ?

46. Group Operator In the group operator, when a group of characters is
enclosed in parentheses, the next operator applies to the
whole group, not only the previous characters.
Note: depends on version of “grep”
use \( and \) instead

47. Grep detail and examples
grep is family of commands
grep
common version
egrep
understands extended REs
(| + ? ( ) don’t need backslash)
fgrep
understands only fixed strings, i.e. is faster
rgrep
will traverse sub-directories recursively

48. Commonly used “grep” options:
Print only a count of matched lines. -i
Ignore uppercase and lowercase distinctions. -l
List all files that contain the specified pattern. -n
Print matched lines and line numbers. -s
Work silently; display nothing except error messages. Useful for checking the exit status. -v
Print lines that do not match the pattern.

49. Example: grep with pipe
% ls -l | grep '^d'
drwxr-xr-x 2 krush csci Feb 8 22:12 assignments
drwxr-xr-x 2 krush csci Feb 5 07:43 feb3
drwxr-xr-x 2 krush csci Feb 5 14:48 feb5
drwxr-xr-x 2 krush csci Dec 18 14:29 grades
drwxr-xr-x 2 krush csci Jan 18 13:41 jan13
drwxr-xr-x 2 krush csci Jan 18 13:17 jan15
drwxr-xr-x 2 krush csci Jan 18 13:43 jan20
drwxr-xr-x 2 krush csci Jan 24 19:37 jan22
drwxr-xr-x 4 krush csci Jan 30 17:00 jan27
drwxr-xr-x 2 krush csci Jan 29 15:03 jan29
% ls -l | grep -c '^d' 10
Pipe the output of the “ls –l” command to grep and list/select only directory entries.
Display the number of lines where the pattern was found. This does not mean the number of occurrences of the pattern.

50. Example: grep with \< \>
% cat grep-datafile
northwest NW Charles Main
western WE Sharon Gray
southwest SW Lewis Dalsass
southern SO Suan Chin
southeast SE Patricia Hemenway
eastern EA TB Savage
northeast NE AM Main Jr
north NO Ann Stephens
central CT KRush
Extra [A-Z]****[0-9]..$5.00
Print the line if it contains the word “north”.
% grep '\<north\>' grep-datafile
north NO Ann Stephens

51. Example: grep with a\|b
% cat grep-datafile
northwest NW Charles Main
western WE Sharon Gray
southwest SW Lewis Dalsass
southern SO Suan Chin
southeast SE Patricia Hemenway
eastern EA TB Savage
northeast NE AM Main Jr
north NO Ann Stephens
central CT KRush
Extra [A-Z]****[0-9]..$5.00
Print the lines that contain either the expression “NW” or the expression “EA”
% grep 'NW\|EA' grep-datafile
northwest NW Charles Main
eastern EA TB Savage
Note: egrep works with |

52. Example: egrep with + Note: grep works with \+ % cat grep-datafile
northwest NW Charles Main
western WE Sharon Gray
southwest SW Lewis Dalsass
southern SO Suan Chin
southeast SE Patricia Hemenway
eastern EA TB Savage
northeast NE AM Main Jr
north NO Ann Stephens
central CT KRush
Extra [A-Z]****[0-9]..$5.00
Print all lines containing one or more 3's.
% egrep '3+' grep-datafile
northwest NW Charles Main
western WE Sharon Gray
southwest SW Lewis Dalsass
Note: grep works with \+

53. Example: egrep with RE: ?
% cat grep-datafile
northwest NW Charles Main
western WE Sharon Gray
southwest SW Lewis Dalsass
southern SO Suan Chin
southeast SE Patricia Hemenway
eastern EA TB Savage
northeast NE AM Main Jr
north NO Ann Stephens
central CT KRush
Extra [A-Z]****[0-9]..$5.00
Print all lines containing a 2, followed by zero or one period, followed by a number.
% egrep '2\.?[0-9]' grep-datafile
southwest SW Lewis Dalsass
Note: grep works with \?

54. Note: grep works with \( \) \+
Example: egrep with ( )
% cat grep-datafile
northwest NW Charles Main
western WE Sharon Gray
southwest SW Lewis Dalsass
southern SO Suan Chin
southeast SE Patricia Hemenway
eastern EA TB Savage
northeast NE AM Main Jr
north NO Ann Stephens
central CT KRush
Extra [A-Z]****[0-9]..$5.00
Print all lines containing one or more consecutive occurrences of the pattern “no”.
% egrep '(no)+' grep-datafile
northwest NW Charles Main
northeast NE AM Main Jr
north NO Ann Stephens
Note: grep works with \( \) \+

55. Example: egrep with (a|b)
% cat grep-datafile
northwest NW Charles Main
western WE Sharon Gray
southwest SW Lewis Dalsass
southern SO Suan Chin
southeast SE Patricia Hemenway
eastern EA TB Savage
northeast NE AM Main Jr
north NO Ann Stephens
central CT KRush
Extra [A-Z]****[0-9]..$5.00
Print all lines containing the uppercase letter “S”, followed by either “h” or “u”.
% egrep 'S(h|u)' grep-datafile
western WE Sharon Gray
southern SO Suan Chin
Note: grep works with \( \) \|

56. Example: fgrep % cat grep-datafile northwest NW Charles Main 300000.00
western WE Sharon Gray
southwest SW Lewis Dalsass
southern SO Suan Chin
southeast SE Patricia Hemenway
eastern EA TB Savage
northeast NE AM Main Jr
north NO Ann Stephens
central CT KRush
Extra [A-Z]****[0-9]..$5.00
Find all lines in the file containing the literal string “[A-Z]****[0-9]..$5.00”. All characters are treated as themselves. There are no special characters.
% fgrep '[A-Z]****[0-9]..$5.00' grep-datafile
Extra [A-Z]****[0-9]..$5.00

57. Print all lines beginning with the letter n.
Example: Grep with ^
% cat grep-datafile
northwest NW Charles Main
western WE Sharon Gray
southwest SW Lewis Dalsass
southern SO Suan Chin
southeast SE Patricia Hemenway
eastern EA TB Savage
northeast NE AM Main Jr
north NO Ann Stephens
central CT KRush
Extra [A-Z]****[0-9]..$5.00
Print all lines beginning with the letter n.
% grep '^n' grep-datafile
northwest NW Charles Main
northeast NE AM Main Jr
north NO Ann Stephens

58. Print all lines ending with a period and exactly two zero numbers.
Example: grep with $
% cat grep-datafile
northwest NW Charles Main
western WE Sharon Gray
southwest SW Lewis Dalsass
southern SO Suan Chin
southeast SE Patricia Hemenway
eastern EA TB Savage
northeast NE AM Main Jr
north NO Ann Stephens
central CT KRush
Extra [A-Z]****[0-9]..$5.00
Print all lines ending with a period and exactly two zero numbers.
% grep '\.00$' grep-datafile
northwest NW Charles Main
southeast SE Patricia Hemenway
Extra [A-Z]****[0-9]..$5.00

59. Example: grep with \char
% cat grep-datafile
northwest NW Charles Main
western WE Sharon Gray
southwest SW Lewis Dalsass
southern SO Suan Chin
southeast SE Patricia Hemenway
eastern EA TB Savage
northeast NE AM Main Jr
north NO Ann Stephens
central CT KRush
Extra [A-Z]****[0-9]..$5.00
Print all lines containing the number 5, followed by a literal period and any single character.
% grep '5\..' grep-datafile
Extra [A-Z]****[0-9]..$5.00

60. Print all lines beginning with either a “w” or an “e”.
Example: grep with [ ]
% cat grep-datafile
northwest NW Charles Main
western WE Sharon Gray
southwest SW Lewis Dalsass
southern SO Suan Chin
southeast SE Patricia Hemenway
eastern EA TB Savage
northeast NE AM Main Jr
north NO Ann Stephens
central CT KRush
Extra [A-Z]****[0-9]..$5.00
Print all lines beginning with either a “w” or an “e”.
% grep '^[we]' grep-datafile
western WE Sharon Gray
eastern EA TB Savage

61. Print all lines ending with a period and exactly two non-zero numbers.
Example: grep with [^]
% cat grep-datafile
northwest NW Charles Main
western WE Sharon Gray
southwest SW Lewis Dalsass
southern SO Suan Chin
southeast SE Patricia Hemenway
eastern EA TB Savage
northeast NE AM Main Jr
north NO Ann Stephens
central CT KRush
Extra [A-Z]****[0-9]..$5.00
Print all lines ending with a period and exactly two non-zero numbers.
% grep '\.[^0][^0]$' grep-datafile
western WE Sharon Gray
southwest SW Lewis Dalsass
eastern EA TB Savage

62. Example: grep with x\{m\}
% cat grep-datafile
northwest NW Charles Main
western WE Sharon Gray
southwest SW Lewis Dalsass
southern SO Suan Chin
southeast SE Patricia Hemenway
eastern EA TB Savage
northeast NE AM Main Jr
north NO Ann Stephens
central CT KRush
Extra [A-Z]****[0-9]..$5.00
Print all lines where there are at least six consecutive numbers followed by a period.
% grep '[0-9]\{6\}\.' grep-datafile
northwest NW Charles Main
southwest SW Lewis Dalsass
southeast SE Patricia Hemenway
eastern EA TB Savage
north NO Ann Stephens
central CT KRush

63. Example: grep with \<
% cat grep-datafile
northwest NW Charles Main
western WE Sharon Gray
southwest SW Lewis Dalsass
southern SO Suan Chin
southeast SE Patricia Hemenway
eastern EA TB Savage
northeast NE AM Main Jr
north NO Ann Stephens
central CT KRush
Extra [A-Z]****[0-9]..$5.00
Print all lines containing a word starting with “north”.
% grep '\<north' grep-datafile
northwest NW Charles Main
northeast NE AM Main Jr
north NO Ann Stephens