1. Week 9 - Nov 7, 2005 1 Week 9 Agenda I/O redirection I/O redirection pipe pipe tee tee

2. 2Week 9 - Nov 7, 2005 Redirection and Piping Most “processes” in UNIX: send their output to the terminal screen get their input from the keyboard send any error messages to the terminal

3. 3Week 9 - Nov 7, 2005 Redirection and Piping Most “processes” in UNIX: send their output to the terminal screen get their input from the keyboard send any error messages to the terminal Do you remember what a “process” is?

4. 4Week 9 - Nov 7, 2005 A process is the execution of a command by the UNIX system. UNIX assigns a unique process identification (PID) number at the creation of each process. As long as a process is in existence, it keeps the same PID number.

5. 5Week 9 - Nov 7, 2005 Redirection Most “processes” in UNIX: send their to the output terminal screen input keyboard get their from the send any to the error messages terminal screen

6. 6Week 9 - Nov 7, 2005 Redirection UNIX allows us to “redirect”where processes send their output where where processes get their input andwhere processes send any error messages.

7. 7Week 9 - Nov 7, 2005

8. 8 I/O Channel 0 - standard input (stdin) I/O Channel 1- standard output (stdout) I/O Channel 2 - standard error (stderr) These “channels” are called file descriptors

9. 9Week 9 - Nov 7, 2005 stdin stdout stderr

10. 10Week 9 - Nov 7, 2005 I/O Channel 0 – stdin -default is the terminal keyboard I/O Channel 1- stdout - default is the terminal screen I/O Channel 2- stderr - default is the terminal screen These “channels” are called file descriptors Let’s learn to redirect ”stdout” first

11. 11Week 9 - Nov 7, 2005 Standard Input, Standard Output, Standard Error..... not a piece of hardware not a keyboard, a screen or a card in your computer Actually “virtual files”...opened by the kernel for every command Remember

12. 12Week 9 - Nov 7, 2005 Redirecting the output of a command to a file Use the greater than symbol > this redirects the “standard output” or “channel 1” of a command this is the same as 1> but you don’t need to type the “1” - it is “understood”.

13. 13Week 9 - Nov 7, 2005 Redirecting the output of a command to a file “The Rules” a command always comes before the > symbol a filename always comes after the > symbol command > file always a command before the > always a file name after the > You can think of “>” symbol as being an arrow pointing from a command to a file

14. 14Week 9 - Nov 7, 2005 Redirecting the output of a command to a file The output of this command is displayed on the screen by default Nothing is displayed on the screen – the output of the date command is redirected to the file “today” Issue the command w >users

15. 15Week 9 - Nov 7, 2005 Now, look at the contents of the file named users Redirecting the output of a command to a file

16. 16Week 9 - Nov 7, 2005 Redirecting the output of a command to a file Use the double greater than symbol >> this appends the “standard output” or “channel 1” of a command to a file this appends the “standard output” or “channel 1” of a command to a file You can think of the symbol as a double arrow from the command to the file, meaning 'add the output from the command to the END of the file'. You can think of the symbol as a double arrow from the command to the file, meaning 'add the output from the command to the END of the file'.

17. 17Week 9 - Nov 7, 2005 We can also redirect the output of one command to be input to another command Redirection and Piping

18. 18Week 9 - Nov 7, 2005 Piping - redirecting the output from a command to a command Use the pipe symbol to “connect” commands Use the pipe symbol to “connect” commands The “pipe” character is a broken vertical line on the keyboard Most fonts display it as a single line that looks like this: “ | ” Data goes in the start of a pipe and flows out the end command1 | command2

19. 19Week 9 - Nov 7, 2005 Using pipes Q. How many files are in my current directory? ls | wc -w

20. 20Week 9 - Nov 7, 2005 Using pipes ls | wc -w The output of the ls command is a list of file names The wc command takes the output of the ls command as its input and displays a count of the words

21. 21Week 9 - Nov 7, 2005 Using Pipes ls | wc –w > file_count The wc command sends its output to a file called file_count The spaces before and after a pipe are optional (you will find your commands easier to read if you use them) The spaces before and after a file redirection symbol (>, >>) are also optional

22. 22Week 9 - Nov 7, 2005 Using Pipes UNIX commands were designed to do “one thing very well” We use a series of pipes between simple UNIX commands to create complex commands lines Much of the power and flexibility of the UNIX command line is built upon “pipelines” of commands!

23. 23Week 9 - Nov 7, 2005 Using Pipes Command a | Command b | Command c The stdout of command ' Command a' is connected to the stdin of ' Command b', and the stdout of ' Command b' is connected to the stdin of ' Command c'. and the stdout of ' Command b' is connected to the stdin of ' Command c'.

24. 24Week 9 - Nov 7, 2005 “Always-Remember-and-Never-Forget”... RULE #1 – Use file redirection symbols such as > and >> before a RULE #2 – Use a pipe before a filename command

25. 25Week 9 - Nov 7, 2005 “Always-Remember-and-Never-Forget”... RULE #1... RULE #2...

26. 26Week 9 - Nov 7, 2005 “Always-Remember-and-Never- Forget”... RULE #3 Don’t use pipes when you don’t need them! example: cat users |more more users WRONG! CORRECT!

27. 27Week 9 - Nov 7, 2005 “Always-Remember-and-Never- Forget”... RULE #3 Don’t use pipes when you don’t need them! example: cat users |head -5 head -5 users WRONG! CORRECT!

28. 28Week 9 - Nov 7, 2005 Skill Testing Question #1 On the command line what comes AFTER a > or >> symbol?

29. 29Week 9 - Nov 7, 2005 Skill Testing Question #2 On the command line what comes AFTER a | symbol?

30. 30Week 9 - Nov 7, 2005 Skill Testing Question #3 When should you NOT use file redirection OR piping?

31. 31Week 9 - Nov 7, 2005 I/O Channel 0 – stdin -default is the terminal keyboard I/O Channel 1- stdout - default is the terminal screen I/O Channel 2- stderr - default is the terminal screen We’ve done this! Now let’s do this! Redirection and Piping

32. 32Week 9 - Nov 7, 2005 Redirecting the input of a command by default commands get their “input” from the terminal keyboard you can redirect “stdin” so that it comes from a file or from another command

33. 33Week 9 - Nov 7, 2005 Redirecting the input of a command Use the less than symbol < this redirects the “standard input” or “channel 0” of a command to come from a file instead of the keyboard this is the same as 0< but you don’t need to type the “0” - it is “understood”. You can also think of the '<' symbol as being an arrow pointing from the file to the command, meaning that the information will flow from the file into the input of the command.

34. 34Week 9 - Nov 7, 2005 Redirecting the input of a command Examples: cat < temp cat temp2 mail –s “week 9 Notes” ling.zhu@senecac.on.ca < week9.html ling.zhu@senecac.on.ca

35. 35Week 9 - Nov 7, 2005 Redirecting the input of a command Are these the same? cat < temp cat temp Are these the same? cat temp2 cp temp temp2 and and

36. 36Week 9 - Nov 7, 2005 The < symbol redirects the input of a command to come from a file Q. What if you want the input of a command to come from another command? A. When you place a command after a pipe you are redirecting its input to come from a command instead of from the keyboard

37. 37Week 9 - Nov 7, 2005 I/O Channel 0 – stdin -default is the terminal keyboard I/O Channel 1- stdout - default is the terminal screen I/O Channel 2- stderr - default is the terminal screen We’ve done this! Redirection and Piping Let’s do this!

38. 38Week 9 - Nov 7, 2005 Error Redirection By default, stderr goes to the terminal screen The “standard error” of a command can be redirected to a file using the 2> symbol Source: Ling Zhu

39. 39Week 9 - Nov 7, 2005 Error Redirection cp file1 file2 2> error.log This would overwrite the file error.log cp file1 file2 2>>error.log This would append the error message of the cp command to the file error.log Source: Ling Zhu Warning: There is no space in between

40. 40Week 9 - Nov 7, 2005 A Special Place of UNIX/Linux /dev/null /dev/null Can be called bit bucket Can be called bit bucket Can be called black hole Can be called black hole Source: Ling Zhu

41. 41Week 9 - Nov 7, 2005 Use > or >> to redirect output to a file Use a | to redirect output to a command Q. What if you want to do both? (this is on the test!) A. The tee command

42. 42Week 9 - Nov 7, 2005 The tee command duplicates the standard output of a command you can then redirect one “copy” of the output to a file and one “copy” of the output to a command example: ls –l | tee listing | more the filename you are sending one copy of the output of the ls –l command to the command you are sending one copy of the output of the ls –l command to

43. 43Week 9 - Nov 7, 2005 The tee command duplicates the standard output of a command Example: ls –l | tee listing | more if this file exists it will be overwritten To append to an existing file use tee –a filename ls –l | tee –a listing | more

44. 44Week 9 - Nov 7, 2005 Never use > within the tee command! ls –l | tee > listings | more ls –l | tee > listings | moreor ls –l | tee >> listings | more WRONG!!!!

45. 45Week 9 - Nov 7, 2005 Let’s review some commands then practice “redirection” with them... 1.sort 2.grep

46. 46Week 9 - Nov 7, 2005 Some Special symbols when you grep a pattern ^ - beginning of a line ^ - beginning of a line $ - end of a line $ - end of a line

47. 47Week 9 - Nov 7, 2005 sort and grep belong to a group of UNIX utilities called “filters” Filters are utilities that: 1.read from stdin 2.write to stdout 3.do not change the contents of the original file

48. 48Week 9 - Nov 7, 2005 How do you learn a new command? 1.Learn its purpose 2.Learn its “syntax” including common options 3.Learn where it gets input and where it sends its output

49. 49Week 9 - Nov 7, 2005 Other UNIX utilities that are “filters” headtailwctee

50. 50Week 9 - Nov 7, 2005 Filters can be used between pipes in a complex command line Example: grep happy diary >childhood sort mymarks | tee sortedmarks | more

51. 51Week 9 - Nov 7, 2005 Commands that are not filters include: cdchmodcprmdateecho ls lsmvpwdwho.... Warning! You can not put these commands between pipes! Moral of the Story: ALWAYS be careful about “what” you put in a pipe!