1. Introduction to Unix – CS 21 Lecture 5

2. Lecture Overview Lab Review Useful commands that will illustrate today’s lecture Streams of input and output File redirection Piping between different programs Putting it all together for some powerful tools

3. Important Points From Lab Get used to both modes of chmod Understand how directory permissions affect access Write permission only doesn’t let you do anything Start thinking about using wildcards to speed up tasks

4. Good Commands To Know: wc Stands for word count Will report the number of lines, words, and characters in a file Works with all sorts of files, but only makes sense with text files Useful for all sorts of applications Either file direction or piping is required…

5. Example Of wc

6. sort Will sort the lines of a file into alphabetical order Useful on data files or output from programs Usage: sort [OPTIONS] [FILE]

7. Example Of sort

8. uniq Will go through a file and remove duplicate copies of a line Will only remove duplicate copies of lines that are adjacent Useful on files that might have repeated unnecessary data Usage: uniq [OPTIONS] [FILE]

9. Example Of uniq

10. Wouldn’t It Be Nice If We Could sort And uniq A File? No command exists that does both By the end of the lecture, we should see if this is possible Do you think this will be possible? Hint: Remember Unix was designed so that tasks could be easily automated and so that tasks could work together

11. What Exactly Is Input And Output? Input Everything a Unix program reads Usually from the keyboard What you type Output Everything a Unix program writes Usually to the screen What you see

12. Streams Of Input And Output stdin Input from the keyboard stdout Output to the screen stderr Error messages that show up on the screen

13. stdin Everything you type into the terminal The programs mostly expect all of their input from the user (you) typing at the keyboard

14. stdout The results of a running program Goes to the terminal you ran the program from

15. stderr Appears the same location as stdout Used for error messages Allows for the error messages to be separated from normal output

16. Redirecting These Streams Deciphering “echo ‘hi’ > helloFile” > Sends all output that normally would go to stdout into a file Completely overwrites whatever file was there or creates a new file if none existed

17. Example Of >

18. A Tricky Catch To Remember You must be careful because ‘>’ will overwrite files

19. Redirecting stdin < Makes all of the input for a file come from a file as opposed to the keyboard Allows you to input a lot of data without a lot of typing Exercise: Go to lab and run cat without any parameters

20. Example Of <

21. Redirection Of Output Without File Overwrite What if you want to add information to a file instead of overwriting it? >> Appends all output of a program to a file Will create a file if it doesn’t exist, but won’t overwrite any information currently in a file

22. Example Of Append Redirection

23. One More Stream Don’t forget about stderr 2> Not some weird smiley This says redirect stderr to a file Important to note why this notation exists 0 = stdin 1 = stdout 2 = stderr

24. Example Of Redirecting stderr

25. Example Of The Difference Between stdout And stderr

26. Is This Ever Useful? Short answer: Yes! Longer answer: Whenever you want to save the output of a program, redirect it Whenever you want to avoid typing input, redirect it Whenever you want to isolate any error messages, redirect it

27. Possible Uses For File Redirection Saving data generated by one program for use by another If you continually type the same input into different programs, place that data into a file and redirect it into the different programs

28. How To Remember Which Arrow Does What cat < inputFile Think of this as the inputFile dumping all of its contents into cat In other words, inputFile is going into cat cat > outputFile In this case, cat is dumping its results into outputFile In other words, cat is going into outputFile

29. What About Other Command Flags? The input or output file must come directly after the redirection operator Example: Right: ls –l * > listingOutput Wrong: ls > -l * listingOutput Question: How do you think this line is interpreted? As a general rule of thumb, redirection should come at the end of a line Redirection should come after all arguments to the program to avoid problems like the above

30. Combining These Flags Entirely possible to have both on the same line Each redirection operator assumes the next argument is what is being redirected Question: Can you have > and >> on the same line?

31. Answer

32. What Does This Command Do? ls /bin/doesntExist 2>> errorFile Appends stderr to the end of errorFile

33. What About These Commands?

34. Piping: Tying Programs Together We can combine some of the previous commands together with a pipe Pipes feed the output of one program into another | Yes, that’s a vertical bar

35. What Does A Pipe Look Like?

36. sort And uniq Answering our previous question, we see that it should be possible to both sort and uniq a file at once using pipes How exactly would we do it?

37. sort And uniq With Redirection

38. Same Example With Piping

39. How To Read Pipes You can have more than one pipe per line if you’d like Always read left to right, and output flows left to right

40. Putting It All Together What if we want to have both file redirection as well as piping? Well, things start to look ugly, but this is entirely doable

41. Complex Example

42. Really Complex Example

43. Is All This Stuff Worth It? Well, let’s compare this with how you do things in Windows With piping, several tasks can be combined and run in one single step What if I wanted to sort and uniq a Word file? Open the file and highlight all of the text Select Table->Sort (not intuitive) Specify how to sort and click o.k. Word doesn’t have a uniq feature…

44. Slowly, Your Power Grows With this information, you are slowly becoming a Unix power user Maybe you’ll save the world from dinosaurs yet…

45. Next Time More thorough explanation of wildcards Regular expression overview grep – what exactly is it? Quiz #1