1. CSC 352– Unix Programming, Spring 2015 February 2015 Unix Filters

2. Simple Filters We will study a subset of Chapter 10. 10.2 comparing files Useful for comparing a program’s actual output to its expected output in a reference file. 10.3 cmp – used primarily on binary data files Compare binary or textual files. -s is silent, results in exit status only. -l gives some additional detail. Exit status is 0 on identical files, 1 for differences, > 1 for an error. Use “-l” with “od –c” or “od –o” or “od –x” to investigate binary file differences.

3. diff 10.5 diff – lots of options Line by line comparisons of text files, with annotations that delimit the differing lines (c for change, d for delete, a for add). We have been using diff for regression tests. 1,3d0 < A < B < C 5c2,3 < 2 --- > two > three 10,11c8 < 8 < 9 --- > eight 13a11,13 > X > Y  Z  -bash-3.00$ echo $?  1

4. head, tail, less, more -bash-3.00$ head -3 junk1.txt A B C -bash-3.00$ tail -3 junk2.txt X Y Z more supports scrolling through a text file and searching forward less also supports searching backward

5. cut, Section 2.8 Cut for extracting fields from a text file. -cSTART-END for selecting character columns. -c-END for front of each line, -cSTART- for the back. START and END are character positions. -d for delimiter in conjunction with –f for field. Useful for cutting out text containing formatting chars. ls -l |cut -c55- base=`echo $file | cut -d_ -f2 | cut -d. -f1` (See my splitstudents handout for the latter.)

6. sort, Section 10.10 Sort the lines of a text file. Sort is useful not only for sorting lines alphabetically or numerically, but also for getting varying-order text into a canonical order for comparison using diff. -n for numeric, -r for reverse. -k for sorting on fields, -t for delimiting fields. -m for merging sorted files.

7. uniq As a filter after sort, uniq passes one copy of each (optionally duplicate) distinct line of text. It provides a way to determine a set of distinct lines. Without sorting it filters adjacent duplicates. -u passes only unique lines, 0 repeated lines. -d passes unique copies of the duplicate lines. -c counts occurrences.

8. Applications of these filters. There are many applications that can use these filters to examine and extract file characteristics and contents. Extracting process properties is likewise possible. Software development applications include automation of file manipulation, including source and object files used by programmers, on the basis of file naming and text field delimiting conventions. For example, colon-separated fields in a file or “_” or “.” delimited fields in a file path.