UNIX Tools and the UNIX File System CMSC 121 Introduction to UNIX The material on submitexec was taken from Dan Hood’s CMSC 121 Lecture Notes.

UNIX File System: Links Every file in the UNIX file system has an associated block of information called an i-node. Every file in the UNIX file system has an associated block of information called an i-node. The i-node contains such all information about a file, except its name. The i-node contains such all information about a file, except its name. The i-node’s information includes: date created, user id, group id, file size, timestamps, the number of links, etc. It also contains the physical location of the file’s blocks on the disk. The i-node’s information includes: date created, user id, group id, file size, timestamps, the number of links, etc. It also contains the physical location of the file’s blocks on the disk. The i-node table maps the filename to the specific i- node. ls –i will display the i-node for a file. The i-node table maps the filename to the specific i- node. ls –i will display the i-node for a file. As each file has its own i-node, there is no reason that more than one name can’t point to the same i-node in the table. This is the idea behind links. As each file has its own i-node, there is no reason that more than one name can’t point to the same i-node in the table. This is the idea behind links.

UNIX File System: Links The ln command creates two names to the same file. The ln command creates two names to the same file. linux2 [4]# touch temp linux2 [5]# ln temp temp_link These are called “hard links” because they are of equal status. Type ls –i and compare the files’ i-nodes. These are called “hard links” because they are of equal status. Type ls –i and compare the files’ i-nodes. A file isn’t actually deleted until all of the hard links to it have been removed. So, rm temp will not remove the file, only this one link to it. A file isn’t actually deleted until all of the hard links to it have been removed. So, rm temp will not remove the file, only this one link to it. One important thing to note is that you cannot hard link across file systems, as i-node numbers are not unique across file systems. One important thing to note is that you cannot hard link across file systems, as i-node numbers are not unique across file systems. You also cannot hard link directories. You also cannot hard link directories.

UNIX File System: Symbolic Links A symbolic link is a sort of alias to a file. Symlinks have a much lower status than hard links. A symbolic link is a sort of alias to a file. Symlinks have a much lower status than hard links. The ln -s command creates a symlink to a file. The ln -s command creates a symlink to a file. linux2 [5]# ln –s temp temp_symlink ls –L will dereference symlinks to the actual files, instead of giving information about where symlinks point. ls –L will dereference symlinks to the actual files, instead of giving information about where symlinks point. Symlinks can work with directories and across file systems, as this is their main purpose. Symlinks can work with directories and across file systems, as this is their main purpose. Note that some programs (e.g. tar) provide options for dealing with symlinks so that they do not break. Note that some programs (e.g. tar) provide options for dealing with symlinks so that they do not break. tar’s –h ensures that symlinks don’t break by saving the referenced files instead of the symlinks. tar’s –h ensures that symlinks don’t break by saving the referenced files instead of the symlinks.

Wildcards Wildcards allow a single command to act upon multiple files with a name that matches a pattern. Wildcards allow a single command to act upon multiple files with a name that matches a pattern. Primarily, wildcards reduce typing and make commands more reusable. Primarily, wildcards reduce typing and make commands more reusable. * – Matches any sequence of characters. * – Matches any sequence of characters. ? – Matches any single character. ? – Matches any single character.

[ ] - Match Single Character [ ] – Matches any single character included in the list. [ ] – Matches any single character included in the list. The list may contain ranges, specified by a dash. E.g. [a-z] The list may contain ranges, specified by a dash. E.g. [a-z] An exclamation point ! matches any character not in the list. E.g. [!a-z] An exclamation point ! matches any character not in the list. E.g. [!a-z] Example: Example: linux3 [6]# ls file.a file.cgi file.html file.lib file.tk file.z file.c file.h file.l file.pl file.txt To list just the *.c and *.h files: To list just the *.c and *.h files: linux3 [8]# ls *.[c,h] file.c file.h Or you can specify ranges of values, such as... Or you can specify ranges of values, such as... linux3 [10]# ls *.[a-z] file.a file.c file.h file.l file.z

{ } - Match Multiple Characters We can also denote multiple files by using the "{" and "}" characters. We can also denote multiple files by using the "{" and "}" characters. This is the multiple character version of the previous example. This is the multiple character version of the previous example. Given the following directory listing: Given the following directory listing: linux3 [11]# ls file.a file.cgi file.html file.lib file.tk file.z file.c file.h file.l file.pl file.txt To list just the *.cgi, *.pl and *.tk files: To list just the *.cgi, *.pl and *.tk files: linux3 [12]# ls *.{cgi,pl,tk} file.cgi file.pl file.tk This could be very helpful when trying to open up an entire project that consists of multiple files in emacs. This could be very helpful when trying to open up an entire project that consists of multiple files in emacs. For example: emacs *.{c,h,txt} &

UNIX Tools Philosophy Each tool should do one thing very well, nothing more and nothing less. Each tool should do one thing very well, nothing more and nothing less. These tools should support chaining using streams. The output of one program should be considered to be the input to another program. These tools should support chaining using streams. The output of one program should be considered to be the input to another program. Use existing tools in combination to do very complex tasks instead of building specialized tools. Use existing tools in combination to do very complex tasks instead of building specialized tools.

xargs: chaining commands xargs takes the output of one command and passes it as command line arguments to another command. xargs takes the output of one command and passes it as command line arguments to another command. find. -name “*.txt” | xargs grep John Using single quotes may overflow the command line, so you shouldn’t use this form. Using single quotes may overflow the command line, so you shouldn’t use this form. grep John ‘find. -name “*.txt”’ Another use of xargs: Another use of xargs: Example: take all the files whose names are in the file “names” and concatenate their contents: xargs cat contents

enscript: more flexible printing The enscript command converts ascii files to postscript and then spools the postscript file to the printer. It is a more flexible way of printing than lpr. The enscript command converts ascii files to postscript and then spools the postscript file to the printer. It is a more flexible way of printing than lpr. enscript options: enscript options: -G: a “gaudy” header with a timestamp, file name, and page numbers -r: rotate the page to a landscape orientation -2: print in two columns -a: print only the specified pages -p: print to the specified file instead of the printer Example: Example: linux2 [5]# enscript -2Gr -p myfile.ps

Working with ps and pdf files ps2pdf generates a pdf file (Adobe Acrobat) from a ps (Postscript) file. ps2pdf generates a pdf file (Adobe Acrobat) from a ps (Postscript) file. linux2 [10]# ps2pdf input.ps output.pdf Viewers: gv, xpdf. Viewers: gv, xpdf.

Working with text files ispell interactively spell checks a file. ispell interactively spell checks a file. linux2 [9]# ispell filename.txt cut extracts specific columns of data from input. cut extracts specific columns of data from input. sed is an advanced tool for editing a stream. sed is an advanced tool for editing a stream. awk is a powerful data manipulation tool. awk is a powerful data manipulation tool.

Working with graphics gimp is a powerful graphics software package. gimp is a powerful graphics software package. convert changes images from one format to another. convert changes images from one format to another. xv is an interactive image viewer. xv is an interactive image viewer.

submitexec Many people wish to do things like compile and run projects once they are submitted. There have been many instances where a student has forgotten to submit a small part of a project, such as a header file, and loses a bunch of points for it not compiling. Many people wish to do things like compile and run projects once they are submitted. There have been many instances where a student has forgotten to submit a small part of a project, such as a header file, and loses a bunch of points for it not compiling. Dan Hood has written a script that allows you to execute arbitrary commands from within your submit directory. Some of courses provide such program, but many of them are written for a specific course and are compiled for a particular architecture (such as Linux and not IRIX or Solaris). So, Dan Hood has written a cross platform (works on any of the UNIX systems that we have here) script that you can use for any course. Dan Hood has written a script that allows you to execute arbitrary commands from within your submit directory. Some of courses provide such program, but many of them are written for a specific course and are compiled for a particular architecture (such as Linux and not IRIX or Solaris). So, Dan Hood has written a cross platform (works on any of the UNIX systems that we have here) script that you can use for any course. It is in his public directory: /afs/umbc.edu/users/d/h/dhood2/pub/submit_utilities/ and the name of the script is called submitexec It is in his public directory: /afs/umbc.edu/users/d/h/dhood2/pub/submit_utilities/ and the name of the script is called submitexec

submitexec The usage for this script is similar to all of the other submit utilities. The first command line argument is the course, the second is the project, and then any remaining arguments are the command(s) that you want to execute. The usage for this script is similar to all of the other submit utilities. The first command line argument is the course, the second is the project, and then any remaining arguments are the command(s) that you want to execute. This is a very versatile script. You can do anything from an ls command to compiling and running your program where the grader will do so. I suggest making a copy of this script and placing it in your own bin directory. This is a very versatile script. You can do anything from an ls command to compiling and running your program where the grader will do so. I suggest making a copy of this script and placing it in your own bin directory. Note: some of you may have used submitmake or heard of it. This script is also capable of doing that as well. If you rename the script (or create a symbolic link) with the name submitmake it will act like the submitmake provided in other courses. Again, it is no longer limited to a particular course or a particular architecture. Note: some of you may have used submitmake or heard of it. This script is also capable of doing that as well. If you rename the script (or create a symbolic link) with the name submitmake it will act like the submitmake provided in other courses. Again, it is no longer limited to a particular course or a particular architecture.

submitexec linux2 [3]# submit cs331 proj2 hello.c Submitting hello.c...OK linux2 [4]# submitexec cs331 proj2 ls Executing the command: ls hello.c Execution complete: exit status = 0 linux2 [5]# submitexec cs331 proj2 ls –l Executing the command: ls –l total 1 -rw-r--r-- 1 dhood2 general 78 Apr 20 10:42 hello.c Execution complete: exit status = 0 linux2 [6]# submitexec cs331 proj2 gcc -Wall -ansi hello.c Executing the command: gcc -Wall -ansi hello.c Execution complete: exit status = 0 linux2 [7]# submitexec cs331 proj2 a.out Executing the command: a.out Hello World! Execution complete: exit status = 256

Common Programming Tools gcc – the GNU compiler for C,C++, and Objective-C. gcc – the GNU compiler for C,C++, and Objective-C. gdb – the GNU debugger. gdb – the GNU debugger. make – automates compilation and execution make – automates compilation and execution java/javac – the Java virtual machine and compiler java/javac – the Java virtual machine and compiler An excellent guide to UNIX tools for the edit-compile- link-debug programming cycle: An excellent guide to UNIX tools for the edit-compile- link-debug programming cycle:

indent: proper whitespace indent formats C/C++ files by adjusting the whitespace to make it easier to read. It is also useful for enforcing consistent formatting for large projects. indent formats C/C++ files by adjusting the whitespace to make it easier to read. It is also useful for enforcing consistent formatting for large projects. linux2 [8]# indent file.c –o file.out linux2 [9]# indent –st file.c > file.out

make: automating compilation and execution make runs using the build instructions in a file called “Makefile” or “makefile” in the working directory. make runs using the build instructions in a file called “Makefile” or “makefile” in the working directory. The makefile includes variable definitions, and build rules for each component. The makefile includes variable definitions, and build rules for each component. Variable definitions facilitate future makefile modifications: Variable definitions facilitate future makefile modifications: CC = gcc CFLAGS = -g -I~/classes/cmsc121/include A build rule consists of one line specifying dependency information and a second line specifying the command to build that component. A build rule consists of one line specifying dependency information and a second line specifying the command to build that component. proj1.o : proj1.c proj1.h aux.h TAB$(CC) $(CFLAGS) –c proj1.c

# Makefile PROJ = Proj1 CC = CC CCFLAGS = -g SUBMITCLASS = cs121_fall05 PRINTPGM = /usr/lib/print/lptops PRINTFLAGS = -G -U -H -M2 -O0.4pt PRINTFILE = $(PROJ).ps SOURCES = \ $(PROJ).C \ Aux1.H \ Aux1.C \ Aux2.H \ Aux2.C OBJECTS = \ Aux1.o \ Aux2.o $(PROJ): $(PROJ).C $(OBJECTS) $(CC) $(CCFLAGS) -o $(PROJ) $(PROJ).C $(OBJECTS) Aux1.o: Aux1.C Aux1.H $(CC) $(CCFLAGS) -c Aux1.C Aux2.o: Aux2.C Aux2.H $(CC) $(CCFLAGS) -c Aux2.C print: $(SOURCES) - $(PRINTPGM) $(PRINTFLAGS) $(SOURCES) Makefile > $(PRINTFILE) submit: submit $(SUBMITCLASS) $(PROJ) $(SOURCES) Makefile clean: - rm -f *~ cleaner: - make clean; rm -f *.o cleanest: - make cleaner; rm -f $(PROJ)