1. Brief introduction to UNIX A. Emerson CINECA, High Performance Systems

2. Contents Using Unix commands −command syntax Getting Started –Getting help, identity, logging out Files and directories −Making, renaming, deleting and copying. −Examining file contents. Further file handling −File compression and making archives. −File permissions Miscellaneous −Jobs and processes −Editing with vi.

3. Unix commands - usage Unix commands are normally used in the form: Where the options are generally included with the – sign. Example ls -l –a /usr/local/bin/files Single letter options can usually be combined : ls -la /usr/local/bin/files

4. Unix commands - wildcards If you want to do the command on multiple files you can use the * “wild card” character. Examples $ mv *.pl programs/perl/ $ ls data/* $ mv *.pl programs/perl/ $ ls data/* rm *.* Be careful with rm (which does a delete) because there is no way to undo it.. AARGH!

5. Getting Started Command − man Purpose − Gives you the manual page for a given command Example $ man pwd pwd(1) NAME pwd - print working directory name SYNOPSIS pwd DESCRIPTION pwd prints the path name of the working (current) directory. pwd is both an explicit command (invoked as /usr/bin/pwd), as well as a builtin $ man pwd pwd(1) NAME pwd - print working directory name SYNOPSIS pwd DESCRIPTION pwd prints the path name of the working (current) directory. pwd is both an explicit command (invoked as /usr/bin/pwd), as well as a builtin

6. Getting Started Command − id Purpose − Tells you your username (!) and what group you belong to. Example $ id uid=50083(aem0) gid=30(cineca) $ id uid=50083(aem0) gid=30(cineca)

7. Getting Started Command − exit (or ctrl-d) Purpose − Logs you out from the system Example $ exit

8. Files and directories Command − mkdir Purpose − Makes a directory Common options − -p creates all the sub-directories in a path if they don’t exist Examples $ mkdir perl-programs $ mkdir –p 2003/jan/data $ mkdir perl-programs $ mkdir –p 2003/jan/data

9. Files and directories Command − cd Purpose − Changes directory. With no arguments changes to home directory. Examples $ cd my_data $ cd /usr/local/bin/programs $ cd $ cd my_data $ cd /usr/local/bin/programs $ cd

10. Files and directories Command − mv Purpose − Moves or renames a file or directory Common options − -i Asks confirmation before overwriting another file or directory Examples $ mv first.pl perl-programs/jan $ mv blast.out blast.out.bak $ mv *.seq sequence-dir $ mv first.pl perl-programs/jan $ mv blast.out blast.out.bak $ mv *.seq sequence-dir

11. Files and directories Command − rm Purpose − Deletes or renames a file Common options − -i Asks confirmation first − -r deletes all sub-directories of a directory (VERY DANGEROUS) Examples $ rm *.old $ rm –i blast.pl File blast.pl. Remove ? (yes/no)[no] : $ rm *.old $ rm –i blast.pl File blast.pl. Remove ? (yes/no)[no] :

12. Files and directories Command − cp Purpose − Makes a copy of a file or directory Common options − -i Asks confirmation first if overwriting another file − -r copies all files of all sub-directories of a directory Examples $ cp program.f90 program.f90.old $ cp blastdir/*.out. $ cp program.f90 program.f90.old $ cp blastdir/*.out.. means the current directory

13. Files and directories Command − ls Purpose − Lists files and directories Common options − -t sort by modification time − -l long format, gives all details of the file (very useful) − -a shows file beginning with. (not visible with just ls) $ ls –lt total 136 -rw-r--r-- 1 bioinf00 cineca 15678 May 20 2002 test.out -rw-r--r-- 1 bioinf00 cineca 2939 May 20 2002 test.bas -rw-r--r-- 1 bioinf00 cineca 53541 May 20 2002 prova.bas $ ls –lt total 136 -rw-r--r-- 1 bioinf00 cineca 15678 May 20 2002 test.out -rw-r--r-- 1 bioinf00 cineca 2939 May 20 2002 test.bas -rw-r--r-- 1 bioinf00 cineca 53541 May 20 2002 prova.bas size in bytes

14. Files and directories Command − more (traditional Unix), less (Linux) Purpose − Allows you to view the contents of a file. $ less fasta_1.seq >THC479287 CAGAACAGTAGCTAAGAGTCAAACCATGCGTTTGAGTCTCAGCTCTGCT CTCCACTTTACCTTTTGAAGGAGATCCGGACTACAAAGGAAAGGTCTTT CTAATTTTTATCTTTTTTTTTTTTTAAACAGGTGAAGGTGCCGAGCTAT AGAAATACAAAATAAAGATCACACATCAAGACTATCTACAAAAATTTAT AGAAGAAAAGCATGCATATCATTAAACAAATAAAATACTTTTTATCACA AGGAA $ less fasta_1.seq >THC479287 CAGAACAGTAGCTAAGAGTCAAACCATGCGTTTGAGTCTCAGCTCTGCT CTCCACTTTACCTTTTGAAGGAGATCCGGACTACAAAGGAAAGGTCTTT CTAATTTTTATCTTTTTTTTTTTTTAAACAGGTGAAGGTGCCGAGCTAT AGAAATACAAAATAAAGATCACACATCAAGACTATCTACAAAAATTTAT AGAAGAAAAGCATGCATATCATTAAACAAATAAAATACTTTTTATCACA AGGAA fasta_1.seq (END)

15. Files and directories Command − head, tail Purpose − Allows you to view the first lines of a file ( head ) or the last lines of a file ( tail) Common options − n The number of lines to show. The default is 10. $ head -5 seq.fasta >THC479329 SWI/SNF complex 155 KDa subunit^^SWI/SNF complex 155 KDa subunit (BAF155)^^SWI/SNF related TTTTAGAATCCAGAAATGGTGTTCCATTTATTCACTGAAAAAGAGAGAG TTCATTCATTTTCTCCATTCTTGCCAAACTCCCTCCCCTCATTTTTTCC ACACTGAGAAACATGTTTGTACAAAAACCACATATTATTCCCCCCCCTC TGGCTGAATTACAGGAATAAAACCAGATCAAAGACATGAAAAGAAAAAG $ head -5 seq.fasta >THC479329 SWI/SNF complex 155 KDa subunit^^SWI/SNF complex 155 KDa subunit (BAF155)^^SWI/SNF related TTTTAGAATCCAGAAATGGTGTTCCATTTATTCACTGAAAAAGAGAGAG TTCATTCATTTTCTCCATTCTTGCCAAACTCCCTCCCCTCATTTTTTCC ACACTGAGAAACATGTTTGTACAAAAACCACATATTATTCCCCCCCCTC TGGCTGAATTACAGGAATAAAACCAGATCAAAGACATGAAAAGAAAAAG

16. Further file handling Command − compress (standard UNIX), gzip (GNU version – faster) − uncompress, gunzip Purpose − Compresses text files to save disk space Common options − -v verbose, gives % compression ratio $ ls -l MAG500 -rw-r--r-- 1 aem0 cineca 3249198 Jan 29 2002 MAG500 $ gzip –v MAG500 gzip -v MAG500 MAG500: 70.1% -- replaced with MAG500.gz $ $ ls –l MAG500.gz -rw-r--r-- 1 aem0 cineca 971325 Jan 29 2002 MAG500.gz $ ls -l MAG500 -rw-r--r-- 1 aem0 cineca 3249198 Jan 29 2002 MAG500 $ gzip –v MAG500 gzip -v MAG500 MAG500: 70.1% -- replaced with MAG500.gz $ $ ls –l MAG500.gz -rw-r--r-- 1 aem0 cineca 971325 Jan 29 2002 MAG500.gz

17. Making archives Command − tar Purpose − Creates an archive of files and directories. Many Unix software packages consist of a tree of sub-directories which can be difficult to transfer between different machines. tar can be used to create a single archive file which when untarred re- creates the original directory structure Common options − -c creates an archive − -f uses a file for the archive (you can also use CDs,tapes, etc) − -x extracts file from an archive − -v verbose – tells the user what tar is doing (RECOMMENDED)

18. Creating archives - Example $ ls -F blast/ $ tar -cvf blast.tar blast blast/ blast/results/ blast/input/ blast/input/input1.dat blast/input/input2.dat blast/input/input3.dat blast/data/ blast/data/blast1.out blast/data/blast2.out blast/data/blast3.out blast/data/blast4.out blast/data/blast5.out blast/data/data-old/blast-old.out blast/data/blast0.out $ ls -F blast/ $ tar -cvf blast.tar blast blast/ blast/results/ blast/input/ blast/input/input1.dat blast/input/input2.dat blast/input/input3.dat blast/data/ blast/data/blast1.out blast/data/blast2.out blast/data/blast3.out blast/data/blast4.out blast/data/blast5.out blast/data/data-old/blast-old.out blast/data/blast0.out directory to archive name of archive file

19. Extracting archives - Example $ cd new-dir $ tar -xvf blast.tar blast/ blast/results/ blast/input/ blast/input/input1.dat blast/input/input2.dat blast/input/input3.dat blast/data/ blast/data/blast1.out blast/data/blast2.out blast/data/blast3.out blast/data/blast4.out blast/data/blast5.out blast/data/data-old/ blast/data/blast0.out $ cd new-dir $ tar -xvf blast.tar blast/ blast/results/ blast/input/ blast/input/input1.dat blast/input/input2.dat blast/input/input3.dat blast/data/ blast/data/blast1.out blast/data/blast2.out blast/data/blast3.out blast/data/blast4.out blast/data/blast5.out blast/data/data-old/ blast/data/blast0.out Often gzip is combined with tar archives to give files like blast.tar.gz or blast.tgz Sometimes called tarballs.

20. File permissions $ ls -l blast.tar -rwxr--r-- 1 aem0 cineca 30720 Mar 13 10:47 blast.tar $ ls -l blast.tar -rwxr--r-- 1 aem0 cineca 30720 Mar 13 10:47 blast.tar File permissions rwxr-- - ownergroupotherspecial r - read access w - write access (for a directory means files can be deleted in the directory, even if the files don’t have write access) x - executable (searchable for a directory) - permission not set

21. File permissions Command − chmod Purpose − Changes the permissions of a file or directory. Only the owner of a file, or root, can change the permissions. Common options − -R changes all the permissions in a directory, including sub- directories $ chmod u+x myprog.pl $ chmod g+w,o-w seq.dat $ chmod +r *.fasta $ chmod 777 *.prog $ chmod u+x myprog.pl $ chmod g+w,o-w seq.dat $ chmod +r *.fasta $ chmod 777 *.prog make file executable for owner write access for group, no write for others add read access for all octal notation, here = +rwx for all

22. Running programs To run a program which is an executable file, just type the name of the file: $ my_prog.pl However, like this the terminal cannot be used until the program finishes. Add an & to return control to the user (running in the background): $ my_prog.pl & [1] 31705 $ my_prog.pl & [1] 31705 Now the user can do other things, logout and go home, etc. Note that Unix assigns a job number and a process number to the running program.

23. Running programs To see what programs are running you can use the jobs command: But this only applies to programs run during the same session (the same shell). To see all programs (processes) being run use the ps command: $ jobs [1] + Running myprog.pl $ jobs [1] + Running myprog.pl $ ps –u aem0 31705 pts/3 00:00:40 myprog.pl 29775 pts/3 00:00:00 tcsh 31738 pts/3 00:00:00 ps $ ps –u aem0 31705 pts/3 00:00:40 myprog.pl 29775 pts/3 00:00:00 tcsh 31738 pts/3 00:00:00 ps

24. Editing with vi vi is the standard Unix text editor and is present on every Unix system. $ vi myprog.pl vi has 3 modes: 1.Command mode −For manipulating and moving through the the text 2.Line mode −For special commands and interacting with Unix. 3.Insert mode −For entering text, i.e. writing programs, entering data, etc.

25. Editing with vi Command mode – the usual and initial mode (i.e. when starting vi ) Commands include − ←↑↓→ arrow keys move the cursor − hjklsame as arrow keys − x delete a character − dw delete a word − dd delete a line − 3dd delete 3 lines − u undo previous change − ZZ exit vi, saving changes

26. Editing with vi Line mode – entered by typing :, /, ? or !. Commands include − :q! save file, discarding changes − :q quit − :e filenameedit a new file − :w filenamewrite with new filename − :wqwrite file and quit − :!cmdrun Unix command − /stringlook for string RETURN executes command and returns to command mode

27. Editing with vi Insert mode – entered by typing any of the following in command mode − a append after cursor − i insert before cursor − o open line below − O open line above − Rtextreplace with text to exit insert mode, and return to command mode, type.