1. Bioinformatics Programming 1 EE, NCKU Tien-Hao Chang (Darby Chang)

2. In the last slide Terminology –Unix, Linux and UNIX –Linux vs. Window Unix-like system –commands, permissions, shell –cd, ls, du, ln, sort, find, tar, wc, … Ubuntu Shell script gcc 2

3. 3 Other features worth to mention http://brownblog.info/wp-content/uploads/2008/02/ignore.JPG

4. More about Shells 4

5. Shells Shell Startup The file.profile (sh) or.login (csh) is used at login to: –set path –define functions –set terminal parameters (stty) –set terminal type Other files such as.bash_profile and.bashrc. You can man shell ($ man bash), since shell is also a program, for more information. 5

6. Shells Sample.profile File PATH=/usr/bin:/usr/ucb:/usr/local/bin:. export PATH PS1="{ ‘hostname‘ ‘whoami‘ }" ls() { /bin/ls -sbF "$@"; } ll() { ls -al "$@"; } stty erase ˆH eval ‘tset -Q -s -m ’:?xterm’‘ umask 077 6

7. Shells.login and.cshrc.login runs only at login time –tell whether you have mail –tell who else is online –configure terminal settings.cshrc runs whenever the shell starts –set environment and shell variables –set aliases Other advanced shells inherit similar concepts. For example, the corresponding files in bash are.bash_profile and.bashrc, respectively. 7

8. Shells Job Control Putting a job into the background by appending & to the command line ˆZto stop while job is running bgcontinue stopped job in background fgreturn the job to the foreground jobslist background jobs killkill a background job 8

9. Shells History C Shell, Korn shell and others retain information about former commands executed within the shell –Use history and savehist variables to set number of commands retained: –in.cshrc: set history=100 savehist=50 –saved in ˜/.history between logins Examples –$ history nn # prints last nn commands –$ !! # repeats the last command –$ !nn # repeats the command numbered nn –$ !string repeats latest command starting with string 9

10. Shells Changing your Shell $ chsh /bin/sh The new shell must be the full path name Frequently standard shells: –Bourne/bin/sh –Korn/bin/ksh –C/bin/csh Alternate shells should be listed in /etc/shells tcsh (/bin/tcsh) and bash (/bin/bash) are the most common alternatives To try some other shell, type it at the system prompt –useful when you want to check some compatibility –type exit to return to normal 10

11. Special Unix Features 11

12. Special Unix Features I/O Redirection and Piping Output redirection to a file Input redirection from a file Piping — output of one command becomes the input of a subsequent command Standard File Descriptors –stdinstandard input to the program –stdoutstandard output from the program –stderrstandard error output 12

13. > redirect standard output to file –$ command > outfile –$ ls > foo >> append standard output to file –$ command >> outfile –$ echo ‘foo’ >> foo < input redirection from file –$ command < infile –$ sort < foo –less useful since most commands accept filenames as arguments | pipe output to another command –$ command1 | command2 –$ ls | sort 13

14. >&redirect stdout and stderr to file >>&append stdout and stderr to file |&pipe stdout and stderr to command 2>redirect stderr to file >file 2>&1redirect both stdout and stderr to file >>file 2>&1append both stdout and stderr to file 2>&1|commandpipe stdout and stderr to command To redirect stdout and stderr to two separate files: –$ (command > outfile) >& errfile –$ command > outfile 2> errfile To discard stderr: –$ command 2 > /dev/null –/dev/null is a “black hole” for bits 14

15. Special Unix Features Other Symbols ;command separator &run the command in the background &&run the following command only if previous command completes successfully ||run the following command only if previous command did not complete successfully ()grouping, commands within parentheses are executed in a subshell 15

16. Special Unix Features Quoting \ escape the following character (take it literally) –$ echo \”\” ‘’ don’t allow any special meaning to characters within single quotes (except ! in csh) –$ echo ‘shell is $SHELL’ “” allow variable and command substitution inside double quotes (does not disable $ and \ within the string) –$ echo “shell is $SHELL” `command‘ backquotes take the output of command and substitute it into the command line (works inside double- quotes) –$ echo `ls` 16

17. Special Unix Features Wild Cards ?match any single character *match any string of zero or more characters [abc]match any one of the enclosed characters [a-z]match any character in the range a through z [!def] (sh)match any characters not one of the [ˆdef] (csh)enclosed characters {ab,bc,cd}match any set of characters separated by comma ˜user’s own home directory ˜userhome directory of specified user 17

18. Remember 18 These symbols may vary from shell to shell — see the man pages

19. Any Questions? 19

20. Screen 20

21. 21 No Alt-Tab to switch between programs http://www.mergersandinquisitions.com/wp-content/uploads/2008/05/alttab-key.jpg

22. Screen Screen is best described as a terminal multiplexer Prevent multiple terminal emulators More important, to live with sessions rather than terminals To start –$ screen 22

23. Screen Commands In screen, all commands begin with ^a (Ctrl+a) –?help –ccreate a new window, each created window is assigned with a number –wlist current windows –[0-9]switch window by number –-switch to an empty window (boss coming) –n, [space]switch to the next window –p, [backspace]switch to the previous window –a, ^aswitch to the last window (recall button) –Aname the current window –', "switch window by name –x, ^xlock the current window 23

24. Screen Sessions Each screen (and all the associated window) is a session –When you type ‘screen’, you start a session. Then you use ‘^a c’ to create some windows. The status (screen, connections, …) of all these windows are logged by the session. Commands –^a ddetach the current session –^a DDdetach the current session and logout –^a z, ^a ^zmake the current session background, of course you can use ‘fg’ to restore it 24

25. Options –$ screen -lslist sessions –$ screen -d [pid]detach a remote session (which is attached, but not to the current terminal) –$ screen -D [pid]detach and logout a remote session –$ screen –r [pid]reattach a session –$ screen –Rreattach the youngest session, create a new one if necessary –$ screen –xattach to a not detached screen session (multi display mode) –$ screen -d -rreattach a session and if necessary detach it first –$ screen -D -Rattach here and now In detail this means: If a session is running, then reattach. If necessary detach and logout remotely first. If it was not running create it and notify the user. This is the author's favorite. –$ screen -wipeclean dead sessions 25

26. Any Questions? 26

27. Relation among 27 Session, terminal and window in screen. One to many, many to one, or many to many? A drawing would be necessary.

28. Text Processing 28

29. Text Processing Editors vi –Visual Editor –No alternative (except you choose emacs), it is the best way to force you guys to learn vi –Pronounce both letters: V-I, never “Vy” –Three modes Command mode (“beep mode”) Insert mode (“no beep mode”) Command line mode (“colon mode”) –Commands are generally case sensitive 29

30. vi Cursor Movement h, j, k, lalternates for arrows [n]hleft [n]character(s) [n]jdown [n] character(s) [n]kup [n] line(s) [n]ldown [n] line(s) ˆFforward one screen ˆBback one screen ˆDdown half screen ˆUup half screen Ggo to last line of file $end of current line ˆbeginning of text on current line 0beginning of current line [n]wforward [n] word(s) [n]bback [n] word(s) 30

31. vi Inserting/Deleting Text iinsert text before the cursor aappend text after the cursor Iinsert text at beginning of line Aappend text at end of line oopen new line after current line Oopen new line before current line dddelete current line [n]dddelete [n] line(s) [n]dwdelete [n] word(s) Ddelete from cursor to end of line xdelete current character [n]xdelete [n] characters Xdelete previous character (like backspace) Confused? Remember range command unit philosophy 31

32. vi Change Commands cwchange word [n]cwchange next [n] word(s) c$change from cursor to end of line ˜change case of character Jjoins current line and next line uundo the last command just done.repeat last change [n]yyyank [n] line(s) to buffer [n]ywyank [n] word(s) to buffer pputs yanked or deleted text after cursor Pputs yanked or deleted text before cursor 32

33. vi File Manipulation :wwrite changes to file :wqwrite changes and quit :w!force overwrite of file :qquit if no changes made :q!quit without saving changes :!shell escape :r!insert result of shell command at cursor position 33

34. Any Questions? 34

35. How to 35 Change the next 10 lines?

36. What is 36 dG

37. Text Processing Commands 37

38. Text Processing Commands grep grep — search the argument for all occurrences of the search string grep [option]... regexp file –search for the number 15 $ grep '15' file –count the number of lines matching the search criterion $ grep -c '15' file –search for lines not matching the search criterion $ grep -v '15' file –search for 11, 12 or 15 $ grep '1[125]' file –search for all lines that begin with a space $ grep '^ ' file –search for lines begin with the characters 1 through 9 grep '^[1-9]' file 38

39. Text Processing Commands Advanced grep Examples $ wget -q -O- http://url.to.web/ | grep 'a href' | head –list the first 10 links of a given web page –wget –power of pipe $ grep MemTotal /proc/meminfo $ grep 'model name' /proc/cpuinfo –show RAM and CPU info –everything is a file $ set | grep $USER –set –using environment variables 39

40. Text Processing Commands sed sed — stream editor for editing files from script or command line sed [option]... edit_command file –changes all incidents of a comma into a comma followed by a space $ sed 's/,/, /g' file –filter for lines containing ‘Date: ’ and ‘From: ’ and replace these without the colon (perform multiple operations) $ sed -e 's/Date: /Date /' -e 's/From: /From /‘ –print only those lines of the file from the one beginning with "Date:" up to, and including, the one beginning with "Name:" $ sed -n '/^Date:/,/^Name:/p‘ 40

41. Text Processing Commands awk — scan for patterns in a file and process the results awk program file –$ cat /etc/passwd | tr a-z A-Z | awk -F: '{printf ("user %-16s %c %5d %5d\n",$1,$2,$3,$3)}‘ –$ awk 'BEGIN { x=0 } /^$/ { x=x+1 } END { print "I found " x " blank lines. :)" }' file awk is good at column processing awk is almost a script language 41

42. Any Questions? 42

43. How Much 43 Do you remember?

44. 2 Things You Should Learn 44 The way to interpret these magic commands Remember that there are such facilities

45. What to Do First 45 When you see this $ sed -n 1,10p

46. 46 Google? Sure But you should try man (or --help) first http://farm1.static.flickr.com/34/108805307_c43af20f59.jpg

47. $ sed -n 1,10p It is similar to head Then, why not use head? Suppose that if you want the ninth and tenth lines $ sed -n 9,10p 47

48. $ sed -n 1,10p 48 It is similar to head. Then, why not use head? Suppose that if you want the ninth and tenth lines. $ sed -n 9,10p

49. UNIX The Last Reminding How to remember these? Just use them! Force yourself to use them! Briefly speaking want to search some lines? try grep want to edit some lines? try sed want to handle columns? try awk something awk script cannot do (I doubt), try perl one-liner 49

50. Any Questions? 50 Our UNIX tutorial is finished

51. Terminology 51 Programming

52. 52 What is programming? What is programming language? What is language?

53. Programming Vs. Language In general, both can be used to –describe how to accomplishes a particular task However, to learn programming –to learn the method (step-by-step) On the other hand, to learn a (programming) language –to learn how to express the method in commonly readable symbols 53

54. Programming Different Languages Express the same concept with different symbols Chinese and English – 想睡覺 –feel sleepy English vs. C –increase i by 1 –{ ++i; } Programming languages are just languages the commonly readable for computers 54

55. Programming Algorithm + Data Structure Some people think learning programming is to learn the method such as sort. It is not wrong. –An algorithm is a finite set of instructions that accomplishes a particular task However, choosing different data structures may result in extremely different algorithms Consider a program of scoring system –it supports insert, delete, min, max, average –two data structures are considered a) array b) sorted array 55

56. 56 Which algorithms of these operations would change between a) and b)? Which operations are faster in a), faster in b) or equal? Which algorithm + data structure combination is better?

57. Dealer 57 In- Out13 cards Requirement - the cards are chosen by chance - but they are displayed in order - each card is represented by two characters - S1 means spade one, HJ means heart J,... - no need to implement a sort algorithm - using C would be the best Bonus - output 52 cards randomly split into 4 sets

58. Deadline 58 2010/3/16 23:59 Zip your code, a step-by-step README of how to execute the code and anything worthy extra credit. Email to darby@ee.ncku.edu.tw. darby@ee.ncku.edu.tw

59. qsort #include #include int comp(const int * a,const int * b) { if (*a *b) return 1; return 0; } int main(int argc, char* argv[]) { int numbers[10]={1892,45,200,-98,4087,5,-12345,1087,88,-100000}; int i; // sort the array qsort(numbers,10,sizeof(int),comp); for (i=0;i<9;i++) printf("Number = %d\n",numbers[ i ]); return 0; } 59