School of Computer Science and Information Technology Devi Ahilya Vishwavidyalaya Shell Scripting

SCSIT DAVV Applications of Shell Scripts  Task 1: There is a text file having data in columns. Store a particular column in a different file.  Task 2: Sort a file.  Task 3: Apply sorting in the output file of Task 1.  Task 4: Store only unique data in the output file.

SCSIT DAVV Applications of Shell Scripts  Task 5: Store the data which satisfy the given condition.  Task 6: Transfer the file to the client’s machine for the further processing.  Task 7: Determine whether the executed program completed successfully or not.  Task 8: In case of failure produce the signal for error.

SCSIT DAVV Solution #!/bin/ksh HOST=‘ftp.myserverid.mydomain’ USER=‘MyUserid’ PASSWD=‘MyPassword‘ FILE=“filename” OUTFILE=“newfile” MAILINGLIST=“supportmail.lst” LOG=“logfile” cut –c5,6 $FILE| sort| uniq > $OUTFILE awk '{if ($2 > 30) print $1}‘ $OUTFILE

SCSIT DAVV Solution ftp -n $HOST > /tmp/ftp.worked 2> /tmp/ftp.failed <<END_SCRIPT user $USER pass $PASSWD put $FILE quit END_SCRIPT EXITSTATUS=$? if [ $EXITSTATUS != "0" ] then for PEOPLE in `cat $MAILINGLIST` do /usr/bin/mailx –s “a process is failed” $PEOPLE [$? –ne 0] && echo “$PEOPLE mailx failed” >>$LOG done fi

SCSIT DAVV Applications of Shell Scripts  Requirement: Need to execute a program on a particular time (System time). ?

SCSIT DAVV Applications of Shell Scripts  Requirement: Logging the information about the execution of a job. ?

SCSIT DAVV Applications of Shell Scripts  Requirement: Need to execute a program on a particular time (System time) but after successful completion of a job. ?

SCSIT DAVV Applications of Shell Scripts  Requirement: Need to execute a program on a particular time (System time) but after successful completion of a job. Also check for the existence of a particular file. ?

SCSIT DAVV Applications of Shell Scripts  Requirement: Need to know that whether a particular job completed successfully or not. ?

SCSIT DAVV Applications of Shell Scripts  Requirement: Sending Mail/ SMS in case of error in the execution. ?

SCSIT DAVV Applications of Shell Scripts  Requirement: Working like database on the text files. ?

SCSIT DAVV Applications of Shell Scripts  Requirement: Fetching columns/ rows from a file, counting the records, filtering the data, sorting the data, Pattern matching/ replacing. ?

SCSIT DAVV Applications of Shell Scripts  Requirement: Handling CSV Files. ?

SCSIT DAVV Applications of Shell Scripts  Requirement: Periodic Monitoring the system activities like disk space utilization etc. ?

SCSIT DAVV Applications of Shell Scripts  Requirement: Sending / Receiving data to / from Remote location/computer. ?

SCSIT DAVV Applications of Shell Scripts  Requirement: Checking for the existence and permission for a file. ?

SCSIT DAVV Applications of Shell Scripts  Requirement: System administrators, for automating many aspects of computer maintenance, user account creation etc. ?

SCSIT DAVV Applications of Shell Scripts  Requirement: Application package installation tools. ?

SCSIT DAVV Applications of Shell Scripts  Requirement: Application startup scripts, especially unattended applications. ?

SCSIT DAVV Applications of Shell Scripts  Requirement: Data Synchronization. ?

SCSIT DAVV Applications of Shell Scripts  Requirement: Interface between Os and other tools/language like Java, Oracle, FTP. ?

SCSIT DAVV Applications of Shell Scripts  Requirement: Log Rotation. ?

SCSIT DAVV Applications of Shell Scripts  Requirement: Purging of old files and data. ?

SCSIT DAVV Applications of Shell Scripts  Requirement: Removing blank files and File comparison. ?

SCSIT DAVV The Shell & Shell Script  A shell is a command interpreter turns the input text in to actions.  Bourne Shell  Bourne Again Shell  Korn Shell  C Shell  etc  A Shell Script is a logical sequence of commands.

SCSIT DAVV The Anatomy of a Command  grep –i localhost /etc/hosts Command Option Arguments Options Change the behavior of a command Arguments control what the command act upon

SCSIT DAVV Running the Shell Script  Type the name of a program and some command line options.  The shell reads this line, finds the program and runs it, feeding it the specified options.  The shell establishes 3 I/O channels:  Standard Input  Standard Output  Standard Error

SCSIT DAVV The Shebang (#!)  The Shebang is a special comment.  It specifies which shell to use to execute this shell script.  If no “#!” found, the current shell will be used to run the script.  Example #!/bin/ksh

SCSIT DAVV Two ways to execute the Shell  Set the permission attributes as a executable file then execute it like a command. OR  Invoke the shell explicitly. sh backup8pm.sh

SCSIT DAVV Debugging the Shell Script  Running a script in debug mode will print each line of shell script before it executes.  Enable debug mode after adding the –v after shell interpreter’s name in Shebang.

SCSIT DAVV Advantages  Writing a shell script is much quicker than writing the equivalent code in other programming or scripting languages.  Shell scripts have no compilation step, so the script can be executed quickly while debugging.

SCSIT DAVV Disadvantages  One significant disadvantage of using shell scripts is that they can run slowly due to the need to create potentially many new sub-processes for each of the many commands executed.  Simple sh scripts can be quite compatible with the extremely diverse range of Unix but more complex shell scripts can fail because of the many subtle differences between shells.

SCSIT DAVV Programs and Standard I/O Program Standard Input (STDIN) Standard Output (STDOUT) Standard Error (STDERR)

SCSIT DAVV Overwriting the Standard I/O Device  Input/ Output Redirection

SCSIT DAVV Pipes A pipe is a holder for a stream of data. A pipe can be used to hold the output of one program and feed it to the input of another. prog1 prog2 STDOUT STDIN

SCSIT DAVV Regular Expression  Regular Expressions provide a concise and flexible means for identifying text of interest.  Examples:  [abc] matches a single a b or c  [a-z] matches any of abcdef…xyz

SCSIT DAVV Regular Expression  Examples: .at matches any three-character string ending with "at", including "hat", "cat", and "bat".  [hc]at matches "hat" and "cat".  [^b]at matches all strings matched by.at except "bat".  ^[hc]at matches "hat" and "cat", but only at the beginning of the string or line.  [hc]at$ matches "hat" and "cat", but only at the end of the string or line.

SCSIT DAVV Regular Expression  Examples: .at matches any three-character string ending with "at", including "hat", "cat", and "bat".  [hc]at matches "hat" and "cat".  [^b]at matches all strings matched by.at except "bat".  ^[hc]at matches "hat" and "cat", but only at the beginning of the string or line.  [hc]at$ matches "hat" and "cat", but only at the end of the string or line.

SCSIT DAVV Regular Expression Used by  grep“Get Regular Expression and Print” – search files line by line  sedSimple Editing tool, right from the command line  awkScripting language, executes “program” on matching lines

SCSIT DAVV Important Commands (UNIX)  touch: create a new file / update timestamp of existing file.  grep: search for a specified string or pattern  chmod/ chown/ chgrp: Change permissions / ownership / group on a file  du/ df: Display hard disk information.  find:find a file  sort: sort a file into alphanumeric order (by lines.)  sed: Invoke the stream editor.  tr: Translate characters.  awk: Invoke the awk scripting language.  split: Split up a file into smaller chunks.

SCSIT DAVV Important Commands (UNIX)  at: Run a command / script at a specified time and date.  Cut: cut specified field(s)/ character(s) from lines in file(s)  more, less, and pg: page through a file  head/ tail: display the start/ end of a file  cmp: compare two files and list where differences occur (T/B)  diff : compare the two files and display the differences (T)  wc: display word (or character or line) count for file(s)  mail/ mailx/ Mail: simple utility available on Unix systems.  paste: The paste command allows two files to be combined side-by-side.

SCSIT DAVV Important Commands (WinNT)  AT: Schedule a command to run at a later time  ATTRIB: Change file attributes  CACLS: Change file permissions.  CleanMgr: Automated cleanup of Temp files, recycle bin  COMP: Compare the contents of two files or sets of files  FC: Compare two files  FDISK: Disk Format and partition  FIND: Search for a text string in a file  Magnify: Display windows magnification  MAPISEND: Send from the command line  MEM: Display memory usage

SCSIT DAVV Important Commands (WinNT)  MORE: Display output, one screen at a time  MSG: Send a message  NET: Manage network resources  PERFMON: Performance Monitor  QGREP: Search file(s) for lines that match a given pattern.  SCHTASKS: Create or Edit Scheduled Tasks  SCLIST: Display NT Services  SORT: Sort input  TOUCH: Change file timestamps  USRSTAT List domain usernames and last login

SCSIT DAVV Book for UNIX

SCSIT DAVV Book for WinNT

School of Computer Science and Information Technology Devi Ahilya Vishwavidyalaya Thank You Any Questions