In the last class, Filters and delimiters The sample database pr command head and tail commands cut and paste commands.

Slides:



Advertisements
Similar presentations
Database Basics. What is Access? Database management system Computer-based equivalent of a manual database Makes it easy to organize and update information.
Advertisements

Unix Trix for Emprirical CL1 CSA405: Unix Trix for Empirical CL How to use Unix as a toolbox for NLP applications.
A Guide to Unix Using Linux Fourth Edition
Uniq command 6/12/2015Gary DeRoest1 report or filter out repeated lines in a file Note: the file needs to be sorted so that repeated lines are adjacent.
 *, ? And [ …] . Any single character  ^ beginning of a line  $ end of the line.
CS 497C – Introduction to UNIX Lecture 24: - Simple Filters Chin-Chih Chang
CS 497C – Introduction to UNIX Lecture 25: - Simple Filters Chin-Chih Chang
Unix Utilities (sort/uniq) CS465 – Unix. The sort command Sorts lines Default behavior: Do a case-sensitive, ascii- alphabetic line sort, starting at.
CS 497C – Introduction to UNIX Lecture 23: - Simple Filters Chin-Chih Chang
Guide To UNIX Using Linux Third Edition
Grep, comm, and uniq. The grep Command The grep command allows a user to search for specific text inside a file. The grep command will find all occurrences.
CSCI 330 T HE UNIX S YSTEM File operations. OPERATIONS ON REGULAR FILES 2 CSCI The UNIX System Create Edit Display Contents Display Contents Print.
© 2006 KDnuggets [16/Nov/2005:16:32: ] "GET /jobs/ HTTP/1.1" "
Unix Files, IO Plumbing and Filters The file system and pathnames Files with more than one link Shell wildcards Characters special to the shell Pipes and.
Unix Filters Text processing utilities. Filters Filter commands – Unix commands that serve dual purposes: –standalone –used with other commands and pipes.
UNIX Filters.
CS 124/LINGUIST 180 From Languages to Information Unix for Poets (in 2014) Dan Jurafsky (From Chris Manning’s modification of Ken Church’s presentation)
©NIIT Pipes and Filters Lesson 2B / Slide 1 of 28 Introduction to Linux Pre-Assessment Questions 1.Consider the following statements: Statement A: A text.
Advanced File Processing
Chapter 10: Working with Large Data Spreadsheet-Based Decision Support Systems Prof. Name Position (123) University Name.
Chapter Four UNIX File Processing. 2 Lesson A Extracting Information from Files.
Guide To UNIX Using Linux Fourth Edition
LIN 6932 Unix Lecture 6 Hana Filip. LIN 6932 HW6 - Part II solutions posted on my website see syllabus.
Introduction to Unix (CA263) File Processing. Guide to UNIX Using Linux, Third Edition 2 Objectives Explain UNIX and Linux file processing Use basic file.
Unix programming Term: III B.Tech II semester Unit-II PPT Slides Text Books: (1)unix the ultimate guide by Sumitabha Das (2)Advanced programming.
Dedan Githae, BecA-ILRI Hub Introduction to Linux / UNIX OS MARI eBioKit Workshop; Nov , 2014.
Jozef Goetz, expanded by Jozef Goetz, 2009 Credits: Parts of the slides are based on slides created by UNIX textbook authors, Syed M. Sarwar, Robert.
CS 403: Programming Languages Lecture 21 Fall 2003 Department of Computer Science University of Alabama Joel Jones.
Regular expressions Used by several different UNIX commands, including ed, sed, awk, grep A period ‘.’ matches any single characters.X. matches any X.
CS 403: Programming Languages Fall 2004 Department of Computer Science University of Alabama Joel Jones.
Advanced File Processing. 2 Objectives Use the pipe operator to redirect the output of one command to another command Use the grep command to search for.
Chapter Five Advanced File Processing Guide To UNIX Using Linux Fourth Edition Chapter 5 Unix (34 slides)1 CTEC 110.
Chapter Five Advanced File Processing. 2 Objectives Use the pipe operator to redirect the output of one command to another command Use the grep command.
Course materials may not be reproduced in whole or in part without the prior written permission of IBM. 5.1 © Copyright IBM Corporation 2008 Unit 10 Linux.
Introduction to Unix (CA263) File Processing (continued) By Tariq Ibn Aziz.
Chapter Five Advanced File Processing. 2 Lesson A Selecting, Manipulating, and Formatting Information.
Chapter Four I/O Redirection1 System Programming Shell Operators.
Advanced Text Processing. 222 Lecture Overview  Character manipulation commands cut, paste, tr  Line manipulation commands sort, uniq, diff  Regular.
TEXT PROCESSING UTILITIES. THE cat COMMAND $ cat emp1.lst $ cat emp1.lst 2233 | shukla | g.m | sales | 12/12/52 | | sharma |d.g.m |product.
CS 124/LINGUIST 180 From Languages to Information Unix for Poets (in 2013) Christopher Manning Stanford University.
Awk- An Advanced Filter by Prof. Shylaja S S Head of the Dept. Dept. of Information Science & Engineering, P.E.S Institute of Technology, Bangalore
– Introduction to the Shell 1/21/2016 Introduction to the Shell – Session Introduction to the Shell – Session 3 · Job control · Start,
CS 124/LINGUIST 180 From Languages to Information
CSC 352– Unix Programming, Spring 2015 February 2015 Unix Filters.
Chapter 10: Working with Large Data Spreadsheet-Based Decision Support Systems Prof. Name Position (123) University Name.
CSC 4630 Meeting 17 March 21, Exam/Quiz Schedule Due to ice, travel, research and other commitments that we all have: –Quiz 2, scheduled for Monday.
ORAFACT Text Processing. ORAFACT Searching Inside Files grep - searches for patterns within files grep [options] [[-e] pattern] filename [...] -n shows.
Uniq The uniq command is useful when you need to find duplicate lines in a file. The basic format of the command is uniq in_file out_file In this format,
UNIX commands Head More (press Q to exit) Cat – Example cat file – Example cat file1 file2 Grep – Grep –v ‘expression’ – Grep –A 1 ‘expression’ – Grep.
FILTERS USING REGULAR EXPRESSIONS – grep and sed.
Lesson 6-Using Utilities to Accomplish Complex Tasks.
CSC 4630 Perl 3 adapted from R. E. Beck. Problem But we worked on it first: Input: Read from a text file named in a command line argument Output: List.
Filters and Utilities. Notes: This is a simple overview of the filtering capability Some of these commands are very powerful ▫Only showing some of the.
Awk 2 – more awk. AWK INVOCATION AND OPERATION the "-F" option allows changing Awk's "field separator" character. Awk regards each line of input data.
SIMPLE FILTERS. CONTENTS Filters – definition To format text – pr Pick lines from the beginning – head Pick lines from the end – tail Extract characters.
Lesson 5-Exploring Utilities
CS 124/LINGUIST 180 From Languages to Information
Agenda Basic Unix Commands (Chapters 2 & 3) Miscellaneous Commands:
Chapter 6 Filters.
CS 403: Programming Languages
Tutorial of Unix Command & shell scriptS 5027
Tutorial of Unix Command & shell scriptS 5027
CS 124/LINGUIST 180 From Languages to Information
UNIT-4 More File Attributes Course code: 10CS44
Guide To UNIX Using Linux Third Edition
Tutorial of Unix Command & shell scriptS 5027
Chapter Four UNIX File Processing.
CS 124/LINGUIST 180 From Languages to Information
Lab 7: Filtering.
Software I: Utilities and Internals
Presentation transcript:

In the last class, Filters and delimiters The sample database pr command head and tail commands cut and paste commands

paste: pasting file (contd.,) We can specify one or more delimiters with -d paste -d “|” cutlist1 cutlist2 Where each field will be separated by the delimiter |

Joining lines (-s) Let us consider that the file address book contains the details of three persons cat addressbook paste -s addressbook -to print in one single line paste -s -d ”| | \n” addressbook -are used in a circular manner

Sort : ordering a file Sorting is the ordering of data in ascending or descending sequence. The sort command orders a file and by default, the entire line is sorted Example: sort shortlist

Sort options -tchar uses delimiter char to identify fields -k n sorts on nth field -k m,n starts sort on mth field and ends sort on nth field -k m.nstarts sort on nth column of mth field -uremoves repeated lines

-n sorts numerically -r reverses sort order -f folds lowercase to equivalent uppercase -m list merges sorted files in list -c checks if file is sorted -o flname places output in file flname

sort –t“|” –k 2 shortlist sort –t”|” –r –k 2 shortlist or sort –t”|” –k 2r shortlist sort –t”|” –k 3,3 –k 2,2 shortlist sort –t”|” –k 5.7,5.8 shortlist sort –n numfile

uniq command Locate repeated and non-repeated lines cat dept.lst uniq dept.lst sort : dept.lst | uniq - uniqlist Selecting the nonrepeated lines (-u) Selecting the duplicate lines (-d) Counting frequency of occurrence (-c)

tr command Manipulates the individual characters in a line. It translates characters using one or two compact expressions tr options expn1 expn2 standard input It takes input only from standard input, it doesn’t take a filename as argument tr ‘|/’ ‘~-’ < emp.lst | head –n 3

exp1=‘|/’ ; exp2=‘~-’ tr “$exp1” “$exp2” < emp.lst Changing case of text head –n 3 emp.lst | tr ‘[a-z]’ ‘[A-Z]’ Deleting charecters (-d) Compressing multiple consecutive charecters (-s) Complementing values of expression (-c) Using ASCII octal values and escape sequences

Summary Filters and delimiters pr command head and tail command cut and paste command sort command uniq command tr command

THANK YOU