1. Perl

2. Introduction Perl stands for "Practical Extraction and Report Language" Perl stands for "Practical Extraction and Report Language" Created by Larry Wall when awk ran out of steam Created by Larry Wall when awk ran out of steam Perl grew at almost the same rate as the Unix operating system Perl grew at almost the same rate as the Unix operating system

3. Introduction (cont.) Perl fills the gaps between program languages of different levels Perl fills the gaps between program languages of different levels A great tool for leverage A great tool for leverage High portability and readily available High portability and readily available Perl can be “write-only,” without proper care during programming Perl can be “write-only,” without proper care during programming

4. Availability It's free and runs rather nicely on nearly everything that calls itself UNIX or UNIX-like It's free and runs rather nicely on nearly everything that calls itself UNIX or UNIX-like Perl has been ported to the Amiga, the Atari ST, the Macintosh family, VMS, OS/2, even MS/DOS and Windows Perl has been ported to the Amiga, the Atari ST, the Macintosh family, VMS, OS/2, even MS/DOS and Windows The sources for Perl (and many precompiled binaries for non-UNIX architectures) are available from the Comprehensive Perl Archive Network (the CPAN). http://www.perl.com/CPAN The sources for Perl (and many precompiled binaries for non-UNIX architectures) are available from the Comprehensive Perl Archive Network (the CPAN). http://www.perl.com/CPAN http://www.perl.com/CPAN

5. Running Perl on Unix Setup path variable to point to the directory where Perl is located Setup path variable to point to the directory where Perl is located Check /usr/local/bin or /usr/bin for “perl” Check /usr/local/bin or /usr/bin for “perl” Run a Perl script by typing “perl ” Run a Perl script by typing “perl ” Alternatively, change the file attribute to executable and include “#!/usr/bin/perl” in the first line of your perl script Alternatively, change the file attribute to executable and include “#!/usr/bin/perl” in the first line of your perl script The.pl extension is frequently associated to Perl scripts The.pl extension is frequently associated to Perl scripts

6. Running Perl on Win32 ActivePerl allows Perl scripts to be executed in MS-DOS/Windows ActivePerl allows Perl scripts to be executed in MS-DOS/Windows Perl was ported faithfully Perl was ported faithfully The #! directive is no longer used because it does not mean anything to MS-DOS/Windows The #! directive is no longer used because it does not mean anything to MS-DOS/Windows Perl scripts are executed by typing “perl Perl scripts are executed by typing “perl Alternatively, double clicking on the file if the extension.pl is associated to the Perl interpreter Alternatively, double clicking on the file if the extension.pl is associated to the Perl interpreter

7. An Example #!/usr/bin/perl print “Hello World!”; The #! directive directs subsequent lines in the file to the perl executable The #! directive directs subsequent lines in the file to the perl executable All statements are terminated with ; as in C/C++/Java All statements are terminated with ; as in C/C++/Java print by default outputs any strings to the terminal console (such as printf in C or cout in C++) print by default outputs any strings to the terminal console (such as printf in C or cout in C++) Perl completely parses and compiles the script before executing it Perl completely parses and compiles the script before executing it

8. Variables Three main types of variables - scalar, hash and array Three main types of variables - scalar, hash and array Examples: $scale, %hash, @array Examples: $scale, %hash, @array Perl is not a strongly typed language Perl is not a strongly typed language Retrieving values from the variables: Retrieving values from the variables: $scale, $hash{key}, $array[offset] $scale, $hash{key}, $array[offset] Variables are all global in scope unless defined to be private or local Variables are all global in scope unless defined to be private or local Note: remember that hash and array are used to hold scalar values Note: remember that hash and array are used to hold scalar values

9. Examples Assigning values to a scalar Assigning values to a scalar $i = “hello world!”; $j = 1 + 1; ($i,$j) = (2, 3) Assigning values to an array Assigning values to an array $array[0] = 1; $array[1] = “hello world!”; push(@array,1); #stores the value 1 in the end of @array $value = pop(@array); #retrieves and removes the last element #from @array @array = (8,@array); #inserts 8 in front of @array

10. Examples (cont.) Assigning values to a hash Assigning values to a hash $ hash{‘greeting’} = “Hello world!”; $hash{‘available’} = 1; #or using a hash slice #or using a hash slice @hash{“greeting”,”available”} = (“Hello world!”, 1); Deleting a key-value pair from a hash: Deleting a key-value pair from a hash: delete $hash{‘key’};

11. Conditional Statements Variables alone will not support switches or conditions Variables alone will not support switches or conditions If-Then-Else like clauses are used to make decisions based on certain preconditions If-Then-Else like clauses are used to make decisions based on certain preconditions Keywords: if, else, elsif, unless Keywords: if, else, elsif, unless Enclosed by ‘{‘ and ‘}’ Enclosed by ‘{‘ and ‘}’

12. A Conditional Statement Example print "What is your name? "; $name = ; chomp ($name); if ($name eq "Randal") { print "Hello, Randal! How good of you to be here!\n"; } else { print "Hello, $name!\n"; # ordinary greeting } unless($name eq “Randal”) { print “You are not Randal!!\n”; #part of the ordinary greeting } $name = reads from standard input $name = reads from standard input chomp is a built-in function that removes newline characters chomp is a built-in function that removes newline characters

13. Loops Conditional statements cannot handle repetitive tasks Conditional statements cannot handle repetitive tasks Keywords: while, foreach, for, until, do-while, do-until Keywords: while, foreach, for, until, do-while, do-until Foreach loop iterates over all of the elements in an array or hash, executing the loop body on each element Foreach loop iterates over all of the elements in an array or hash, executing the loop body on each element For is a shorthand of while loop For is a shorthand of while loop until is the reverse of while until is the reverse of while

14. Loops (cont.) Do-while and do-until loops execute the loop body once before checking for termination Do-while and do-until loops execute the loop body once before checking for termination Statements in the loop body are enclosed by ‘{‘ and ‘}’ Statements in the loop body are enclosed by ‘{‘ and ‘}’

15. While Loop Syntax: Syntax: while(some expression){ statements;…} Example: Example: #prints the numbers 1 – 10 in reverse order $a = 10; while ($a > 0) { print $a; $a = $a – 1; $a = $a – 1;}

16. Until Loop Syntax: Syntax: while(some expression){ statements;…} Example: Example: #prints the numbers 1 – 10 in reverse order $a = 10; until ($a <= 0) { print $a; $a = $a – 1; $a = $a – 1;}

17. Foreach Loop Syntax: Syntax: foreach [ ] (@some-list){ statements… statements…} Example: Example: #prints each elements of @a @a = (1,2,3,4,5); foreach $b (@a) { print $b; }

18. Foreach Loop (cont.) Accessing a hash with keys function: Accessing a hash with keys function: foreach $key (keys (%fred)) { # once for each key of %fred print "at $key we have $fred{$key}\n"; # show key and value } Alternatively: Alternatively: while (($first,$last) = each(%lastname)) { print "The last name of $first is $last\n"; }

19. For Loop Syntax: Syntax: For(initial_exp; test_exp; re-init_exp ) { statements; …} Example: Example: #prints numbers 1-10 for ($i = 1; $i <= 10; $i++) { print "$i "; }

20. Do-While and Do-Until Loops Syntax: Syntax: do {statments; do{ statements; } while some_expression; }until some_expression; Prints the numbers 1-10 in reverse order: Prints the numbers 1-10 in reverse order: $a = 10;$a = 10; do{do{ print $a;print $a; print $a;print $a; $a = $a – 1;$a = $a - 1; $a = $a – 1;$a = $a - 1; }while ($a > 0);}until ($a 0);}until ($a <= 0);

21. Built-in functions shift function shift function Ex: $value = Shift(@fred) is similar to ($x,@fred) = @fred; Ex: $value = Shift(@fred) is similar to ($x,@fred) = @fred; unshift function unshift function Ex: unshift(@fred,$a); # like @fred = ($a,@fred); Ex: unshift(@fred,$a); # like @fred = ($a,@fred); reverse function reverse function @a = (7,8,9); @a = (7,8,9); @b = reverse(@a); # gives @b the value of (9,8,7) @b = reverse(@a); # gives @b the value of (9,8,7) sort function sort function @y = (1,2,4,8,16,32,64); @y = (1,2,4,8,16,32,64); @y = sort(@y); # @y gets 1,16,2,32,4,64,8 @y = sort(@y); # @y gets 1,16,2,32,4,64,8

22. Built-In Functions (cont.) qw function qw function Ex: @words = qw(camel llama alpaca); # is equivalent to @words = (“camel”,”llama”,”alpaca”); Ex: @words = qw(camel llama alpaca); # is equivalent to @words = (“camel”,”llama”,”alpaca”); defined function defined function Returns a Boolean value saying whether the scalar value resulting from an expression has a real value or not Returns a Boolean value saying whether the scalar value resulting from an expression has a real value or not Ex: defined $a; Ex: defined $a; undefined function undefined function Inverse of the defined function Inverse of the defined function

23. Built-In Functions (cont.) uc and ucfirst functions –vs- lc and lcfirst functions uc and ucfirst functions –vs- lc and lcfirst functions = uc( ) = uc( ) = ucfirst( ) = ucfirst( ) $string = “abcde”; $string2 = uc($string); #ABCDE $string3 = ucfirst($string); #Abcde Lc and lcfirst has the reverse effect as uc and ucfirst functions Lc and lcfirst has the reverse effect as uc and ucfirst functions

24. Basic I/O STDIN and STDOUT STDIN and STDOUT STDIN Examples: STDIN Examples: $a = ; $a = ; @a = ; @a = ; while (defined($line = )) while (defined($line = )) { # process $line here } STDOUT Examples: STDOUT Examples: print(list of arguments); print(list of arguments); print “text”; print “text”; printf ([HANDLE], format, list of arguments); printf ([HANDLE], format, list of arguments);

25. Regular Expressions Template to be matched against a string Template to be matched against a string Patterns are enclosed in ‘/’s Patterns are enclosed in ‘/’s Matching against a variable is done by the =~ operator Matching against a variable is done by the =~ operator Syntax: / / Syntax: / / Examples: Examples: $string =~/abc/ #matches “abc” anywhere in $string $string =~/abc/ #matches “abc” anywhere in $string =~ /abc/ #matches “abc” from standard #input =~ /abc/ #matches “abc” from standard #input

26. Creating Patterns Single character patterns: Single character patterns: “.” matches any single character except newline (\n), for example: /a./ “.” matches any single character except newline (\n), for example: /a./ “?” matches zero or one of the preceding characters “?” matches zero or one of the preceding characters Character class can be created by using “[“ and “]”. Range of characters can be abbreviated by using “-”, and a character class can be negated by using the “^” symbol. Character class can be created by using “[“ and “]”. Range of characters can be abbreviated by using “-”, and a character class can be negated by using the “^” symbol. For examples: For examples: [aeiouAEIOU] matches any one of the vowels [aeiouAEIOU] matches any one of the vowels [a-zA-Z] matches any single letter in the English alphabet [a-zA-Z] matches any single letter in the English alphabet [^0-9] matches any single non-digit [^0-9] matches any single non-digit

27. Creating Patterns (cont.) Predefined character class abbreviations: Predefined character class abbreviations: \d == [0-9] \d == [0-9] \D == [^0-9] \D == [^0-9] \w == [a-zA-Z0-9] \w == [a-zA-Z0-9] \W == [^a-zA-Z0-9] \W == [^a-zA-Z0-9] \s == [ \r\t\n\f] \s == [ \r\t\n\f] \s == [^ \r\t\n\f] \s == [^ \r\t\n\f]

28. Creating Patterns (cont.) Multipliers: *, + And {} Multipliers: *, + And {} * matches 0 or more of the preceding character * matches 0 or more of the preceding character ab*c matches a followed by zero or more bs and followed by a c ab*c matches a followed by zero or more bs and followed by a c + Matches 1 or more of the preceding character + Matches 1 or more of the preceding character ab+c matches a followed by one or more bs and followed by a c ab+c matches a followed by one or more bs and followed by a c {} is a general multiplier {} is a general multiplier a{3,5} #matches three to five “a”s in a string a{3,5} #matches three to five “a”s in a string a{3,} #matches three of more “a”s a{3,} #matches three of more “a”s

29. Creating Patterns (cont.) a{3} #matches any string with more than three “a”s in it a{3} #matches any string with more than three “a”s in it Complex patterns can be constructed from these operators Complex patterns can be constructed from these operators For examples: For examples: /a.*ce.*d/ matches strings such as “ a sdffds ce dfssadf z ” /a.*ce.*d/ matches strings such as “ a sdffds ce dfssadf z ”

30. Creating Patterns: Exercises Construct patterns for the following strings: Construct patterns for the following strings: 1. "a xxx c xxxxxxxx c xxx d“ 2. a sequence of numbers 3. three or more digits followed by the string “abc” 4. Strings that have an “a”, one or more “b”s and at least five “c”s 4. Strings that have an “a”, one or more “b”s and at least five “c”s 5. Strings with three vowels next to each other. Hint: try character class and general multiplier

31. Creating Patterns: Exercises Answers: Answers: /a.*c.*d/ /a.*c.*d/ /\d+/ or /[0-9]+/ /\d+/ or /[0-9]+/ /\d\d\d.*abc/ or /\d{3,}abc/ /\d\d\d.*abc/ or /\d{3,}abc/ /ab+c{5,}/ /ab+c{5,}/ /[aeiouAEIOU]{3}/ /[aeiouAEIOU]{3}/ Other possible answers? Other possible answers?

32. Anchoring Patterns No boundaries are defined by the previous patterns No boundaries are defined by the previous patterns Word boundary: \w and \W Word boundary: \w and \W \b and \B is used to indicate word boundaries and vice verse \b and \B is used to indicate word boundaries and vice verse Examples: Examples: /fred\b/ #matches fred, but not frederick /fred\b/ #matches fred, but not frederick /\b\+\b/ #matches “x+y”, but not “x + y”, “++” and ”+”. Why? /\b\+\b/ #matches “x+y”, but not “x + y”, “++” and ”+”. Why? /\bfred\B/ #matches “frederick” but not “fred /\bfred\B/ #matches “frederick” but not “fred

33. Anchoring Patterns (cont.) ^ and $ ^ and $ ^ matches beginning of a string ^ matches beginning of a string $ matches end of a string $ matches end of a string Exampls: Exampls: /^Fred$/ #matches only “Fred” /^Fred$/ #matches only “Fred” /aaa^bbb/ #matches nothing /aaa^bbb/ #matches nothing

34. More on matching operators Additional flags for the matching operator: Additional flags for the matching operator: / /i #ignores case differences / /i #ignores case differences /fred/i #matches FRED,fred,Fred,FreD and etc… /fred/i #matches FRED,fred,Fred,FreD and etc… / /s #treat string as single line / /s #treat string as single line / /m #treat string as multiple line / /m #treat string as multiple line

35. More on Matching Operators (cont.) “(“ and “)” can be used in patterns to remember matches “(“ and “)” can be used in patterns to remember matches Special variables $1, $2, $3 … can be used to access these matches Special variables $1, $2, $3 … can be used to access these matches For example: For example: $string = “Hello World!”; if( $string =~/(\w*) (\w*)) { #prints Hello World print “$1 $2\n”; print “$1 $2\n”;}

36. More on Matching Operators (cont.) Alternatively: Alternatively: $string = “Hello World!”; ($first,$second) = ($string =~/(\w*) (\w*)); print “$first $second\n”; #prints Hello World Line 2: Remember that the =~ returns values just like a function. Normally, it returns 0 or 1, which stands for true or false, but in this case, the existence of “(“ and “)” make it return values of the matching patterns

37. Substitution Replacement of patterns in string Replacement of patterns in string s/ / /ig s/ / /ig i indicates case insensitive i indicates case insensitive g enables the matching to be performed more than once g enables the matching to be performed more than once Examples: Examples: $which = “this this this”; $which =~ s/this/that/; #produces “that this this”

38. Substitution (cont.) $which =~ s/this/that/g; #produces “that that that” $which =~ s/THIS/that/i; #produces “that this this” $which =~ s/THIS/that/ig; #produces “that that that” Multipliers, anchors and memory operators can be used as well: Multipliers, anchors and memory operators can be used as well: $string = “This is a string”; $string =~ s/^/So/; # “So This is a string” $string =~ s/(\w{1,})/I think $1/; # “I think This is a string”

39. Split and Join Functions Syntax: Syntax: = split(/ /[, ]); = split(/ /[, ]); = join(“ ”, ); = join(“ ”, ); Examples: Examples: $string = “This is a string”; @words = split(/ /,$string); #splits the string into #separate words @words = split(/\s/,$string); #same as above $string = join(“ “,@words); #”This is a string” Great functions in parsing formatted documents Great functions in parsing formatted documents

40. Functions Automates certain tasks Automates certain tasks Syntax: Syntax: sub sub {…<statements>} Global to the current package. Since we are not doing OOP and packages, functions are “global” to the whole program Global to the current package. Since we are not doing OOP and packages, functions are “global” to the whole program

41. Functions (cont.) Example: Example: sub say_hello { print “Hello world!\n”; } Invoking a function: Invoking a function: say_hello(); #takes in parameters &say_hello; #no parameters

42. Functions (cont.) Return values Return values Two types of functions: void functions (also known as routine or procedure), and functions Two types of functions: void functions (also known as routine or procedure), and functions void functions have no return values void functions have no return values Functions in Perl can return more than one variable: Functions in Perl can return more than one variable: sub threeVar { return ($a, $b, $c); #returns a list of 3 variables }

43. Functions (cont.) ($one,$two,$three) = threeVar(); Alternatively: Alternatively: @list = threeVar(); #stores the three values into a list Note: Note: ($one, @two, $three) = threeVar(); #$three will not have #any value, why?

44. Functions (cont.) Functions can’t do much without parameters Functions can’t do much without parameters Parameters to a function are stored as a list with the @_ variable Parameters to a function are stored as a list with the @_ variable Example: Example: sub say_hello_two { $string = @_; #gets the value of the parameter $string = @_; #gets the value of the parameter} Invocation: Invocation: say_hello_two(“hello world!\n”);

45. Functions (cont.) For example: For example: sub add { ($left,$right) = @_; return $left + $right; return $left + $right;} $three = add(1,2);

46. Functions (cont.) Variables are all global even if they are defined within a function Variables are all global even if they are defined within a function my keyword defines a variable as being private to the scope it is defined my keyword defines a variable as being private to the scope it is defined For example: For example: sub add { my($left,$right) = @_; return $left + $right; return $left + $right;}

47. Functions (cont.) $three = add(1,2); #$three gets the value of 3 print “$one\n”; #prints 0 print “$one\n”; #prints 0 Print “$two\n”; #prints 0 Print “$two\n”; #prints 0

48. Exercises A trim() function that removes leading and trailing spaces in a string A trim() function that removes leading and trailing spaces in a string Hint: use the s/// operator in conjunction with anchors Hint: use the s/// operator in conjunction with anchors A date() function that converts date string, “DD:MM:YY” to “13th of December, 2003” A date() function that converts date string, “DD:MM:YY” to “13th of December, 2003” Hint: use a hash table to create a lookup table for the month strings. Hint: use a hash table to create a lookup table for the month strings.

49. File I/O Filehandle Filehandle Automatic filehandles: STDIN, STDOUT and STDERR Automatic filehandles: STDIN, STDOUT and STDERR Syntax: Syntax: open(,”( |>>)filename”); close( ); Example: Example: open(INPUTFILE,”<inputs.txt”); #opens file handle … Close(INPUTFILE); #closes file handle

50. File I/O (cont.) Handle access does not always yield true Handle access does not always yield true Check for return value of the open function Check for return value of the open function Example: Example: if(open(INPUT,”<inputs.txt”)) if(open(INPUT,”<inputs.txt”)) … #do something else print “File open failed\n”;

51. File I/O (cont.) The previous method is the standard practice The previous method is the standard practice Unlike other languages, Perl is for lazy people Unlike other languages, Perl is for lazy people Ifs can be simplified by the logical operator “||” Ifs can be simplified by the logical operator “||” For example: For example: open(INPUT,”<inputs.txt”) ||die “File open failed\n”; Use $! variable to display additional operating system errors Use $! variable to display additional operating system errors die “cannot append $!\n”; die “cannot append $!\n”;

52. File I/O (cont.) Filehandles are similar to standard I/O handles Filehandles are similar to standard I/O handles <> operator to read lines <> operator to read lines For example: For example:open(INPUT,”<inputs.txt”);while(<INPUT>){chomp; print “$_\n”; } Use print to output to a file Use print to output to a file

53. File I/O (cont.) File copy example: File copy example: open(IN,$a) || die "cannot open $a for reading: $!"; open(OUT,">$b") || die "cannot create $b: $!"; while ( ) { # read a line from file $a into $_ print OUT $_; # print that line to file $b print OUT $_; # print that line to file $b} close(IN) || die "can't close $a: $!"; close(OUT) || die "can't close $b: $!";

54. File I/O (cont.) File tests provides convenience for programmers File tests provides convenience for programmers -e –r –w –x –d –f –l –T –B -e –r –w –x –d –f –l –T –B For example: For example: if(-f $name){ print “$name is a file\n”; } elsif(-d $name){ print “$name is a directory\n”; }

55. Special Variables $_, @_ $_, @_ $1, $2… - backreferencing variables $1, $2… - backreferencing variables $_ = "this is a test"; $_ = "this is a test"; /(\w+)\W+(\w+)/; # $1 is "this" and $2 is "is" $`, $& and $’ - match variables $`, $& and $’ - match variables $string = “this is a simple string”; /si.*le/; #$& is now “sample”, $` is “this is a” and $’ is #“string” And many more…refer to ActivePerl’s online documentation for more functions And many more…refer to ActivePerl’s online documentation for more functions

56. Packages and Modules Concentrate only on their usage in the Greenstone environment Concentrate only on their usage in the Greenstone environment Package: a mechanism to protect code from tampering with other package’s variables Package: a mechanism to protect code from tampering with other package’s variables Module: reusable package that is stored in.dm Module: reusable package that is stored in.dm The ppm (Perl Package Manager) for Linux and Win32 version of Perl manages installation and uninstallation of Perl packages The ppm (Perl Package Manager) for Linux and Win32 version of Perl manages installation and uninstallation of Perl packages

57. Packages and Modules (cont.) Install the module and put “use ModuleName” or “require ModuleName” near the top of the program Install the module and put “use ModuleName” or “require ModuleName” near the top of the program :: qualifying operator allows references to things in the package, such as $Module::Variable :: qualifying operator allows references to things in the package, such as $Module::Variable So “use Math::Complex module” refers to the module Math/Complex.pm So “use Math::Complex module” refers to the module Math/Complex.pm new creates an instance of the object, then use the handle and operator -> to access its functions new creates an instance of the object, then use the handle and operator -> to access its functions

58. Packages and Modules (cont.) use accepts a list of strings as well, so we can access the elements directly without the qualifying operator use accepts a list of strings as well, so we can access the elements directly without the qualifying operator For example: For example: use Module qw(const1 const2 func1 func2 func3); const1, const2, func1, func2 and func3 can now be used directly in the program const1, const2, func1, func2 and func3 can now be used directly in the program

59. Packages and Modules (cont.) Perl locates modules by searching the @INC array Perl locates modules by searching the @INC array The first instance found will be used for the module referenced within a program The first instance found will be used for the module referenced within a program Where to locate modules is an automatic process as the Makefiles and PPM take care to place modules in the correct path Where to locate modules is an automatic process as the Makefiles and PPM take care to place modules in the correct path

60. Packages and Modules (cont.) An example that uses the package CGI.pm: An example that uses the package CGI.pm: use CGI; #uses the CGI.pm module $query = CGI::new(); #creates an instance of CGI $bday = $query->param("birthday"); #gets a named parameter print $query->header(); #outputs html header print $query->p("Your birthday is $bday."); #outputs text to html

61. Packages and Modules (cont.) Advantages: Encourages code reuse and less work Advantages: Encourages code reuse and less work Disadvantages: 33% as fast as procedural Perl according to the book “object-oriented Perl”, generation of Perl modules involves some ugly code Disadvantages: 33% as fast as procedural Perl according to the book “object-oriented Perl”, generation of Perl modules involves some ugly code

62. Packages and Modules (cont.) Huge library Huge library