Programming the Apache Lifecycle

Programming the Apache Lifecycle
Geoffrey Young

Overview

Overview Apache and mod_perl 101

Overview Apache and mod_perl 101 mod_perl Handler Basics

Using the Apache Framework

Using the Apache Framework Advanced mod_perl API Features

Apache's Pre-fork Model

Apache parent process

Apache parent process httpd (parent)

Apache parent process forks multiple child processes httpd (parent) httpd (child) httpd (child) httpd (child) httpd (child)

Roles and Responsibilities

httpd parent processes no actual requests

httpd parent processes no actual requests all requests are served by the child processes

httpd parent processes no actual requests all requests are served by the child processes requests are handled by any available child process, not necessarily the one that handled the previous request

httpd parent processes no actual requests all requests are served by the child processes requests are handled by any available child process, not necessarily the one that handled the previous request remember, the HTTP is stateless by default

Children are Individuals
httpd (parent) httpd (child) httpd (child) httpd (child) httpd (child)

Nice, Responsible Children

each httpd child processes one incoming request at a time

each httpd child processes one incoming request at a time only when the request is over is the child free to serve the next request

each httpd child processes one incoming request at a time only when the request is over is the child free to serve the next request over time httpd child processes are terminated and replaced with fresh children

Request Phases

Request Phases Apache breaks down request processing into separate, logical parts called phases

Request Phases Apache breaks down request processing into separate, logical parts called phases client request logging URI-based init content URI translation fixups file-based init MIME setting resource control

Request Phases Apache breaks down request processing into separate, logical parts called phases each request is stepped through the phases until...

Request Phases Apache breaks down request processing into separate, logical parts called phases each request is stepped through the phases until... all processing is complete

Request Phases Apache breaks down request processing into separate, logical parts called phases each request is stepped through the phases until... all processing is complete somebody throws an "error"

Request Phases Apache breaks down request processing into separate, logical parts called phases each request is stepped through the phases until... all processing is complete somebody throws an "error" developers are given the chance to hook into each phase to add custom processing

Apache Request Cycle client request

Apache Request Cycle client request GET /perl-status HTTP/1.1
Accept: text/html, image/png, image/jpeg, image/gif, image/x-xbitmap, */* Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0 Accept-Language: en Cache-Control: no-cache Connection: Keep-Alive, TE Host: User-Agent: Mozilla/4.0 (compatible; MSIE 5.0; Windows 2000) Opera [en]

Apache Request Cycle client request URI-based init

Apache Request Cycle client request URI-based init URI translation

file-based init

file-based init resource control

file-based init MIME setting resource control

fixups file-based init MIME setting resource control

Apache Request Cycle client request URI-based init content
URI translation fixups file-based init MIME setting resource control

Apache Request Cycle content HTTP/1.1 200 OK
Keep-Alive: timeout=15, max=100 Connection: Keep-Alive Transfer-Encoding: chunked Content-Type: text/html

Apache Request Cycle client request URI-based init content

Apache Request Cycle client request logging URI-based init content

So What? most Apache users don't worry about the request cycle too much...

So What? most Apache users don't worry about the request cycle too much... ...but they do use modules that plug into it

... for instance mod_rewrite:
client request URI-based init mod_rewrite: RewriteRule /favicon.ico$ /images/favicon.ico URI translation

... for instance mod_auth: AuthUserFile .htpasswd client request
URI-based init URI translation file-based init mod_auth: AuthUserFile .htpasswd resource control

... for instance mod_cgi: SetHandler cgi-script client request
URI-based init content URI translation fixups file-based init MIME setting resource control

That's great breaking down the request into distinct phases has many benefits

That's great breaking down the request into distinct phases has many benefits gives each processing point a role that can be easily managed and programmed

That's great breaking down the request into distinct phases has many benefits gives each processing point a role that can be easily managed and programmed makes Apache more like an application framework rather than a content engine

That's great, but... breaking down the request into distinct phases has many benefits gives each processing point a role that can be easily managed and programmed makes Apache more like an application framework rather than a content engine but you have to code in C

Enter mod_perl

Enter mod_perl mod_perl offers an interface to each phase of the request cycle

Enter mod_perl mod_perl offers an interface to each phase of the request cycle opens up the Apache API to Perl code

Enter mod_perl mod_perl offers an interface to each phase of the request cycle opens up the Apache API to Perl code allows you to program targeted parts of the request cycle using Perl

Enter mod_perl mod_perl offers an interface to each phase of the request cycle opens up the Apache API to Perl code allows you to program targeted parts of the request cycle using Perl we like Perl

mod_perl Interface client request PerlCleanupHandler PerlLogHandler
PerlPostReadRequestHandler logging URI-based init PerlHandler PerlTransHandler content URI translation PerlFixupHandler PerlHeaderParserHandler fixups file-based init PerlAccessHandler PerlAuthenHandler PerlAuthzHandler PerlTypeHandler MIME setting resource control

mod_perl Interface world's ugliest slide client request
PerlCleanupHandler PerlLogHandler PerlPostReadRequestHandler world's ugliest slide logging URI-based init PerlHandler PerlTransHandler content URI translation PerlFixupHandler PerlHeaderParserHandler fixups file-based init PerlAccessHandler PerlAuthenHandler PerlAuthzHandler PerlTypeHandler MIME setting resource control

mod_perl Interface client request PerlCleanupHandler PerlLogHandler
PerlPostReadRequestHandler logging URI-based init PerlHandler PerlTransHandler content URI translation PerlFixupHandler PerlHeaderParserHandler fixups file-based init PerlAccessHandler PerlAuthenHandler PerlAuthzHandler PerlTypeHandler MIME setting resource control

What is mod_perl?

What is mod_perl? mod_perl is the Perl interface to the Apache API

a C extension module, just like mod_cgi or mod_rewrite

a C extension module, just like mod_cgi or mod_rewrite creates a persistent perl environment embedded within Apache

What's the Big Deal?

What's the Big Deal? mod_perl allows you to interact with and directly alter server behavior

What's the Big Deal? mod_perl allows you to interact with and directly alter server behavior gives you the ability to "program within Apache's framework instead of around it"

What's the Big Deal? mod_perl allows you to interact with and directly alter server behavior gives you the ability to "program within Apache's framework instead of around it" let's you do it in Perl instead of C

What's the Big Deal? mod_perl allows you to interact with and directly alter server behavior gives you the ability to "program within Apache's framework instead of around it" let's you do it in Perl instead of C allows you to intercept basic Apache functions and replace them with your own (sometimes devious) Perl substitutes

Registry is just a handler

Apache::Registry is merely an (incredibly clever and amazing) mod_perl handler

Apache::Registry is merely an (incredibly clever and amazing) mod_perl handler its performance gains are made possible due to what mod_perl really is

Apache::Registry is merely an (incredibly clever and amazing) mod_perl handler its performance gains are made possible due to what mod_perl really is let's take a peek inside...

Apache::Registry

Apache::Registry Client side http://localhost/perl-bin/bar.pl

Apache::Registry Client side Server side
Server side

Server side mod_perl intercepts content generation

Server side mod_perl intercepts content generation for Apache/Registry.pm

Server side mod_perl intercepts content generation for Apache/Registry.pm calls Apache::Registry::handler(Apache->request)

Server side mod_perl intercepts content generation for Apache/Registry.pm calls Apache::Registry::handler(Apache->request) inserts wizardry

Server side mod_perl intercepts content generation for Apache/Registry.pm calls Apache::Registry::handler(Apache->request) inserts wizardry returns response to client

Wizardry, you say?

Wizardry, you say? the wizardry is basically just putting the CGI script into it's own package

Wizardry, you say? the wizardry is basically just putting the CGI script into it's own package package Apache::ROOT::perl_2dbin::foo_2epl; sub handler { BEGIN { $^W = 1; }; ... your script here... } 1;

Wizardry, you say? the wizardry is basically just putting the CGI script into it's own package package Apache::ROOT::perl_2dbin::foo_2epl; sub handler { BEGIN { $^W = 1; }; ... your script here... } 1; because the perl interpreter is persistent the (compiled) package is already in memory when called

"The dream is always the same"

the basic process for mod_perl is the same for the other request phases as for content generation (eg, Registry)

the basic process for mod_perl is the same for the other request phases as for content generation (eg, Registry) Apache passes control to mod_perl

the basic process for mod_perl is the same for the other request phases as for content generation (eg, Registry) Apache passes control to mod_perl mod_perl passes control to your Perl handler

the basic process for mod_perl is the same for the other request phases as for content generation (eg, Registry) Apache passes control to mod_perl mod_perl passes control to your Perl handler your Perl subroutine defines the status

the basic process for mod_perl is the same for the other request phases as for content generation (eg, Registry) Apache passes control to mod_perl mod_perl passes control to your Perl handler your Perl subroutine defines the status mod_perl passes status back to Apache

the basic process for mod_perl is the same for the other request phases as for content generation (eg, Registry) Apache passes control to mod_perl mod_perl passes control to your Perl handler your Perl subroutine defines the status mod_perl passes status back to Apache Apache continues along

Key Questions

Key Questions What is the Apache default behavior for the phase?

What is a typical mod_perl usage of the phase?

What is a typical mod_perl usage of the phase? What happens on success?

What is a typical mod_perl usage of the phase? What happens on success? What happens on error?

The Client Request client request

The Client Request client request GET /perl-status HTTP/1.1
Accept: text/html, image/png, image/jpeg, image/gif, image/x-xbitmap, */* Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0 Accept-Language: en Cache-Control: no-cache Connection: Keep-Alive, TE Host: User-Agent: Mozilla/4.0 (compatible; MSIE 5.0; Windows 2000) Opera [en]

URI-based Initialization
client request URI-based init

Request URI and headers are known

Request URI and headers are known Apache request record has been populated

Request URI and headers are known Apache request record has been populated The first place you can insert processing

Request URI and headers are known Apache request record has been populated The first place you can insert processing Apache has no default behavior

Request URI and headers are known Apache request record has been populated The first place you can insert processing Apache has no default behavior All configured handlers are run

client request URI-based init

client request PerlPostReadRequestHandler

client request PerlInitHandler

PerlPostReadRequestHandler

Non-specific hook

Non-specific hook Useful for adding processing that needs to occur on every request

Non-specific hook Useful for adding processing that needs to occur on every request every request means every request

A sample... Object: to protect our name-based virtual hosts from HTTP/1.0 requests Apache can't handle

HTTP/1.0 and Host

HTTP/1.0 and Host HTTP/1.0 does not require a Host header

assumes a “one host per IP" configuration

assumes a “one host per IP" configuration this limitation "breaks" name-based virtual host servers for browsers that follow HTTP/1.0 to the letter

assumes a “one host per IP" configuration this limitation "breaks" name-based virtual host servers for browsers that follow HTTP/1.0 to the letter most send the Host header, so all is well

Connected to www.apache.org. Escape character is '^]'.
$ telnet 80 Trying Connected to Escape character is '^]'. GET /foo.html HTTP/1.0

$ telnet 80 Trying Connected to Escape character is '^]'. GET /foo.html HTTP/1.0 HTTP/ Found Date: Tue, 04 Jun :52:55 GMT Server: Apache/ dev (Unix) Location: Content-Length: 289 Connection: close Content-Type: text/html; charset=iso <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>302 Found</title> </head><body> <h1>Found</h1> <p>The document has moved <a href=" <hr /> <address>Apache/ dev Server at dev.apache.org Port 80</address> </body></html> Connection closed by foreign host.

$ telnet 80 Trying Connected to Escape character is '^]'. GET /foo.html HTTP/1.0 Host:

$ telnet 80 Trying Connected to Escape character is '^]'. GET /foo.html HTTP/1.0 Host: HTTP/ Not Found Date: Tue, 04 Jun :56:40 GMT Server: Apache/ dev (Unix) Content-Length: 283 Content-Type: text/html; charset=iso <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>404 Not Found</title> </head><body> <h1>Not Found</h1> <p>The requested URL /foo.html was not found on this server.</p> <hr /> <address>Apache/ dev Server at Port 80</address> </body></html>

A sample... Object: to protect our name-based virtual hosts from HTTP/1.0 requests Apache can't handle

A sample... Object: to protect our name-based virtual hosts from HTTP/1.0 requests Apache can't handle Method: intercept every request prior to content-generation and return an error unless...

A sample... Object: to protect our name-based virtual hosts from HTTP/1.0 requests Apache can't handle Method: intercept every request prior to content-generation and return an error unless... there is a Host header

A sample... Object: to protect our name-based virtual hosts from HTTP/1.0 requests Apache can't handle Method: intercept every request prior to content-generation and return an error unless... there is a Host header the request is an absolute URI

package Cookbook::TrapNoHost;
use Apache::Constants qw(DECLINED BAD_REQUEST); use Apache::URI; use strict; sub handler { my $r = shift; # Valid requests for name based virtual hosting are: # requests with a Host header, or # requests that are absolute URIs. unless ($r->headers_in->get('Host') || $r->parsed_uri->hostname) { $r->custom_response(BAD_REQUEST, "Oops! Did you mean to omit a Host header?\n"); return BAD_REQUEST; } return DECLINED; 1;

use Apache::Constants qw(DECLINED BAD_REQUEST); use Apache::URI; use strict; sub handler { my $r = Apache->request; # Valid requests for name based virtual hosting are: # requests with a Host header, or # requests that are absolute URIs. unless ($r->headers_in->get('Host') || $r->parsed_uri->hostname) { $r->custom_response(BAD_REQUEST, "Oops! Did you mean to omit a Host header?\n"); return BAD_REQUEST; } return DECLINED; 1;

Setup add TrapNoHost.pm

Setup add TrapNoHost.pm to @INC
ServerRoot/lib/perl/Cookbook/TrapNoHost.pm

Setup add TrapNoHost.pm to @INC add to httpd.conf
ServerRoot/lib/perl/Cookbook/TrapNoHost.pm add to httpd.conf

ServerRoot/lib/perl/Cookbook/TrapNoHost.pm add to httpd.conf PerlModule Cookbook::TrapNoHost

ServerRoot/lib/perl/Cookbook/TrapNoHost.pm add to httpd.conf PerlModule Cookbook::TrapNoHost PerlInitHandler Cookbook::TrapNoHost

Setup add TrapNoHost.pm to @INC add to httpd.conf that's it!
ServerRoot/lib/perl/Cookbook/TrapNoHost.pm add to httpd.conf PerlModule Cookbook::TrapNoHost PerlInitHandler Cookbook::TrapNoHost that's it!

Intercept the Request client request PerlPostReadRequestHandler

Intercept the Request client request PerlPostReadRequestHandler HTTP/ Bad Request Date: Tue, 04 Jun :17:52 GMT Server: Apache/ dev (Unix) mod_perl/1.27_01-dev Perl/v5.8.0 Connection: close Content-Type: text/html; charset=iso Oops! Did you mean to omit a Host header?

Intercept the Request client request logging PerlPostReadRequestHandler

Key Concepts

Key Concepts The Apache request object, $r

Key Concepts The Apache request object, $r The Apache::Table class

Return values and the Apache::Constants class

Return values and the Apache::Constants class Stacked handlers

Apache Request Object

Apache Request Object passed to handlers or available via Apache->request()

Apache Request Object passed to handlers or available via Apache->request() $r = shift; # passed to handler()

Apache Request Object passed to handlers or available via Apache->request() $r = shift; # passed to handler() $r = Apache->request();

Apache Request Object passed to handlers or available via Apache->request() $r = shift; # passed to handler() $r = Apache->request(); provides access to the Apache class, which provides access to request attributes

Apache Request Object passed to handlers or available via Apache->request() $r = shift; # passed to handler() $r = Apache->request(); provides access to the Apache class, which provides access to request attributes singleton-like constructor, always returning the same object

Apache::Table

Apache::Table the Apache::Table class provides the underlying API for the following request attributes...

Apache::Table the Apache::Table class provides the underlying API for the following request attributes... $r->headers_in() $r->headers_out() $r->err_headers_out() $r->dir_config() $r->subprocess_env() $r->notes() Apache::Request::param() Apache::Request::parms() in old releases

Apache::Table to manipulate Apache::Table objects, you use the provided methods

Apache::Table to manipulate Apache::Table objects, you use the provided methods get() set() add() unset() do() merge() new() clear()

Apache Table Properties

Apache tables have some nice properties

Apache tables have some nice properties case insensitive

Apache tables have some nice properties case insensitive allow for multi-valued keys

Apache tables have some nice properties case insensitive allow for multi-valued keys they also have one important limitation

Apache tables have some nice properties case insensitive allow for multi-valued keys they also have one important limitation can contain only simple strings

Apache tables have some nice properties case insensitive allow for multi-valued keys they also have one important limitation can contain only simple strings use pnotes() for Perl scalars

keeps the Net flowin'...

keeps the Net flowin'... both case-insensitivity and multiple values are key to manipulating headers

keeps the Net flowin'... both case-insensitivity and multiple values are key to manipulating headers $r->headers_out->set('Set-Cookie' => 'name=foo');

keeps the Net flowin'... both case-insensitivity and multiple values are key to manipulating headers $r->headers_out->set('Set-Cookie' => 'name=foo'); $r->headers_out->add('set-cookie' => 'name=bar');

keeps the Net flowin'... both case-insensitivity and multiple values are key to manipulating headers $r->headers_out->set('Set-Cookie' => 'name=foo'); $r->headers_out->add('set-cookie' => 'name=bar'); = $r->headers_out->get('Set-cookie');

Table Iteration

Table Iteration Apache::Table objects use a special idiom when you need to operate on every item in a table

Table Iteration Apache::Table objects use a special idiom when you need to operate on every item in a table my $input = $apr->param; # Apache::Table object $input->do(sub { my ($key, $value) $log->info("input: name = $key, value = $value"); 1; });

More Apache::Table Fun

Tired

Tired PerlSetVar Sails "jib"

Tired PerlSetVar Sails "jib" my $sail = $r->dir_config('Sails');

Tired PerlSetVar Sails "jib" my $sail = $r->dir_config('Sails'); Wired PerlSetVar Sails "spinnaker"

Tired PerlSetVar Sails "jib" my $sail = $r->dir_config('Sails'); Wired PerlSetVar Sails "spinnaker" PerlAddVar Sails "blooper"

Tired PerlSetVar Sails "jib" my $sail = $r->dir_config('Sails'); Wired PerlSetVar Sails "spinnaker" PerlAddVar Sails "blooper" = $r->dir_config->get('Sails');

Trickery

Trickery really understanding the Apache::Table class will make you a better mod_perl programmer

Trickery really understanding the Apache::Table class will make you a better mod_perl programmer every table can be set

Trickery really understanding the Apache::Table class will make you a better mod_perl programmer every table can be set $r->headers_in() $r->headers_out() $r->err_headers_out() $r->dir_config() $r->subprocess_env() $r->notes() Apache::Request::param()

Trickery handle "what if?" cases

Trickery handle "what if?" cases
$r->headers_in->set('Set-Cookie' => 'name=foo'); my $sub = $r->lookup_uri('/scripts/foo.html');

Trickery handle "what if?" cases gratuitous exploitation
$r->headers_in->set('Set-Cookie' => 'name=foo'); my $sub = $r->lookup_uri('/scripts/foo.html'); gratuitous exploitation

$r->headers_in->set('Set-Cookie' => 'name=foo'); my $sub = $r->lookup_uri('/scripts/foo.html'); gratuitous exploitation # configure "PerlSetVar Filter On" on-the-fly

$r->headers_in->set('Set-Cookie' => 'name=foo'); my $sub = $r->lookup_uri('/scripts/foo.html'); gratuitous exploitation # configure "PerlSetVar Filter On" on-the-fly $r->dir_config->set(Filter => 'On');

Apache::Constants

Apache::Constants Apache::Constants class provides over 90 runtime constants use Apache::Constants qw(DECLINED BAD_REQUEST);

Apache::Constants Apache::Constants class provides over 90 runtime constants use Apache::Constants qw(DECLINED BAD_REQUEST); the most common are:

Apache::Constants Apache::Constants class provides over 90 runtime constants use Apache::Constants qw(DECLINED BAD_REQUEST); the most common are: OK

Apache::Constants Apache::Constants class provides over 90 runtime constants use Apache::Constants qw(DECLINED BAD_REQUEST); the most common are: OK SERVER_ERROR

Apache::Constants Apache::Constants class provides over 90 runtime constants use Apache::Constants qw(DECLINED BAD_REQUEST); the most common are: OK SERVER_ERROR REDIRECT

Apache::Constants Apache::Constants class provides over 90 runtime constants use Apache::Constants qw(DECLINED BAD_REQUEST); the most common are: OK SERVER_ERROR REDIRECT DECLINED

Return Values handlers are expected to return a value

the return value of the handler defines the status of the request

the return value of the handler defines the status of the request Apache defines three "good" return values

the return value of the handler defines the status of the request Apache defines three "good" return values OK – all is well

the return value of the handler defines the status of the request Apache defines three "good" return values OK – all is well DECLINED – forget about me

the return value of the handler defines the status of the request Apache defines three "good" return values OK – all is well DECLINED – forget about me DONE – we're finished, start to log

the return value of the handler defines the status of the request Apache defines three "good" return values OK – all is well DECLINED – forget about me DONE – we're finished, start to log All other values are "errors" and trigger the ErrorDocument cycle

When handlers turn bad...

When handlers turn bad... "error" return codes are not always errors

instead, they indicate a new route for the request

instead, they indicate a new route for the request errors codes take effect immediately

instead, they indicate a new route for the request errors codes take effect immediately other scheduled handlers are not run

package My::Redirect; use Apache::Constants qw(REDIRECT);
sub handler { my $r = shift; $r->headers_out->set(Location => '/foo'); return REDIRECT; }; 1;

package My::Redirect; use Apache::Constants qw(REDIRECT);
sub handler { my $r = shift; $r->headers_out->set(Location => '/foo'); return REDIRECT; }; 1; <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <HTML><HEAD> <TITLE>302 Found</TITLE> </HEAD><BODY> <H1>Found</H1> The document has moved <A HREF="/foo">here</A>.<P> <HR> <ADDRESS>Apache/ dev Server at mainsheet.laserlink.com Port 80</ADDRESS> </BODY></HTML>

Stacked Handlers

Stacked Handlers Apache allows for more than one module to handle each phase

Stacked Handlers Apache allows for more than one module to handle each phase except content-generation

Stacked Handlers Apache allows for more than one module to handle each phase except content-generation (in 1.3)

Stacked Handlers Apache allows for more than one module to handle each phase except content-generation (in 1.3) For example, both mod_rewrite and mod_alias schedule fixup processing

Stacked Handlers Apache allows for more than one module to handle each phase except content-generation (in 1.3) For example, both mod_rewrite and mod_alias schedule fixup processing mod_perl is just another Apache module

Stacked Handlers Apache allows for more than one module to handle each phase except content-generation (in 1.3) For example, both mod_rewrite and mod_alias schedule fixup processing mod_perl is just another Apache module happens to schedule processing for every phase

Perl Stacked Handlers

Perl Stacked Handlers for each phase of the request, mod_perl will run any registered Perl handlers for that phase

Perl Stacked Handlers for each phase of the request, mod_perl will run any registered Perl handlers for that phase you can register more than one Perl handler per phase

Perl Stacked Handlers for each phase of the request, mod_perl will run any registered Perl handlers for that phase you can register more than one Perl handler per phase whether all handlers are called depends on the syntax of the phase itself in Apache

Perl Stacked Handlers some phases run until the handler list is exhausted

Perl Stacked Handlers some phases run until the handler list is exhausted PerlInitHandler Cookbook::TrapNoHost PerlInitHandler Apache::Reload

Perl Stacked Handlers some phases run until the handler list is exhausted PerlInitHandler Cookbook::TrapNoHost PerlInitHandler Apache::Reload some phases run until one handler returns OK

Perl Stacked Handlers some phases run until the handler list is exhausted PerlInitHandler Cookbook::TrapNoHost PerlInitHandler Apache::Reload some phases run until one handler returns OK PerlTransHandler My::TranslateHTML PerlTransHandler My::TranslateText

Perl Stacked Handlers some phases run until the handler list is exhausted PerlInitHandler Cookbook::TrapNoHost PerlInitHandler Apache::Reload some phases run until one handler returns OK PerlTransHandler My::TranslateHTML PerlTransHandler My::TranslateText all phases terminate on "error"

Return values and the Apache::Constants class Stacked handlers

URI Translation client request URI-based init URI translation

URI Translation Apache needs to map the URI to a physical file on disk

URI Translation Apache needs to map the URI to a physical file on disk
Default is to prepend DocumentRoot to the URI DocumentRoot /usr/local/apache/htdocs

URI Translation Directives like Alias override the default

DocumentRoot /usr/local/apache/htdocs Alias /manual/ /usr/local/apache/manual/" <Directory /usr/local/apache/manual> ...

DocumentRoot /usr/local/apache/htdocs Alias /manual/ /usr/local/apache/manual/ <Directory /usr/local/apache/manual> ... Some URIs have no associated file, but Apache tries anyway

DocumentRoot /usr/local/apache/htdocs Alias /manual/ /usr/local/apache/manual/ <Directory /usr/local/apache/manual> ... Some URIs have no associated file, but Apache tries anyway <Location server-status>

URI Translation client request URI-based init URI translation

URI Translation client request URI-based init PerlTransHandler

PerlTransHandler

PerlTransHandler Useful for overriding the Apache default

Allows you to be extremely devious

Allows you to be extremely devious There are a few pitfalls of which to be aware

Simple PerlTransHandler

Object: be rid of those silly favicon.ico requests that end up 404

Object: be rid of those silly favicon.ico requests that end up 404 Method: translate the incoming URI to a common place if it matches favicon.ico

package Cookbook::Favicon; use Apache::Constants qw(DECLINED);
use strict; sub handler { my $r = shift; $r->uri("/images/favicon.ico") if $r->uri =~ m!/favicon\.ico$!; return DECLINED; } 1;

Setup add Favicon.pm

Setup add Favicon.pm to @INC ServerRoot/lib/perl/Cookbook/Favicon.pm

Setup add Favicon.pm to @INC add to httpd.conf
ServerRoot/lib/perl/Cookbook/Favicon.pm add to httpd.conf

ServerRoot/lib/perl/Cookbook/Favicon.pm add to httpd.conf PerlModule Cookbook::Favicon

ServerRoot/lib/perl/Cookbook/Favicon.pm add to httpd.conf PerlModule Cookbook::Favicon PerlTransHandler Cookbook::Favicon

Setup add Favicon.pm to @INC add to httpd.conf that's it!
ServerRoot/lib/perl/Cookbook/Favicon.pm add to httpd.conf PerlModule Cookbook::Favicon PerlTransHandler Cookbook::Favicon that's it!

Why Not Use mod_rewrite?

our Cookbook::Favicon is pretty much the same as RewriteRule /favicon.ico$ /images/favicon.ico

our Cookbook::Favicon is pretty much the same as RewriteRule /favicon.ico$ /images/favicon.ico Let's look at a more clever example...

Mischievous Behavior

Mischievous Behavior Simple URI re-mapping is only the beginning

Apache has this neat, built-in functionality called proxying

Apache has this neat, built-in functionality called proxying provided you have mod_proxy installed

Apache has this neat, built-in functionality called proxying provided you have mod_proxy installed With mod_perl and mod_proxy you can proxy just about anything...

Advanced PerlTransHandler
Object: create a proxy that uses our local Apache documentation instead of ASF servers

Advanced PerlTransHandler
Object: create a proxy that uses our local Apache documentation instead of ASF servers Method: intercept proxy requests and silently replace calls to with /usr/local/apache/htdocs/manual

Client Setup

package My::ManualProxy; use Apache::Constants qw(OK DECLINED);
use strict; sub handler { my $r = shift; return DECLINED unless $r->proxyreq; my (undef, $file) = $r->uri =~ m!^ if ($file =~ m!^docs/!) { $file =~ s!^docs/!manual/!; $file = join "/", ($r->document_root, $file); if (-f $file) { $r->filename($file); # use local disk $r->proxyreq(0); # fool mod_mime return OK; } return DECLINED; 1;

Proxy in Action GET HTTP/1.1 Accept: text/html, image/png, image/jpeg, image/gif, image/x-xbitmap, */* Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0 Accept-Language: en Connection: Keep-Alive, TE Host: httpd.apache.org User-Agent: Mozilla/4.0 (compatible; MSIE 5.0; Windows 2000) Opera 5.12 [en] HTTP/ OK Last-Modified: Sat, 25 May :15:27 GMT ETag: "240c cf00cff" Accept-Ranges: bytes Content-Length: 13152 Keep-Alive: timeout=15, max=100 Connection: Keep-Alive Content-Type: text/html

Winner Takes All

Winner Takes All The first URI translation handler to return OK ends the phase

Winner Takes All The first URI translation handler to return OK ends the phase for both mod_perl and Apache!

Winner Takes All The first URI translation handler to return OK ends the phase for both mod_perl and Apache! Return OK only when you map the file to disk yourself

Winner Takes All The first URI translation handler to return OK ends the phase for both mod_perl and Apache! Return OK only when you map the file to disk yourself Return DECLINED all other times

File-based Initialization
client request URI-based init URI translation file-based init

URI has been mapped to a file $r->filename is now known

URI has been mapped to a file $r->filename is now known We also know to which <Location> the request belongs

URI has been mapped to a file $r->filename is now known We also know to which <Location> the request belongs Apache has no default behavior

URI has been mapped to a file $r->filename is now known We also know to which <Location> the request belongs Apache has no default behavior All configured handlers are run

client request URI-based init URI translation file-based init

client request URI-based init URI translation PerlHeaderParserHandler

client request URI-based init URI translation PerlInitHandler

PerlHeaderParserHandler

Non-specific hook

Non-specific hook Useful for adding processing that needs to occur on every request to a given URI

Sample Usage...

Sample Usage... Parsing out the query string or POST data on each request is a pain

Sample Usage... Parsing out the query string or POST data on each request is a pain For the most part, you know you need it to every request to a given <Location>

Sample Usage... Parsing out the query string or POST data on each request is a pain For the most part, you know you need it to every request to a given <Location> Modularize the parsing code

Apache::RequestNotes

Apache::RequestNotes parses cookies and input parameters

Apache::RequestNotes parses cookies and input parameters stores the data in pnotes() for later retrieval

Interface

Interface in httpd.conf

Interface in httpd.conf Alias /perl-bin /usr/local/apache/perl-bin
<Location /perl-bin/> SetHandler perl-script PerlHandler Apache::Registry Options +ExecCGI PerlInitHandler Apache::RequestNotes </Location>

Interface in httpd.conf in handler
Alias /perl-bin /usr/local/apache/perl-bin <Location /perl-bin/> SetHandler perl-script PerlHandler Apache::Registry Options +ExecCGI PerlInitHandler Apache::RequestNotes </Location> in handler

Interface in httpd.conf in handler
Alias /perl-bin /usr/local/apache/perl-bin <Location /perl-bin/> SetHandler perl-script PerlHandler Apache::Registry Options +ExecCGI PerlInitHandler Apache::RequestNotes </Location> in handler my $input = $r->pnotes('INPUT'); # Apache::Table reference my $uploads = $r->pnotes('UPLOADS'); # Apache::Upload array ref my $cookies = $r->pnotes('COOKIES'); # hash reference

Resource Control client request URI-based init URI translation

Resource Control

Resource Control Request is inside a particular <Location> container

Resource Control Request is inside a particular <Location> container Apache provides three different layers of control to determine who gets access to the resource

Resource Control Request is inside a particular <Location> container Apache provides three different layers of control to determine who gets access to the resource Client access checker

Resource Control Request is inside a particular <Location> container Apache provides three different layers of control to determine who gets access to the resource Client access checker User ID checker

Resource Control Request is inside a particular <Location> container Apache provides three different layers of control to determine who gets access to the resource Client access checker User ID checker User authorization checker

file-based init Client Access User ID User Authorization

file-based init PerlAccessHandler PerlAuthenHandler PerlAuthzHandler

User Access

User Access Used to make access decisions based on Client information

Client IP

Client IP Client User-Agent

Client IP Client User-Agent Request URI

Client IP Client User-Agent Request URI mod_access controls Apache's default

Client IP Client User-Agent Request URI mod_access controls Apache's default Allow from localhost

Client IP Client User-Agent Request URI mod_access controls Apache's default Allow from localhost All configured handlers run

PerlAccessHandler

PerlAccessHandler Useful for making same decisions as mod_auth

PerlAccessHandler Useful for making same decisions as mod_auth
do it in Perl

Simple PerlAccessHandler

Object: get debugging telnet sessions past Basic authentication

Object: get debugging telnet sessions past Basic authentication Method: set the Authorization header to a known user if coming from localhost

package My::DefaultLogin; use Apache::Constants qw(OK);
use MIME::Base64 (); use Socket qw(sockaddr_in inet_ntoa); use strict; sub handler { my $r = shift; my $c = $r->connection; my $local_ip = inet_ntoa((sockaddr_in($c->local_addr))[1]); if ($c->remote_ip eq $local_ip) { my $user = 'bug'; my $passwd = 'squashing'; # Join user and password and set the incoming header. my $credentials = MIME::Base64::encode(join(':', $user, $passwd)); $r->headers_in->set(Authorization => "Basic $credentials"); } return OK; 1;

Setup add DefaultLogin.pm to @INC add to httpd.conf that's it!
ServerRoot/lib/perl/My/DefaultLogin.pm add to httpd.conf PerlModule My::DefaultLogin PerlAccessHandler My::DefaultLogin that's it!

How to Deny Access

How to Deny Access Each access handler returns OK if the client meets its conditions

How to Deny Access Each access handler returns OK if the client meets its conditions Access handlers return FORBIDDEN to decline access

Reality... In the real world, you could accomplish the same thing with the core Satisfy directive

Reality... In the real world, you could accomplish the same thing with the core Satisfy directive AuthType Basic AuthName "cookbook" AuthUserFile .htpasswd Require valid-user PerlAccessHandler My::DefaultLogin

Reality... In the real world, you could accomplish the same thing with the core Satisfy directive AuthType Basic AuthName "cookbook" AuthUserFile .htpasswd Require valid-user Allow from localhost Satisfy any

file-based init Client Access User ID

User Authentication

User Authentication Apache default authentication mechanism is mod_auth

User Authentication Apache default authentication mechanism is mod_auth Winner takes all

User Authentication Apache default authentication mechanism is mod_auth Winner takes all uses a password file generated using Apache's htpasswd utility

User Authentication Apache default authentication mechanism is mod_auth Winner takes all uses a password file generated using Apache's htpasswd utility geoff:zzpEyL0tbgwwk

User Authentication configuration placed in .htaccess file or httpd.conf

User Authentication configuration placed in .htaccess file or httpd.conf AuthUserFile .htpasswd AuthName "cookbook" AuthType Basic Require valid-user

How Authentication Works
client requests a document

client requests a document GET /perl-status HTTP/1.1 Accept: text/xml, image/png, image/jpeg, image/gif, text/plain Accept-Charset: ISO , utf-8;q=0.66, *;q=0.66 Accept-Encoding: gzip, deflate, compress;q=0.9 Accept-Language: en-us Connection: keep-alive Host: Keep-Alive: 300 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US)

client requests a document GET /perl-status HTTP/1.1 Accept: text/xml, image/png, image/jpeg, image/gif, text/plain Accept-Charset: ISO , utf-8;q=0.66, *;q=0.66 Accept-Encoding: gzip, deflate, compress;q=0.9 Accept-Language: en-us Connection: keep-alive Host: Keep-Alive: 300 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US) server denies request

client requests a document GET /perl-status HTTP/1.1 Accept: text/xml, image/png, image/jpeg, image/gif, text/plain Accept-Charset: ISO , utf-8;q=0.66, *;q=0.66 Accept-Encoding: gzip, deflate, compress;q=0.9 Accept-Language: en-us Connection: keep-alive Host: Keep-Alive: 300 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US) server denies request HTTP/ Authorization Required WWW-Authenticate: Basic realm="my site" Keep-Alive: timeout=15, max=100 Connection: Keep-Alive Transfer-Encoding: chunked Content-Type: text/html; charset=iso

client request client request URI-based init URI translation file-based init Client Access User ID

client request client request URI-based init URI translation file-based init Client Access User ID HTTP/ Authorization Required

client request client request logging URI-based init URI translation file-based init Client Access User ID

client sends a new request

client sends a new request GET /perl-status HTTP/1.1 Accept: text/xml, image/png, image/jpeg, image/gif, text/plain Accept-Charset: ISO , utf-8;q=0.66, *;q=0.66 Accept-Encoding: gzip, deflate, compress;q=0.9 Accept-Language: en-us Authorization: Basic Z2VvZmY6YWZha2VwYXNzd29yZA== Connection: keep-alive Host: Keep-Alive: 300 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US)

client sends a new request GET /perl-status HTTP/1.1 Accept: text/xml, image/png, image/jpeg, image/gif, text/plain Accept-Charset: ISO , utf-8;q=0.66, *;q=0.66 Accept-Encoding: gzip, deflate, compress;q=0.9 Accept-Language: en-us Authorization: Basic Z2VvZmY6YWZha2VwYXNzd29yZA== Connection: keep-alive Host: Keep-Alive: 300 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US) server sends document

client sends a new request GET /perl-status HTTP/1.1 Accept: text/xml, image/png, image/jpeg, image/gif, text/plain Accept-Charset: ISO , utf-8;q=0.66, *;q=0.66 Accept-Encoding: gzip, deflate, compress;q=0.9 Accept-Language: en-us Authorization: Basic Z2VvZmY6YWZha2VwYXNzd29yZA== Connection: keep-alive Host: Keep-Alive: 300 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US) server sends document HTTP/ OK Keep-Alive: timeout=15, max=99 Connection: Keep-Alive Transfer-Encoding: chunked Content-Type: text/html

file-based init Client Access User ID

file-based init Client Access PerlAuthenHandler

Who Uses Flat Files?

Who Uses Flat Files? flat files are limiting, hard to manage, difficult to integrate, and just plain boring

Who Uses Flat Files? flat files are limiting, hard to manage, difficult to integrate, and just plain boring we can use the Apache API and Perl to replace flat files with our own authentication mechanism

Do it in Perl since mod_perl gives us the ability to intercept the request cycle before Apache, we can authenticate using Perl instead

Do it in Perl since mod_perl gives us the ability to intercept the request cycle before Apache, we can authenticate using Perl instead Apache provides an API, making the job easy

Do it in Perl since mod_perl gives us the ability to intercept the request cycle before Apache, we can authenticate using Perl instead Apache provides an API, making the job easy mod_perl provides access to the Apache API

package My::Authenticate;
use Apache::Constants qw(OK DECLINED AUTH_REQUIRED); use strict; sub handler { my $r = shift; # Let subrequests pass. return DECLINED unless $r->is_initial_req; # Get the client-supplied credentials. my ($status, $password) = $r->get_basic_auth_pw; return $status unless $status == OK; # Perform some custom user/password validation. return OK if authenticate_user($r->user, $password); # Whoops, bad credentials. $r->note_basic_auth_failure; return AUTH_REQUIRED; }

No Authorization Header
client requests a document GET /perl-status HTTP/1.1 Accept: text/xml, image/png, image/jpeg, image/gif, text/plain Accept-Charset: ISO , utf-8;q=0.66, *;q=0.66 Accept-Encoding: gzip, deflate, compress;q=0.9 Accept-Language: en-us Connection: keep-alive Host: Keep-Alive: 300 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US) server denies request HTTP/ Authorization Required WWW-Authenticate: Basic realm="my site" Keep-Alive: timeout=15, max=100 Connection: Keep-Alive Transfer-Encoding: chunked Content-Type: text/html; charset=iso

package My::Authenticate;
use Apache::Constants qw(OK DECLINED AUTH_REQUIRED); use strict; sub handler { my $r = shift; # Let subrequests pass. return DECLINED unless $r->is_initial_req; # Get the client-supplied credentials. my ($status, $password) = $r->get_basic_auth_pw; return $status unless $status == OK; # Perform some custom user/password validation. return OK if authenticate_user($r->user, $password); # Whoops, bad credentials. $r->note_basic_auth_failure; return AUTH_REQUIRED; }

Configuration

Configuration change AuthUserFile .htpasswd AuthName "cookbook"
AuthType Basic Require valid-user

Configuration change to AuthUserFile .htpasswd AuthName "cookbook"
AuthType Basic Require valid-user to PerlAuthenHandler My::Authenticate

The Choice is Yours how you decide to authenticate is now up to you

sub authenticate_user { my ($user, $pass) return $user eq $pass; }

sub authenticate_user { my ($user, $pass) return $user eq $pass; } are you seeing the possibilities yet?

The Power of CPAN over 25 Apache:: shrink-wrapped modules on CPAN for authentication SecureID Radius SMB LDAP NTLM

To Infinity and Beyond!

To Infinity and Beyond! this example only covered Basic authentication via popup box

To Infinity and Beyond! this example only covered Basic authentication via popup box the same techniques can be used to authenticate via a login form plus cookies, munged URLs, or hidden fields

To Infinity and Beyond! this example only covered Basic authentication via popup box the same techniques can be used to authenticate via a login form plus cookies, munged URLs, or hidden fields extended to use Digest authentication as well

User Authorization client request URI-based init URI translation

User Authorization We now know the user has supplied a valid password

User Authorization We now know the user has supplied a valid password
now it's up to us to decide if we want the user to have access

User Authorization

User Authorization Apache's default behavior varies, depending on the syntax of Require Require valid-user Require user foo Require group bar

User Authorization Apache's default behavior varies, depending on the syntax of Require Require valid-user Require user foo Require group bar Require file-owner Require file-group

User Authorization Apache's default behavior varies, depending on the syntax of Require Require valid-user Require user foo Require group bar Require file-owner Require file-group Winner takes all

file-based init Client Access User ID PerlAuthzHandler

PerlAuthzHandler

PerlAuthzHandler Key is the requires() method

$r->requires() returns an array of hashes representing all Require directives

$r->requires() returns an array of hashes representing all Require directives Require user grier ryan Require group admiral

$r->requires() returns an array of hashes representing all Require directives Require user grier ryan Require group admiral [ { requirement => `user grier ryan', method => -1}, { requirement => `group admiral', ];

PerlAuthzHandler Once you get the Require directive back, you can decide which users meet the authorization requirement

PerlAuthzHandler Once you get the Require directive back, you can decide which users meet the authorization requirement foreach my $requires { my = split " ", $requires->{requirement}; # We're ok if only valid-user was required. return OK if lc($directive) eq 'valid-user'; # Likewise if the user requirement was specified and # we match based on what we already know. return OK if lc($directive) eq 'user' && grep { $_ eq $r->user }

PerlAuthzHandler Once you get the Require directive back, you can decide which users meet the authorization requirement foreach my $requires { my = split " ", $requires->{requirement}; # We're ok if only valid-user was required. return OK if lc($directive) eq 'valid-user'; # Likewise if the user requirement was specified and # we match based on what we already know. return OK if lc($directive) eq 'user' && grep { $_ eq $r->user } [ { requirement => `user grier ryan', method => -1} ]

MIME-type Checking client request URI-based init URI translation

MIME-type Checking We now have the physical resource and have decided the user can see it

MIME-type Checking We now have the physical resource and have decided the user can see it time to set the Content-Type header

MIME-type Checking We now have the physical resource and have decided the user can see it time to set the Content-Type header Apache's default is mod_mime, which examines the file extension

MIME-type Checking We now have the physical resource and have decided the user can see it time to set the Content-Type header Apache's default is mod_mime, which examines the file extension mod_mime also decides which content handler will run

MIME-type Checking We now have the physical resource and have decided the user can see it time to set the Content-Type header Apache's default is mod_mime, which examines the file extension mod_mime also decides which content handler will run AddHandler server-parsed

MIME-type Checking We now have the physical resource and have decided the user can see it time to set the Content-Type header Apache's default is mod_mime, which examines the file extension mod_mime also decides which content handler will run AddHandler server-parsed Winner takes all

file-based init PerlTypeHandler resource control

PerlTypeHandler

PerlTypeHandler mod_mime has a stranglehold on the request

if you set the Content-Type and return OK, mod_mime won't set the content handler

if you set the Content-Type and return OK, mod_mime won't set the content handler if you set the Content-Type and return DECLINED, mod_mime will clobber the Content-Type

if you set the Content-Type and return OK, mod_mime won't set the content handler if you set the Content-Type and return DECLINED, mod_mime will clobber the Content-Type Best to just forget about the PerlTypeHandler

Fixups client request URI-based init URI translation fixups

Fixups The final chance to fiddle with the request before content is written to the client

Fixups The final chance to fiddle with the request before content is written to the client non-specific phase

Fixups The final chance to fiddle with the request before content is written to the client non-specific phase Apache has no default behavior

Fixups The final chance to fiddle with the request before content is written to the client non-specific phase Apache has no default behavior All configured handlers will be run

Fixups client request URI-based init URI translation fixups

Fixups client request URI-based init URI translation PerlFixupHandler

PerlFixupHandler

PerlFixupHandler Good place to do anything you might have wanted to do in the PerlTypeHandler

PerlFixupHandler Good place to do anything you might have wanted to do in the PerlTypeHandler especially setting $r->handler()

Sample Fixup Object: re-implement XBitHack in Perl
Method: after some basic checks, turn the request over to mod_include using $r->handler()

package Cookbook::XBitHack;
use Apache::Constants qw(OK DECLINED OPT_INCLUDES); use Apache::File; use Fcntl qw(S_IXUSR S_IXGRP); use strict; sub handler { my $r = shift; return DECLINED unless (-f $r->finfo && # the file exists $r->content_type eq 'text/html' && # and is HTML $r->allow_options & OPT_INCLUDES); # and we have Options +Includes # Find out the user and group execution status. my $mode = (stat _)[2]; # We have to be user executable specifically. return DECLINED unless ($mode & S_IXUSR); # Set the Last-Modified header if group executable. $r->set_last_modified((stat _)[9]) if ($mode & S_IXGRP); # Make sure mod_include picks it up. $r->handler('server-parsed'); return OK; } 1;

Content Generation client request URI-based init content

Content Generation Apache's default content handler is default-handler, which takes care of all HTTP/1.1 events

Content Generation Apache's default content handler is default-handler, which takes care of all HTTP/1.1 events byteserving

Content Generation Apache's default content handler is default-handler, which takes care of all HTTP/1.1 events byteserving cache headers

Content Generation Apache's default content handler is default-handler, which takes care of all HTTP/1.1 events byteserving cache headers one and only one C module gets to handle content-generation

Content Generation Apache's default content handler is default-handler, which takes care of all HTTP/1.1 events byteserving cache headers one and only one C module gets to handle content-generation mod_cgi mod_perl mod_include

Content Generation client request URI-based init content

Content Generation client request URI-based init PerlHandler

Sample PerlHandler

Sample PerlHandler Object: save bandwidth universally

Method: module HTML::Clean to "clean" outgoing documents

Method: module HTML::Clean to "clean" outgoing documents Afterward: alter the handler to take advantage of more advanced mod_perl features

use Apache::Constants qw(OK DECLINED); use Apache::File;
package My::Clean; use Apache::Constants qw(OK DECLINED); use Apache::File; use HTML::Clean; use strict; sub handler { my $r = shift; my $fh = Apache::File->new($r->filename) or return DECLINED; my $dirty = do {local $/; <$fh>}; my $h = HTML::Clean->new(\$dirty); $h->level(3); $h->strip; $r->send_http_header('text/html'); print ${$h->data}; return OK; } 1;

Configuration

Configuration add directives to httpd.conf to mirror DocumentRoot

Configuration add directives to httpd.conf to mirror DocumentRoot
Alias /clean /usr/local/apache/htdocs <Location /clean> SetHandler perl-script PerlHandler My::Clean </Location>

Results original: 202 bytes <html> <body>
<form method="GET" action="/foo"> Text: <input type="text" name="foo"><br> <input type="submit"> </form> <strong>hi there </strong> </body> </html>

Results original: 202 bytes clean: 145 bytes <html> <body>
<form method="GET" action="/foo"> Text: <input type="text" name="foo"><br> <input type="submit"> </form> <strong>hi there </strong> </body> </html> clean: 145 bytes <html><body><form method="GET" action="/foo"> Text: <input type="text" name="foo"><br><input type="submit"></form><b>hi there </b></body></html>

Dynamic or Static?

Dynamic or Static? we often think of dynamic content as "could be different on any given access"

Dynamic or Static? we often think of dynamic content as "could be different on any given access" "dynamic" content can also be static with clearly defined factors that can change its meaning

Dynamic or Static? we often think of dynamic content as "could be different on any given access" "dynamic" content can also be static with clearly defined factors that can change its meaning by properly managing HTTP/1.1 cache headers, we can reduce strain on our servers

Conditional GET Request

HTTP/1.1 allows for a conditional GET request

HTTP/1.1 allows for a conditional GET request clients are allowed to use cached content based on information about the resource

HTTP/1.1 allows for a conditional GET request clients are allowed to use cached content based on information about the resource information is provided by both the client and the server

GET /manual/index.html HTTP/1.1
Accept: text/html, image/png, image/jpeg, image/gif, image/x-xbitmap, */* Accept-Charset: windows-1252;q=1.0, utf-8;q=1.0, utf-16;q=1.0, iso ;q=0.6, *;q=0.1 Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0 Accept-Language: en Connection: Keep-Alive, TE Host: mainsheet.laserlink.net TE: deflate, gzip, chunked, identity, trailers User-Agent: Mozilla/4.0 (compatible; MSIE 5.0; Windows XP) Opera [en]

Accept: text/html, image/png, image/jpeg, image/gif, image/x-xbitmap, */* Accept-Charset: windows-1252;q=1.0, utf-8;q=1.0, utf-16;q=1.0, iso ;q=0.6, *;q=0.1 Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0 Accept-Language: en Connection: Keep-Alive, TE Host: mainsheet.laserlink.net TE: deflate, gzip, chunked, identity, trailers User-Agent: Mozilla/4.0 (compatible; MSIE 5.0; Windows XP) Opera [en] HTTP/ OK Last-Modified: Thu, 01 Nov :35:27 GMT ETag: "4c be179cf" Accept-Ranges: bytes Content-Length: 9268 Connection: close Content-Type: text/html

Accept: text/html, image/png, image/jpeg, image/gif, image/x-xbitmap, */* Accept-Charset: windows-1252;q=1.0, utf-8;q=1.0, utf-16;q=1.0, iso ;q=0.6, *;q=0.1 Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0 Accept-Language: en Connection: Keep-Alive, TE Host: mainsheet.laserlink.net If-Modified-Since: Thu, 01 Nov :35:27 GMT If-None-Match: "4c be179cf TE: deflate, gzip, chunked, identity, trailers User-Agent: Mozilla/4.0 (compatible; MSIE 5.0; Windows XP) Opera [en]

Accept: text/html, image/png, image/jpeg, image/gif, image/x-xbitmap, */* Accept-Charset: windows-1252;q=1.0, utf-8;q=1.0, utf-16;q=1.0, iso ;q=0.6, *;q=0.1 Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0 Accept-Language: en Connection: Keep-Alive, TE Host: mainsheet.laserlink.net If-Modified-Since: Thu, 01 Nov :35:27 GMT If-None-Match: "4c be179cf TE: deflate, gzip, chunked, identity, trailers User-Agent: Mozilla/4.0 (compatible; MSIE 5.0; Windows XP) Opera [en] HTTP/ Not Modified Last-Modified: Thu, 01 Nov :35:27 GMT ETag: "4c be179cf" Accept-Ranges: bytes Connection: close

Accept: text/html, image/png, image/jpeg, image/gif, image/x-xbitmap, */* Accept-Charset: windows-1252;q=1.0, utf-8;q=1.0, utf-16;q=1.0, iso ;q=0.6, *;q=0.1 Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0 Accept-Language: en Connection: Keep-Alive, TE Host: mainsheet.laserlink.net If-Modified-Since: Thu, 01 Nov :35:27 GMT If-None-Match: "4c be179cf TE: deflate, gzip, chunked, identity, trailers User-Agent: Mozilla/4.0 (compatible; MSIE 5.0; Windows XP) Opera [en]

Accept: text/html, image/png, image/jpeg, image/gif, image/x-xbitmap, */* Accept-Charset: windows-1252;q=1.0, utf-8;q=1.0, utf-16;q=1.0, iso ;q=0.6, *;q=0.1 Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0 Accept-Language: en Connection: Keep-Alive, TE Host: mainsheet.laserlink.net If-Modified-Since: Thu, 01 Nov :35:27 GMT If-None-Match: "4c be179cf TE: deflate, gzip, chunked, identity, trailers User-Agent: Mozilla/4.0 (compatible; MSIE 5.0; Windows XP) Opera [en] HTTP/ OK Last-Modified: Thu, 06 Jun :51:11 GMT ETag: "4c cff4caf" Accept-Ranges: bytes Content-Length: 9268 Connection: close

for static documents, Apache takes care of making our response cache-friendly

for static documents, Apache takes care of making our response cache-friendly since the file is on disk, Apache can determine when the file was last changed

for static documents, Apache takes care of making our response cache-friendly since the file is on disk, Apache can determine when the file was last changed with static files, local modification is the only factor

for static documents, Apache takes care of making our response cache-friendly since the file is on disk, Apache can determine when the file was last changed with static files, local modification is the only factor still too many rules to keep straight

for static documents, Apache takes care of making our response cache-friendly since the file is on disk, Apache can determine when the file was last changed with static files, local modification is the only factor still too many rules to keep straight Apache provides an API to use so we don't have to think too much

Now for the Fun Part

Now for the Fun Part modify our PerlHandler to be "cache friendly"

send 304 when the document hasn't changed

send 304 when the document hasn't changed properly handle If-* header comparisons

How do you define change?

when dynamically altering static documents there are a number of factors to consider

when dynamically altering static documents there are a number of factors to consider when the file changes on disk

when dynamically altering static documents there are a number of factors to consider when the file changes on disk when the code changes

when dynamically altering static documents there are a number of factors to consider when the file changes on disk when the code changes when the options to the code change

when dynamically altering static documents there are a number of factors to consider when the file changes on disk when the code changes when the options to the code change all of these affect the "freshness" of the document

Code Changes

Code Changes in order to determine when the code itself changes, we need to mark the modification time of the package

Code Changes in order to determine when the code itself changes, we need to mark the modification time of the package at request time, we call an API to compare the package modification to the If-Modified-Since header

Code Changes in order to determine when the code itself changes, we need to mark the modification time of the package at request time, we call an API to compare the package modification to the If-Modified-Since header on reloads, we regenerate the package modification time

package My::Clean; use Apache::Constants qw(OK DECLINED); use Apache::File; use HTML::Clean; use strict; sub handler { my $r = shift; my $fh = Apache::File->new($r->filename) or return DECLINED; my $dirty = do {local $/; <$fh>}; my $h = HTML::Clean->new(\$dirty); $h->level(3); $h->strip; $r->send_http_header('text/html'); print ${$h->data}; return OK; } 1;

package My::Clean; use Apache::Constants qw(OK DECLINED); use Apache::File; use HTML::Clean; use strict; # Get the package modification time... (my $package = __PACKAGE__) =~ s!::!/!g; my $package_mtime = (stat $INC{"$package.pm"})[9];

Configuration Changes

in order to determine when the options to the code change, we need to mark the modification time of httpd.conf

in order to determine when the options to the code change, we need to mark the modification time of httpd.conf at request time, we call an API to compare the configuration modification to the If-Modified-Since header

in order to determine when the options to the code change, we need to mark the modification time of httpd.conf at request time, we call an API to compare the configuration modification to the If-Modified-Since header on restarts, we regenerate the configuration modification time

package My::Clean; use Apache::Constants qw(OK DECLINED); use Apache::File; use HTML::Clean; use strict; # Get the package modification time... (my $package = __PACKAGE__) =~ s!::!/!g; my $package_mtime = (stat $INC{"$package.pm"})[9];

package My::Clean; use Apache::Constants qw(OK DECLINED); use Apache::File; use HTML::Clean; use strict; # Get the package modification time... (my $package = __PACKAGE__) =~ s!::!/!g; my $package_mtime = (stat $INC{"$package.pm"})[9]; # ...and when httpd.conf was last modified my $conf_mtime = (stat Apache->server_root_relative('conf/httpd.conf'))[9]; # When the server is restarted we need to # make sure we recognize config file changes and propigate # them to the client to clear the client cache if necessary. Apache->server->register_cleanup(sub { $conf_mtime = (stat Apache->server_root_relative('conf/httpd.conf'))[9]; });

Resource Changes

Resource Changes in order to determine when the resources changes, we need to mark the modification time of $r->filename

Resource Changes in order to determine when the resources changes, we need to mark the modification time of $r->filename at request time, we call an API to compare the resource modification to the If-Modified-Since header

Resource Changes in order to determine when the resources changes, we need to mark the modification time of $r->filename at request time, we call an API to compare the resource modification to the If-Modified-Since header resource modification is checked on each request

package My::Clean; use Apache::Constants qw(OK DECLINED); use Apache::File; use HTML::Clean; use strict; # Get the package modification time... (my $package = __PACKAGE__) =~ s!::!/!g; my $package_mtime = (stat $INC{"$package.pm"})[9]; # ...and when httpd.conf was last modified my $conf_mtime = (stat Apache->server_root_relative('conf/httpd.conf'))[9]; # When the server is restarted we need to # make sure we recognize config file changes and propigate # them to the client to clear the client cache if necessary. Apache->server->register_cleanup(sub { $conf_mtime = (stat Apache->server_root_relative('conf/httpd.conf'))[9]; });

package My::Clean; use Apache::Constants qw(OK DECLINED); use Apache::File; use HTML::Clean; use strict; # Get the package modification time... (my $package = __PACKAGE__) =~ s!::!/!g; my $package_mtime = (stat $INC{"$package.pm"})[9]; # ...and when httpd.conf was last modified my $conf_mtime = (stat Apache->server_root_relative('conf/httpd.conf'))[9]; # When the server is restarted we need to # make sure we recognize config file changes and propigate # them to the client to clear the client cache if necessary. Apache->server->register_cleanup(sub { $conf_mtime = (stat Apache->server_root_relative('conf/httpd.conf'))[9]; }); sub handler { ... } 1;

my $fh = Apache::File->new($r->filename) or return DECLINED;
sub handler { my $r = shift; my $fh = Apache::File->new($r->filename) or return DECLINED; my $dirty = do {local $/; <$fh>}; my $h = HTML::Clean->new(\$dirty); $h->level(3); $h->strip; $r->send_http_header('text/html'); print ${$h->data}; return OK; }

my $fh = Apache::File->new($r->filename) or return DECLINED;
sub handler { my $r = shift; my $fh = Apache::File->new($r->filename) or return DECLINED; my $dirty = do {local $/; <$fh>}; my $h = HTML::Clean->new(\$dirty); $h->level(3); $h->strip; $r->update_mtime($package_mtime); $r->update_mtime((stat $r->finfo)[9]); $r->update_mtime($conf_mtime); $r->set_last_modified; $r->set_etag; $r->set_content_length(length ${$h->data}); # only send the file if it meets cache criteria if ((my $status = $r->meets_conditions) == OK) { $r->send_http_header('text/html'); } else { return $status; print ${$h->data}; return OK;

Logging client request logging URI-based init content URI translation

Logging Apache's default is to use mod_log_config in common format

LogFormat "%h %l %u %t \"%r\" %>s %b" common CustomLog logs/access_log common

LogFormat "%h %l %u %t \"%r\" %>s %b" common CustomLog logs/access_log common Most people tweak this to combined

LogFormat "%h %l %u %t \"%r\" %>s %b" common CustomLog logs/access_log common Most people tweak this to combined LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

LogFormat "%h %l %u %t \"%r\" %>s %b" common CustomLog logs/access_log common Most people tweak this to combined LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined The connection to the client is still open!

LogFormat "%h %l %u %t \"%r\" %>s %b" common CustomLog logs/access_log common Most people tweak this to combined LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined The connection to the client is still open! All configured handlers run

Logging client request logging URI-based init content URI translation

Logging client request PerlLogHandler URI-based init content

PerlLogHandler

PerlLogHandler Useful for logging using interfaces in which Perl shines

PerlLogHandler Useful for logging using interfaces in which Perl shines like databases

Logging to a Database

Logging to a Database Logging directly to a database makes life easier if you have an application for which you need lots of reports

Logging to a Database Logging directly to a database makes life easier if you have an application for which you need lots of reports DBI rules

package Cookbook::SiteLog; use Apache::Constants qw(OK); use DBI;
use strict; sub handler { my $r = shift; my $dbh = DBI->connect($r->dir_config('DBASE'), {RaiseError => 1, AutoCommit => 1, PrintError => 1}) or die $DBI::errstr; my %columns = ( status => $r->status, bytes => $r->bytes_sent, language => $r->headers_in->get('Accept-Language'), ); my $fields = join "$_,", keys %columns; my $values = join ', ', ('?') x values %columns; my $sql = qq( insert into (hit, servedate, $fields) values (hitsequence.nextval, sysdate, $values) my $sth = $dbh->prepare($sql); $sth->execute(values %columns); return OK; } 1;

Cleanups client request cleanups URI-based init logging content

Cleanups Apache doesn't really have a cleanup phase

It calls a function when the request memory pool is destroyed

It calls a function when the request memory pool is destroyed The connection to the client is closed

Cleanups client request cleanups URI-based init logging content

Cleanups client request PerlCleanupHandler URI-based init logging
content URI translation fixups file-based init MIME setting resource control

PerlCleanupHandler

PerlCleanupHandler Generally used to do any end of request cleanups

Apache::File::tmpfile() removes its temporary file here

Apache::File::tmpfile() removes its temporary file here Also good for logging

Apache::File::tmpfile() removes its temporary file here Also good for logging no active browsers

Debugging

Debugging Let's examine a very conceptual debugging cleanup handler

Debugging Let's examine a very conceptual debugging cleanup handler
I actually did use it for a while

package Cookbook::TraceError;
use Apache::Constants qw(OK SERVER_ERROR DECLINED); use Apache::Log; use strict; sub handler { my $r = shift; # Don't do anything unless the main process errors. return DECLINED unless $r->is_initial_req && $r->status == SERVER_ERROR; my $old_loglevel = $r->server->loglevel(Apache::Log::DEBUG); my $old_trace = DBI->trace(2); # Start the debuggging request. my $sub = $r->lookup_uri($r->uri); # run() would ordinarily send content to the client, but # since we're in cleanup, the connection is already closed. $sub->run; # Reset things back to their original state - # loglevel(N) will persist for the lifetime of the child process. DBI->trace($old_trace); $r->server->loglevel($old_loglevel); return OK; } 1;

Fine Manuals Writing Apache Modules with Perl and C
mod_perl Developer's Cookbook mod_perl Pocket Reference mod_perl Guide mod_perl at the ASF

Materials These slides My modules
My modules

Programming the Apache Lifecycle

Similar presentations

Presentation on theme: "Programming the Apache Lifecycle"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Programming the Apache Lifecycle

Similar presentations

Presentation on theme: "Programming the Apache Lifecycle"— Presentation transcript:

Similar presentations

About project

Feedback