Presentation is loading. Please wait.

Presentation is loading. Please wait.

Writing Pluggable Software Tatsuhiko Miyagawa Six Apart, Ltd. / Shibuya Perl Mongers YAPC::Asia 2007 Tokyo.

Similar presentations


Presentation on theme: "Writing Pluggable Software Tatsuhiko Miyagawa Six Apart, Ltd. / Shibuya Perl Mongers YAPC::Asia 2007 Tokyo."— Presentation transcript:

1 Writing Pluggable Software Tatsuhiko Miyagawa Six Apart, Ltd. / Shibuya Perl Mongers YAPC::Asia 2007 Tokyo

2 Tatsuhiko Miyagawa For non-JP attendees … If you find \ in the code, Replace that with backslash. (This is MS' fault)

3 Tatsuhiko Miyagawa Plaggable Software

4 Tatsuhiko Miyagawa Plaggable Software

5 Tatsuhiko Miyagawa Pluggable Software

6 Tatsuhiko Miyagawa Agenda

7 Tatsuhiko Miyagawa #1 How to make your app pluggable

8 Tatsuhiko Miyagawa #2 TMTOWTDP There's More Than One Way To Deploy Plugins Pros/Cons by examples

9 Tatsuhiko Miyagawa First-of-all: Why pluggable?

10 Tatsuhiko Miyagawa Benefits

11 Tatsuhiko Miyagawa #1 Keep the app design and code simple

12 Tatsuhiko Miyagawa #2 Let the app users customize the behavior (without hacking the internals)

13 Tatsuhiko Miyagawa #3 It's fun to write plugins for most hackers (see: Plagger and Kwiki)

14 Tatsuhiko Miyagawa "Can your app do XXX?" "Yes, by plugins."

15 Tatsuhiko Miyagawa "Your app has a bug in YYY" "No, it's the bug in plugin YYY, Not my fault." (Chain Of Responsibilities)

16 Tatsuhiko Miyagawa Good Enough Reasons, huh?

17 Tatsuhiko Miyagawa #1 Make your app pluggable

18 Tatsuhiko Miyagawa Example

19 Tatsuhiko Miyagawa ack (App::Ack)

20 Tatsuhiko Miyagawa grep –r for programmers

21 Tatsuhiko Miyagawa Ack is a "full-stack" software now.

22 Tatsuhiko Miyagawa By "full-stack" I mean: Easy install No configuration No way to extend

23 Tatsuhiko Miyagawa Specifically: These are hardcoded Ignored directories Filenames and types

24 Tatsuhiko Miyagawa Ignored = qw( blib CVS RCS SCCS.svn _darcs.git );

25 Tatsuhiko Miyagawa Filenames and languages mapping %mappings = ( asm => [qw( s S )], binary => …, cc => [qw( c h xs )], cpp => [qw( cpp m h C H )], csharp => [qw( cs )], … perl => [qw( pl pm pod tt ttml t )], … );

26 Tatsuhiko Miyagawa What if making these pluggable?

27 Tatsuhiko Miyagawa DISCLAIMER

28 Tatsuhiko Miyagawa Don't get me wrong Andy, I love ack the way it is… Just thought it can be a very good example for the tutorial.

29 Tatsuhiko Miyagawa Quickstart: Class::Trigger Module::Pluggable © Six Apart Ltd. Employees

30 Tatsuhiko Miyagawa Class::Trigger SYNOPSIS package Foo; use Class::Trigger; sub foo { my $self = shift; $self->call_trigger('before_foo'); # some code... $self->call_trigger('after_foo'); } package main; Foo->add_trigger(before_foo => \&sub1); Foo->add_trigger(after_foo => \&sub2);

31 Tatsuhiko Miyagawa Class::Trigger Helps you to implement Observer Pattern. (Rails calls this Observer)

32 Tatsuhiko Miyagawa Module::Pluggable SYNOPSIS package MyClass; use Module::Pluggable; use MyClass; my $mc = MyClass->new(); # returns the names of all plugins installed under MyClass::Plugin::* = $mc->plugins(); package MyClass::Plugin::Foo; sub new { … } 1;

33 Tatsuhiko Miyagawa Setup plugins in App::Ack package App::Ack; use Class::Trigger; use Module::Pluggable require => 1; __PACKAGE__->plugins;

34 Tatsuhiko Miyagawa Setup plugins in App::Ack package App::Ack; use Class::Trigger; use Module::Pluggable require => 1; __PACKAGE__->plugins; # "requires" modules

35 Tatsuhiko Miyagawa Ignored Directories = qw( blib CVS RCS SCCS.svn _darcs.git );

36 Tatsuhiko Miyagawa Ignored Directories (After) # lib/App/Ack.pm __PACKAGE__->call_trigger('ignore_dirs.add',

37 Tatsuhiko Miyagawa Ignored Directories (plugins) # lib/App/Ack/Plugin/IgnorePerlBuildDir.pm package App::Ack::Plugin::IgnorePerlBuildDir; App::Ack->add_trigger( "ignore_dirs.add" => sub { my($class, $ignore_dirs) qw( blib ); }, ); 1;

38 Tatsuhiko Miyagawa Ignored Directories (plugins) # lib/App/Ack/Plugin/IgnoreSourceControlDir.pm package App::Ack::Plugin::IgnoreSourcdeControlDir; App::Ack->add_trigger( "ignore_dirs.add" => sub { my($class, $ignore_dirs) qw( CVS RCS.svn _darcs.git ); }, ); 1;

39 Tatsuhiko Miyagawa Filenames and languages (before) %mappings = ( asm => [qw( s S )], binary => …, cc => [qw( c h xs )], cpp => [qw( cpp m h C H )], csharp => [qw( cs )], … perl => [qw( pl pm pod tt ttml t )], … );

40 Tatsuhiko Miyagawa Filenames and languages (after) # lib/App/Ack.pm __PACKAGE__->call_trigger('mappings.add', \%mappings);

41 Tatsuhiko Miyagawa Filenames and languages (plugins) package App::Ack::Plugin::MappingCFamily; use strict; App::Ack->add_trigger( "mappings.add" => sub { my($class, $mappings) $mappings->{asm} = [qw( s S )]; $mappings->{cc} = [qw( c h xs )]; $mappings->{cpp} = [qw( cpp m h C H )]; $mappings->{csharp} = [qw( cs )]; $mappings->{css} = [qw( css )]; }, ); 1;

42 Tatsuhiko Miyagawa Works great with few lines of code!

43 Tatsuhiko Miyagawa Now it's time to add Some useful stuff.

44 Tatsuhiko Miyagawa Example Plugin: Content Filter

45 Tatsuhiko Miyagawa sub _search { my $fh = shift; my $is_binary = shift; my $filename = shift; my $regex = shift; my %opt if ($is_binary) { my $new_fh; App::Ack->call_trigger('filter.binary', $filename, \$new_fh); if ($new_fh) { return _search($new_fh, 0, $filename, }

46 Tatsuhiko Miyagawa Example: Search PDF content with ack

47 Tatsuhiko Miyagawa PDF filter plugin package App::Ack::Plugin::ExtractContentPDF; use strict; use CAM::PDF; use File::Temp; App::Ack->add_trigger( 'mappings.add' => sub { my($class, $mappings) $mappings->{pdf} = [qw(pdf)]; }, );

48 Tatsuhiko Miyagawa PDF filter plugin (cont.) App::Ack->add_trigger( 'filter.binary' => sub { my($class, $filename, $fh_ref) if ($filename =~ /\.pdf$/) { my $fh = File::Temp::tempfile; my $doc = CAM::PDF->new($file); my $text; for my $page (1..$doc->numPages){ $text.= $doc->getPageText($page); } print $fh $text; seek $$fh, 0, 0; $$fh_ref = $fh; } }, );

49 Tatsuhiko Miyagawa PDF search > ack --type=pdf Audrey yapcasia2007-pugs.pdf:3:Audrey Tang

50 Tatsuhiko Miyagawa Homework Use File::Extract To handle arbitrary media files

51 Tatsuhiko Miyagawa Homework 2: Search non UTF-8 files (hint: use Encode::Guess) You'll need another hook.

52 Tatsuhiko Miyagawa Summary Class::Trigger + Module::Pluggable = Pluggable app easy

53 Tatsuhiko Miyagawa #2 TMTOWTDP There's More Than One Way To Deploy Plugins

54 Tatsuhiko Miyagawa Module::Pluggable + Class::Trigger = Simple and Nice but has limitations

55 Tatsuhiko Miyagawa In Reality, we need more control over how plugins behave

56 Tatsuhiko Miyagawa 1) The order of plugin executions

57 Tatsuhiko Miyagawa 2) Per user configurations for plugins

58 Tatsuhiko Miyagawa 3) Temporarily Disable plugins Should be easy

59 Tatsuhiko Miyagawa 4) How to install & upgrade plugins

60 Tatsuhiko Miyagawa 5) Let plugins have storage area

61 Tatsuhiko Miyagawa Etc, etc.

62 Tatsuhiko Miyagawa Examples: Kwiki Plagger qpsmtpd Movable Type

63 Tatsuhiko Miyagawa I won't talk about Catalyst plugins (and other framework thingy)

64 Tatsuhiko Miyagawa Because they're NOT "plug-ins"

65 Tatsuhiko Miyagawa Install plugins And now you write MORE CODE

66 Tatsuhiko Miyagawa 95% of Catalyst plugins Are NOT "plugins" But "components" 95% of these statistics is made up by the speakers.

67 Tatsuhiko Miyagawa Kwiki 1.0

68 Tatsuhiko Miyagawa Kwiki Plugin code package Kwiki::URLBL; use Kwiki::Plugin -Base; use Kwiki::Installer -base; const class_id => 'urlbl'; const class_title => 'URL Blacklist DNS'; const config_file => 'urlbl.yaml'; sub register { require URI::Find; my $registry = shift; $registry->add(hook => 'edit:save', pre => 'urlbl_hook'); $registry->add(action => 'blacklisted_url'); }

69 Tatsuhiko Miyagawa Kwiki Plugin (cont.) sub urlbl_hook { my $hook = pop; my $old_page = $self->hub->pages->new_page($self->pages- >current->id); my $this = $self->hub->urlbl; = $this->get_urls($old_page->content); = $this->get_urls($self->cgi->page_content); = if && { $hook->cancel(); return $self->redirect("action=blacklisted_url"); }

70 Tatsuhiko Miyagawa Magic implemented in Spoon(::Hooks)

71 Tatsuhiko Miyagawa "Install" Kwiki Plugins # order doesn't matter here (according to Ingy) Kwiki::Display Kwiki::Edit Kwiki::Theme::Basic Kwiki::Toolbar Kwiki::Status Kwiki::Widgets # Comment out (or entirely remove) to disable # Kwiki::UnnecessaryStuff

72 Tatsuhiko Miyagawa Kwiki plugin config # in Kwiki::URLBL plugin __config/urlbl.yaml__ urlbl_dns: sc.surbl.org, bsb.spamlookup.net, rbl.bulkfeeds.jp # config.yaml urlbl_dns: myowndns.example.org

73 Tatsuhiko Miyagawa Kwiki plugins are CPAN modules

74 Tatsuhiko Miyagawa Install and Upgrade plugins cpan> install Kwiki::SomeStuff

75 Tatsuhiko Miyagawa Using CPAN as a repository Pros #1: reuse most of current CPAN infrastructure.

76 Tatsuhiko Miyagawa Using CPAN as a repository Pros #2: Increasing # of modules = good motivation for Perl hackers

77 Tatsuhiko Miyagawa Cons #1: Installing CPAN deps could be a mess (especially for Win32)

78 Tatsuhiko Miyagawa Cons #2: Whenever Ingy releases new Kwiki, lots of plugins just break.

79 Tatsuhiko Miyagawa Kwiki plugin storage return if grep my $html = io->catfile( $self->plugin_directory,$page->id )->utf8;

80 Tatsuhiko Miyagawa Kwiki 2.0

81 Tatsuhiko Miyagawa Same as Kwiki 1.0

82 Tatsuhiko Miyagawa Except: plugins are now in SVN repository

83 Tatsuhiko Miyagawa

84 Plagger plugin package Plagger::Plugin::Publish::iCal; use strict; use base qw( Plagger::Plugin ); use Data::ICal; use Data::ICal::Entry::Event; use DateTime::Duration; use DateTime::Format::ICal; sub register { my($self, $context) $context->register_hook( $self, 'publish.feed' => \&publish_feed, 'plugin.init ' => \&plugin_init, ); }

85 Tatsuhiko Miyagawa Plagger plugin (cont) sub plugin_init { my($self, $context) my $dir = $self->conf->{dir}; unless (-e $dir && -d _) { mkdir $dir, 0755 or $context->error("Failed to mkdir $dir: $!"); }

86 Tatsuhiko Miyagawa Plagger plugin storage $self->conf->{invindex} ||= $self->cache->path_to('invindex');

87 Tatsuhiko Miyagawa Plagger plugin config # The order matters in config.yaml # if they're in the same hooks plugins: - module: Subscription::Config config: feed: - - module: Filter::DegradeYouTube config: dev_id: XYZXYZ - module: Publish::Gmail disable: 1

88 Tatsuhiko Miyagawa Plugins Install & Upgrade > notest cpan Plagger # or … > svn co plagger > svn update

89 Tatsuhiko Miyagawa Plagger impl. ripped off by many apps now

90 Tatsuhiko Miyagawa qpsmtpd

91 Tatsuhiko Miyagawa mod_perl for SMTP Runs with tcpserver, forkserver or Danga::Socket standalone

92 Tatsuhiko Miyagawa Plugins: Flat files rock:/home/miyagawa/svn/qpsmtpd> ls -F plugins async/ greylisting auth/ hosts_allow check_badmailfrom http_config check_badmailfromto ident/ check_badrcptto logging/ check_badrcptto_patterns milter check_basicheaders parse_addr_withhelo check_earlytalker queue/ check_loop quit_fortune check_norelay rcpt_ok check_relay relay_only check_spamhelo require_resolvable_fromhost content_log rhsbl count_unrecognized_commands sender_permitted_from dns_whitelist_soft spamassassin dnsbl tls domainkeys tls_cert* dont_require_anglebrackets virus/

93 Tatsuhiko Miyagawa qpsmtpd plugin sub hook_mail { my ($self, $transaction, $sender, %param) = $self->qp->config("badmailfrom") or return (DECLINED); for my $bad { my $reason = $bad; $bad =~ s/^\s*(\S+).*/$1/; next unless $bad; $transaction->notes('badmailfrom', $reason) … } return (DECLINED); }

94 Tatsuhiko Miyagawa Actually qpsmtpd Plugins are "compiled" to modules

95 Tatsuhiko Miyagawa my $eval = join("\n", "package $package;", 'use Qpsmtpd::Constants;', "require Qpsmtpd::Plugin;", 'use vars 'use strict;', = qw(Qpsmtpd::Plugin);', ($test_mode ? 'use Test::More;' : ''), "sub plugin_name { qq[$plugin] }", $line, $sub, "\n", # last line comment without newline? ); $eval =~ m/(.*)/s; $eval = $1; eval $eval; die "eval if

96 Tatsuhiko Miyagawa qpsmtpd plugin config rock:/home/miyagawa/svn/qpsmtpd> ls config.sample/ config.sample: IP logging require_resolvable_fromhost badhelo loglevel rhsbl_zones badrcptto_patterns plugins size_threshold dnsbl_zones rcpthosts tls_before_auth invalid_resolvable_fromhost relayclients tls_ciphers

97 Tatsuhiko Miyagawa config/plugins # content filters virus/klez_filter # rejects mails with a SA score higher than 2 spamassassin reject_threshold 20

98 Tatsuhiko Miyagawa config/badhelo # these domains never uses their domain when greeting us, so reject transactions aol.com yahoo.com

99 Tatsuhiko Miyagawa Install & Upgrade plugins Just use subversion

100 Tatsuhiko Miyagawa

101 MT plugins are flat-files (or scripts that call modules)

102 Tatsuhiko Miyagawa MT plugin code package MT::Plugin::BanASCII; our $Method = "deny"; use MT; use MT::Plugin; my $plugin = MT::Plugin->new({ name => "BanASCII v$VERSION", description => "Deny or moderate ASCII or Latin-1 comment", }); MT->add_plugin($plugin); MT->add_callback('CommentFilter', 2, $plugin, \&handler);

103 Tatsuhiko Miyagawa MT plugin code (cont) sub init_app { my $plugin = shift; my($app) return unless $app->isa('MT::App::CMS'); $app->add_itemset_action({ type => 'comment', key => 'spam_submission_comment', label => 'Report SPAM Comment(s)', code => sub { }, } );

104 Tatsuhiko Miyagawa

105

106 MT plugin storage require MT::PluginData; my $data = MT::PluginData->load({ plugin => 'sidebar-manager', key => $blog_id }, ); unless ($data) { $data = MT::PluginData->new; $data->plugin('sidebar-manager'); $data->key($blog_id); } $data->data( \$modulesets ); $data->save or die $data->errstr;

107 Tatsuhiko Miyagawa Order control MT->add_callback('CMSPostEntrySave', 9, $rightfields, \&CMSPostEntrySave); MT->add_callback('CMSPreSave_entry', 9, $rightfields, \&CMSPreSave_entry); MT::Entry->add_callback('pre_remove', 9, $rightfields, \&entry_pre_remove); Defined in plugins. No Control on users end

108 Tatsuhiko Miyagawa Conclusion Flat-files vs. Modules

109 Tatsuhiko Miyagawa Flat-files: ☺ Easy to install (Just grab it) ☻ Hard to upgrade OK for simple plugins

110 Tatsuhiko Miyagawa Modules: ☺ Full-access to Perl OO goodness ☺ Avoid duplicate efforts of CPAN ☻ Might be hard to resolve deps. Subversion to the rescue (could be a barrier for newbies)

111 Tatsuhiko Miyagawa Nice-to-haves: Order control Temporarily disable plugins Per plugin config Per plugin storage

112 Tatsuhiko Miyagawa Resources Class::Trigger Module::Pluggable Ask Bjorn Hansen: Build Easily Extensible Perl Programs qpsmtpd MT plugins Kwiki Plagger


Download ppt "Writing Pluggable Software Tatsuhiko Miyagawa Six Apart, Ltd. / Shibuya Perl Mongers YAPC::Asia 2007 Tokyo."

Similar presentations


Ads by Google