1. CPAN6 Mark Overmeer YAPC::EU 2007 LinuxConf EU 2007 (subset of slide)

2. index history collecting collecting collections CPAN6 and Pause6
releases
use-cases
initial results
status

3. history
Perl5 has CPAN
CPAN is very successful: >12k modules by >3500 authors
designed in 1995: very simple web-interface, based on “old technologies”
strongly entangled with Perl5

4. history
Perl5 has CPAN
CPAN is very successful: >12k modules by >3500 authors
designed in 1995: very simple web-interface, based on “old technologies”
strongly entangled with Perl5
Perl6?
modules, PIR, PASM, ...
ParTcl, Pieton, Lua, ... parrot friends where wil we collect it?  CPAN6

5. history
CPAN6 started as non-(perl-)community investigation into needs and possiblities

6. history
CPAN6 started as non-community investigation into needs and possiblities
CPAN6 can be used to collect and distribute everything (also Perl modules)

7. history
CPAN6 started as non-community investigation into needs and possiblities
CPAN6 can be used to collect and distribute everything (also Perl modules)
CPAN6 may never be used for Perl (and may need a different name)

8. CPAN, the goods and bads What makes CPAN successful
simple to use; anyone can contribute
open and accessible
at lot of useful stuff
very stable interface

9. CPAN, the goods and bads What makes CPAN successful
simple to use; anyone can contribute
open and accessible
at lot of useful stuff
very stable interface
What are the problems
simple to use: no protections
no described rules; needs GBDs
too much useless stuff; name-space polution
no major changes since long

10. abstraction
Goal: distribute Perl software releases from the publisher to the user.

11. abstraction
Goal: distribute Perl software releases from the publisher to the user.

12. abstraction
Goal: distribute Perl software releases from the publisher to the user.
Goal: distribute Perl software releases from the publisher to the user. release = a set of files

13. Nederlands Openluchtmuseum
Nederlands Openluchtmuseum (Dutch Open-Air Museum)
“European Museum of the Year” in 2005
Old history (windmills!) and Modern history (1950-)

14. “Spaarstation Dingenliefde”
Exposition about people collecting things
garden gnomes

15. shopping lists
money boxes
crucifixes

16. sick bags
pipes
tattoos

17. Collecting People are collectors, in nature
We collect more than we can handle
archivists, librarians, and bookkeepers come to the rescue.
Not everyone is well organized
have a look at your attic! (or agenda, home-dir, brain, desk-top, kitchen, ...)

19. where do we keep out things?
Let us focus on digital collections:
software
photos, films
music
documentation
...

20. digital collections In a directory structure
on (removable) disk, private use accessible on the local system
on a network disk, group use accessible on the local network
on an ftp-server, global use for world-wide access

21. digital collections In a directory structure
on (removable) disk, private use accessible on the local system
on a network disk, group use accessible on the local network
on an ftp-server, global use for world-wide access
Applications do not care about the needs for collecting. Who does care?

22. three levels of CPAN6 the CPAN6 infrastructure
distribution of “released” digital information over the network. Platform and implementation independent.

23. three levels of CPAN6 the CPAN6 infrastructure
distribution of “released” digital information over the network. Platform and implementation independent.
the Pause6 archiver implementation
upload/download rights, trust, name-space administration, rules.

24. three levels of CPAN6 the CPAN6 infrastructure
distribution of “released” digital information over the network. Platform and implementation independent.
the Pause6 archiver implementation
upload/download rights, trust, name-space administration, rules.
Application integration
like “open” and “save” menu entries

25. overview
Once installed, CPAN6 let you start a new archive as simple as creating a directory.
The archives can be local or remote; no difference.
Each archive has rules, which control the behavior.
Rule set from very strict to very forgiving.

26. identities
Creators: people (using a VCS?) create files which belong together: distributed as end-product, a release
Publisher: someone uploads the release to the archive
A release as a project name (name-space location) and version label, together unique, both unicode
Authors own the project (name), and have to give their consent to the releases
Finally, the Users download the data

27. archiver
the rules of an archive are set by the Board; a group of people.
releases are uploaded to the Commissioner (daemon) which checks the rules
releases are mirrored by Deployers, which can be searched by users, and then used for download
uploading, downloading, and (smart) mirroring is performed by Scribe processes

28. scribes Scribes organize the best way of transport:
compression: gzip, bzip2, ..., none
packaging: zip, tar, rpm, ..., none
protocol: HTTP, FTP, , CD/DVD,  no new protocols  no fixed discissions

29. scribes Scribes organize the best way of transport:
compression: gzip, bzip2, none
packaging: zip, tar, rpm, none
protocol: HTTP, FTP, , CD/DVD
Scribes can do filtering
only the last releases from a source archive
only certain mime-types or file-sizes
exclude the alphas and beta releases  building complex hierarchies

31. release states Any release has a state: uploading published embargo
released
deprecated
expired
rejected
Release state change require signatures

32. release types
What can be put in files, can be released into an archive:
publications (of course)
identities; PGP public keys, ...
references to related archives  they all follow the same rules

33. release types
What can be put in files, can be released into an archive:
publications (of course)
identities; PGP public keys, ...
references to related archives
archive configuration; the Constitution, as written by the board
storage configuration, for Stores  they all follow the same rules

34. project / release relations

35. target
Improving my data organization

36. target
Improving my data organization 

37.  target Improving my wifes data organization
if it works for her, it works for everyone 

38. My Photos

39. My Photos 
Our
Photos

40. use-cases Software distribution; like CPAN
FTP-server replacement or added security, search, and maintenance
Paper trails through the company
Developing and distributing Linux distributions

41. status > 120 pages detailed (XML Schema) design
Start with implementation in Perl5
error handling/translation framework
compliant XML schema processor
CPAN6 pluggable components
jQuery graphical interface
Promotion
I need help!

42. short term needs Schema design proofreaders
Trust / security algorithms
License text archive
Developer support in time or money
Your ideas...

43. progress, design Requirements and terminology with Sam Vilain
Open architecture and functionality
Data design quite detailed
Websites, mailinglist

44. too many changes?
Install a perl distribution using CPAN6: $ perl -MCPAN6 -e 'install DBIx::Simple' CPAN: Storable loaded ok Going to read /root/.cpan/Metadata Database was generated on Sun, 19 Aug :38:06 Running install for module DBIx::Simple Running make for J/JU/JUERD/DBIx-Simple-1.31.tar.gz CPAN: LWP::UserAgent loaded ok Fetching with LWP: failed Fetching with CPAN6: pause6 get perl5.perl.org DBIx-Simple tar xzvf DBIx-Simple-1.31.tar.gz DBIx-Simple-1.31/ DBIx-Simple-1.31/lib/ DBIx-Simple-1.31/lib/DBIx/ ...

46. CPAN6 standard services
Simple data selection and distribution
no download of package list needed
no packaging into tar.gz needed
Smart dependencies
dependency `prediction'
parallel installation of modules (for Perl6)
dependency classes: run, test, install, optional
Clear and strict rules
GDBs become board, with transparent rules

47. graphical interface “Web2.0”: JavaScript & AJAX
resembles Open/Save with optional meta- data
talks to local daemon

48. challenge
All Pause6 features are implemented the same way, but the user will not understand this: translate it into user space. Example, equivalent are
use a new public key
opload a set of holiday photos
start a new archive
change the rules of an archive
initiate a mirror
submit a change request form

49. progress, general purpose
Building general purpose modules first:
OODoc
generating heterogenous sets of manual pages
Log::Report
exception handling / error reporting
translations
XML::Compile
interpreting data definitions

50. progress in core Implementing “Stores”
archive, project, and version names are all UTF-8 of unlimited length (restrictions possible)
plain (mutulated) projection
encoded (%2E) projection
mapped projection (name -> something)
Implementing “Identities”
(UNIX) username
PauseID
public-key

51. soon Log::Report::Views
combine translated (error) messages with tooltips and detailed documentation.
Frameworks to merge
interface components
jQuery needs
...
Licenses
filling a license database

52. whishes
Perl(5) installation tools are (mostly) outside the scope of CPAN6, but could we please:
install multiple module versions in parallel
first download all dependencies, and test them together, before installing them
be much less verbose in the process; we frighten people off!