1. XML
(and Perl)
Mark Overmeer, MARKOV Solutions YAPC::EU August 2012

2. Plan XML Schemata (horrors) XML::Compile SOAP and WSDL

3. Be realistic
“XML” is just a like lines on paper, it's nothing more than structural notation:
YAML
JSON
Data::Dumper
Serialize
INI files

4. (dis-)advantages name-spaces utf-8 readable editable industry standard
widely available

5. (dis-)advantages name-spaces utf-8 readable editable industry standard
widely available

6. Name-spaces
Avoid name collisions when multiple specifications get mixed.

7. Name-spaces
Avoid name collisions when multiple specifications get mixed.
Data elements and types are always pairs of (namespace, name)
{ns}name

8. Name-spaces
Avoid name collisions when multiple specifications get mixed.
Data elements and types are always pairs of (namespace, name)
Name-spaces are uri's, usually very long. Introduce abbreviations in your file <otm:xyz xmlns:otm=”

9. Name-spaces
Avoid name collisions when multiple specifications get mixed.
Data elements and types are always pairs
Name-spaces are uri's, usually very long. Introduce abbreviations in your file <otm:xyz xmlns:otm=”
Your abbreviation can differ from other people's abbreviation XML::LibXML::XPathContext

10. Structure XML is syntax, structure and namespaces, nothing more.
<ns:name1 a=”13” xmlns:ns=” <ns:name2>42</ns:name2> <ns2:name2/> </ns:name1>
Terms: container, element, attribute qualified, unqualified

11. Technologies
Validate the structure of a message
DTD
Schema's
RelaxNG

12. Technologies Validate the structure of a message DTD Schema's RelaxNG
Postscript specialists write webservers in postscript. XML people use XML where they can.

13. Example <addressBook> <card>
<name>John Smith</name>
</card>
<name>Fred Bloggs</name>
</addressBook>

14. Description in DTD <!DOCTYPE addressBook [
<!ELEMENT addressBook (card*)>
<!ELEMENT card (name, )>
<!ELEMENT name (#PCDATA)>
<!ELEMENT (#PCDATA)>
]>

15. Description in RelaxNG
<element name="addressBook">
<zeroOrMore>
<element name="card">
<element name="name">
<text/>
</element>
<element name=" ">
</zeroOrMore>

16. Description in Schema <element name="addressBook">
<complexType>
<sequence>
<element name="card"
minOccurs="0" maxOccurs="unbounded">
<element name="name" type="string" />
<element name=" " type="string" />
</sequence>
</complexType>
</element>

17. Description in Schema
<element name="addressBook" type="my:addressBookType"/>
<complexType name="addressBookType">
<sequence>
<element ref="my:card"
minOccurs="0" maxOccurs="unbounded" />
</sequence>
</complexType>
<element name="card" type="my:card" />
<complexType name="cardType">
<element name="name" type="string" />
<element name=" " type="string" />

18. Schema's
A fast way to learn new things is to focus on the unexpected! Let's discuss the problems first.

19. Mistakes
unqualified elements / default prefix <tic xmlns=” <tac /> # default namespace? <toe /> # namespace-less?
unqualified attributes
namespace collisions

20. Mistakes Versioning via namespace http://www.w3.org/1999/XMLSchema

21. Mistakes Versioning via namespace http://www.w3.org/1999/XMLSchema
<xs:schema version="1.0" ...
targetNamespace="
<xs:annotation>
<xs:documentation>
Part 1 version: structures.xsd,v /01/15 11:34:25 ht Exp
Part 2 version: datatypes.xsd,v /01/23 18:11:13 ht Exp

22. Mistakes Version lock-in
schema's use a certain version of an other schema... and do not want to upgrade.

23. Mistakes Version lock-in
schema's use a certain version of an other schema... and do not want to upgrade.
Many protocols start schema-less
KML
most Perl implementations

24. Mistakes validation is optional schema's loaded dynamically
use of uri as name-space confusing
readable or computer processing

25. Mistakes designed by librarians:
base-type is anyType: simple? complex?
in many steps to boolean
Integer is huge 18 digits(!)
Default type is anyType <element name=”city” />

26. Mistakes
Extension via xsi:type <pc xsi:type=”pcNL”>6816 BH</pc>

27. Mistakes
Extension via xsi:type <pc xsi:type=”pcNL”>6816 BH</pc>
Extension via any <sequence> <element name=”address” type=”string”/> <any minOccurs=”0” />

28. substitutionGroups! <element name="base" type="my:basetype"
abstract="true" />
<complexType name="basetype" />
<element name="ext" type="my:exttype"
substitutionGroup="my:base" />
<complexType name="my:exttype">
<complexContent>
<extension base="my:basetype" />
<sequence>
<element ref="my:base" minOccurs="0" />

29. Schema set-up <schema targetNamespace="http://my-uri"
xmlns="
xmlns:mine="
<element name="my-elem" type="mine:my-type"/>
<complexType name="my-type">
...
</complexType>
</schema>

30. Schema set-up <xsd:schema targetNamespace="http://my-uri"
xmlns:xsd="
xmlns="
<xsd:element name="my-elem" type="my-type" />
<xsd:complexType name="my-type">
...
</xsd:complexType>
</xsd:schema>

31. XML Node Attribute or child element? <stock>
<product id=”f0134”>
<name lang=”en”>apple
<quantity>10
<product id=”g2133”>
<name lang=”nl”>voetbal
<quantity>12

32. XML Node Attribute or child element? <stock>
<product id=”f0134”>
<name lang=”en”>apple
<quantity>10
<product id=”g2133”>
<name lang=”nl”>voetbal
<quantity>12
product =>
[ { id => 'f0134'
, name => ...
, quantity => 10 }
, { id => '92133'

33. Three Constructs SimpleType <quantity>10</quantity>
ComplexType/SimpleContent (“tagged”) <name lang=”en”>Perl</name>
ComplexType <person id=”12314”> <name>Damian

34. Three Constructs
SimpleType <quantity>10</quantity> quantity => 10
ComplexType/SimpleContent <name lang=”en”>Perl</name>
ComplexType <person id=”12314”> <name>Damian person => {id => 12314, name => 'Damian'}

35. Three Constructs
SimpleType <quantity>10</quantity> quantity => 10
ComplexType/SimpleContent <name lang=”en”>Perl</name> name => { lang => 'en', _ => 'Perl' }
ComplexType <person id=”12314”> <name>Damian person => {id => 12314, name => 'Damian'}

36. SimpleType
<simpleType name=”xyz”> <restriction base=”int”> <minimum value=”42”> # facets
<simpleType name=”abc”> <list itemType=”int”>
<simpleType name=”timestamp”> <union memberTypes=”date dateTime”>

37. ComplexType/SimpleContent
Simple value extension
<complexType name=”b”>
<simpleContent>
<extension base="int">
<attribute name="a" type="int"/>
</extension>
</simpleContent>
</complexType>

38. ComplexType/SimpleContent
Simple value restriction
<complexType name="t">
<simpleContent>
<restriction base="int">
<attribute name="a" type="string"/>
</restriction>
</simpleContent>
</complexType>

39. ComplexType normal <complexType name="t"> <sequence>
<element name="ta" type="int" />
<element name="tb" type="int" />
</sequence>
<attribute name="aa" type="int" />
<attribute name="ab" type="int"
use="required" />
</complexType>

40. complexType normal <complexType name="t"> <sequence>
<element name=”a” type=”int” />
<choice>
<element name=”b” type=”int” />
<element name=”c” type=”int” />
<all>
<element name=”d” type=”int” />
<element name=”e” type=”int” />

41. Particles normal <choice minOccurs=”0” maxOccurs=”50”>
<element name=”a” type=”int”/>
<element name=”b” type=”int”
minOccurs=”0” />
<sequence>
<element name=”c” type=”int” />
<element name=”d” type=”int” />

42. Particles normal <choice minOccurs=”0” maxOccurs=”50”>
<element name=”a” type=”int”/>
cho_a => [ { a => 10 }
, { b => 42 }
, { a => 7 }
, { c => 1, d => 2 } ]

43. ComplexType/ComplexContent
extension
<complexType name="t2">
<complexContent>
<extension base="me:t1">
<sequence>
<element name="t2" type="int"/>
</sequence>
<attribute name="a2" type="int"/>
</extension>
</complexContent>
</complexType>

44. <schema xmlns="http://www.w3.org/2001/XMLSchema"
xmlns:sidn="urn:ietf:params:xml:ns:sidn-ext-epp-1.0"
xmlns:epp="urn:ietf:params:xml:ns:epp-1.0"
elementFormDefault="qualified" >
<!-- Extensions on existing commands -->
<element name="ext" type="sidn:extType"/>
<complexType name="extType">
<choice>
<element name="create" type="sidn:createType"/>
<element name="response" type="sidn:responseType"/>
<complexType name="createType">
<element name="contact" type="sidn:contactType"/>
<element name="domain" type="sidn:domainType"/>
<complexType name=”domainType”>
<sequence>
<element name=”optOut”>

45. <!-- in epp.xsd -->
<any minOccurs="0"/>
<element ref="epp:command-extension" />
<element name="command-extension" abstract=”true”
type="epp:commandExtType"/>
<complexType name="commandExtType" />
<!-- in sidn.xsd -->
<element name="createDomainExt" type="sidn:crType"
substitutionGroup="epp:command-extension">
<complexType name="crType">
<complexContent>
<extension base="epp:commandExtType">
<sequence>
<element name="optOut" type="boolean">

46. <complexType name="responseType">
<sequence>
<element name="msg" type="sidn:msgType" minOccurs="0" maxOccurs="unbounded"/>
</sequence>
</complexType>
<complexType name="msgType">
<attribute name="code" use="required"/>
<attribute name="field"/>

47. ComplexType/ComplexContent
restriction
<complexType name="t2">
<complexContent>
<restriction base="me:t1">
<sequence>
<element name="t" type="int" />
</sequence>
<attribute name="a" type="int" />
</restriction>
</complexContent>
</complexType>

48. Conclusions Extremely verbose syntax for only a dozen constructs.
Very powerful
Hard to validate

49. Let's get started This saves your day:
my $schema = XML::Compile::Cache->new($xsd);
$schema->prefix(p => '
# $schema->addKeyRewrite('PREFIXES');
# $schema->addKeyRewrite('UNDERSCORES');
print $schema->template(PERL => 'p:elname');

50. XML::(LibXML::)Simple
use XML::LibXML::Simple 'XMLin';
my $hash = XMLin($xml);
no validation
no smart handling
blanks
auto-encoding/-decoding time, base64, ...
repeats

51. # is a xs:boolean
# defaults to 0
open => 0,
# is an atom:atomPersonConstruct
# is optional
author =>
{ # choice of name, uri,
# occurs any number of times
cho_name =>
[ { # is a xs:string
name => "example",
# is a xs:string
uri => "example",
# Pattern:
=> "example", },
], },

52. XML → Perl From String/File/FileHandle to HASHes # compile once
my $rules = XML::Compile::Schema->new($xsd);
my $elem = pack_type($myns, $myname);
my $reader = $rules->compile(READER => $elem);
# call often
my $hash = $reader->($xml);
$elem == “{$myns}$myname”

53. XML → Perl From String/File/FileHandle to HASHes # compile once
my $rules = XML::Compile::Cache->new($xsd);
$rules->prefixes(mine => $myns);
# call often
my $hash = $rules->reader(“mine:$myname”) >($xml);

54. Perl → XML More complex # compile once
my $rules = XML::Compile::Schema->new($xsd);
my $elem = packtype($myns, $myname);
my $writer = $rules->compile(WRITER => $elem);
# call often
my $doc = XML::LibXML::Document
->new('1.0', 'UTF-8');
my $xml = $writer->($doc, $hash);
$doc->setDocumentElement($xml);
$doc->toString(1);

55. Perl → XML More complex # compile once
my $rules = XML::Compile::Cache->new($xsd);
$rules->namespaces(mine => $myns);
# call often
my $doc = XML::LibXML::Document
->new('1.0', 'UTF-8');
my $xml = $rules->write(“mine:$myname”) >($doc, $hash);
$doc->setDocumentElement($xml);
$doc->toString(1);

56. To NIL or not to NIL Conceptual difference between
<person>
<first>Mark
<surname>Overmeer
<pets />
<parents xsi:nil="true" />
Conceptual difference between
not specified (children) minOccurs=0
none (pets) sub-elem minOccurs=0
missing (parents) nillable

57. SOAP / WSDL SOAP 1.1 SOAP 1.2 WSDL 1.1 WSDL 2
SOAP is like RPC (what: procedures)
rpc-encoded
rpc-literal
document-literal
WSDL (how, where)

58. SOAP message <SOAP-ENV:Envelope
xmlns:SOAP-ENV=" xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header>
<x0:Transaction xmlns:x0=" 5
</x0:Transaction>
</SOAP-ENV:Header>
<SOAP-ENV:Body>
<x0:GetLastTradePrice xmlns:x0="
<x0:symbol>DIS</x0:symbol>
</x0:GetLastTradePrice>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

59. SOAP message <SOAP-ENV:Envelope
xmlns:SOAP-ENV=" org/soap/envelope/">
<SOAP-ENV:Header>
<x0:Transaction xmlns:x0="
<SOAP-ENV:Body>
<x0:GetLastTradePrice xmlns:x0="...”>
<x0:symbol>DIS

60. XML::Compile::SOAP my $wsdl = XML::Compile::WSDL11->new($wsdl);
my $call = $wsdl->compileClient('tradePrice');
# run often
my ($answer, $trace) = $call->($request);
if($answer->{Fault}) …
$trace->printTimings;
$trace->printRequest;
$trace->printerrors;

61. XML::Compile::WSDL11 contains definition: schema, elements and types
message: message parts
portType: operations, input and output parts become structures.
binding: operations, encoding (document literal), protocol (http), soapAction
service: application name and server

62. example (continued) my $wsdl = XML::Compile::WSDL11->new($wsdl);
my $call = $wsdl->compileClient('tradePrice');
my $ask =
{ transaction => 5
, request => { symbol => 'DIS' } };
my ($answer, $trace) = $call->($ask);
print Dumper $answer;
# { response => …
# , headerparts => }
operation
message parts

63. example (continued) my $wsdl = XML::Compile::WSDL11->new($wsdl);
my $call = $wsdl->compileClient('tradePrice');
my $answer = $call->(symbol => 'DIS');
if($answer->{Fault}) …
print “$answer->{response}{price}\n”;

64. Create examples
foreach my $op ($wsdl->operations(service => 'MYAPI')) {
my $name = $op->name;
print "produce operation explanation for $name\n";
open OUT, '>:encoding(utf8)', "$name.pl"
or die "cannot write op-explain for $name";
print OUT "**** request by the client\n";
print OUT $op->explain($wsdl, PERL => 'INPUT');
print OUT "\n\n**** response by the server\n";
print OUT $op->explain($wsdl, PERL => 'OUTPUT');
close OUT; }

65. Server $d = XML::Compile::SOAP::Daemon::AnyDaemon->new
( pid_file => '/var/run/appl.pid' );
$wsdl = XML::Compile::WSDL11->new(...);
$d->operationsFromWSDL
( $wsdl
, callbacks => { tradePrice => \&trade_price } );
$d->run
( name => 'my server'
, host => 'localhost'
, port => 4545
, max_childs => 5

66. Handler sub trade_price($$$) { my ($daemon, $msgin, $http_req) = @_;
my $bodyin = $msgin->{request};
my $symbol = $bodyin->{symbol};
my $price = lookup_price_of($symbol);
my $bodyout = { price => $price };
+{ response => $bodyout }; }
message
part
names