1. Semantics in Declarative System The Evolution of Business Unit Empowerment Dan McCreary Dan McCreary & Associates Wednesday, 5/23/2007 8:00 AM - 9:00 AM Level: Business/Strategic M D Metadata Solutions

2. M D 2 Presentation Summary Declarative programming has become the latest buzzword to describe languages that abstractly define systems requirements (the what) and leave the implementation (the how) to be determined by an independent process. This makes the semantics (meaning) of declarative data elements even more critical as these systems are shared between organizations. This presentation: –Provides a background of declarative programming –Describes why understanding the semantic aspects of declarative systems is critical to cost-effective software development Note: All opinions stated in this paper are solely those of the author.

3. M D Metadata Solutions 3 Presentation Summary (cont) –Discusses declarative and semantic aspects of common development systems such as; XHTML, CSS, XForms, XML transforms, XML Schemas, OWL, metadata registries, web services, composition, service-oriented architectures (SOA) and the enterprise service bus (ESB) –Discusses how social networking software and Wikis are used to quickly build consensus on precise semantics –Presents ten specific recommendations to lower costs of agile information systems

4. M D Metadata Solutions 4 Presentation Includes Definitions of declarative systems and contrasts with them with traditional procedural systems and stand- alone declarative languages A critical analysis of semantics in declarative systems Case studies using XForms, Wikis and other collaborative software The role of social networking systems, reputation and trust in the development of semantically precise declarative frameworks Specific recommendations of how organizations can be more effective by integrating semantics and declarative systems into their software development processes

5. M D Metadata Solutions 5 Evolution Metaphors Specialization of Languages Generalization of Languages

6. M D Metadata Solutions 6 Evolution: Specialization Darwins Galapagos Finches Beaks are highly adapted to different food sources Finches adapted to specific ecological "niches over millions of years of isolated evolution Similar to domain-specific declarative languages See Wikipedia "Darwin's Finches"

7. M D Metadata Solutions 7 Evolution: Generalization Generalization: The Raccoon –The world has a higher population of raccoons today due to their ability to quickly adapt to changing urban environments –Similar to highly adaptive procedural languages

8. M D Metadata Solutions 8 Computer Science Abstractions Assembly 1010001010 Fortran FOR I = 1 TO 10 DO Structured Function(A, B) Object Class MyClass( Method MyMethod Declarative time Higher abstractions

9. M D Metadata Solutions 9 The Software Development Process Requirements are about What Design and Build is about How Requirements (BA) Design (Architect) Build (Programmer) Test (QA Staff) Declarative Languages

10. M D Metadata Solutions 10 A Declarative System Is… A software development system, tailored to a specific domain (such as web applications), used to capture precise business requirements within the context of a problem domain (the implicit context) Declarative systems do not specify how requirements are implemented to build working systems. Declarative systems only define the requirements Declarative systems document requirements in specialized vocabularies and can be used to generate entire working systems including user interfaces, persistence and test data Declarative systems specifically omit some assumed requirements (such as system availability, performance, reliability, security etc) A Declarative system is a set of "little languages" with precise semantics that fit together like a puzzle to solve a problem

11. M D Metadata Solutions 11 Computer Science Definition Do not confuse a Declarative System with the computer science language taxonomy Declarative Language Declarative languages" are used to describe a group of programming languages and to contrast them against imperative languages. Declarative Languages Functional Languages Logic Languages Constraint Languages Have sub-types See Wikipedia Declarative Programming

12. M D Metadata Solutions 12 Declarative Systems and Context Declarative Systems are specialized languages for capturing requirements within a specific domain Just as the word play connotes meaning based on context* (i.e. theater vs. a playground), a given vocabulary has the ability to capture requirements based on the current problem The vocabulary for capturing electronic form requirements (XML Schema) may not be appropriate for expressing your build process (Apache Ant) * See: http://wordnet.princeton.edu

13. M D Metadata Solutions 13 HTML, CSS and SQL HTML, CSS, XQuery and SQL are declarative languages using these definitions The semantics or meaning of each tag in the file is determined by an external organization The syntax does not have to be XML Declarative Systems Introduction to Declarative Systems Dan McCreary April 2007 HTML /* global CSS used by all web pages */ body { font-family: Arial, Helvetica, sans-serif; font-size: 75%; margin: 0; padding: 0; width: 1000px; } h1 { color: blue; padding: 0 15px; } CSS -- Old expensive books SELECT title, author, price, pubyear FROM books WHERE price > 100 AND pubyear BEFORE 1960 ORDER BY title SQL xquery version "1.0"; Old Expensive Books { for $book in doc("books.xml")//book order by $book/title return {$book /title}, {$book}/author}, {$book}/price}, {$book}/pubyear} } XQuery http://www.w3.org/TR/REC-html32#body

14. M D Metadata Solutions 14 XML Schema Sample Screen capture of Altova XML Spy 30 minutes to learn graphical notation See: http://www.altova.com/products/xmlspy/graphical_xml_schema_editor.html

15. M D Metadata Solutions 15 Use Case: Electronic Forms User fills out a web- based form Typical requirements may include listing of data to be gathered, data types, validation codes and data repetition patterns Examples: –HTML Forms –XForms –InfoPath

16. M D Metadata Solutions 16 Forms Players My-XForm.xhtml Rendered Form XForms Players XForms Extension Mobile Client Forms Server Netfront Mobile See Wikipedia XForms

17. M D Metadata Solutions 17 What Is Declarativeness for A Context? Efficiency at capturing the testable business requirements in a semantically precise and concise manner Example: Is a field optional? Will validation of the data fail if the field is missing? Ability of each data element to have precise meaning over time and within organizations Schema Drawing Tool XML Schema File (.xsd) XForms xf|input:required {font-weight: bold;} xf|input:required.xf-value {background-color:#fff6af;} *:required::after {font-weight:bold; font-size:1.5em; content: "*"; color: red; } XForms CSS tags

18. M D Metadata Solutions 18 Declarative Spectrum For any given context different languages have different levels of declarativeness General purpose languages are less abstract but can solve a wide variety of problems Declarative languages have a more narrow purpose and target a specific problem like styling a web page or selecting data HTML Java Assembly Language CC++ JavaScript C# Narrow Purpose More Abstract General Purpose Less Abstract CSS XSLXML Schema Apache Ant XForms XQuery XQuery UpdateSQL XPath Ruby perl Groovy Python XUL

19. M D Metadata Solutions 19 Middle-Tiers Remain Procedural When the interface is consistent, declarative languages flourish Middle tiers tend to have the most variation Wikipedia lists over 200 web application frameworks CMSMatrix.org lists over 700 content management systems user database declarativeness Presentation/style (client tier) Data definition, insert, selection and update (persistence tier) Business Logic (middle tier) See: http://en.wikipedia.org/wiki/List_of_web_application_frameworks and http://www.cmsmatrix.org

20. M D Metadata Solutions 20 The Application Development Puzzle Semantics Constraints Presentation Build Publish Transform Workflow Query Update Forms

21. M D Metadata Solutions 21 Filling In Each Piece With A Declarative Language XMLSchema CSS HTML Apache Ant Metadata Registry Cocoon XSL BPEL XQuery XQuery Update XForms

22. M D Metadata Solutions 22 Semantically Precise Vocabularies StandardDomain Atom/RSSSyndication/Notification ACORDInsurance Industry Forms Data FpMLFinancial Products Markup Language FixMLBanking (Financial Information eXchange) HL7Healthcare (now using XML) MISMOMortgage Industry NIEMNational Information Exchange Model (registry) OTAOpenTravel Alliance (Travel industry) SIFSchool Interoperability Framework GJXDMUS Department of Justice (registry) XBRLXML Business Reporting Language See: http:// en.wikipedia.org/wiki/Category:XML-based_standards

23. M D Metadata Solutions 23 Phone Metadata Shopping Tools Address FirstName See http://niem.gtri.gatech.edu/iepd-ssgt/SSGT-SearchSubmit.do You dont need to know about 100,000 SKUs to purchase 10 items from a grocery store Sub-schema generation tools give you exactly what you need and nothing more

24. M D Metadata Solutions 24 Criteria for Semantic Precision Is there a published standard? Are there ISO/IEC 11179 definitions? –concise, precise, non-circular, distinct Are people using it? –Do a Google search –> 100,000 and you are safe –< 10,000 and you should be concerned Examples: filetype:owl, filetype:xsd See Metadata publishing Wikipedia

25. M D Metadata Solutions 25 If You Use Industry Standards… You Could Be Almost Done… If you use industry standards… –and these standards publish their documents in XML Schema format… –and these standards have been transformed from XML Schema to XForms… –and you use native XML databases to store and XQuery to report on the data… …then sample applications have been created and do not require additional procedural code –just change the constraints in the XML Schema and rerun the transforms See also: http:// www.alphaworks.ibm.com/tech/purexml See: http:// www.exist-db.org

26. M D Metadata Solutions 26 Accessibility Business Analysts SMEs and GUI Tools Users Precisely specify business requirements Requires data stewardship training Procedural Programmers Extend declarative vocabularies and provide web service glue Lower costs by moving routine logic maintenance to lower levels in the pyramid IT Strategists Architecture and Strategy (prevent unnecessary procedural code)

27. M D Metadata Solutions 27 Java Libraries 10,000 class and 100,000+ methods available …but which ones are relevant to your business problem?

28. M D Metadata Solutions 28 Less is More XForms 1.1 has only 21 XML elements Much of the presentation of XForms is deferred to CSS Event management is deferred to the XML Events XML binding is deferred to the XBL standard See Wikipedia Minimalism Mies van der Rohe Reconstruction of the German Pavilion in Barcelona

29. M D Metadata Solutions 29 Learning XForms Vocabulary 1.Bind 2.Case 3.Input 4.Instance 5.Group 6.Help 7.Hint 8.Label 9.Load 10.Output 11.Message 12.Model 13.Repeat 14.Secret 15.Select 16.Select1 17.Switch 18.Submission 19.Submit 20.Textarea 21.Trigger Source: W3C XForms Quick Reference http://www.w3.org/MarkUp/Forms/2006/xforms-qr.html

30. M D Metadata Solutions 30 Recognizing XForms Structures 1.Bind 2.Case 3.Input 4.Instance 5.Group 6.Help 7.Hint 8.Label 9.Load 10.Output 11.Message 12.Model 13.Repeat 14.Secret 15.Select 16.Select1 17.Switch 18.Submission 19.Submit 20.Textarea 21.Trigger Color coding limited vocabularies can increase the speed of pattern matching. Look for advanced text editors to provide custom element coloring.

31. M D Metadata Solutions 31 The New Semantics of "Nutshell" The 1.4 release of Java 2 Standard edition increases the size of the platform by 50%, to 2,757 classes in 135 packages 1.5 and 1.6 add additional classes nutshell: something of small size, amount, or scope in a nutshell : in a very brief statement 992 pages

32. M D Metadata Solutions 32 Procedural Programming is Not Poison It would be a mistake to tell all your procedural programmers that the programs they are creating are fundamentally evil The relevant questions are: -How closely does it fit the problem domain? -Can BAs, SMEs and other non- programmers maintain the business rules? -What are the chances that others will be able to maintain it in future years? -How good are the development tools for your system?

33. M D Metadata Solutions 33 Popular Language Have Better Tools Editor –Syntax coloring Debugger –Set breakpoints –View internal state variables Refactoring tools –Can recognize reoccurring patterns and suggest alternatives Performance –Code profiling demand Limited supply curve $ quantity Limited resource cost curve Whuffie curve Whuffie is a reputation based currency. Prices drop as demand increases. The higher demand for a good debugger, the better open-source products will become.

34. M D Metadata Solutions 34 If You Give a Kid a Hammer… People solve problems using familiar tools People develop specific Cognitive Styles* based on training and experience What are we teaching the next generation of developers? * Source: Shoshana Zuboff: In the Age of the Smart Machine (1988) …the whole world becomes a nail

35. M D Metadata Solutions 35 Use Case: Build Scripts Instructions for compiling source code or transforming data Vocabulary includes terms such as build, compile, transform, copy or clean Examples: –Apache Ant –Apache Maven –UNIX make

36. M D Metadata Solutions 36 Use Case: Data Selection With XQuery Ways to specify what data you want to extract from a data set Typical tasks include selecting attributes (columns), filtering, restricting results and changing sort order Examples –Structured Query Language (SQL) –XQuery (w3c standard) –FLOWR

37. M D Metadata Solutions 37 Semantics The science of meaning What you mean when you say cat How do you associate meaning with symbols (verbal, physical, textual) How do we know if we both mean the same thing when we use a word? What if a word has multiple meanings? Reference: WordNet

38. M D Metadata Solutions 38 Semantic Triangle Symbols can only link to referents through concepts You can not link directly from a symbol to a referent referent concept symbol cat Wikipedia: Semiotic triangle

39. M D Metadata Solutions 39 Communication Communication involves exchanging symbols that describe common attributes A one-to-one match of attributes that describe a common symbol match implies a high precision match cat Domestic feline House pet Has fur Has whiskers Sometimes has fleas Chases mice Domestic feline House pet Has fur Has whiskers Sometimes has fleas Chases mice

40. M D Metadata Solutions 40 Same symbol – different meaning Little match of common attributes Low precision semantic match customer First name Last name Home address E-mail Cell phone number Gender Company name Home office address Branch office address CEO name Web site Industry code

41. M D Metadata Solutions 41 High and Low Precision The ideal is to have a precise standard and to use the data elements exactly as they were intended vague standard vague usage Not-so-good precise standard vague usage better precise standard precise usage best

42. M D Metadata Solutions 42 Semantic Precision Semantic mappings are relative in time and between groups of people (organization) Semantic variability over time –Something that has precise meaning to you today may not have the same precise meaning a year from now –Our memories are imperfect and change over time Semantic variability across organizations (project and organization) –A customer to one organization may denote a person but to another organization it may denote a company

43. M D Metadata Solutions 43 Semantics of an XML Data Element A developer puts an XML data element in an xml file The tag has some meaning and the data within the tag has some meaning when it was created by the developer What is the probability –That the developer will know the same meaning of the code 47 one year later –That another project that opens file will understand and be able to use the meaning of the tag –Vague standards often trigger vague interpretations of the meaning of data 47

44. M D Metadata Solutions 44 Semantic Precision in Space and Time time space: (projects, organizations) Small Semantic Footprint (rapid prototype) Large Semantic Footprint (long lifetime systems) weeksmonthsyears10+ years person team dept. world enter- prise

45. M D Metadata Solutions 45 DRY Coding, XSL and MDA If developers cant quickly transform it… …they will copy it. DRY: Dont Repeat Yourself Documentation –Is always kept up to date? –Do developers communicate their intent? –JavaDoc – generated from the source code The tendency to copy and paste is just too common –Developers must be diligent –Budgets must be adequate –Time must be sufficient You can promote Model-Driven Architecture (MDA) by reducing the effort of transformation from specifications captured in declarative languages

46. M D Metadata Solutions 46 Use Procedural Glue XForms uses a REST interface to send XML to a server Some XML databases (DB2 v9) sill use JDBC interfaces to insert XML documents Use procedural glue to build custom interfaces between systems with incompatible interfaces Procedural Glue XQuery Update (JDBC) XForms XML (REST)

47. M D Metadata Solutions 47 How Quickly Can We Create New Declarative Languages? Configuration files are really just small languages Configuration files are easy to parse, validate data using XML tools and build custom forms to use Easy to teach non-programmers to graphically build XML Schemas to validate XML files –Answer – About a week

48. M D Metadata Solutions 48 How Quickly Can We Create Consensus? Example: XForms standard –Work started in 2001 –XForms 2.0 is still a two to three years away How can we accelerate this consensus building process? What factors impact the rate that species evolve? What impact does life span have on a species? How are design ideas exchanged between species?

49. M D Metadata Solutions 49 Solution: Wikis and Collaboration How long does it take to build consensus on the semantics of a new data element? How many people might use this declarative language? The larger the stakeholder group, the longer it takes

50. M D Metadata Solutions 50 Parker Projection Time 100% Relative Code Base Procedural code (Java, JavaScript, VB, C#, C++) Declarative code (XHTML, CSS, XSLT, XForms) Source: Jason Parker, Minnesota Department of Revenue, November 2006

51. M D Metadata Solutions 51 Top 10 Recommendations 1.Use semantically-precise declarative systems over procedural systems when the business problem is in the right context 2.Aggressively promote data stewardships within business units 3.Use controlled vocabularies and metadata shopping tools to allow non- programmers to select data elements from lists 4.Separate semantics (leaf elements) from constraints (branches) 5.Leverage the pattern matching portion of the brain using graphical development tools and element-based syntax coloring editors 6.Train programmers on ways to extend declarative vocabularies using technologies such as XML Binding Language (XBL) 7.Use procedural code to build interfaces between incompatible systems 8.Isolate procedural by providing XML/REST service wrappers code behind semantically precise web services 9.Dont expect your procedural programmers to jump on the declarative bandwagon without some training 10.Use social networking tools (voting, wikis) to rank and find appropriate tags and extensions

52. M D Metadata Solutions 52 Unanswered Questions 1.How will declarative systems evolve? Will there be a few or many? 2.How quickly will declarative languages evolve? Will they continue to specialize? 3.Can we deliver entire working systems without ever leaving a single family of standards? (see XForms/eXist and w3c example) 4.What will be the impact of semantic wikis and micro-formats? 5.Can individual tags stand alone or do they need to be integrated into larger standards to be accepted? 6.Will metadata shopping tools and graphical constraint tools make the process of building entire systems accessible to mere mortals? 7.Will declarative systems halt overseas out-sourcing of procedural software developers? 8.Will there be a large increase in the demand for staff that can extend declarative systems? 9.Will declarative systems allow applications to run anywhere a Forms Players can run? 10.Will vendors used to locking customers into proprietary APIs be forced to compete by having more robust and more reliable execution platforms?

53. M D Metadata Solutions 53 Selected References Evolution –Charles Darwin, Origin of the Species –Richard Dawkins, The Blind Watchmaker Cognitive Styles –Shoshana Zuboff: In the Age of the Smart Machine (1988) Building Trust –Succeeding with Objects: Decision Frameworks for Project Management, Adele Goldberg and Kenneth S. Rubin XML Data Standards –NIEM Concept of Operations at NIEM.gov –ISO/IEC 11179 (wikipedia) XForms –XForms.org, wikipedia Domain Specific Languages –Niel Ford, Language Oriented Programming –Martin Flowler http://www.infoq.com/presentations/domain-specific-languages