1. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 1 Web development processes A/Prof David Lowe University of Technology, Sydney

2. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 2 Software Engineering Wisdom “Some Facts: Before software and computing systems can be developed, their requirements must be reasonably well understood. Before requirements can be finalised the application domain, as it is, must be fairly well understood” ‘Domain Engineering: A “Radical Innovation” for Software and Systems Engineering? A Biased Account’, Dines Bjorner

3. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 3 Software Engineering Heresy  For Web projects, design and requirements can only be jointly resolved!  I will explain why this heresy is appropriate within Web development, and the (incorrect) assumptions that underlies conventional SE wisdom. Domain Understanding RequirementsDesign

4. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 4 Intro / Overview  What makes Web devel. difficult  Web systems impacting on the system domain: mutual constitution  Dealing with the differences PrototypingPrototyping Design modelsDesign models Agile DevelopmentAgile Development  Other issues  …

5. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 5 Web Development  Discrepancy between research and commercial practice!  Research has focused on design and information architectures  Commercial practice has focused on the broader development process, and in particular understanding the develop-client interaction.  Strong divergence of views on numerous issues

6. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 6 Web Development We interview the clients to determine requirementsWe interview the clients to determine requirements We interview intended users to determine requirementsWe interview intended users to determine requirements It is important to respond to changes in reqt’s as they occurIt is important to respond to changes in reqt’s as they occur Changes in user reqt’s require the application to be renegotiated.Changes in user reqt’s require the application to be renegotiated. We often have difficulty in the relationship with clientsWe often have difficulty in the relationship with clients We prefer a single client liaisonWe prefer a single client liaison It is important to identify technologies as soon as possibleIt is important to identify technologies as soon as possible It is important to be able to modify the system once it is completedIt is important to be able to modify the system once it is completed Technical, design and management staff work well togetherTechnical, design and management staff work well together The project size/costs are tracked and corrective actions are takenThe project size/costs are tracked and corrective actions are taken

7. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 7 How is Web development different?  Minor Differences Increased emphasis on UIIncreased emphasis on UI Uninformed competitivenessUninformed competitiveness Fine-grained organic evolution.Fine-grained organic evolution. Open modularised architectures and component-based developmentOpen modularised architectures and component-based development Rapidly changing technologiesRapidly changing technologies Variable client understandingVariable client understanding Diverse teamsDiverse teams

8. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 8 How is Web development different?  Three key differences: Poor understanding of link between business needs and application functionalityPoor understanding of link between business needs and application functionality Tighter synergism between business needs and technologyTighter synergism between business needs and technology Tighter development timeframes and rapid pace of changeTighter development timeframes and rapid pace of change

9. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 9 How is Web development different?  These key issues all relate back to one single “killer” issue:  The system changes the domain! business workflowsbusiness workflows business modelbusiness model interactions with clients and customersinteractions with clients and customers … and ultimately even your clients!… and ultimately even your clients!

10. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 10 But is this really different?  So what – isn't this true for normal SE / IT development? YES! But impact is more immediate and greater in scope.YES! But impact is more immediate and greater in scope. “Conventional” IT development largely ignores this impact, and addresses it through subsequent system refinement and/or BPR!“Conventional” IT development largely ignores this impact, and addresses it through subsequent system refinement and/or BPR!  We ignore it at our peril for Web projects!

11. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 11 Organisation Changing the domain Workflows Roles Activities Resources Tools Communications Expectations

12. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 12 Organisation Changing the domain

13. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 13 Changing the domain Organisation Domain of Influence Domain of Change

14. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 14 Organisation Changing the domain

15. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 15 Organisation Changing the domain

16. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 16 Organisation Changing the domain Domain of Influence Domain of Change

17. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 17 Mutual constitution The problem drives the solution The solution changes (creates?) the problem © MC Escher

18. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 18 So how do we deal with this?  Various approaches Prototypes to explore the impactsPrototypes to explore the impacts Design models that link design to business issuesDesign models that link design to business issues Agile development and evolvable architecturesAgile development and evolvable architectures  Web developers understand much of this inherently … but little of it is rigorous or formally managed.… but little of it is rigorous or formally managed.

19. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 19 Prototyping… Prototyping…  Allows impact exploration  But different from conventional prototyping: 1.Validation of requirements 2.Design exploration 3.Impact exploration  Useful sources of concepts IBM Patterns for E-BusinessIBM Patterns for E-Business Rapid Application DevelopmentRapid Application Development Information modellingInformation modelling

20. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 20 Prototyping  Issues raised: What prototypes should we use?What prototypes should we use? When should these be used?When should these be used? What system characteristics should emerge at different stages of the development?What system characteristics should emerge at different stages of the development? How is this integrated into the development process?How is this integrated into the development process? How does this affect contractual negotiations?How does this affect contractual negotiations? How do clients interact with prototypes?How do clients interact with prototypes? …

21. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 21 Prototyping Analysis

22. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 22 Prototyping Contract Prototype Development Specification Evaluation Prototypes / Partial Designs Client Feedback Specification

23. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 23 Prototyping  Preliminary research  What factors influence the ability to understand the impacts of a proposed system when presented with a series of early web design artefacts?  Lessons Use of storyboards, screen shots and business process models was valuable!Use of storyboards, screen shots and business process models was valuable! WebML content models, example site navigation maps, database schemas were not useful!WebML content models, example site navigation maps, database schemas were not useful! Clients made assumptions about business elements which are not covered by the prototypes (and outside the system scope).Clients made assumptions about business elements which are not covered by the prototypes (and outside the system scope).

24. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 24 Prototyping  Lessons Clients had variable understanding of functional aspects. This was typically not well addressed by the prototypes.Clients had variable understanding of functional aspects. This was typically not well addressed by the prototypes. Weak correlation between understanding of existing business and understanding of impacts! And between understanding of modelling approaches and understanding of impacts.Weak correlation between understanding of existing business and understanding of impacts! And between understanding of modelling approaches and understanding of impacts. Way in which the prototypes were explored appears to be crucial!Way in which the prototypes were explored appears to be crucial! There is little difference between Engineering/IT and Business participants in ability to understand business impactsThere is little difference between Engineering/IT and Business participants in ability to understand business impacts

25. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 25 Design Models  Need to provide a clearer link between low-level design and business issues i.e. how do specific functional or informational elements provide business value, and if they are changed, what are the impacts on the value chains.i.e. how do specific functional or informational elements provide business value, and if they are changed, what are the impacts on the value chains. ? ?

26. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 26 Design Models

27. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 27 Design Models

28. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 28 Agile development  Agile development eXtreme ProgrammingeXtreme Programming  But… How do we define an overall architecture (particularly one which is evolvable)?How do we define an overall architecture (particularly one which is evolvable)? What elements should be designed first (i.e. interplay between providing business value and understanding business impacts)?What elements should be designed first (i.e. interplay between providing business value and understanding business impacts)?

29. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 29 Other Issues  Necessity for changes to the tendering process and contractual arrangements  When can full quotations be provided?  How do we managed evolutionary / incremental development?  Client engagement with the development / Developer engagement with the business  How do we educate clients about technological impacts?  How do we educate developers about business drivers?  External drivers for domain change  How is this handled?  Different technological / business evolution paths?  How do we select between different possibilities?

30. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 30 Take-home messages  Web development is still software development BUT  Web projects have unique characteristics Poor understanding of link between business needs and application functionality and contentPoor understanding of link between business needs and application functionality and content Tighter synergism between business needs and technologyTighter synergism between business needs and technology Tighter development timeframes and rapid pace of changeTighter development timeframes and rapid pace of change  But especially: Inter-dependence if problem and solution domainsInter-dependence if problem and solution domains  Conventional approaches work, but need to be adapted appropriately!

31. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 31 Invitation  Ongoing research: Role of prototypesRole of prototypes How do clients understand system impactsHow do clients understand system impacts How can we relate designs and business processes and models more explicitlyHow can we relate designs and business processes and models more explicitly  Activities: Case studies of system evolution (emergence of understanding, and what affects this)Case studies of system evolution (emergence of understanding, and what affects this) Impact prediction toolsImpact prediction tools  We are looking for commercial collaborators Critique of (and report on) your processesCritique of (and report on) your processes Model your systems and business processesModel your systems and business processes Use of our tools and knowledgeUse of our tools and knowledge

32. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 32 References / Additional Info  Textbook: D. Lowe and W. Hall, Hypermedia and the Web: An Engineering Approach, Wiley, 1998  Other References: M. Abrams, World Wide Web: Beyond the Basics, Prentice-Hall, 1998M. Abrams, World Wide Web: Beyond the Basics, Prentice-Hall, 1998 J. Nielson, Multimedia and Hypertext, 1995, Academic PressJ. Nielson, Multimedia and Hypertext, 1995, Academic Press  Design models: WebML: http://www.webml.org (and especially http://webml.elet.polimi.it/webml/bibliography.htm)WebML: http://www.webml.org (and especially http://webml.elet.polimi.it/webml/bibliography.htm)http://www.webml.org http://webml.elet.polimi.it/webml/bibliography.htmhttp://www.webml.org http://webml.elet.polimi.it/webml/bibliography.htm UML: http://citeseer.nj.nec.com/conallen99modeling.htmlUML: http://citeseer.nj.nec.com/conallen99modeling.htmlhttp://citeseer.nj.nec.com/conallen99modeling.html UML+UID: http://www.inf.puc-rio.br/~schwabe/papers/UML2000.pdfUML+UID: http://www.inf.puc-rio.br/~schwabe/papers/UML2000.pdfhttp://www.inf.puc-rio.br/~schwabe/papers/UML2000.pdf OOHDM: http://www.telemidia.puc-rio.br/oohdm/OOHDM.htmlOOHDM: http://www.telemidia.puc-rio.br/oohdm/OOHDM.htmlhttp://www.telemidia.puc-rio.br/oohdm/OOHDM.html

33. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 33 References / Additional Info  Design methods XP: http://www.extremeprogramming.orgXP: http://www.extremeprogramming.orghttp://www.extremeprogramming.org RUP and UML: http://www.rational.com/products/whitepapers/101057.jsp and http://www.rational.com/products/whitepapers/101066.jspRUP and UML: http://www.rational.com/products/whitepapers/101057.jsp and http://www.rational.com/products/whitepapers/101066.jsphttp://www.rational.com/products/whitepapers/101057.jsp http://www.rational.com/products/whitepapers/101066.jsphttp://www.rational.com/products/whitepapers/101057.jsp http://www.rational.com/products/whitepapers/101066.jsp JAD and PD: http://sern.ucalgary.ca/courses/seng/613/F97/grp1/report.html and http://sern.ucalgary.ca/~paulson/SENG613/seng613-jad.htmJAD and PD: http://sern.ucalgary.ca/courses/seng/613/F97/grp1/report.html and http://sern.ucalgary.ca/~paulson/SENG613/seng613-jad.htmhttp://sern.ucalgary.ca/courses/seng/613/F97/grp1/report.htmlhttp://sern.ucalgary.ca/~paulson/SENG613/seng613-jad.htmhttp://sern.ucalgary.ca/courses/seng/613/F97/grp1/report.htmlhttp://sern.ucalgary.ca/~paulson/SENG613/seng613-jad.htm  Miscellaneous Web Configuration Management: http://www.susandart.com/publicat.htmWeb Configuration Management: http://www.susandart.com/publicat.htmhttp://www.susandart.com/publicat.htm SUE: Garzotto F and Matera M (1997) "A Systematic Method for Hypermedia Usability Inspection", The New Review of Hypermedia and Multimedia, Vol 3, pp39-65SUE: Garzotto F and Matera M (1997) "A Systematic Method for Hypermedia Usability Inspection", The New Review of Hypermedia and Multimedia, Vol 3, pp39-65 TAM: Davis, F.D. “Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology”, MIS Quarterly, 1989, 13(3):319-342.TAM: Davis, F.D. “Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology”, MIS Quarterly, 1989, 13(3):319-342.  Numerous Web Engineering workshops and conferences: See http://www.teco.edu/~gaedke/webe/See http://www.teco.edu/~gaedke/webe/http://www.teco.edu/~gaedke/webe/

34. July 29-31, 2003 WebTech'03 - "Web Development Processes", David Lowe, UTS Slide 34 Questions ??????