1. (1) Introduction to Software Build Technology Philip Johnson Collaborative Software Development Laboratory Information and Computer Sciences University of Hawaii Honolulu HI 96822

2. (2) Objectives Understand motivation for build and packaging technologies. Why was running your classmate’s Robocode robot so hard? Become familiar with a sample of build technologies. Get introduced to the build technologies and standards to be used in this class.

3. (3) Motivation Modern team-based, cross-platform development involves: Multiple developers and users. System is developed/used on multiple platforms: - Win2K, Linux, Win98, Solaris Developers/users have different environments: - c:\jdk1.5.0_08, d:\jdk1.5, c:\myjava Differences between user and developer distributions: - Source distribution for developers - Binary distribution for users Developers may prefer different IDEs: - Eclipse, JBuilder, Emacs, RationalRose Users may not have any IDE at all. Frequent releases and distributions. Dependence on third party libraries (which may themselves be released frequently). These issues are not unique to Java development!

4. (4) Motivation (cont.) Build/packaging techniques should allow: Users to build and test the system without IDE. Developers to build and test the system using their IDE of choice. Developers to build and test the way users build and test. Configuration to different platforms and different environments within a platform. Multiple release configurations Automated build to prevent errors Rapid release, deployment, testing Dependency management for 3 rd party libraries

5. (5) A quick tour of build tools

6. (6)

7. (7) Make ‘Make’ is the canonical build tool. Make features: File-level dependency analysis - Example: compile only changed files Unix-based (but ported to other platforms) Language independent (Java, C++, etc.) ASCII configuration file syntax - Tab characters required—Yikes! Supports invocation of any shell command. Does not support library-level dependency management. Related: imake, cmake, scons

8. (8)

9. (9) Ant Java-based build tool by Apache http://ant.apache.org/ Features: Java-based Cross-platform Extensible XML configuration files Open source Dependency management provided by Ivy.

10. (10)

11. (11) Maven “Project Management” technology Build based on Ant. Documentation and reporting plugins. File-level dependency management Library-level dependency management Library repository definition & management “Convention over configuration” Maven builds in conventions for project layout and workflow to simplify use. - Great if you like them.

12. (12) Ant and Maven Both based upon XML configuration files. Both provide a high-level "Domain Specific Language" for building systems. DSL makes assumptions about what you want to do and how you want to do it. Extensible via "plugins" More difficult to write

13. (13) Rake

14. (14) Rake "Thin coating" of build functionality on top of Ruby language: Predefined libraries for packaging. Prerequisites for dependencies. Not a DSL: If you know Ruby, you're good to go.

15. (15) Buildr

16. (16) Buildr A "rake for Java" Underlying language is Ruby Understands Java files and Maven file structure

17. (17) rake and buildr Do not attempt to provide a DSL for builds. Requires you to (eventually) learn a general purpose programming language (Ruby). No "plugins" required---just write code for your special situations.

18. (18) The basic divide Domain-specific language Ant Maven Syntactic Sugar Make Rake Buildr Read "Why everyone (eventually) hates (or leaves) Maven for a good discussion of the divide: DSLs get you farther, faster, initially Ultimately, the DSL gets in your way as your needs get more sophisticated.

19. (19) Our approach We will use Maven You get "farther, faster" using Maven. Many projects use it. Our projects do not have (many) idiosyncracies that make Maven life difficult.

20. (20) Basic Ant concepts Every project requires an Ant build file named build.xml by default. Each build file is composed of targets. Typical targets: compile, junit, etc. Each target is composed of tasks Ex: copy.java files to build/src, invoke javac... Targets can have dependencies Ex: modified sources must be recompiled before testing.

21. (21) Ant targets Dependent targets are executed exactly once. Example: - test depends upon compile - deploy depends upon compile - all depends upon test, deploy Compile will be executed only once. Some targets are executed only when needed. Sources recompiled only if changed.

22. (22) Example Ant target <target name="compile" depends=“init" description="Compiles code."> <javac srcdir="${src.dir}" destdir="${build.dir}/classes“/>

23. (23) Best practices Ant provides a simple, powerful language for building Java-based systems. How do we use this language effectively? The next section describes my best approach so far on how to employ Ant to achieve the Three Prime Directives of Open Source Software Engineering. The “robocode-pmj-dacruzer” example project provides you with a template structure to get you off and running quickly.

24. (24) robocode-pmj-dacruzer: An example Ant-based build system Implements a pretty lame Robocode robot. Not very interesting. It’s the build system that’s interesting: Provides a template - Easy to adapt to your own projects. Integrates third-party build tools - Checkstyle, PMD, FindBugs, etc. Is modular and extensible - You can grow or shrink it to your needs. Well suited to build automation and project hosting: - Continuous integration (Hudson) Uses Ivy to download/install/manage third-party libraries. Result is Maven-ish, but without the “black box” If you want to change something, it’s just Ant code.

25. (25) Top-level directory contents build.xml top-level build file. *.build.xml “module” build files, generally one per tool. src/ the system source files lib/ library files: maintained by Ivy. build/ derived files: generated by build process

26. (26) Build operations compile compile system jar create jar file junit run junit tests javadoc build javadoc docs clean delete build dir. dist create zip dist file. checkstyle analyze source code findbugs analyze byte code pmd analyze source code sclc compute size of system jacoco compute coverage of test cases. (Others to be added later in semester!)

27. (27) Build System Scalability Issues Modularity: putting all targets in one build.xml file creates a very large build.xml file that is difficult to understand and maintain. Incrementality: you may not want to require all developers to have to install every possible third party tool. Coupling: dependencies between target and property definitions should be minimal and easily visible.

28. (28) Build system architecture build.xml pmd.build.xml findbugs.build.xml dist.build.xml checkstyle.build.xml javadoc.build.xml sclc.build.xml *.build.xml build.xml contains most project- specific definitions. *.build.xml files import the build.xml file and provide an implementation of a single extension. common.build.xml boilerplate code

29. (29) build.xml build.xml provides most (but unfortunately not all) of the project-specific definitions: version number dependency libraries project name etc. Most other *.build.xml files can be used with little to no changes. Typical exceptions: junit.build.xml, emma.build.xml, jar.build.xml

30. (30) *.build.xml responsibilities Implements support for a single tool/function: Checkstyle, Jar, JavaDoc, PMD, Distribution, etc. Typical tasks (assume a tool called 'foo'): Download foo using Ivy, install in lib/foo Download a configuration file for foo, store in lib/configfiles. Provide three standard tasks: - foo.tool: runs foo tool, generates data. - foo.report: creates HTML file. - foo: runs foo.tool and foo.report. Make 'foo' the default task.

31. (31) User Interface To do initial installation and compilation: ant For other operations, use -f : ant -f dist.build.xml ant -f junit.build.xml ant -f checkstyle.build.xml ant -f pmd.build.xml ant -f emma.build.xml etc.

32. (32) Considerations This approach provides the following: Minimal coupling: *.build.xml files depend only on definitions in build.xml Modularity: individual *.build.xml files are small. Reporting: "report" targets provide details on what the "tool" targets generated. One problem: How to "verify" that all analyses "passed"?

33. (33) verify.build.xml Provides “one stop shop” for system analysis: Imports not just build.xml, but other *.build.xml files as well. Runs all "tool" analyses, failing build if they generate any warnings/errors. - Compile, Junit, PMD, Checkstyle, JavaDoc, FindBugs User interface: ant -f verify.build.xml

34. (34) The lib/ directory Contains third-party libraries required by project: robocode checkstyle pmd findbugs etc. Contains configuration files for analysis tools. lib/configfiles/checkstyle.modules.xml lib/configfiles/pmd.rulesets.xml lib/configfiles/findbugs.filter.xml

35. (35) The build/ directory Contains files derived from build process. Can be deleted and reconstructed. Not included in version control or distribution.zip. Build subdirs: Targets creating them: build/javadoc/javadoc build/classes/compile build/dist/dist build/sclc/size build/junit/junit

36. (36) build/ vs. bin/ Both Ant and Eclipse (or any other IDE) compiles files. It causes problems if Ant and Eclipse are using the same output directories for compilation/etc. Therefore, Eclipse writes to the bin/ directory, not the build/ directory. Set in Project | Properties | Java Build Path | Source

37. (37) Testing Notes There are two kinds of “tests” in this system: Manual QA using JUnit - Ensures your code satisfies its specifications (if you write good test cases.) Automated QA using Checkstyle, FindBugs, PMD - Ensures your code obeys best practices for coding style. Encourages incremental testing and formatting.

38. (38) Traditional release numbering Traditional approach (i.e. jakarta-tomcat-5.1.0.zip) MM.mm.bb, where: - MM = Major release number (major redesign or incompatible changes) - mm = Minor release number (minor enhancements, backward compatible) - bb = Bugfix release number (bug fix, no new functionality) All numbers maintained manually. Advantage: Release number comparison indicates degree of change. Disadvantage: Requires manual editing of release value every time which leads to errors (easy to forget to do this.)

39. (39) Our release numbering Our approach (i.e. robocode-pmj-dacruzer-1.3.2011.09.11.12.32.zip) Major.minor.timestamp, where: - Major = major release number - Minor= minor release number - Timestamp= time of bugfix release (yyyy.mm.dd.hh.mm) Major and minor release number maintained manually. Bugfix timestamp generated automatically. Advantages: Automatic except when incrementing major/minor release. Release number provides timestamp of release. Disadvantages: Number of bugfix releases not indicated by release number. Very long version number.

40. (40) IDE integration Important to maintain independence from IDE. All build functions should be possible without any IDE. But it’s fine to provide IDE-specific files to facilitate development. Example:.class and.project files in top-level directory for Eclipse. Provide additional files/directories for other IDEs if desired.

41. (41) robocode-pmj-dacruzer as a standard From now on, all projects must be developed, packaged, and submitted using the standard build conventions embodied in this package.

42. (42) robocode-pmj-dacruzer as a template You can use robocode-pmj-dacruzer as a template for your own projects: Download a fresh version. Remove dacruzer-specific info Add your project info. See the template instruction page in the course website for step-by-step instructions.

43. (43) Ant/Eclipse integration Eclipse has nice support for Ant, but: You will also need to tell Eclipse about ANT_HOME: - Window | Preferences | Ant | Runtime | Ant Home… Make sure Eclipse writes to the bin/ directory - Project | Properties | Java Build Path | Source

44. (44) Limitations of example The robocode-pmj-dacruzer template provides a framework plus basic build functionality, but has limitations: No SVN integration No support for web app development. No support for automated metrics collection. We will enhance this initial template during the semester.

45. (45) Meta-level goal Ant provides an efficient solution to satisfying both PD#2 (users can install system) and PD#3 (developers can understand and enhance system) Other approaches exist: InstallAnywhere for PD#2 (more work) IDE-specific solution for PD#3 (limited) With good installation documentation, I believe Ant-based build can satisfy both PD#2 and PD#3.

46. (46)