1. Karabo installation and software organization
Karabo Workshop
Hamburg,
Janusz Szuba
WP76
How to edit the title slide
Upper area: Title of your talk, max. 2 rows of the defined size (55 pt)
Lower area (subtitle): Conference/meeting/workshop, location, date, your name and affiliation, max. 4 rows of the defined size (32 pt)

2. Outline
Overview of development environment
Structure and organization of Karabo software
Step by step installation

3. Big picture Package repository Source code maintenance
Continuous Integration
Code development
Documentation
Bug tracker

4. Big picture Package repository Source code maintenance
Continuous Integration
Code development
Documentation
Bug tracker

5. Code development Supported operating systems:
Ubuntu 10.04, – 32 and 64 bit architecture
Scientific Linux 6.3, 6.4 – 32 and 64 bit architecture
Mac OS X, 10.7 (Lion), 10.8 (Mountain Lion) – 64 bit architecture
Languages
C++ (gcc 4.6, 4.7, 4.2 – native Mac OS X compiler)
Python 2.7
Build system
Make, GNU autotools (for dependencies)
NetBeans projects

6. SVN repository Central SVN repository https://svnsrv.desy.de
Authentication – kerberos , basic and certificate
Preferred kerberos for all having DESY accounts
For external collaborators – basic authentication
Web interface
Command line access
With kerberos authentication
With basic authentication

7. Special arrangement for tutorial SVN repository
Special Karabo Tutorial repository
Web interface:
(only for selected users)
Command line:
(only for selected users)
User name and password
24Sep2013
When you plan to develop karabo devices or contribute to framework after the workshop
Create your own account in SVN repository if you don’t have one already
Instructions – Getting Started on Karabo framework site in Alfresco Documentation server
Contact us to get access to karabo repository
Url to karabo package development

8. SVN repository structure
Karabo repository structure
karaboFramework
Base libraries, binaries, python bindings, GUI
karaboPackages
Plugins – devices, sorted in different categories like controlDevices, computeDevices, etc.
Additional dependencies for plugins
karaboPackageDevelopment
Environment for developing and testing plugins
Helper script for creating new plugins, commiting, checking out from repository etc.
Basic templates with netbeans projects for devices or dependencies
Running environment – scripts for starting device servers, gui, broker etc

9. Code maintenance conventions
Follow best practices for a project structure:
Trunk, branches, tags folders for each individual project/plugin/dependency
Assured when using „karabo” helper script
Follow best practices for release versioning:
Major, minor, patch level: e.g
Major number change – possible compatibility break in API, communication protocols
Minor numer change – assured API compatibility, new features
Patch level change – bug fixes
Naming conventions for device plugins, dependencies
Camel case for package name, like simpleMotor, simulatedCamera
Capitalization for class names, like TestMessageReader
Camel case for executables, like tcpClientAsync
Coding conventions, netbeans customization

10. Continuous Integration - Jenkins
Automated or on demand tasks:
Compiling on different OS
Testing – unit tests
Creating installation packages
Deploying and running functional tests
Current setup with Jenkins
Nightly builds of karaboFramework from trunk
Keeping trunk in shape all the time
Release builds
karaboFramework installation packages creation, testing and copying to web/ftp repository
Jobs on demands, parametrized with svn version, unit tests options, build type, etc
Plugin for triggering jobs from netbeans

11. Installation packages repository
Currently contains karaboFramework binary installation packages
Binary packages for plugins and dependencies, as well as packaging into rpms and debian packages will follow in future
Repository location
Internal network
Accesible from outside
Repository structure
Nightly builds: date subfolder, filename indicates svn revision number
Released builds: trunk(branch)/version/date, filename indicates version number
Links for karabo framework installation packages

12. Bug tracker Mantis bug tracker
Currently available within internal network
Planned migration to a web service available world-wide
Register first, then create a new bug report
Don’t submit a bug anonymously

13. Documentation Karabo Framework site on Alfresco documentation server
All EuXFEL emploees have account by default
For external collaborators, tutorial participants, accounts are created and invitations to join the site sent
Wiki pages, Document library
Plugin/devices documentation site – Karabo Plugins
Accesible from within GUI as a documentation panel
Editable in GUI and on the web

14. Karabo support email karabo@xfel.eu
Request Tracker distribution list
Serves as a support entry point

15. Karabo software use cases
Karabo Framework developer
Needs all the source code
Development environment fully equipped with all necessary dependencies (mainly development packages – header files)
Karabo Device Plugin developer
Needs Karabo Framework binary package
Binaries, libraries, header files, python source
Needs plugins source code templates, examples
Needs environment for testing developed plugins
Karabo software deployment
Needs precompiled set of plugins and their dependencies
Needs configuration/versioning tool for selecting compatible set of karabo framework and plugins versions
For the first two use cases we provide consistent environment setup
Deployment use case is in preparation

16. Karabo Framework software structure
Separate netbeans projects
Karabo library – base library
Code structured to functional groups
util, net, io, core, xms, log, xip, webAuth
Karathon library – python bindings
Binaries
Device server, broker message logger
Python karabo
Natively implemented device server and FSM
Python GUI and command line interface
Third-party dependencies – compiled and shipped with installation package
Python 2.7 with addtional modules: numpy, scipy, matplotlib, ipython (with qtconsole and notebook), pyqt4, h5py
Mac OS X needs its own python installation (preferred way through macports)
Boost, hdf5, log4cpp, cppunit
openMQ C API and JMS broker (currently broker on Linux only)

17. Karabo Framework binary installation package
Precompiled Karabo Framework is available as a self extracting tar file
Always installs under single folder karabo-version or karabo- revision_number in case of nightly builds
Interactive or batch (needs --prefix=top_installation folder) installation
During post-installation phase two configuration files are created
$HOME/.karabo/karaboFramework
Contains full path to karabo framework installation directory
Used in netbeans makefiles for finding location of libraries and header files
Every new installation of karabo framework overwrites this file with a new location
$HOME/.karabo/karaboSvnPath
Contains url pointing to karabo SVN repository for svn command line client
Needed as different authentication schemes (kerberos, basic, certificate) have different urls

18. Karabo Package Development environment
Helps with creation, development and running/testing device plugins
Set of templates for new packages
karabo script
new Create a new package
checkout - Checkout an existing package (use list to retrieve name)
install - Install a packages configuration
update - Update (svn) an installed packages configuration
rebuild - Rebuild packages within installed configuration
list Create listings of various type
tag Creates a svn tag for a given package
branch - Creates a svn branch for a given package
setsvn - Sets karabo svn repository path
Run folder
Set of scripts for starting up all components: device servers, GUI, CLI, local broker
Packages configuration files – describing a logical set of packages to be installed

19. Karabo Packages Every device plugin is a separate netbeans project
In SVN repository it is placed in appropriate category
controlDevices, computeDevices, testDevices, etc
Can be checkout separately
self-contained Netbeans project
Can be checkout within Karabo Package Development
Helps in development and managing many plugins and their possible additional dependencies
Helps with setting-up self-contained running environment

20. JMS broker Currently using Oracle JMS openMQ 4.5.2 broker
Standalone application
Or within GlassFish server
Also available within full Netbeans installation including GlassFish server
For our development team we have centrally maintained broker instance running all the time
Communication through different topics – default is a username
For standalone development
Included in Karabo Framework binary installation package
Start/stop/query scripts in Karabo Package Development

21. Netbeans development environment
All karabo software ingredients comes as a separate netbeans project
C++ as well as Python
Note: there is no official Netbeans support for Python, user community plugin needs to be installed, description on Karabo Framework Wiki
Support for command line compilation
automated tasks, batch compilation
Currently we require Netbeans 7.3
Version comes with upgraded JMS broker v5.0, not compatible with v4.5.2, currently used with karabo software
This is important when running broker from Netbeans

22. Step by step installation
Live demo