1. Installation - Plus Loretta Auvil
National Center for Supercomputing Applications University of Illinois at Urbana-Champaign

2. Outline Installation Meandre servers and clusters
Development Tools: Eclipse Plugin
Hands-On

3. Considerations Do you want to use SEASR-powered services?
May not need to install anything (besides a browser)
Do you want to run analytics on your laptop?
Quick 3 step process
Do you want to provide SEASR-powered services?
Start simple
Scale as needed
Deploying all the extra goodies

4. Using SEASR-Powered Services
SEASR provides some demo services
Requires a browser
You can access them from
Community Hub to execute a flow
Zotero to analyze your collections with existing flows
Meandre Server Client to execute a flow; or tune properties and execute a flow
Hosted at
Meandre Workbench to execute a flow; or tune properties and execute a flow; or create a flow
Hosted at

5. I Need To Run SEASR on my laptop
I want to run on my laptop (server)
I have copyrighted information
I have collection for analysis that is too big to be moved
I just want to test it and have fun with it
Getting a Meandre server up and running in 3 steps
Install Java
Download the Meandre into a new directory
Use the “Start-Infrastructure” or type “java –jar meandre-server jar”
Access your new installation at

6. Specialized Downloadable Bundles
On the SEASR/Meandre download site
Installation bundles available for:
Mac OS
Linux
Windows
Bundles contain:
Zip file that includes executable files
Set of demo components and flows
Requires Java (1.5 or greater) to be installed

7. Bundles Include The bundle comes with Provides simple scripts to
Meandre Server
ZigZag console/compiler/runtime
Meandre Workbench (also provided as a war file)
Provides simple scripts to
Start/stop the Meandre server
Start/stop the Meandre Workbench

8. What About Setting Up My Own Server?
You can also deploy the bundles on a server using the same approach.

9. Customization of My Server
This will support
Moderated traffic ??we don’t know what this means??
Persistent web services can be provided using this server
Application Server
Workbench can be deployed alongside the Meandre Server in the embedded Jetty Application Server
Workbench can be deployed using your favorite application server using the .war file
Database Options
Meandre uses an embedded Derby as the database
Meandre can be also be setup to use MySQL for the database

10. Backend Using Derby Database
default meandre-config-store.xml
<entry key="DB_USER"></entry> <entry key="DB_DRIVER_CLASS"> org.apache.derby.jdbc.EmbeddedDriver</entry> <entry key="DB">Derby</entry> <entry key="DB_PASSWD"></entry> <entry key="DB_URL”> jdbc:derby:./MeandreStore;create=true;logDevice=./DerbyLog </entry>

11. Backend Using MySQL meandre-config-store.xml to
<entry key="DB_USER">USERNAME</entry> <entry key="DB_DRIVER_CLASS”>com.mysql.jdbc.Driver</entry> <entry key="DB">MySQL</entry> <entry key="DB_PASSWD">PASSWORD</entry> <entry key="DB_URL”> <![CDATA[jdbc:mysql://your- server.com/YOURDB?useUnicode=yes&characterEncoding=utf8&aut oReconnect=true]]> </entry>
Changing from Derby to MySQL
Stop the server
Change the meandre-config-store.xml file
Restart the server
Now your server is backend on MySQL

12. Scaling Up Two possible routes
Deploy a farm of self-contained services (via zigzag)
Use the Meandre Cluster solution
Both require your sysadmin/netadmin to provide a highly available load balancer (some virtual appliances available)
To create a cluster
Use the previous MySQL set up
Point all the servers to the same database
The server interface pages will allow you to monitor of the servers

13. Installing The Workbench
Use the installation bundles
Use the war file
Install your favorite application server
Deploy the war file against the application server

14. Installing the Community Hub
Deploy the Wordpress plugin
Unzip file into Wordpress plugins directory

15. Community Hub Customization (1)
Makes flows available for exploration and execution
Renders the description of the flow information
Provides a simple execute button to allow visitors to run the flow
Exploration via wordpress shortcode to expose the Keyword Cloud functionality
[MeandreTagCloud store='
[MeandreListSelectedTags]
[MeandreListFlowsByTags store='

16. Community Hub Customization (2)
Pages and posts
Add the wordpress shortcode to display the meta information and execute buttons
[MeandreDescribeFlow]
[MeandreListFlowsByFlowTags]
Add custom fields for specifying meta information
FlowURI: specifies the uri of the flow
ImageURI: specifies a uri to associate with this flow
StoreURI: specifies the uri of the flow rdf
CustomFlowURI: specifies the uri of the flow that allows user to input their data
ExecuteURI: specifies the uri of the server where flow should run

17. Eclipse Plugin for Developers
On the SEASR/Meandre download site
Steps for installation
Exit Eclipse
Download zip file into Eclipse/dropins directory
Unzip file
Restart Eclipse

18. Developers: Eclipse Plugin
Uploads components to the Meandre Server
Lists components installed
Allows for removal of components
Shows additional data of interest to a programmer

19. Meandre: The Architecture
The design of the Meandre architecture follows three directives:
provide a robust and transparent scalable solution from a laptop to large-scale clusters
create an unified solution for batch and interactive tasks
encourage reusing and sharing components
To ensure such goals, the designed architecture relies on four stacked layers and builds on top of service-oriented architectures (SOA)

20. Meandre: Basic Single Server

21. Meandre MDX: Cloud Computing
Servers can be
instantiated on demand
disposed when done or on demand
A cluster is formed by at least one server
The Meandre Distributed Exchange (MDX)
Orchestrates operational integrity by managing cluster configuration and membership using a shared database resource.

22. Meandre MDX: The Picture
MDX Backbone

23. Meandre MDX: The Architecture
Virtualization infrastructure
Provide a uniform access to the underlying execution environment. It relies on virtualization of machines and the usage of Java for hardware abstraction.
IO standardization
A unified layer provides access to shared data stores, distributed file-system, specialized metadata stores, and access to other service-oriented architecture gateways.

24. Meandre MDX: The Architecture
Data-intensive flow infrastructure
Provide the basic Meandre execution engine for data-intensive flows, component repositories and discovery mechanisms, extensible plugins and web user interfaces (webUIs).
Interaction layer
Can provide self-contained applications via webUIs, create plugins for third-party services, interact with the embedding application that relies on the Meandre engine, or provide services to the cloud.

25. Meandre: ZigZag Script Language
ZigZag is a simple language for describing data- intensive flows
Modeled on Python for simplicity.
ZigZag is declarative language for expressing the directed graphs that describe flows.
Command-line tools allow ZigZag files to compile and execute.
A compiler is provided to transform a ZigZag program (.zz) into Meandre archive unit (.mau).
Mau(s) can then be executed by a Meandre engine.

26. Meandre: ZigZag Script Language
As an example the Flow Diagram
The flow below pushes two strings that get concatenated and printed to the console

27. Meandre: ZigZag Script Language
ZigZag code that represents example flow: #
# Imports the three required components and creates the component aliases
import <
alias < as PUSH
alias < as CONCAT
alias < as PRINT
# Creates four instances for the flow
push_hello, push_world, concat, print = PUSH(), PUSH(), CONCAT(), PRINT()
# Sets up the properties of the instances
push_hello.message, push_world.message = "Hello ", "world!"
# Describes the data-intensive flow
= push_hello(), push_world()
@cres = concat( string_one: phres.string; string_two: pwres.string )
print( object: cres.concatenated_string )

28. Meandre: ZigZag Script Language
Automatic Parallelization
Multiple instances of a component could be run in parallel to boost throughput.
Specialized operator available in ZigZag Scripting to cause multiple instances of a given component to used
Consider a simple flow example show in the diagram
The dataflow declaration would look like #
# Describes the data-intensive flow
@pu = push()
@pt = pass( string:pu.string )
print( object:pt.string )

29. Meandre: ZigZag Script Language
Automatic Parallelization
Adding the operator [+AUTO] to middle component
[+AUTO] tells the ZigZag compiler to parallelize the “pass component instance” by the number of cores available on system.
[+AUTO] may also be written [+N] where N is an numeric value to use for example [+10].
# Describes the data-intensive flow #
@pu = push()
@pt = pass( string:pu.string ) [+AUTO]
print( object:pt.string )

30. Meandre: ZigZag Script Language
Automatic Parallelization
Adding the operator [+4] would result in a directed grap
# Describes the data-intensive flow #
@pu = push()
@pt = pass( string:pu.string ) [+4]
print( object:pt.string )
# Describes the data-intensive flow #
@pu = push()
@pt = pass( string:pu.string ) [+4!]
print( object:pt.string )

31. Scaling Genetic Algorithms with Meandre
Intel 2.8Ghz QuadCore, 4Gb RAM. Average of 20 runs.

32. And Beyond with Hadoop
60 Dual Quad Core Xeons with 8GB RAM. GB Ethernet
Resources exhaustion

33. Meandre: Flows to MAU
Flows can be executed using their RDF descriptors
Flows can be compiled into MAU
MAU is:
Self-contained representation
Ready for execution
Portable
The base of flow execution in grid environments

34. Compile and Run MAU Compile zigzag to mau creating my_file.mau
java -jar ~/meandre/zzc jar my_file.zz
Run the mau file
java -jar ~/meandre/zzre jar -port 1816 my_file.mau

35. Demonstration Installation of Meandre Meandre Eclipse Plugin
JIRA, Confluence, Bamboo - what they are and what we use them for
Usage of ZigZag
Compiling and executing flows using ZigZag
Usage of ZigZag for Zotero-enabled flows
Usage of ZigZag for Fedora flows

36. Learning Exercises Open an existing ZigZag flow
Convert your flow from yesterday to ZigZag
Compile the script
Execute the script
Have participants download and install SEASR on their personal computers
Have participants sign up for accounts to access the SEASR suite of Atlassian tools
Use JIRA to log a support request

37. Discussion Questions
What challenges (if any) would scholars have installing the SEASR software?
Do you see your institution's IT department running the SEASR environment or would it be your research group?
Which environment would you most likely use, the Meandre Workbench or the ZigZag scripting language?