Science Studio and ANISE (Jan 2010 update) Elder Matias Canadian Light Source

Overview Beamline Control Traditional Technology Beamline Control – ScienceStudio Nan-Fabrication – ScienceStudio User Office Grid Computing User Configurable Light Paths

Where is Saskatoon?

Scientific American May 2008 Science 2.0 – The Risk and Reward of Web-Based Research “Our real mission isn’t to publish journals but to facilitate scientific communication” Timo Hannay – Head of Web Publishing at Nature Publishing Group

What Is the Web 2.0? In plain English …. –Automating tedious tasks using web technology –Tools to help people and software collaborate

Project Team Members Dionisio Medrano Dylan Maxwell Dong Liu Elder Matias Chris Armstrong Khalid Subratee Jinhui Qin Yuzhen Xie Mike Bauer Stewart McIntyre Marina Suominen Fuller Ludeng (Eric) Zhao Yuhong Yan

Goals of the Two Projects Science Studio –User Office Automation –Integrated Experiment Management –Remote Access to VESPERS (XRD and XRF) –Access Scanning Electron Microscope (UWO) –Grid Computing (Cell Processor) ANISE –Integrated Stream Computing to providing near real-time data processing

Beamline Control Traditional Technology

Detectors Cameras etc. Modbus TCP/IP GPIB RS-232 Channel Access Protocol IOC State Machine Engine CA Single Board Computer CA Operator Workstation User Applications Touch Panels CA Telemecanique Momentum PLC VME IOC CA

Beamline Controls Interfaces EPICS Tools Configuration Tool User GUI & ROOT

Beamline Visualization As Map - Raster scan with fluorescence spectroscopy of a pine needle contaminated with arsenic. The technique allows multiple elements to be detected simultaneously at each point of the raster. These images show distributions of arsenic, iron, and manganese, respectively, near the tip. The highest intensity displays in red, lowest in blue. - Custom on-line plotting application scripted in CERN Root, with data stream from the CLS data acquisition application.(G. Wright, R. Igarashi, K. Chang-Yong, N. Chen) Fe Map

Traditional Beamline Visualization Cross section views of the beam spot intensity distribution for varying degrees of monochromator detuning (50-80%). (Only one image shown). (G. Wright, R. Igarashi, K. Chang-Yong, N. Chen)

Beamline Control ScienceStudio Technology

New User Office Functionality –Proposal submission –Peer review –User Feedback Tracking –Experiment Management –User Training/ Safety Testing Remote Beamline Access Integration with grid data-storage Grid computing

Science Studio and ANISE

System Architecture Web Application Beamline Control Module DB SAN JMSCA VESPERS HTTP 1.VESPERS Beamline 2.EPICS control system 3.Beamline Control Module (BCM) 4.Web Application 5.Database 6.File Storage 7.Web Interface

VESPERS Beamline VESPERS — Very Sensitive Elemental and Structural Probe Employing Radiation from a Synchrotron A bending magnet beamline on sector 6 at the Canadian Light Source synchrotron in Saskatoon, Saskatchewan. A hard x-ray microprobe with an energy range of 6 to 30keV. Techniques: X-Ray Fluorescence (XRF) & X-Ray Diffraction (XRD) Web Application Beamline Control Module DB SAN JMSCA VESPERS HTTP

EPICS Low-level Control System EPICS — Experimental Physics and Industrial Control System The standard control system at the CLS. EPICS consists of a network of Input-Output Controls (IOCs) which are connected to directly to devices. An IOC provides many Process Variables (PVs) which relate to either an input or output from a device and have a unique name. Channel Access (CA) is used to read or write to any PV without knowing which IOC provides the PV. More than 50,000 PVs in the CLS control system. Web Application Beamline Control Module DB SAN JMSCA VESPERS HTTP

Beamline Control Module (BCM) The BCM provides a high-level interface to the low-level control system (EPICS). Logical and physical separation of business logic and control logic. Virtual device abstraction that provides independence from low-level control system. Virtual devices can be logically organized into a device hierarchy. Basic devices can be combined to build more functional devices. Communication with external applications using two message queues (ActiveMQ). Web Application Beamline Control Module DB SAN JMSCA VESPERS HTTP

Web Application A J2EE Servlet application that provides a web-based interface Science Studio. Tools: Spring (MVC), iBATIS (ORM), JSecurity (Apache Ki), Apache Tomcat Divided into two parts: the Core application and the VESPERS beamline application. Core application is responsible for providing access to the business objects. VESPERS application is responsible for remote control of the VESPERS beamline. Web Application Beamline Control Module DB SAN JMSCA VESPERS HTTP

Metadata associated with the operation of a remote controlled beamline and the organization of experimental data collected on that beamline. A project is the top level organizational unit and is associated with a project team. A session defines a period of time allocated to a project team to conduct experiments. An experiment relates a sample and the technique being applied to that sample. A scan records the location of the acquired experimental data. Web Application Beamline Control Module DB SAN JMSCA VESPERS HTTP

Experimental Data Storage Experimental data is stored at the CLS. Common directory structure shared with other beamlines. A large data storage facility is now operational at the University of Saskatchewan as part of WestGrid. Web Application Beamline Control Module DB SAN JMSCA VESPERS HTTP

VESPERS Web Interface Rich web interface to Science Studio and the VESPERS beamline. Designed to be used over commodity broadband internet. Developed for the Firefox web browser without any additional plugins or extensions. Known to work with other browsers, but requires the Canvas HTML tag. AJAX is used for the VESPERS interface to provide device values in pseudo real time. ExtJS, a JavaScript framework, provides many advanced GUI elements. Web Application Beamline Control Module DB SAN JMSCA VESPERS HTTP

Beamline Setup

Experiment Setup

XRF (X-Ray Fluorescence)

Beamline Hutch Cameras

Experimental Data Viewer

ScienceStudio

Scanning

XRD Screen

(a) X ray Fluorescence spectra and mapping (b) Laue X ray Diffraction and mapping OrientationStrain

VM strain XRF ImageXRD: orientationXRD:strain Optical image

University of Western Ontario Nan-Fabrication Facility Using ScienceStudio

Nanofabrication Facility The Western Nanofabrication Facility is an open user facility at the University of Western Ontario for the fabrication of micro and nano structures. LEO 1540XB Scanning Electron Microscope (SEM) with an integrated Oxford x-ray system. Oxford x-ray system is controlled by INCA. INCA provides limited control of the SEM, and spectral analysis and quantification.

Nanofab Integration Design

Nanofab Implementation

Initial Test: INCA Remotely

User Office

User Office Modules User Profile/Authentication –User ProfileDeployed –User RolesDeployed –Authentication (LDAP/AD)Deployed Information Model –BeamlinesDone –End-stationsDone –TechniquesDone Notification ModuleDeployed

User Office Modules Proposal SubmissionIn Progress User Feedback SurveyPrototype Scheduling ModulePrototype Workflow EngineEvaluation Sample ManagementRequirements Publication TrackingRequirements

User Office - Training Module First User Office Module Deployed User Profile 1,400 + user records/accounts created Training Module –53 Training Modules Setup and past user records manually transferred

Calendar view in JQuery

User Office Workflow Example Prototype Implementation 1. CLS issues a call for proposals and gives deadline 2. Beamline users submit proposals 3. User Office administrator ends registration or extends deadline 4. User Office administrator assigns proposals to user office reviewers 5. Reviewers look at proposals and rank them 6. User Office looks at ranking and chooses the proposals to accept 7. Accepted proposals contact persons are notified 8. Beamline User completes training (web service) 9. After training is completed (simulated by a delay) the CLS is notified

User Login

Admin – Add Proposal

Admin - Setup for Cycle

Admin – Creating Roles

Grid Computing

HPC Computing in Canada Compute Canada –Responsible for High Performance Computing in Canada –Each regional grid is a member of Compute Canada ACEnet – Atlantic Canada CLUMEQ - Quebec SCINET - UofT HPCVL – Queens, Royal Military Collage St. Lawrence, Carlson, Ottawa, … RQCHP - Quebec SHARCNET - Ontario WESTGRID – Western Canada CLUMEQ Facility

Grid Data Storage? UofS is the host for the new WestGrid data storage facility Cost: $3.2 M Includes on-line and archival storage Two sites on campus Photo: tape backup unit holding 6,000 tape

IBM Cell Processor (3.2 GHz)

User Configurable Light Paths

Lightpath Accelerator controls a software virtual cross-connect that commands UCLP. Integrating LPA with User Configurable Light Paths In effect, CA*Net4 is treated as a single lightpath cross-connect real device virtual device

File Transfer Demo