National University Community Research Institute (NUCRI) NU Community Research Institute (NUCRI) HASTAC (higher education)/HASS grid National School Board.

Slides:



Advertisements
Similar presentations
National University Community Research Institute (NUCRI) PRAGMA 18 Cyberinfrastructure for CHOIS a Global Health initiative for obesity surveillance and.
Advertisements

San Diego Supercomputer Center & National Partnership for Advance Computational Infrastructure Storage Resource Broker Reagan W. Moore San Diego Supercomputer.
National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center Data Grids for Collection Federation Reagan W. Moore University.
The Storage Resource Broker and.
The Storage Resource Broker and.
Overview of the SDSC Storage Resource Broker Wayne Schroeder (and other SRB team members) May, 2004 San Diego Supercomputer Center, University of California.
Peter Berrisford RAL – Data Management Group SRB Services.
Data Grid: Storage Resource Broker Mike Smorul. SRB Overview Developed at San Diego Supercomputing Center. Provides the abstraction mechanisms needed.
National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center Data Grids Reagan W. Moore San Diego Supercomputer Center.
NATIONAL PARTNERSHIP FOR ADVANCED COMPUTATIONAL INFRASTRUCTURE SAN DIEGO SUPERCOMPUTER CENTER Particle Physics Data Grid PPDG Data Handling System Reagan.
San Diego Supercomputer CenterNational Partnership for Advanced Computational Infrastructure1 Grid Based Solutions for Distributed Data Management Reagan.
Security Requirements for Shared Collections Storage Resource Broker Reagan W. Moore
MTA SZTAKI Hungarian Academy of Sciences Grid Computing Course Porto, January Introduction to Grid portals Gergely Sipos
Complex Systems Engineering for the Global information Grid Bob Marcus
VL-e PoC Introduction Maurice Bouwhuis VL-e work shop, April 7 th, 2006.
Applying Data Grids to Support Distributed Data Management Storage Resource Broker Reagan W. Moore Ian Fisk Bing Zhu University of California, San Diego.
Office of Science U.S. Department of Energy Grids and Portals at NERSC Presented by Steve Chan.
On Developing Data Grid Workflows using Storage Resource Broker (SRB) and Kepler Tim H. Wong - UC Davis Efrat Frank - SDSC Bertram Ludäscher - UC Davis.
Robust Tools for Archiving and Preserving Digital Data Joseph JaJa, Mike Smorul, and Mike McGann Institute for Advanced Computer Studies Department of.
Mike Smorul Saurabh Channan Digital Preservation and Archiving at the Institute for Advanced Computer Studies University of Maryland, College Park.
On Developing Data Grid Workflows using Storage Resource Broker (SRB) and Kepler Tim H. Wong - UC Davis Efrat Frank - SDSC Dr. Bertram Ludäscher - UC Davis.
Web-based Portal for Discovery, Retrieval and Visualization of Earth Science Datasets in Grid Environment Zhenping (Jane) Liu.
Slide 1 of 9 Presenting 24x7 Scheduler The art of computer automation Press PageDown key or click to advance.
Tools for e-Research Mat Wyatt. 2 e-Research Sensor nets data compute… Models/ software/ workflows colleagues instruments.
Scientific Data Infrastructure in CAS Dr. Jianhui Scientific Data Center Computer Network Information Center Chinese Academy of Sciences.
- 1 - Grid Programming Environment (GPE) Ralf Ratering Intel Parallel and Distributed Solutions Division (PDSD)
By: Roman Olschanowsky An Introduction to the.
Core SRB Technology for 2005 NCOIC Workshop By Michael Wan And Wayne Schroeder SDSC SDSC/UCSD/NPACI.
1 School of Computer, National University of Defense Technology A Profile on the Grid Data Engine (GridDaEn) Xiao Nong
EGEE-II INFSO-RI Enabling Grids for E-sciencE Data Grid Services/SRB/SRM & Practical Hai-Ning Wu Academia Sinica Grid Computing.
ICTP, April 2007 CIMA in Australia Ian Atkinson HPRC Manager, ITR School of Maths, Physics and IT James Cook University.
EzHPC ERDC, ITL ezHPC Workshop April 23, Problem HPC tools are primarily command line Hard to remember syntax to accomplish things Not everyone.
CYBERINFRASTRUCTURE FOR THE GEOSCIENCES Data Replication Service Sandeep Chandra GEON Systems Group San Diego Supercomputer Center.
Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007 JAINIS (JCU and Indiana Instrument Services): A Grid.
GEM Portal and SERVOGrid for Earthquake Science PTLIU Laboratory for Community Grids Geoffrey Fox, Marlon Pierce Computer Science, Informatics, Physics.
San Diego Supercomputer Center National Partnership for Advanced Computational Infrastructure SRB + Web Services = Datagrid Management System (DGMS) Arcot.
Grid Architecture William E. Johnston Lawrence Berkeley National Lab and NASA Ames Research Center (These slides are available at grid.lbl.gov/~wej/Grids)
National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center Persistent Management of Distributed Data Reagan W. Moore.
National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center Persistent Archive for the NSDL Reagan W. Moore Charlie Cowart.
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
IODE Ocean Data Portal - ODP  The objective of the IODE Ocean Data Portal (ODP) is to facilitate and promote the exchange and dissemination of marine.
MTA SZTAKI Hungarian Academy of Sciences Introduction to Grid portals Gergely Sipos
GRID Overview Internet2 Member Meeting Spring 2003 Sandra Redman Information Technology and Systems Center and Information Technology Research Center National.
PoC Induction 19-April VBrowser (VL-e Toolkit) The single point of access to the grid  Medical use case: functional MRI (fMRI)  VBrowser design  VBrowser.
The Global Land Cover Facility is sponsored by NASA and the University of Maryland.The GLCF is a founding member of the Federation of Earth Science Information.
SAN DIEGO SUPERCOMPUTER CENTER By: Roman Olschanowsky An Introduction to the.
Michael Doherty RAL UK e-Science AHM 2-4 September 2003 SRB in Action.
Center for Computational Visualization University of Texas, Austin Visualization and Graphics Research Group University of California, Davis Molecular.
1 e-Science AHM st Aug – 3 rd Sept 2004 Nottingham Distributed Storage management using SRB on UK National Grid Service Manandhar A, Haines K,
Introduction to The Storage Resource.
NeuroLOG ANR-06-TLOG-024 Software technologies for integration of process and data in medical imaging A transitional.
Development of e-Science Application Portal on GAP WeiLong Ueng Academia Sinica Grid Computing
Biomedical Informatics Research Network The Storage Resource Broker & Integration with NMI Middleware Arcot Rajasekar, BIRN-CC SDSC October 9th 2002 BIRN.
SDSC Storage Resource Broker & Meta-data Catalog SRB Archives HPSS, ADSM, UniTree, DMF Databases DB2, Oracle, Sybase File Systems Unix, NT, Mac OSX Application.
AHM04: Sep 2004 Nottingham CCLRC e-Science Centre eMinerals: Environment from the Molecular Level Managing simulation data Lisa Blanshard e- Science Data.
Partnerships in Innovation: Serving a Networked Nation Grid Technologies: Foundations for Preservation Environments Portals for managing user interactions.
Rights Management for Shared Collections Storage Resource Broker Reagan W. Moore
The Storage Resource Broker and.
Collection-Based Persistent Archives Arcot Rajasekar, Richard Marciano, Reagan Moore San Diego Supercomputer Center Presented by: Preetham A Gowda.
Campus grids: e-Infrastructure within a University Mike Mineter National e-Science Centre 22 February 2006.
Preservation Data Services Persistent Archive Research Group Reagan W. Moore October 1, 2003.
1 eScience Grid Environments th May 2004 NESC - Edinburgh Deployment of Storage Resource Broker at CCLRC for E-science Projects Ananta Manandhar.
Building Preservation Environments from Federated Data Grids Reagan W. Moore San Diego Supercomputer Center Storage.
Problem: Ecological data needed to address critical questions are dispersed, heterogeneous, and complex Solution: An internet-based mechanism to discover,
Arcot Rajasekar Michael Wan Reagan Moore (sekar, mwan,
OGCE Portal Applications for Grid Computing
VORB Virtual Object Ring Buffers
Presentation transcript:

National University Community Research Institute (NUCRI) NU Community Research Institute (NUCRI) HASTAC (higher education)/HASS grid National School Board (NCC/FRC) UC Humanities Research Institute UCOP Envision (San Diego) Creative Community (KPBS)/SDSU NCSA/University of Urbana- Champaign K12 (AmeriCorps) ARTCA, PRAGMA ? HASTAC (higher education)/HASS grid National School Board (NCC/FRC) UC Humanities Research Institute UCOP Envision (San Diego) Creative Community (KPBS)/SDSU NCSA/University of Urbana- Champaign K12 (AmeriCorps) ARTCA, PRAGMA ? focused on digital technology need based Research for the community NUCRI is an outreach to the Community for National University (

National University Community Research Institute (NUCRI) NU Community Research Institute (NUCRI) ( i-Datawarei-Dataware – web based integrated database system for providing social services ezSRB ezSRB – web based interface for managing data through Storage Resource Broker (SRB) middleware; used in HASSgrid deployment VCMSSVCMSS – virtual collaboratory tool for working on collaborative projects FastAppsFastApps – web based software system for grant management CyberEdWorldCyberEdWorld – immersive 3D space for virtual classroom Open source software technologies (Portal Technology, Content management Software/Drupal, audio/video communication software/Access Grid, PHP, MySQL…) Tools and Technologies

National University Community Research Institute (NUCRI) CHOMS : A web based database system for obesity management for children enter PDX online PDX online – for professional development through Online or Hybrid courses

National University Community Research Institute (NUCRI) HASTAC Community NSF funded InFormation Year /InCommunityInCommunity

National University Community Research Institute (NUCRI) SOFTWARE THAT POWERS THE FRC i- DATAWARE TM SOFTWARE THAT POWERS THE Family Resource Centers

National University Community Research Institute (NUCRI) demo

National University Community Research Institute (NUCRI) LA Jewish History Portal and Digital LibraryPortal

National University Community Research Institute (NUCRI) FastApps for Coastal (

National University Community Research Institute (NUCRI) Solution? - grid technology Data grid (SRB) Grid computing (Globus, Condor) Communication grid (Access grid) Virtual Collaborative Research Environment Connecting heterogeneous digital devices /cell phones/ PDA/ sensors, etc. to data repositories, telecommunications and tele-research tools with the purpose of supporting collaborative research

National University Community Research Institute (NUCRI) Objectives Research and Collaboration Shared Global Resource Creating Virtual Space for others to join in Data manipulation, analyzing Enhancing Human-to-Human interaction

National University Community Research Institute (NUCRI) Requirements: Universal access Data Management Job submission Security Secured Login Technology transfer Gateway to other links, databases and applications

National University Community Research Institute (NUCRI) Data Grid DATA Grid is Grid Computing systems that deals with controlled sharing and management of huge amounts of distributed data sets. The data grid creates virtual but collaborative environments that support distributed scientific and engineering research. Current large scale projects are; Biomedical Informatics and Research Network (BIRN) Southern California Earthquake Center (SCEC) Data Grid needs a middleware called Storage Resource Broker (SRB), which acts as the underlying data grid technology.

National University Community Research Institute (NUCRI) Features SRB Data Virtualization – Uniform Data Name Space & Virtual Collection-concept Resource Transparency – Local or Remote, Resource Type & Access Method Location/Format Transparency – Path Names, Schemas, Containers User Transparency – Uniform User Name Space – Tickets & Groups Method Transparency Other Features – Rich Access Control – Cross-Domain Authentication – Segmented & Replicated Data Mgmt – Data Discovery – … Virtual Data Grid (SRB) MCAT UC-Berkley UCLA UCHRI NUCRI SDSC

National University Community Research Institute (NUCRI) Solution SRB SDSC Storage Resource Broker & Meta-data Catalog SRB Archives HPSS, ADSM, UniTree, DMF Databases DB2, Oracle, Sybase File Systems Unix, NT, Mac OSX Application C, C++, Linux I/O Unix Shell Dublin Core Resource, Mthd, User User Defined Application Meta-data Remote Proxies DataCutter Metadata Extraction Java, NT Browsers Web Prolog Predicate MCAT

National University Community Research Institute (NUCRI) Visualization of SRB Mainframes Supercomputers Any platform Distributed Server SQL server/Oracle/MySQL

National University Community Research Institute (NUCRI) SRBSRB Clients Scommands Unix file system like interface Version available for Unix, Dos and Mac Inquisitor (inQ) Windows Explorer style interface Version available for Windows Client access API C API JARGON – Java API to SRB MATRIX – SRB Workflow system MySRB Web Browser interface

National University Community Research Institute (NUCRI) MySRB Web-bases Access to the SRB Secure HTTP Uses Cookies for Session Control Self Registration of Users Supported – Currently limited to SDSC users Self Registration of Resources (soon) Access to Both Data and Metadata

National University Community Research Institute (NUCRI) Types of Meta data System-level Metadata – Size, resource, owner, date, access control, … User-defined Meta data – for data & collections – triples – No limits in number of metadata – Support for Collection-level schemas Comments, default values, drop-down lists – Support for Standardized Schemas (eg. Dublin Core) Annotations – Supports textual annotations – Annotator, date, context also registered

National University Community Research Institute (NUCRI) enter

National University Community Research Institute (NUCRI) MySRB disadvantages.. Non User-friendly Too Many complexity in accessing data Too many features High definition language used

National University Community Research Institute (NUCRI) About Scommands Command line interface >> Scripting Available for Unix and Dos platform Most flexible and powerful for client SRB Fastest and reliable Great for scripts, batch jobs etc. Example: Sinit= establishes a session Senv= displays connection information Smkdir= makes a new directory Sget= copy file from remote SRB Master to local Sput= upload file from local to SRB Master server Sexit= ends session

National University Community Research Institute (NUCRI) PHP Wrapper PHP Scommand Wrapper function sinit() { $retval=array(); putenv("mdasEnvFile=".$GLOBALS['MDASENV_PATH']." /.MdasEnv"); putenv("mdasAuthFile=".$GLOBALS['MDASENV_PATH']. "/.MdasAuth"); $scmd_result=shell_exec($GLOBALS['SCMD_PATH'].'/ Sinit -v'); if (stristr($scmd_result, "Connection to srbMaster failed.")) { $retval['status']=false; } else { $retval['status']=true; } $retval['data']=$scmd_result; return $retval; } function sls($param) { $retval=array(); $scmd_result=shell_exec($GLOBALS['SCMD_P ATH']."/Sls -l $param"); if (stristr($scmd_result, "No such dataset or collection")) { $retval['status']=false; } else { $retval['status']=true; } $retval['data']=$scmd_result; return $retval; }

National University Community Research Institute (NUCRI) ezSRBezSRB – web based interface for managing data through Storage Resource Broker (SRB) middleware; used in HASSgrid deployment enter