A User’s Introduction to the Grand Challenge Software STAR-GC Workshop Oct 1999 D. Zimmerman.

Slides:



Advertisements
Similar presentations
Evaluation of Relational Operators CS634 Lecture 11, Mar Slides based on “Database Management Systems” 3 rd ed, Ramakrishnan and Gehrke.
Advertisements

How to access AUXDATA March If you have not received a Username from your DSO-IS, do not go any further. Send a request to your FSO-IS/SO-IS asking.
File System Interface CSCI 444/544 Operating Systems Fall 2008.
Grid Collector: Enabling File-Transparent Object Access For Analysis Wei-Ming Zhang Kent State University John Wu, Alex Sim, Junmin Gu and Arie Shoshani.
Reconstruction and Analysis on Demand: A Success Story Christopher D. Jones Cornell University, USA.
Java Server Faces Model/View/Controller Design Pattern for Web Development Slides adapted from “Core JavaServer Faces” by Geary and Horstmann and the J2EE.
Client – Server Architecture A Basic Introduction Kathleen R. Murray, Ph.D. May 2002.
Introduction to DBMS Purpose of Database Systems View of Data
Information storage: Introduction of database 10/7/2004 Xiangming Mu.
Configuration Management and Server Administration Mohan Bang Endeca Server.
Database System Concepts and Architecture Lecture # 2 21 June 2012 National University of Computer and Emerging Sciences.
Lecture 9 Methodology – Physical Database Design for Relational Databases.
JQUERY | INTRODUCTION. jQuery  Open source JavaScript library  Simplifies the interactions between  HTML document, or the Document Object Model (DOM),
CHAPTER 3 TOP LEVEL VIEW OF COMPUTER FUNCTION AND INTERCONNECTION
Central Reconstruction System on the RHIC Linux Farm in Brookhaven Laboratory HEPIX - BNL October 19, 2004 Tomasz Wlodek - BNL.
ESP workshop, Sept 2003 the Earth System Grid data portal presented by Luca Cinquini (NCAR/SCD/VETS) Acknowledgments: ESG.
Client – Server Architecture. Client Server Architecture A network architecture in which each computer or process on the network is either a client or.
Jerome Lauret RCF Advisory Committee Meeting The Data Carousel what problem it’s trying to solve the data carousel and the grand challenge the bits and.
Lecture 8 – Cookies & Sessions SFDV3011 – Advanced Web Development 1.
IT 456 Seminar 5 Dr Jeffrey A Robinson. Overview of Course Week 1 – Introduction Week 2 – Installation of SQL and management Tools Week 3 - Creating and.
Oracle 10g Database Administrator: Implementation and Administration Chapter 2 Tools and Architecture.
1 New developments in the HENP-GC HENP-GC Collaboration New Capabilities in the HENP Grand Challenge Storage Access System and its Application at RHIC.
Grand Challenge MDC1 Plans Doug Olson Nuclear Science Division, Berkeley Lab for the HENP-GC Collaboration RCF Meeting September 24, 1998.
Ingres Version 6.4 An Overview of the Architecture Presented by Quest Software.
Introduction to dCache Zhenping (Jane) Liu ATLAS Computing Facility, Physics Department Brookhaven National Lab 09/12 – 09/13, 2005 USATLAS Tier-1 & Tier-2.
NOVA Networked Object-based EnVironment for Analysis P. Nevski, A. Vaniachine, T. Wenaus NOVA is a project to develop distributed object oriented physics.
Using Bitmap Index to Speed up Analyses of High-Energy Physics Data John Wu, Arie Shoshani, Alex Sim, Junmin Gu, Art Poskanzer Lawrence Berkeley National.
An RTAG View of Event Collections, and Early Implementations David Malon ATLAS Database Group LHC Persistence Workshop 5 June 2002.
Grand Challenge and PHENIX Report post-MDC2 studies of GC software –feasibility for day-1 expectations of data model –simple robustness tests –Comparisons.
HPSS for Archival Storage Tom Sherwin Storage Group Leader, SDSC
1 GCA Application in STAR GCA Collaboration Grand Challenge Architecture and its Interface to STAR Sasha Vaniachine presenting for the Grand Challenge.
A university for the world real R © 2009, Chapter 9 The Runtime Environment Michael Adams.
Integrating QDEC with Slicer3 Click to add subtitle.
 Registry itself is easy and straightforward in implementation  The objects of registry are actually complicated to store and manage  Objects of Registry.
Scientific Data Management Research Group National Energy Research Scientific Computing Center, L B N L 1 Henrik Nordberg, June 1998 Query Estimator Henrik.
STAR C OMPUTING STAR Analysis Operations and Issues Torre Wenaus BNL STAR PWG Videoconference BNL August 13, 1999.
Introduction to Morpho RCN Workshop Samantha Romanello Long Term Ecological Research University of New Mexico.
1 Andrea Sciabà CERN Critical Services and Monitoring - CMS Andrea Sciabà WLCG Service Reliability Workshop 26 – 30 November, 2007.
STAR Collaboration, July 2004 Grid Collector Wei-Ming Zhang Kent State University John Wu, Alex Sim, Junmin Gu and Arie Shoshani Lawrence Berkeley National.
Grand Challenge in MDC2 D. Olson, LBNL 31 Jan 1999 STAR Collaboration Meeting
STAR C OMPUTING Plans for Production Use of Grand Challenge Software in STAR Torre Wenaus BNL Grand Challenge Meeting LBNL 10/23/98.
1 fileCatalog, tagDB and GCA A. Vaniachine Grand Challenge STAR fileCatalog, tagDB and Grand Challenge Architecture A. Vaniachine presenting for the Grand.
CCCognos Connection RSReport Studio ASAnalysis Studio QSQuery Studio ESEvent Studio CSContent Store FWM Framework.
1 Grid in STAR: Needs and Plans A. Vaniachine STAR Grid Components in STAR: Experience, Needs and Plans A. Vaniachine Lawrence Berkeley National Laboratory.
Super Computing 2000 DOE SCIENCE ON THE GRID Storage Resource Management For the Earth Science Grid Scientific Data Management Research Group NERSC, LBNL.
SPI NIGHTLIES Alex Hodgkins. SPI nightlies  Build and test various software projects each night  Provide a nightlies summary page that displays all.
ITM © Port,Kazman 1 ITM 352 Cookies. ITM © Port,Kazman 2 Problem… r How do you identify a particular user when they visit your site (or any.
Client – Server Architecture A Basic Introduction 1.
Collaborative Systems Developing Collaborative Systems with a Reuse Driven Process.
Em Spatiotemporal Database Laboratory Pusan National University File Processing : Database Management System Architecture 2004, Spring Pusan National University.
1 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Lawrence.
How Web Servers and The Internet Work The Basic Process.
Building Preservation Environments with Data Grid Technology Reagan W. Moore Presenter: Praveen Namburi.
1 Copyright © 2008, Oracle. All rights reserved. Repository Basics.
The HENP Grand Challenge Project and initial use in the RHIC Mock Data Challenge 1 D. Olson DM Workshop SLAC, Oct 1998.
Introduction to DBMS Purpose of Database Systems View of Data
Experience of PROOF cluster Installation and operation
Processes and threads.
Database System Concepts and Architecture
CSE-291 Cloud Computing, Fall 2016 Kesden
DNS.
Methodology – Physical Database Design for Relational Databases
Net 323 D: Networks Protocols
Object-Oriented Analysis
Spatial Online Sampling and Aggregation
國立臺北科技大學 課程:資料庫系統 fall Chapter 18
Database Systems Instructor Name: Lecture-3.
Introduction to DBMS Purpose of Database Systems View of Data
Database System Architectures
Presentation transcript:

A User’s Introduction to the Grand Challenge Software STAR-GC Workshop Oct 1999 D. Zimmerman

What does the Grand Challenge do for you? l Optimizes access to HPSS tape store, That is: The GC Architecture allows users to stage files out of HPSS onto local disk (the GC disk cache) without worrying about Username and Password for HPSS Access Tape Numbers File Names Improves data access for individual users Allows event access by query: Present query string to GCA. (e.g. NumberLambdas>1) receive iterator over events which satisfy query as files are extracted from HPSS. Pre-fetches files so that “the next” file is requested from HPSS while you are analysing the data in your first file Coordinates data access among multiple users Coordinates ftp requests so that a tape is staged only once per set of queries which request files on that tape

Use Case #1 Pt. 1 Requesting events via query Query NumberOfLambdas>1 and TriggerNumber=XXXXX Steps 1) Determine if the tags of interest are contained in the GCIndex If yes: Launch root Load GCALibrary gSystem->Load(“GCALib”); Instantiate QueryBuilder Object Qbuilder qb; Submit query to GCA Token MyToken = qb->EstimatePredicate(“Nlambdas>1&&TriggerType=XXXXX”, “Trk;Hit”); If the Query Estimate is acceptable to you: Execute Query qb->Execute(MyToken);

Use Case #1 Pt. 2 If your selection variables are not part of the Grand Challenge Index: 1) Analyze disk resident (DST) data to create a list of events which satisfy your query. Use this data to perform a query by collection with the GCA. 2) Write the OIDs (RunNumber, EventNumber) for the events of interest to a file: MyList.txt Launch root Load GCALibrary gSystem->Load(“GCALib”); Instantiate QueryBuilder Object Qbuilder qb; Submit query to GCA Token MyToken = qb->EstimateCollection(“MyList.txt ”, “Trk;Hit”); If the Query Estimate is acceptable to you: Execute Query qb->Execute(MyToken);

Use Case #2 After analyzing data from other queries user (or users such as a Physics Working Group) wishes to use grand challenge for queryable access to the event store. Steps 1) Write Information to be used for tagging to a root tree 2) Use IndexFeeder IndexBuilder connection to build personalized GCA Index. 3) Coordinate with STAR Software leader to get an ok to create an additional instance of GCA Create an instance of the GCA which will use your index (via personalized gc.config file) 4) In order to add information from the individual index to the STAR index, the tag information must be calculated for all events and a new index written using both the new and the old tag information.