Algorithmic Finance and Tools for Grid Execution (the Swift Grid Scripting/Workflow tool) Tiberiu (Tibi) Stef-Praun.

Slides:



Advertisements
Similar presentations
Introduction to Grid Application On-Boarding Nick Werstiuk
Advertisements

Copyright © 2008 SAS Institute Inc. All rights reserved. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks.
Parallel Scripting on Beagle with Swift Ketan Maheshwari Postdoctoral Appointee (Argonne National.
A Dynamic World, what can Grids do for Multi-Core computing? Daniel Goodman, Anne Trefethen and Douglas Creager
Investigating Approaches to Speeding Up Systems Biology Using BOINC-Based Desktop Grids Simon J E Taylor (1) Mohammadmersad Ghorbani (1) David Gilbert.
HST 952 Computing for Biomedical Scientists Lecture 10.
GridRPC Sources / Credits: IRISA/IFSIC IRISA/INRIA Thierry Priol et. al papers.
Distributed, parallel web service orchestration using XSLT Peter Kelly Paul Coddington Andrew Wendelborn.
GridFlow: Workflow Management for Grid Computing Kavita Shinde.
The new The new MONARC Simulation Framework Iosif Legrand  California Institute of Technology.
Software Development Unit 6.
Making Database Applications Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama MIT Owen Arden Andrew C. Myers Cornell.
GRID COMPUTING: REPLICATION CONCEPTS Presented By: Payal Patel.
TeraGrid Gateway User Concept – Supporting Users V. E. Lynch, M. L. Chen, J. W. Cobb, J. A. Kohl, S. D. Miller, S. S. Vazhkudai Oak Ridge National Laboratory.
Applying Twister to Scientific Applications CloudCom 2010 Indianapolis, Indiana, USA Nov 30 – Dec 3, 2010.
Apache Airavata GSOC Knowledge and Expertise Computational Resources Scientific Instruments Algorithms and Models Archived Data and Metadata Advanced.
Fabien Viale 1 Matlab & Scilab Applications to Finance Fabien Viale, Denis Caromel, et al. OASIS Team INRIA -- CNRS - I3S.
Database Systems: Design, Implementation, and Management Eighth Edition Chapter 10 Database Performance Tuning and Query Optimization.
Resource Allocation using Java RMI Amrish Kaushik Minal Malde CS599-Grid Computing Project Report USC Computer Science.
Scalable Cluster Management: Frameworks, Tools, and Systems David A. Evensky Ann C. Gentile Pete Wyckoff Robert C. Armstrong Robert L. Clay Ron Brightwell.
Introduction to Hadoop and HDFS
The Limitation of MapReduce: A Probing Case and a Lightweight Solution Zhiqiang Ma Lin Gu Department of Computer Science and Engineering The Hong Kong.
Invitation to Computer Science 5 th Edition Chapter 6 An Introduction to System Software and Virtual Machine s.
PNPI HEPD seminar 4 th November Andrey Shevel Distributed computing in High Energy Physics with Grid Technologies (Grid tools at PHENIX)
© Janice Regan, CMPT 300, May CMPT 300 Introduction to Operating Systems Operating Systems Overview Part 2: History (continued)
The european ITM Task Force data structure F. Imbeaux.
Cost Estimation. Problem Our ability to realistically plan and schedule projects depends on our ability to estimate project costs and development efforts.
ETLity Speed up your ETL development! → faster time to market → guaranteed quality → fix price development.
Andrey Meeting 7 October 2003 General scheme: jobs are planned to go where data are and to less loaded clusters SUNY.
1 Large-Scale Profile-HMM on the Grid Laurent Falquet Swiss Institute of Bioinformatics CH-1015 Lausanne, Switzerland Borrowed from Heinz Stockinger June.
CS 460/660 Compiler Construction. Class 01 2 Why Study Compilers? Compilers are important – –Responsible for many aspects of system performance Compilers.
Grid Execution Management for Legacy Code Applications Grid Enabling Legacy Applications.
The Global Land Cover Facility is sponsored by NASA and the University of Maryland.The GLCF is a founding member of the Federation of Earth Science Information.
David Adams ATLAS DIAL: Distributed Interactive Analysis of Large datasets David Adams BNL August 5, 2002 BNL OMEGA talk.
Matlab for Engineers Gari Clifford © Centre for Doctoral Training in Healthcare Innovation Institute of Biomedical Engineering Department of.
TeraGrid Gateway User Concept – Supporting Users V. E. Lynch, M. L. Chen, J. W. Cobb, J. A. Kohl, S. D. Miller, S. S. Vazhkudai Oak Ridge National Laboratory.
Economic and On Demand Brain Activity Analysis on Global Grids A case study.
Copyright © 2010, SAS Institute Inc. All rights reserved. SAS ® Using the SAS Grid.
LabVIEW.com.tw LabVIEW Community Speeding Up Your VIs 參考 NI 官方教材: LabVIEW Intermediate II for 7.
CS 351/ IT 351 Modeling and Simulation Technologies Review ( ) Dr. Jim Holten.
Scheduling MPI Workflow Applications on Computing Grids Juemin Zhang, Waleed Meleis, and David Kaeli Electrical and Computer Engineering Department, Northeastern.
Securing the Grid & other Middleware Challenges Ian Foster Mathematics and Computer Science Division Argonne National Laboratory and Department of Computer.
Higher Computing Science 2016 Prelim Revision. Topics to revise Computational Constructs parameter passing (value and reference, formal and actual) sub-programs/routines,
Millions of Jobs or a few good solutions …. David Abramson Monash University MeSsAGE Lab X.
Parallel Computation of Skyline Queries Verification COSC6490A Fall 2007 Slawomir Kmiec.
Holding slide prior to starting show. Lessons Learned from the GECEM Portal David Walker Cardiff University
Climate-SDM (1) Climate analysis use case –Described by: Marcia Branstetter Use case description –Data obtained from ESG –Using a sequence steps in analysis,
STAR Scheduler Gabriele Carcassi STAR Collaboration.
© Geodise Project, University of Southampton, Workflow Support for Advanced Grid-Enabled Computing Fenglian Xu *, M.
Alien and GSI Marian Ivanov. Outlook GSI experience Alien experience Proposals for further improvement.
Large-scale accelerator simulations: Synergia on the Grid turn 1 turn 27 turn 19 turn 16 C++ Synergia Field solver (FFT, multigrid) Field solver (FFT,
Simulation of O2 offline processing – 02/2015 Faculty of Electrical Engineering, Mechanical Engineering and Naval Architecture Eugen Mudnić.
PROTEIN IDENTIFIER IAN ROBERTS JOSEPH INFANTI NICOLE FERRARO.
Introduction to Makeflow and Work Queue Nicholas Hazekamp and Ben Tovar University of Notre Dame XSEDE 15.
EGI-InSPIRE RI EGI Compute and Data Services for Open Access in H2020 Tiziana Ferrari Technical Director, EGI.eu
EGI-InSPIRE RI An Introduction to European Grid Infrastructure (EGI) March An Introduction to the European Grid Infrastructure.
Hello everyone I am rajul and I’ll be speaking on a case study of elastic slurm Case Study of Elastic Slurm Rajul Kumar, Northeastern University
Invitation to Computer Science 6th Edition
Pegasus WMS Extends DAGMan to the grid world
Spark Presentation.
Azure Functions and Automation: The SQL Agent in the Cloud
System Design.
Building Analytics At Scale With USQL and C#
Abstract Machine Layer Research in VGrADS
Haiyan Meng and Douglas Thain
Tonga Institute of Higher Education IT 141: Information Systems
MATERI PL/SQL Procedures Functions Packages Database Triggers
Overview of Workflows: Why Use Them?
Tonga Institute of Higher Education IT 141: Information Systems
Frieda meets Pegasus-WMS
Presentation transcript:

Algorithmic Finance and Tools for Grid Execution (the Swift Grid Scripting/Workflow tool) Tiberiu (Tibi) Stef-Praun

Outline l The main idea u Extend your research with Grid computing u Means to do parallel execution l How does it help you u speed up, reuse code, increase input sizes l What do you get u Free infrastructure (lots of machines) u Simple tools to manage large executions u Scaled-up research capabilities u The CI expertise and support l Example u MaxPain u Econ results

Others are doing it already o Algorithmic trading o Community investing o Industry produces parallel cores o clusters are everywhere, and are affordable o Grid computing in finance is already happening (see references) o You probably know others better

Grid computing and why do you want it Clusters of machines: o Uniform access o data o execution o It has: o security o applications o your own space o Think “Fragmented Computing Resources” Machines Cluster Machines

How would you use grid computing o Split up your application into components of durations 5min-2 hours o Distribute parts of your application onto the grid nodes o In the proper order, as determined by the logic and dependencies between the app components: o send the inputs to a node containing the component o invoke the execution on that node o copy back the outputs

What do you need? l The Glue u Globus (middleware) l Execution tools l Data transfer tools u Swift (workflow) l Jobs dependencies description language (for your problem) l Execution Engine l The Examples u Simple Financial App u Complex Econ App l US (the Comp. Inst.) u Experience with Finance u Experience with Computing

Example: The MaxPain model l Inspired from the trading folklore u Assume all the options sellers could influence stock price to minimize their cost at ITM options expiry u Find the stock price at which this cost is minimum l Extended to address more realistic assumptions u What if all ITM options are sold at any point in time, instead of having them wait for the expiry l Issues: u Data acquisition and processing u Options pricing (binomial, Black-Scholes), implied volatility computation - Computationally Intensive

MaxPain on a single machine l DATABASE-DRIVEN u Scripts to collect the data (50 symbols) (bash) u Scripts to process the data (SQL, PLSQL) l Trigger data processing within the DB u All the steps use tables as sources, generate intermediary tables u Results are displayed later u Sequential processing l Issues u Running time: 50 symbols, 8 hours u Fixes to data acquisition, adding more symbols

MaxPain in distributed computing l Split into self-sufficient components: u Data acquisition (in batches of ) u Per-symbol computing u Per-procedure execution (Volatility, option pricing, Interpolation) (Currently R with QuantLib) u Grouping for higher level results (arbitrage, portfolio optimizations) l Use Swift to u Catalog and use the remote procedures u Drive the execution u Manage the inputs, outputs and dependencies

Managing distributed executions “You provide the problem code, I’ll provide the execution environment” (Run DEMO)

Workflow facilities in Swift l General language constructs u Procedures (atomic, composite) u Flow control (loops, conditions) u Data structures (arrays, structs) u Variable manipulation utilities (strcat. etc) l Virtualization support u Applications (declared in catalog files) u Data (mappers) u Just-in-time execution

Scaling it up: Moral Hazard l Stage 1 26x26 runs ->2,4 l Stage 2 102x52 runs ->3 l Stage 3 40x40 runs ->5 l Stage 4 40x40 runs ->5 l Stage 5 30x30 runs

Conclusions l It helps others u Econ (Robert Townsend) u Molecular Sciences (Bennoit Roux) l Build a set of finance and financial mathematics tools, continuous validation l Can share your tools/work easily l Our initial research goals: u Options Indicators (like MaxPain) u Portfolio optimization u Time Series Analysis l Similar work: u