Presentation is loading. Please wait.

Presentation is loading. Please wait.

A Comprehensive Model for Arbitrary Result Extraction Neal Sample, Gio Wiederhold Stanford University Dorothea Beringer Hewlett-Packard.

Published byTravis Exley Modified over 9 years ago

Similar presentations

Presentation on theme: "A Comprehensive Model for Arbitrary Result Extraction Neal Sample, Gio Wiederhold Stanford University Dorothea Beringer Hewlett-Packard."— Presentation transcript:

1 A Comprehensive Model for Arbitrary Result Extraction Neal Sample, Gio Wiederhold Stanford University Dorothea Beringer Hewlett-Packard

2 SAC 2002 2 Shift in Programming Tasks Coding Integration/Composition 1970 1990 2010

3 SAC 2002 3 Sample Composition Tasks Logistics Reservation and distribution systems, “find the best transportation route from A to B” Genomics Framework for composing various processing tools and repositories Modeling Weather prediction, complex chemical systems, basin modeling Composition of processes (vs. components, data)

4 SAC 2002 4 CLAM Composition Language Purely compositional no primitives for arithmetic no primitives for I/O, etc. Splitting up CALL-statement parallelism by asynchrony in sequential program novel possibilities for optimizations reduction in complexity of invocation statements Higher-level language assembly  HLLs HLLs  compositional paradigm Intent: Enable domain experts

5 SAC 2002 5 CLAM Primitives Pre-invocation: SETUP: set up the connection to a service SET-, GETPARAM: in a service ESTIMATE: for optimization Invocation and result gathering: INVOKE: begin execution EXAMINE: test progress of an invoked method EXTRACT: extract results from an invoked method Termination: TERMINATE: terminate a method invocation/connection to a service

6 SAC 2002 6 Data Dependencies & Scheduling // begin program A = service1(); B = service2(); C = service3(A,B); D = service4(C); E = service5(C); // end of program START END service1service2 service3 service4service5

7 SAC 2002 7 Runtime: data extraction is hard Data extraction with native modules worked No language-level specifications in CLAM E.g., Polling, threading, exception handling… Multiple middleware for transport  difficult mapping CORBA-RMI, RMI-COM, COM-CPAM, etc. Crisis of legacy services To generalize or restrict? Refine the strategy…

8 SAC 2002 8 Strategy: hide it & depend on it Have to respect service capabilities Or suffer the LCD… (more in a bit) Simple and flexible programming Data extractions is a runtime issue, it is not central to composition task Simplified Integration Legacy ambivalence Simple bridging for middleware Increase audience for services Better scheduling Declarative language, data dependencies

9 SAC 2002 9 Where are we? Declarative language for composition Data is used synchronization No primitives to support synchronization Apparent “mismatch” in data extraction methods & capabilities among various actors What does the data look like? How can data be extracted?

10 SAC 2002 10 Data View: Services RESULTS Result AResult BResult C

11 SAC 2002 11 Extraction Techniques Asynchrony Explicitly controlled: spin-locks, polling, interrupt handling, etc. Can use with any DAG schedule Partial extraction web browsing - HTML text as a schema SQL cursors ( thanks to the reviewer ) Progressive extraction (exceptional) Adaptive mesh refinements, JPEG interleaving

12 SAC 2002 12 Current Focus Pre-invocation: SETUP: set up the connection to a service SET-, GETPARAM: in a service ESTIMATE: for optimization Invocation and result gathering: INVOKE: begin execution EXAMINE: test progress of an invoked method EXTRACT: extract results from an invoked method Termination: TERMINATE: terminate a method invocation/connection to a service

13 SAC 2002 13 Current Focus Pre-invocation: SETUP: set up the connection to a service SET-, GETPARAM: in a service ESTIMATE: for optimization Invocation and result gathering: INVOKE: begin execution EXAMINE: test progress of an invoked method EXTRACT: extract results from an invoked method Termination: TERMINATE: terminate a method invocation/connection to a service

14 SAC 2002 14 EXAMINE Primitive in CLAM Returns “status” and “progress” Status – 2 bits of state status = {DONE, NOT_DONE, PARTIAL, ERROR} Progress – open descriptor Indicates progress in application specific-way Could be variance, mean, amplitude, etc. Default assumption: integer 0-100 = % done Resolution of EXAMINE Can apply per service (black box) Can apply per result (white box) Not complete for many legacy systems: only “status”, no “progress”

15 SAC 2002 15 EXAMINE Service ABC Service.EXAMINE()  {NOT_DONE, 0} Service.EXAMINE(A)  {NOT_DONE, 0} Service.EXAMINE(B)  {NOT_DONE, 0} Service.EXAMINE(C)  {NOT_DONE, 0} Service ABC Service.EXAMINE()  {PARTIAL, 40} Service.EXAMINE(A)  {DONE, 100} Service.EXAMINE(B)  {NOT_DONE, 0} Service.EXAMINE(C)  {PARTIAL, 20} Service ABC Service.EXAMINE()  {DONE, 100} Service.EXAMINE(A)  {DONE, 100} Service.EXAMINE(B)  {DONE, 100} Service.EXAMINE(C)  {DONE, 100}

16 SAC 2002 16 EXTRACT Primitive Extracts data from a service Per service (black box) (var) = Service.EXTRACT(); Per result (white box) (varA = A, varC = C) = Service.EXTRACT(); Allows partial data extraction saves volume: abandon uninteresting elements saves time: termination of useless invocation Allows progressive data extraction with 2-value EXAMINE (status+progress) Steering, time saving

17 SAC 2002 17 Examine-Extract Relationship per service status only per serviceper result EXTRACT per service status+ progress per result status only per result status+ progress asynchronous procedure call, Java RMI limited Partial Extraction, (binary) thumbnails partitioned progressive extract (full result set) ? semantic partial extraction (full result set) partial extraction browsing, SQL cursor (no progressive) progressive extraction (full result set) progressive and partial extraction CLAM

18 SAC 2002 18 Examine-Extract Relationship per service status only per serviceper result EXTRACT per service status+ progress per result status only per result status+ progress asynchronous procedure call, Java RMI limited Partial Extraction, (binary) thumbnails partitioned progressive extract (full result set) ? semantic partial extraction (full result set) partial extraction browsing, SQL cursor (no progressive) progressive extraction (full result set) progressive and partial extraction CLAM

19 SAC 2002 19 Examine-Extract Relationship per service status only per serviceper result EXTRACT per service status+ progress per result status only per result status+ progress asynchronous procedure call, Java RMI limited Partial Extraction, (binary) thumbnails partitioned progressive extract (full result set) ? semantic partial extraction (full result set) partial extraction browsing, SQL cursor (no progressive) progressive extraction (full result set) progressive and partial extraction *CLAM

20 SAC 2002 20 Examine-Extract Relationship per service status only per serviceper result EXTRACT per service status+ progress per result status only per result status+ progress asynchronous procedure call, Java RMI limited Partial Extraction, (binary) thumbnails partitioned progressive extract (full result set) ? semantic partial extraction (full result set) partial extraction browsing, SQL cursor (no progressive) progressive extraction (full result set) progressive and partial extraction *CLAM

21 SAC 2002 21 Examine-Extract Relationship per service status only per serviceper result EXTRACT per service status+ progress per result status only per result status+ progress asynchronous procedure call, Java RMI limited Partial Extraction, (binary) thumbnails partitioned progressive extract (full result set) ? semantic partial extraction (full result set) partial extraction browsing, SQL cursor (no progressive) progressive extraction (full result set) progressive and partial extraction *CLAM

22 SAC 2002 22 Examine-Extract Relationship per service status only per serviceper result EXTRACT per service status+ progress per result status only per result status+ progress asynchronous procedure call, Java RMI limited Partial Extraction, (binary) thumbnails partitioned progressive extract (full result set) ? semantic partial extraction (full result set) partial extraction browsing, SQL cursor (no progressive) progressive extraction (full result set) progressive and partial extraction *CLAM

23 SAC 2002 23 Examine-Extract Relationship per service status only per serviceper result EXTRACT per service status+ progress per result status only per result status+ progress asynchronous procedure call, Java RMI limited Partial Extraction, (binary) thumbnails partitioned progressive extract (full result set) ? semantic partial extraction (full result set) partial extraction browsing, SQL cursor (no progressive) progressive extraction (full result set) progressive and partial extraction *CLAM

24 SAC 2002 24 Examine-Extract Relationship per service status only per serviceper result EXTRACT per service status+ progress per result status only per result status+ progress asynchronous procedure call, Java RMI limited Partial Extraction, (binary) thumbnails partitioned progressive extract (full result set) ? semantic partial extraction (full result set) partial extraction browsing, SQL cursor (no progressive) progressive extraction (full result set) progressive and partial extraction *CLAM

25 SAC 2002 25 Conclusions Data extraction hiding is bueno! User is not responsible for data management Synchronizing extractions not in the language  simplicity Enables effective service scheduling Simplified integration Blueprint for proactive design pattern for future services

Download ppt "A Comprehensive Model for Arbitrary Result Extraction Neal Sample, Gio Wiederhold Stanford University Dorothea Beringer Hewlett-Packard."

Similar presentations

A Workflow Engine with Multi-Level Parallelism Supports Qifeng Huang and Yan Huang School of Computer Science Cardiff University 2005.9.

A Workflow Engine with Multi-Level Parallelism Supports Qifeng Huang and Yan Huang School of Computer Science Cardiff University
RPC Robert Grimm New York University Remote Procedure Calls.

RPC Robert Grimm New York University Remote Procedure Calls.
Towards a Practical Composition Language Oscar Nierstrasz Software Composition Group University of Bern.

Towards a Practical Composition Language Oscar Nierstrasz Software Composition Group University of Bern.
June 1998 CHAIMS1 Interpreter instead of Compiler CHAIMS execution machine (interpreter and scheduler) user Interpreter: network CHAIMS-protocol complete.

June 1998 CHAIMS1 Interpreter instead of Compiler CHAIMS execution machine (interpreter and scheduler) user Interpreter: network CHAIMS-protocol complete.
What iS RMI? Remote Method Invocation. It is an approach where a method on a remote machine invokes another method on another machine to perform some computation.

What iS RMI? Remote Method Invocation. It is an approach where a method on a remote machine invokes another method on another machine to perform some computation.
Copyright © 2011 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Chapter 13 Introduction to SQL Programming Techniques.

Copyright © 2011 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Chapter 13 Introduction to SQL Programming Techniques.
Semantic description of service behavior and automatic composition of services Oussama Kassem Zein Yvon Kermarrec ENST Bretagne France.

Semantic description of service behavior and automatic composition of services Oussama Kassem Zein Yvon Kermarrec ENST Bretagne France.
CHAIMS: Compiling High-level Access Interfaces for Multisite Software Neal Sample Stanford University.

CHAIMS: Compiling High-level Access Interfaces for Multisite Software Neal Sample Stanford University.
Gio Wiederhold SimQL 1 SimQL Accessing Simulation as Services to Information Systems Gio Wiederhold July 2001.

Gio Wiederhold SimQL 1 SimQL Accessing Simulation as Services to Information Systems Gio Wiederhold July 2001.
490dp Synchronous vs. Asynchronous Invocation Robert Grimm.

490dp Synchronous vs. Asynchronous Invocation Robert Grimm.
November 1999 CHAIMS1 Compiling High-level Access Interfaces for Multi-site Software Stanford University Objective: Investigate revolutionary approaches.

November 1999 CHAIMS1 Compiling High-level Access Interfaces for Multi-site Software Stanford University Objective: Investigate revolutionary approaches.
June 1998 CHAIMS1 Challenge 1: Composition of Processes... versus composition and integration of Data »data-warehouses »wrapping data available on web.

June 1998 CHAIMS1 Challenge 1: Composition of Processes... versus composition and integration of Data »data-warehouses »wrapping data available on web.
Business Process Orchestration

Business Process Orchestration
June 1998 CHAIMS1 execution of a remote method invoke a method i e extract results setup / set attributes s s e i time decomposed (no benefit for one.

June 1998 CHAIMS1 execution of a remote method invoke a method i e extract results setup / set attributes s s e i time decomposed (no benefit for one.
Demystifying Architectural Styles Nikunj Mehta 3/11/02Demystifying Architectural Styles2 Agenda Architectural Styles The Alfa Project Architectural framework.

Demystifying Architectural Styles Nikunj Mehta 3/11/02Demystifying Architectural Styles2 Agenda Architectural Styles The Alfa Project Architectural framework.
April 1999 CHAIMS1 Prof. Gio Wiederhold, Dr. Dorothea Beringer, Composing Autonomous Internet Services with CHAIMS CHAIMS Objective: Using and composing.

April 1999 CHAIMS1 Prof. Gio Wiederhold, Dr. Dorothea Beringer, Composing Autonomous Internet Services with CHAIMS CHAIMS Objective: Using and composing.
University of Kansas Construction & Integration of Distributed Systems Jerry James Oct. 30, 2000.

University of Kansas Construction & Integration of Distributed Systems Jerry James Oct. 30, 2000.
Scheduling Under Uncertainty: Planning for the Ubiquitous Grid Neal Sample Pedram Keyani Gio Wiederhold Stanford University.

Scheduling Under Uncertainty: Planning for the Ubiquitous Grid Neal Sample Pedram Keyani Gio Wiederhold Stanford University.
Megamodules domain expert writes megaprogram for composition CHAIMS automizes generation of client for distributed system megamodule provider provides.

Megamodules domain expert writes megaprogram for composition CHAIMS automizes generation of client for distributed system megamodule provider provides.
January 1999 CHAIMS1 Objectives C H A I M S CLAM CPAM Scheduling ESTIMATE EXTRACT Provide high-level, composition-only language (or graphical front-end)

January 1999 CHAIMS1 Objectives C H A I M S CLAM CPAM Scheduling ESTIMATE EXTRACT Provide high-level, composition-only language (or graphical front-end)

Similar presentations

Ads by Google