1. Modeling Interactive Web Sources for Information Mediation Information Mediation Framework/Motivation Modeling Interactive Sources with Interaction Diagrams Computing Derived Capabilities Conclusions & Future Work Bertram Ludäscher, Amarnath Gupta San Diego Supercomputer Center, UCSD

2. Modeling Web Sources for Information Mediation 2 Mediation of Information Using XML (MIX) Wrapper RDB Mediator Wrapper Mediator BBQ UI OODB WWW XML-answers XMAS View Def. XML-queries MIX Mediator Architecture

3. Modeling Web Sources for Information Mediation 3 Motivation Wrappers export schema, capabilities, and data of sources Specifics of Web sources (e.g., vs RDB/OODB sources): – – limited query capabilities, i.e., * * restricted i/o binding patterns * * need for user interaction * * complex navigations – – data extraction – – accessibility – –...

4. Modeling Web Sources for Information Mediation 4 Example: ATM/bank locator "Find some ATM/bank locations close to a location given by street, city, state, user interaction.""Find some ATM/bank locations close to a location given by street, city, state, user interaction." involves –inputoutput – input and output attributes – – forms, navigations, user interaction – – data extraction

5. Modeling Web Sources for Information Mediation 5 "Non-Wrappable" User Interactions Wrapper RDB Mediator Wrapper Mediator BBQ UI OODB WWW XML-answers XMAS View Def. XML-queries user/source interaction Modified MIX Mediator Architecture

6. Modeling Web Sources for Information Mediation 6 Modeling Interactive Sources... Model input/output behavior of (HTML) elements as select-project queries: Given values for input attributes a, select tuples satisfying  (a) from source relation R, project on output attributes b :    b (   (a) (R))

7. Modeling Web Sources for Information Mediation 7 Modeling Interactive Sources... Example (HTML Link): clicking on an airport code apc produces the airport address with zip code:  zip (   (apc) (R)) where  ( apc ):= ( apc=$apc ) and $apc represents the actual parameter (here: link label) (or: answer(ZIP)  APC=$APC, R (APC,ZIP,...) )

8. Modeling Web Sources for Information Mediation 8... Modeling Interactive Sources... Example (HTML Forms): G Given a street and city name, extract the zip code:  zip (   (street,city) (R)) where  ( street,city ):= ( street=$s  city=$c ). By default, forms are modeled as conjunctions of equalities. (similar for menus and other elements)

9. Modeling Web Sources for Information Mediation 9... Modeling Interactive Sources... Example (Non-Wrappable Elements): interactions requiring explicit user interaction with the source, and where the result depends on a 1,..., a n are denoted by !ui(a1,..., a n ) The user interaction is modeled by a new, internal attribute x:  y (   ( a 1,..., a n, x ) (R))

10. Modeling Web Sources for Information Mediation 10... Modeling Interactive Sources... Use select-project queries to model i/o behavior of individual user interaction elements (links, forms, menus, maps,...) Missing: how to "glue" together... navigations through a Web site and data extraction... in order to model source capabilities of complex interactions   model a Web source using interaction diagrams:   nodes  source page(s) and exported data   edges  transitions and required interactions

11. Modeling Web Sources for Information Mediation 11 Interaction Diagrams diagram: labeled graph over given attributes node  source page(s) with same i/o behavior   node label  exported attributes (tuple/set/list): (a_1,...,a_n), {(a_1,...,a_n)}, [(a_1,...,a_n)]   edge  transition, edge label  required interaction: i[,u\v]i[,u\v] t: x y interaction i  {href, form, menu, !ui,...}  transition condition t's result depends on attributes u but not v [a1,... ] {b1,... }

12. Modeling Web Sources for Information Mediation 12 Example Interaction Diagram transition path t1.t3.t5: fill in form with street, city, menu with state, fill in form s.t. radius>0, follow all bankid links   derivable query template q(t1.t3.t5) = (+street,+city,+state,+radius,-bname,-bstreet, -bcity)

13. Modeling Web Sources for Information Mediation 13 Deriving Query Capabilities: Single Transitions input atts in(t) := (atts(x)  atts(i)  u) \ v output atts out(t) := atts(y) interaction requirements act(i) := i   i[,u\v]i[,u\v] t: x y

14. Modeling Web Sources for Information Mediation 14 Deriving Query Capabilities:Transition Paths t 1.t 2.t (t = t 3.....t n ) in(t 1.t 2.t) := in(t 1 )  in(t 2.t) \ propagate(  con (t 1.t 2 ))) out(t 1.t 2.t) := out(t 2.t) act(t 1.t 2.t) := act(t 1 )   con (t 1.t 2 ))  act(t 2.t) where – –  con (t 1.t 2 )) connects out(t 1 ) with in(t 2 ) (e.g., ) – – propagate (...) are attributes whose values are passed along (e.g., out(t 1 )  in(t 2 )) – –  denotes serial conjunction

15. Modeling Web Sources for Information Mediation 15 Use of Interaction Diagrams modeling tool complex query capabilities (binding patterns) along paths t 1.....t n  q t (in(t), out(t)) t = t 1.....t n  q t (in(t), out(t)) sequence of interaction requirements  execution plans subsumption and equivalence of binding patterns  check for supported queries distinction between wrappable vs. unwrappable queries using the number of !ui nodes in paths

16. Modeling Web Sources for Information Mediation 16 Interacting with the Mediator Diagram-enabled wrapper can do local query optimizations Models of wrapper-mediator interaction: – –Wrapper exports complete interaction diagram, mediator plans query – –Mediator hands complete subquery to wrapper, wrapper optimizes query – –Wrapper gives mediator partial plans, mediator computes overall query plan An interesting model –Touting wrapper –Touting wrapper: identifies missing element in a query that would make an infeasible query feasible

17. Modeling Web Sources for Information Mediation 17 Conclusions & Future Work Ongoing: Stand-alone Web site modeling tool (SiteModel: P. Nguyen) which exports an interaction diagram Definition & analysis of query capabilities of complex sources Future Work: support for query evaluation at the MIX mediator and user/source interaction use interactions diagrams for semi-automatic wrapper generation