1. Query Processing in Connectivity- Challenged Environments Priyanka Puri Sharma Chakravarthy Gururaj Poornima Mohan Kumar Information Technology Laboratory Computer Science and Engineering Department The University of Texas at Arlington, Arlington, TX 76009 Email: sharma@cse.uta.edu URL: http://itlab.uta.edu/sharma

2. This effort is supported by AFRL under Contract Number: FA8750-09-2-0199 Sanjay Madria and Raytheon (Waseem Naqvi) are also involved in this project May 23, 2010Sharma: AF Mobility Workshop

3. Query Processing Has been addressed in the context of centralized DBMSs Has been addressed in the context of distributed DBMSs Cost-based plan generation is typically used So, is there anything more/new to do? May 23, 2010Sharma: AF Mobility Workshop

4. Ground Controller 2 Ground Controller n Ground Controller 1 UAV 1 UAV 4 UAV 3 UAV 5 UAV 2 May 23, 2010Sharma: AF Mobility Workshop

5. Ground Controller 2 Ground Controller 1 Ground Controller n UAV 5 UAV 3 UAV 1 UAV 2 UAV 6 May 23, 2010Sharma: AF Mobility Workshop

6. Currently Data is dumped into a central server and queried Bandwidth, QoS issues are not addressed No collaboration among nodes No continuous query processing, notification, fusion, context usage, and real- or near real- time support May 23, 2010Sharma: AF Mobility Workshop

7. Network of computing nodes: Unmanned vehicles, Sensors, Robots, PCs, Servers, Ground Controlling devices Fault Tolerance Services Context/ Knowledge Base Local fusion/Materiali zation Publish Subscribe Capability Query Capability Raw Data / fused data /data from other nodes Queries, Tasks, Requests, Continuous Queries Publish/Subscribe SOA Distributed Middleware Task planningJoin computation Compositionpub/sub Context-awareNotification Resource ManagementData management Limited Resources Mobility Heterogeneity Disconnections Proposed long-term Architecture May 23, 2010Sharma: AF Mobility Workshop

8. Query Processing May 23, 2010Sharma: AF Mobility Workshop

9. MyObjects Table at each node TimestampNode_idLongitudeLatitudeObj_typeObj_descObject_ptr 8 bytes4 bytes 8 charsVarchar (64) Pointer (8 bytes) Total width: 100 bytes Cardinality (number of tuples), Selectivity, replication site of data are known (part of meta data) May 23, 2010Sharma: AF Mobility Workshop

10. Query Plan Format May 23, 2010Sharma: AF Mobility Workshop Operation 1ParamOperand1 Operand1 Loc Operand2 Operand 2 Location Result Name Result Loc Operation 2ParamOperand1 Operand1 Loc Operand1 Operand2 Loc Result Name Result Loc …………………… Operation nParamOperand1Operand1 Loc Operand1Operand2 Loc Result Name Result Loc

11. Operations in Plan format OperationParam Operand 1 Operand 1 Loc Operand 2 Operand 2 Loc Result Name Result Loc SelectA > 100R1R1 1Null R1’R1’1 ProjectA 1, A 3, A 4 R1’R1’1Null R 1 ’’1 MoveNullR 1 ’’1Null R’’2 CopyNullR1”R1”1 R 14 4 SemiJoinA = CR”2R2R2 2SR 1 2 JoinB = DR 12 2R 2 ’’2JR 1 2 May 23, 2010Sharma: AF Mobility Workshop

12. Plan using Semijoin chains SELECT c 1 R 1 MOVE R 11 To Site2 SELECT c 2 R 2 SJ R 11 R 21 : J 1 MOVE J 1 To Site3 SELECT c 3 R 3 SJ J 1 R 31 : J 2 MOVE J 2 To Site2 SJ J 2 R 21 : J 3 MOVE J 3 To Site1 SJ J 3 R 11 : J 4 COPY R To Site7:J Total Cost= 14720 + 32000 = 46720 May 23, 2010Sharma: AF Mobility Workshop 1 2 3 [lat] [long] R 1 [1000]R 2 [5000] R 3 [3000] R 11 [800] R 21 [3000] R 31 [600] select project select project select project Cost=3200 Cost=4800 Cost=1920 Cost=4800 7 J Cost=32000 J 1 [1200] J 3 [1200] J 2 [240] [lat,nodeid] [long,nodeid] J 4 [320]

13. Semi-join/join plan generation We are developing algorithms for generating the plan space and pruning it for generating “best” (or “good”) plan for each input query (expressed as a join query) It is a cost-based algorithm based on System R and SDD approaches extended to include connectivity and bandwidth issues The complexity of plan generation is k n ; n is number of joins and k is the number of alternatives for each join. Assuming less than 5 joins in a query Integrate replication into the algorithm May 23, 2010Sharma: AF Mobility Workshop

14. Plan Generation Alternatives A Query Plan (QP) is a numbered sequence of operations for executing a Query A QP includes how data is moved as part of execution Plan generation alternatives  Static Plan: generated once and executed in a distributed manner  Dynamic plan: generated incrementally at each node as the query progresses using current connectivity information  Parallel plan: partial plans are executed in parallel  Interactive plan: get some estimate by asking nodes that has relevant data May 23, 2010Sharma: AF Mobility Workshop

15. Static plan The physical plan generated will have node information for data propagation. This will be mapped to “actual connectivity” by the physical layer for execution It is possible that no connectivity exists by the time execution is performed for a generated query plan In that case, either a new plan can be generated (using the same algorithm, but using current meta data) or an alternative approach can be used to incrementally modify the plan May 23, 2010Sharma: AF Mobility Workshop

16. Dynamic plan Generate plan for the first join and defer the rest of the plan  Join plans are generated one at a time  Current connectivity information can be used  Result size estimation will also be more accurate Query execution and (partial) plan generation are intertwined Does not increase the complexity of plan generation or plan execution (compared to static) May 23, 2010Sharma: AF Mobility Workshop

17. Parallel plan All local operations/computations (select, project, and even some joins) can be done in parallel  Join plans are still generated one at a time  Increases message/information exchange  Current connectivity information can be used  Result size estimation will also be more accurate Deal with responses and plan generation and execution may be slightly more complicated than the previous cases May 23, 2010Sharma: AF Mobility Workshop

18. Interactive plan When a query comes in, send out requests for local processing and get processing time and size information Use the above to generate partial plans  Join plans are still generated using information obtained interactively  Increases message/information exchange  Current connectivity information can be used  Result size estimation will also be more accurate Combines Dynamic and parallel execution in an interactive manner May 23, 2010Sharma: AF Mobility Workshop

19. Replication Issues Algorithm for Replication  Single copy replication that “minimizes” the data transmission cost and “maximizes” the number of paths (to deal with connectivity) Algorithm for Replication utilization  Given a replication, determine the utility of that replica in terms of query evaluation cost for a reasonable load Reconcile the above two to come up with a replication strategy that balances the competing tradeoffs May 23, 2010Sharma: AF Mobility Workshop

20. Thank You ! Sharma: AF Mobility Workshop May 23, 2010