Algebraic Transformations Page 1 © 2013 Hortonworks HIVE-784: Sub Query in Where or Having clause HIVE-5555: Alt. Join Syntax; Join conditions in the Where Clause

Sub Query transformation Page 2 © 2013 Hortonworks Support for In, Not In, Exists, Not Exists in Where or Having clause But lots of restrictions –Sub Query predicate must be a top level conjunct –Only 1 Sub Query predicate –No Sub Query nesting –Correlation condition must be valid join conditions –And many more: See Spec on HIVE-784; 17 Restrictions so far. Transformation at a high level are: –In/Exists => Left Outer Join –Not In/Exists => Left Outer Join + null check + null count for Not In –Correlation converted to Gby in Sub Query In spite of long list of Restrictions, possibly useful –See HIVE-784 for TPCH Queries Q4, Q15, Q16, Q18 written with SQs –TPCDS Query 45

Some Examples Page 3 © 2013 Hortonworks -- non agg, corr select * from src b where b.key in (select a.key from src a where b.value = a.value and a.key > '9' ) ; -- non agg, non corr select key, count(*) from src group by key having count(*) in (select count(*) from src s1 where s1.key > '9' group by s1.key ) ; -- tpch Q4 select o_orderpriority, count(*) as order_count from orders o where unix_timestamp(o_orderdate, 'yyyy-MM-dd') >= unix_timestamp(' ', 'yyyy-MM-dd') and unix_timestamp(o_orderdate, 'yyyy-MM-dd') < unix_timestamp(' ', 'yyyy-MM-dd') and exists ( select * from lineitem where l_orderkey = o.o_orderkey and l_commitdate < l_receiptdate ) group by o_orderpriority order by o_orderpriority;

Join syntax Page 4 © 2013 Hortonworks I want to use old-style Join syntax: join conditions in Where clause Sub problems: –HIVE-5556: Push Join conditions up Join tree. –So A join B join C on A.x = B.x and A.y = C.y should be handled as: –A join B on A.x = B.x join C on A.y = C.y –Fix holes in handling of Join Tree merging –HIVE-5557: Push ‘qualifying’ predicates from Where Clause up Join Tree –HIVE-5558: support alternate syntax for cross product (allow use of comma).