1. 2 Old-Fashioned Problems
Surajit Chaudhuri
Microsoft

2. Query Optimization Revisited
Great engineering, poorly understood
What makes our search algorithm unique?
Is our Cardinality Estimation consistent?
What are the limits of effective query optimization?
Separate first order and second order assumptions..

3. Need for Evaluation Frameworks
DB community working on too many “exciting” problems
Hard to know if we are making progress
Isn’t this true of Query Optimizers as well?
+ Correctness of transformation can be verified
Impact on “real” workload/search efficiency unclear
Automated Physical design
+ A Technical definition of “optimality”/“goodness” exists
- DB2 and Oracle sucks, but I cannot convince you

4. Need for Evaluation Frameworks (2)
Example : Data Integration (Data Cleaning/Schema Matching)
“Fuzzy” problem specification
No notion of goodness except for “user” validation
Call to Action
Resist temptation to define another new problem with “overlapping” definition
Separate “heuristic”/”knobs” from a concrete server/middleware API
Define evaluation infrastructure (TREC like?)
Otherwise
We will be such like the AI folks..