Presentation is loading. Please wait.

Presentation is loading. Please wait.

A lightweight framework for testing database applications Joe Tang Eric Lo Hong Kong Polytechnic University.

Similar presentations


Presentation on theme: "A lightweight framework for testing database applications Joe Tang Eric Lo Hong Kong Polytechnic University."— Presentation transcript:

1 A lightweight framework for testing database applications Joe Tang Eric Lo Hong Kong Polytechnic University

2 Our focus System testing (or black-box testing) A database application  its correctness/behavior depends on –The application code + –The information in the database 2

3 How to test a database application? Test preparation: –In a particular “correct” release A tester ‘plays’ the system and the sequence of actions (e.g., clicks) is recorded as a test trace/case T –E.g., T1: A user queries all product –E.g., T2: A user add a product The output of the system is recorded as the “expected results” of that trace –For database applications, “the output of the system” depends on the database content A test trace may modify the database content –For ease of managing multiple test traces, we reset the database content at the beginning of recording each test trace 3

4 How to test a database application? Test execution: –For each test trace T Reset the database content Runs the sequence of actions (e.g., clicks) recorded in T Match the system output with the expected output Problem: Resetting the DB content is expensive –Involves content recovery + log cleaning + thread resetting [ICSE 04] –About 1-2 minutes for each reset –If 1000 test traces  2000 minutes (33 hours) 4

5 DBA testing optimization 1.Test automation –Execute the test traces (and DB resets) automatically (vs. manually one-by-one) 2.Test execution strategies 3.Test optimization algorithms  aims to minimize the number of DB resets 5

6 Related work 2. Test execution strategies Optimistic [vldbj]: Execute reset lazily –T1 T2 T3 R T3 3. Test optimization algorithms SLICE Algorithm [vldbj]: –If T1 T2 R this time –Next time we try T2 T1 … 6

7 Problems 2. Test execution strategies Optimistic [vldbj]: Execute reset lazily –T1 T2 T3 R T3 May introduce false positives E.g., T2 covers a bug but it says nothing! 3. Test optimization algorithms SLICE Algorithm [vldbj]: –If T1 T2 R this time –Next time we try T2 T1 … 7

8 Problems 2. Test execution strategies Optimistic [vldbj]: Execute reset lazily –T1 T2 T3 R T3 May introduce false positives E.g., T2 covers a bug but it says nothing! 3. Test optimization algorithms SLICE Algorithm [vldbj]: –If T1 T2 R this time –Next time we try T2 T1 … Large overhead  keep swapping info Get worse when +/- test traces 8

9 This paper Test execution strategy –SAFE-OPTIMISTIC No false positives Test optimization algorithm –SLICE* No overhead Comparable performance to SLICE Better than SLICE when +/- test traces 9

10 Test execution strategy SAFE-OPTIMISTIC Also “execute resets lazily” –Test preparation: Record not only the system output + query results –Test execution Match not only the system output + Match query results 10

11 Implementation 11

12 Test optimization algorithm SLICE* algorithm Collection of “slices” If T1 T2 T3 R T3 T4 T5 Then we know and are good Next time: swap the slices, and thus try: T3 T4 T5 T1 T2 12

13 Evaluation A real world case study –An on-line procurement system –Test database  1.5GB –A database reset 1.9 min Synthetic experiments –Vary the number of test cases –Vary the degree of “conflicts” between test cases –Vary % of update in the test suite 13

14 Real case study 14

15 1000 test traces, 100K conflicts 15

16 Conclusion SLICE* and SAFE-OPTIMISTIC –Run tests on database applications Efficiently Safe (no false positives) Able to deal with test suite update 16

17 References [vldbj]Florian Haftmann, Donald Kossmann, Eric Lo: A framework for efficient regression tests on database applications. VLDB J. 16(1): (2007) [ICSE04] R. Chatterjee, G. Arun, S. Agarwal, B. Speckhard, and R. Vasudevan. Using data versioning in database application development. ICSE


Download ppt "A lightweight framework for testing database applications Joe Tang Eric Lo Hong Kong Polytechnic University."

Similar presentations


Ads by Google