Presentation on theme: "INITIALIZATION AND ONGOING ACCEPTANCE TESTING. In order to use the contractors test results for acceptance and payment, ODOT must ensure that the contractors."— Presentation transcript:
INITIALIZATION AND ONGOING ACCEPTANCE TESTING
In order to use the contractors test results for acceptance and payment, ODOT must ensure that the contractors test results compare favorably with their own. Favorable comparability will be determined by means of paired testing of split samples.
INITIALIZATION TESTING Minimum of 10 tests 1 st Lot testing divided into 10 sublots Optionally, initialization can be made up of the first 2 lots of 5 sublots Include plant start-up testing
Each set of asphalt testing includes: INITIALIZATION TESTING 1 AIR VOID TEST 1 AC CONTENT TEST 3 ROADWAY DENSITY TESTS
At the conclusion of the sampling and associated testing, a statistical paired-t test will be performed by ODOT using the pairs of initialization test data.
If there is a significant bias: ODOTs test results will be relied upon until the source of the bias has been identified and corrected. The lack of bias must then be validated by further paired-t testing
If Independent Evaluation requested: The Contractor may request evaluation of the testing bias via three-way split sample testing with an independent laboratory. ODOT will select the independent laboratory. All initialization testing will be performed by all three laboratories using split samples.
If ODOT testing shows a significant bias: The Contractor will not be held responsible for any additional costs incurred by ODOT in conjunction with the 3-way split-sample testing The Contractors test results will be relied upon for acceptance and pay adjustments
If Contractor testing shows a significant bias: The Contractor shall reimburse ODOT for any additional costs incurred by ODOT in conjunction with the 3-way split- sample testing ODOTs test results will be relied upon for acceptance and pay adjustments
If paired-t testing validates the Contractors test methods, the Contractors test results will be used for acceptance and pay adjustment subject to: ODOT will perform ongoing testing at a minimum frequency of 1 independent test per lot. Ongoing process verification over time will be performed using the F-test and the t-test.
If ODOTs testing-bias Control Charts indicate an out-of-control condition for testing bias: Both the Contractor and ODOT will immediately investigate the probable cause.
If the probable cause is identified and corrected or if the resulting bias is, in ODOTs judgment, not likely to exceed the Allowable Testing Bias: Acceptance and pay adjustments will continue to be based on the Contractors results At ODOTs discretion, errant test results may be discarded. If replacement test results are available, they may be substituted for discarded test results
If the probable cause is not corrected or if the resulting bias is, in ODOTs judgment, likely to exceed the allowable testing bias: All subsequent testing will be paired testing until the source of the unaccepatable testing bias is corrected. During this period, acceptance and pay adjustments will be based on ODOTs test results.
GUIDELINES FOR INITIAL VALIDATION OF CONTRACTORS TEST METHODS A paired test shall consist of 2 separate tests by ODOT and the Contractor on a split sample
PAIRED TESTING On split samples test: Air Voids Asphalt Cement Content For Roadway Densities, perform paired tests on the same cores. The first lab to test the cores should make sure to oven- dry the cores at 140°F instead of 230°F
GENERAL PROCEDURE U Calculate, the average of the differences between the paired tests STEP 1
Where: n p = Number of paired tests X d = ODOTs individual test result for X p = The average of the difference between paired tests X c = The Contractors individual test result for each sample. each sample.
GENERAL PROCEDURE U Calculate S p, the standard deviation of the differences between the paired tests STEP 2
Where: S p = The Standard Deviation of the n p - 1 = Degrees of freedom, and difference between paired tests, X c, X d, n p, and X p as previously defined.
GENERAL PROCEDURE U Calculate t p, the paired-t test statistic STEP 3
Where: t p = The paired-t test statistic, and n p, X p, and S p as previously defined. The vertical lines on either side of the equation indicate the need to take the ABSOLUTE VALUE of the equation Simply put, if the result of the calculation is NEGATIVE, change it to a POSITIVE number
GENERAL PROCEDURE U Obtain the critical t-value, t crit, from Table 1 in Appendix A using n p -1 degrees of freedom STEP 4
GENERAL PROCEDURE U Compare the calculated paired-t statistic (t p ) to the critical t-value t crit STEP 5 U If t p is less than t crit, there is no statistically significant bias U If t p is more than t crit, there is a statistically significant bias
Compare the testing bias (X p ) to the associated allowable testing bias (ATB) from the table in Appendix A: If t p is more than t crit : AC Quality Char. ATB Units AC Content± 0.15 % by weight Air Voids± 0.50 % by volume Roadway Density ± 0.50 % of G mm Note: ATBs for aggregates are for control charts and information only.
If the magnitude of X p is less than the magnitude of the corresponding ATB, the Contractors bias is statistically, but not practically significant. Contractor test results shall then be relied upon for acceptance and pay adjustments
If the magnitude of X p is greater than or equal to the magnitude of the corresponding ATB, the Contractors bias is practically significant and unacceptable. Contractor test results shall not be relied upon until certified by a qualified independent calibration specialist and validated through further paired-t testing.
EXAMPLE - % Air Voids TEST # XcXc XdXd
Step 1 – Average Differences TEST # XcXc XdXd X c - X d X p = -0.1
Step 2 – Standard Deviation TEST # XpXp -0.1 X c - X d - X p X c - X d (X c - X d - X p ) 2 = 0.58 (X c - X d - X p )
S p =
t p = 1.246
Degrees of Freedom (n p – 1) Critical t-value (t crit )
t p = < t crit = Since t p < t crit, there is no statistical bias. Therefore the contractors tests on this quality characteristic are valid and will be relied upon for acceptance and pay adjustments.
ONGOING VALIDATION OF CONTRACTORS TEST METHODS
GUIDELINES FOR ONGOING VALIDATION OF CONTRACTORS TEST METHODS Ongoing process verification over time will be performed using the F-test and the t-test. The F-test provides a method for comparing the variances of the two data sets. The t-test provides a method for comparing the means of the two data sets.
Contractor Department One independent Contractor: Collect samples to split and test ¼, discard ¼, store 2 nd half. ODOT: Collect one sample per lot, independent of Contractors samples. Test ¼, discard ¼, store 2 nd half.
GUIDELINES FOR ONGOING VALIDATION OF CONTRACTORS TEST METHODS First two lots do not yield enough data for F-test and t-test. These lots will be validated by comparing to D2S limits. If X c – X d D2S limit, assumed from same population – Contractors accepted. If X c – X d > D2S limit, assumed from different populations – ODOTs used.
THIRD - FIFTH LOT AFTER INITIALIZATION Using the F-test and t-test for process verification. Use all data after Initialization, including that used for D2S comparisons. By third lot after Initialization there should be 15 contractor results and 3 ODOT results.
SIXTH LOTS & FOLLOWING AFTER INITIALIZATION Still using the F-test and t-test for process verification. Use the current lot results plus the previous four for a total of five lots. Each of these analyses should include 25 contractor results and 5 ODOT results.
F-test Calculate the mean of each test set. X c = the mean of the Contractors results X d = the mean of the Departments results X c = individual Contractor result X d = individual Department result n c = the number of Contractor results n d = the number of Department results
F-test Calculate the variance of each test set. n c - 1 = degrees of freedom of Contractor results = the variance of the Contractors results n c - 1 = degrees of freedom of Department results = the variance of the Departments results
F-test Calculate the F-statistic. Use the larger of the variances in the numerator so that the F-statistic is always greater than one. or
U Compare the calculated F-statistic to the critical F-value F crit U If F is less than F crit, the data is assumed from the same population. U If F is more than F crit, the data is assumed from a different population F-test U Determine F crit from the F-distribution table using a 0.01 significance level
t-test Calculate the pooled variance,. Where n c + n d - 2 = pooled degrees of freedom
t-test Calculate the t-statistic.
U Compare the calculated t-statistic to the critical t-value t crit U If t is less than t crit, the data is assumed from the same population. U If t is more than t crit, the data is assumed from a different population t-test UDetermine t crit from the t-distribution table using a 0.01 significance level and the pooled degrees of freedom
Both the F-test and the t-test must show the results to be from the same population to continue accepting the Contractors results
If either test shows the results to be from different populations, ODOT will test the remaining half of the Contractors split samples for acceptance until the reason for the difference is resolved.
EXAMPLE - % Asphalt Content 1 st Lot after Initialization – D2S Limits Test #XcXc XdXd After initialization, ODOT and Contractor samples are no longer split samples but obtained independently
Step 1: Compute Averages Test #XcXc XdXd Avg
Step 2: Compute D2S Comparison (D2S=0.30 from Appendix A Part 2 for Asphalt Cement content) Since is true, we assume that the test results come from the same population. The Contractors results are accepted.
EXAMPLE - % Asphalt Content 2 nd Lot after Initialization – D2S Limits Test #XcXc XdXd
Step 1: Compute Averages Test #XcXc XdXd Avg
Step 2: Compute D2S Comparison Since is true, we assume that the test results came from the same population. The Contractors results are accepted. (D2S=0.30 from Appendix A Part 2 for Asphalt Cement content)
EXAMPLE - % Asphalt Content 3 rd Lot after Initialization – Begin F & t Test #XcXc XdXd XcXc XcXc
Step 1: Compute Mean for Each Set
Step 2: Compute s 2 (Variance) for Each Set
Step 3: Compute s c 2 Test # XcXc (X c -X c ) Sum Sum/(n c -1) / (15-1) scsc 2 X c 4.22
Step 4: Compute s d Test #XdXd (X d -X d ) Sum Sum/(n d -1)0.0093/(3-1) sdsd X d 4.13
Step 5: Compute F-statistic or FNumeratorDenominatorResult Note: Pick F with largest numerator (highest result).
Step 6: Pick F crit from F-distribution (2-tailed, =0.01) Table n n For this example, contractor variance was in the numerator with 14 degrees of freedom and ODOTs variance was in the denominator with 2 degrees of freedom. Read the table like this Not like this!
Step 7: Perform F-test Since F-test is true, we must perform the T-test. Had F-test been false, the tests would not have come from the same population. The Contractors results would not be accepted. F F crit ? = True
Step 8: Compute Pooled Variance
Step 9: Compute t-statistic Note: Without rounding, t 1.25.
Step 10: Pick t crit from t-distribution (2- tailed, =0.01) Table DFt crit Degrees of freedom = n c + n d – 2 DF = – 2 = 16
Step 11: Perform t-test Since both t-test and F-test are true, we assume that the test results came from the same population. The Contractors results are accepted. t t crit ? = True