Accuracy and Precision

Accuracy- indicates the difference between measured value and the expected (true) value. If observed (measured) values are near the expected (true) value, the measurement has high accuracy. Precision- indicates the differences among each measured value; if measured values are close together the measurement has high precision.

A Precise Score 4 Arrows from Contestant #2 14th Annual Archery Contest 4 Arrows from Contestant #1 The two contestants are highly precise, however, their accuracy is not very good. There is some subtle problem with their equipment or how they are using it.

A Precise and Accurate Score 14th Annual Archery Contest 4 Arrows from Contestant #3 The third contestant has a score that is precise and accurate.

Neither Precise nor Accurate Score 14th Annual Archery Contest 14th Annual Archery Contest Neither Precise nor Accurate Score Archery Contest 14th Annual 4 Arrows from Contestant # 4 Archery Contest 14th Annual The fourth contestant should not quit their other sport. (This contestant needs to work on the concept of 4 arrows as well)

Accuracy of a measurement describes the difference between an observed value and the expected (true) value. If observed value is near the expected (true) value, measurement has high accuracy. Precision of a measurement describes the differences among individual measurements themselves; if they are close together there is high precision.

Accuracy and Precision How do we quantify these observations so that we can attach a number to the ideas of precision and accuracy of each event? Accuracy and precision have no quantitative significance, and are used in a rough descriptive sense only.

Error and Deviation Error expresses accuracy in the form of a number that relates the measured value to the true (expected) value. Deviation expresses precision in the form of a number that relates the measured value to its closeness to other measured values.

Error Error expresses accuracy in the form of a number that relates the measured value to the true (expected) value. Error is the numerical difference between the observed value, xi and expected (true) value, .

Error Error is the numerical difference between the observed value, xi and expected (true) value, . For this case, the expected position on the target for the expert is assigned the true (expected) score. This position is assigned a distance value of 0 and distance from this perfect score point on the target to an actual contestant point on the target is the error.

Error Error is the numerical difference between the observed value, xi and expected (true) value, . This is the center point that is used to measure a radius to each of the 8 target points shown.

Error Contestant # 1 try # error 1 8.0 2 9.5 3 9.5 4 8.3 Error is the numerical difference between the observed value, xi and expected (true) value, . 11 9 7 4 6 10 Contestant # 1 try # error 2 6 1 8.0 2 9.5 3 9.5 4 8.3 This is the center point that is used to measure a radius to each of the 8 target points shown.

Error Contestant # 2 try # error X5 ? 10.5 X6 10.5 X7 9.5 X8 9.5 In this example, the perfect score, , is equal to zero. A contestant's score, xi, is subtracted from to indicate the error. X5 ? 10.5 X6 10.5 11 X7 9.5 9 6 X8 9.5 7 5 = perfect score = 0 Error =(observed score, xi ) - 2 X6 - = 10.5 -0 = 10.5 X1 - = 8.0 -0= 8.0 Error Calculation X2 - = 9.5 - 0 = 9.5 X3 - = 9.5 - 0 = 9.5 X4 - = 8.3 - 0 = 8.3 X5 - = 10.5 -0 =10.5 X7 - = 9.5 - 0 = 9.5 X8 - = 9.5 - 0 = 9.5 2 Contestant # 1 try # error X1 (8.0 - 0) = 8.0 X2 (9.5- 0) = 9.5 X3 X4 (8.3- 0) = 8.3 6 8 10

Deviation 11 9 6 7 5 2 2 6 11 6 8 10

Deviation Contestant # 1 1 8.0 2 9.5 3 4 8.3 try # Xi Score One measure of precision is the numerical difference between the observed value, xi , and mean value for that group of measurements. This is known as the deviation. Deviation Deviation Calculations X1 8.0 - 8.8 = - 0.8 = X2 9.5 - 8.8 = 0.7 X3 X4 8.3 - 8.8 = -0.5 11 Contestant # 1 1 8.0 2 9.5 3 4 8.3 try # Xi Score X1 X2 X3 X4 9 6 7 5 2 Note: X1 = 8.83 = 8.8 2 6 11 6 8 10 Mean score for contestant # 1 (8.0+ 9.5 + 9.5 + 8.3) n = (X1 + X2 + X3 + X4) (35.3) 4 (8.83) X1 =

Deviation Contestant # 2 1 10.5 2 3 9.5 4 try # Xi Score X5 X6 X7 X8 11 Deviation Calculations 9 6 7 X5 10.5 - 10.0 = 0.5 X2 = X7 9.5 - 10.0 = - 0.5 X8 9.5 - 10.0 = -0.5 X6 5 2 2 6 Note: X2 = 10.0 8 10 Mean score for contestant # 2 (10.5+ 10.5 + 9.5 + 9.5) n = (X5 + X6 + X7 + X8) (40.0) 4 (10.0) X2 =

Why? Who won? 14th Annual Archery Contest # 2 ? Error and Deviation Accuracy and Precision Contestant # 2 Note: X2 = average score for contestant #2 = 10.0 Average Error Average Deviation 10.0 0.1 14th Annual Archery Contest 10 8 6 5 2 9 7 11 # 2 ? Why is Contestant #1 average deviation = 0? This person did not always hit the exact same spot on the target! Why? Who won? Which contestant is a better archer? # 1 ? Note: X1 = average score for contestant #1 = 8.8 Contestant # 1 Average Error Average Deviation 8.8 0.0 Why are the average scores equal to the average errors? How can we change the average deviation calculation so that a zero value only happens when all the arrows hit the same spot? Does that always happen?

Precision Repeatability Hysteresis (As the idea is used in high tech manufacturing). Repeatability A term often used to indicate if the measuring system, the measuring device, the computer and the person, give the same measurement value each time a measurement is made under the same set of condition. Hysteresis A term used to indicate that a measurement system recording values of temperature, for example, does not give the same temperature value as the temperature is going up when compared to when the temperature is going down.

An Example for the term Repeatability Precision An Example for the term Repeatability In the process of manufacturing 300 or so chip on the surface of each silicon wafer, a set of wafers are placed in a very hot furnace. In fact, the wafers shown entering this furnace will be placed in and out of the furnace at lest 10 time before the chips are completely fabricated.

Precision (Repeatability) The process specialist must operate the furnace computer control system and also analyze the temperature data as the furnace temperature goes up and then down. The wafers enter the furnace when the furnace temperature is at 700 oC. The temperature is then raised slowly a rate of , 5-10 oC per minute, until it gets over 1000 oC.

Precision (Repeatability) The table below shows the performance of two different temperature sensors. Both sensors measure temperature from 700 oC to 1200oC. Either one could be use in the furnace. Which one would you recommend be use as a replacement sensor? Device #1 #2 1010 oC 1011 oC 1012 oC 1009 oC 1013oC 1008 oC 1011 oC 1010 oC 1014 oC 1008 oC Note: Data in table represents 5 repeated measurements for each device. All measurements taken at the same conditions

The table showed the performance of two new temperature sensors taking replicate measurements under the same furnace conditions. Either one could be used as a replacement for the temperature sensor now in use. Which one would you recommend be use as a replacement sensor? Device #1 #2 1010 oC 1012 oC 1013oC 1011 oC 1014 oC 1009 oC 1008 oC The correct one to select as a replacement is the one that is more repeatable. Therefore, the only issue now is figure out how to calculate the repeatability. Repeatability and deviation are calculated in a similar but not identical fashion.

Repeatability The calculation for Repeatability is exactly like the calculation for determining a percent grade on a quiz or test. } { Ggrade = (the grade you got on your test) (the range of possible scores on the test) 100 } { Rrepeatability = (the measured temperature value) (the range of possible temperatures that could be measured) 100

} { } { } { ? Grade ( %) Calculation examples Ggrade = (the grade you got on your test) (the range of possible scores on the test) 100 } { } { 52 points (test #1 Student #2) 53 (test 2, student #1) 54 (test 1, student #1) 100 = 90% 88% ? 87% Ggrade = 60 points Scores from 0 to 60 are possible Student #1 Student #2 Test number 1 54 points 90% 52 points 87% 2 53 points 88% 51 points 55 points 49 points 92% 82% 85% 3 56 points 55 points 93% 92% 4 5

} { } { } { It does not make any difference which sensor is installed. Repeatability (%) Calculation examples } { Rrepeatability = (the measured temperature value) (the range of possible temperatures that could be measured) 100 It does not make any difference which sensor is installed. } { } { 1010 oC (temp 1, Device #1) 1011 oC (temp1, Device 2) 1012 oC (temp 2, Device #1) 100 = 84% ? 84% 84.3% Rrepeatability = 1200 oC Device #1 #2 1010 oC 1012 oC 1013oC 1011 oC 1014 oC 1009 oC 1008 oC Temperatures from 0 to 1200 oC are possible 1 2 3 4 5 Device #1 #2 Replicate measurement 1010 oC 84% 1011 oC 84% 1012 oC 84% 1009 oC 1008 oC 1010 oC 84% 1013 oC 1011 oC 1014 oC 84% 85%

Repeatability } { Rrepeatability = (the measured temperature value) (the range of possible temperatures that could be measured) 100 There are many types of temperature sensors that can be used to measure the temperature inside a very hot furnace. Most of these devices will measure temperature over a shorter range than 0 oC to 1200 oC. When this is the case, the Repeatability calculation is done using a more general rule (equation).

Repeatability } { Rrepeatability = (the measured temperature value) (the range of possible temperatures that could be measured) 100 Many temperature sensor can measure temperature over a shorter range than 0 oC to 1200 oC. When this is the case, the Repeatability calculation is slightly different. } { Rrepeatability = [ (the measured temperature value) - (lowest measurable temperature in range )] (the range of possible temperatures that could be measured) 100

What is the repeatability for the following two temperature devices if they have a temperature measurement range from 700 oC to 1200oC? } { Rrepeatability = 100 (1010 oC - 700 oC) Temp # 1, Device # 1 (1200 oC - 700 oC) 100 } { Rrepeatability = = 1012 oC (temp 2, Device #1) 311 oC (temp1, Device 2) 310 oC (temp 1, Device #1) 62% 500 oC Device #1 #2 1010 oC 1012 oC 1013oC 1011 oC 1014 oC 1009 oC 1008 oC 1010 oC Temperatures from 700 oC to 1200 oC possible 1 2 3 4 5 Device #1 #2 Replicate measurement 62% 1011 oC 62% 1012 oC 62% 1009 oC 1008 oC 1010 oC 62% 1013 oC 1011 oC 1014 oC 63% 62%

Temperatures from 700 oC to 1200 oC possible 3 4 5 Device #1 #2 Replicate measurement 1012 oC 1013 oC 1011 oC 1014 oC 1010 oC 62% 63% 1011 oC 1009 oC 1008 oC 1010 oC 62% Thus when the two sensors have a shorter range, Device # 2 has a more consistence performance.

Accuracy and Precision ( a quick concept review)

Accuracy and Precision ( a quick concept review)

An example for Accuracy or Precision? Accuracy and Precision ( a quick concept review) An example for Accuracy or Precision? An example for Error or Deviation?

Accuracy and Precision ( a quick concept review)

An example for Accuracy or Precision? Accuracy and Precision ( a quick concept review) An example for Accuracy or Precision? An example for Error or Deviation?

An example for Accuracy or Precision? Accuracy and Precision ( a quick concept review) An example for Accuracy or Precision? An example for Error or Deviation?

Accuracy and Precision ( a quick concept review)