1. Tracking Signal A Measure of Forecast Accuracy
Tyler Hedin
Brigham Young University
Business Management 361, Section1
Dr. Tom Foster
December 14, 2006
Prepared by:
Tyler Hedin

2. Agenda Tracking Signal Defined Application, Advantages & Disadvantages
Tracking Signal and Forecasting
Application, Advantages & Disadvantages
How it works
Step by step formula
Company XYZ Example
Exercise
Summary
Readings list
Useful websites
Appendix A
The objective of this presentation is to provide a basic explanation of a tracking signal as an accuracy measure for forecasting and describe its application through a realistic example and interactive exercise.
Prior knowledge of forecasting methods and forecasting in general is assumed.

3. What is Tracking Signal?
A measure that indicates whether the forecast average is keeping pace with any genuine upward or downward changes in demand*
*definition taken from the glossary of Operations Management for Competitive Advantage, see Readings List for reference
A tracking signal is a tool used to measure how well an individual forecasting method is performing.

4. Tracking Signal and Forecasting
Continuous control indicator
Monitor effectiveness of forecasting method
Provide control limits
A tracking signal is used as a continuous indicator of the forecasting method in order to determine if the established method needs modification. Its objective is to enhance the effectiveness of the forecasting effort by providing control limits. These limits are set by the organization and will be discussed later. A tracking signal that goes outside of established limits is an indicator that the forecasting method being used should be modified.
Graphic obtained from

5. Application Evaluates forecasting method
Indicator of change in demand patterns
Used in conjunction with anything dependent on future demand
Sales
Inventory
Because a tracking signal requires that actual data be compared with previously made forecasts, it is not normally used in deciding between forecasting methods, but rather in evaluating forecasting methods that have already been established. By nature, a forecast is based on historic patterns and a tracking signal is a common method for determining whether or not a pattern in demand has changed.
A tracking signal can monitor any forecasts that have been made by an organization regarding sales, inventory, or anything pertaining to the organization’s future demand.

6. Advantages Unbiased Versatile
Can be used with any type of forecasting method (time series, regression line, etc.)
Some measures of forecast accuracy, such as MAD, measure the average size of the forecast error and do not take into account whether the forecast is too high or too low. An important component of a tracking signal is a continuous sum of the differences between forecasted and actual values, so as to measure whether a forecast is consistently too high or too low. If a forecast is consistently producing values that are too high or too low, a tracking signal will designate the forecasting method to be out of control, similar to the function of quality control charts.
A tracking signal is also very versatile in that it can be used with a variety of forecasting methods. The only required numbers are an actual demand value and its corresponding forecasted value.

7. Disadvantages Could wrongfully flag perfect forecasts
Unlikely
Small differences in the same direction could cause signal to go outside of control limits
The nature of a tracking signal assumes the existence of error. Though unlikely, a tracking signal could flag a forecasting method that is producing values that are very close to actual demand levels. As will be shown later, a tracking signal is calculated by dividing the most recent sum of errors by the most recent estimate of the mean absolute deviation (MAD). If forecasts are close to actual values, the mean absolute deviation would approach zero while the sum of the errors remained unchanged, launching the tracking signal towards infinity.
Comparable to quality control charts, small deviations in the same direction over time could falsely indicate that the forecasting method is out of control. A closer look at forecast values and actual data following a tracking signal outside of designated limits would rectify this problem.

8. How it Works – Forecast Error
Difference between actual demand and forecast
The first step in calculating a tracking signal is to find the forecast error. The forecast error is found by simply subtracting the forecasted value from the actual demand value. Forecasts that are below the actual demand values will have a positive forecast error and forecasts that are above the actual demand values will have a negative forecast error.
Week
Actual Demand
Forecasted Demand
Forecast Error 1 21 19 2 25 22 3 24 -2

9. How it Works – Absolute Values
Express the forecast errors as absolute values
Week
Actual Demand
Forecasted Demand
Forecast Error
Absolute Value 1 21 19 2 25 22 3 24 -2
After finding the forecast errors, express these errors in there absolute values. These values will be used in a later step to calculate the mean absolute deviation (MAD).

10. How it Works – Running Sum
Keep a continuous running sum of the forecast errors
Do not add absolute values
A running sum of the forecast errors is kept with each forecasted value. This running sum is cumulative and is found by summing the value of the forecast errors from the first period to the most recent period. You should make sure to not sum the absolute values found in the previous step.
Week
Actual Demand
Forecasted Demand
Forecast Error
Absolute Value
Running Sum 1 21 19 2 25 22 3 5 24 -2

11. How it Works - MAD
Divide the summed absolute values by the number of periods to calculate MAD.
Week
Actual Demand
Forecasted Demand
Forecast Error
Absolute Value
Running Sum
MAD 1 21 19 2
2.00 25 22 3 5
2.50 24 -2
2.33
The next step is to calculate the mean absolute deviation. Calculating MAD is done by dividing the sum of the absolute values found in the second step by the number of periods in the data set. This calculation must be done on a period by period basis in order for a tracking signal to be created. For each row, sum the absolute values for each period before it and divide that number by the period indicated by that row. For example, to calculate the MAD for row number three in the table above, one would need to sum the absolute values from the third row and each row that came before it (2+3+2) and divide that number by the total number of periods to that point (3).
The MAD is a measure of the average error size for the forecasts and indicates the average deviation from actual demand. Because the MAD is calculated with absolute values, it does not take into account the direction of those deviations (i.e. whether they are above or below actual demand).

12. The Equation
Tracking signal is mathematically defined as the sum of the forecast errors divided by the mean absolute deviation
Tracking signal =
 (Dt – Ft)
MAD
Once the MAD is calculated for each period, you are able to find the tracking signal. A tracking signal is found by dividing the most recent sum of forecast errors by the most recent estimate of MAD.

13. How it Works – Tracking Signal
Divide the running sum of forecast errors by the corresponding MAD value
As an example, the tracking signal for week 3 in the table above is found by dividing the running sum of errors as it stands in week 3 (3) by the estimate of MAD in week 3 (2.33). This value comes out to be 1.29.
Week
Actual Demand
Forecasted Demand
Forecast Error
Absolute Value
Running Sum
MAD
Tracking Signal 1 21 19 2
2.00
1.00 25 22 3 5
2.50 24 -2
2.33
1.29

14. What Do These Values Mean?
Ratio of cumulative error to average deviation
0.8 σ ~ 1.25 MAD
Limits are usually between 2 to 5 standard deviations
Because a tracking signal is a ratio of the cumulative error produced by individual forecasts to the average deviation of those forecasts from their actual values, it typically “tracks” forecasts to see if they stay within a similar pattern to the corresponding demand. Companies often use the relationship between MAD and the standard deviation to establish limits for the tracking signal. Because 1 standard deviation is approximately equivalent to 1.25 MAD, a common boundary of 3 standard deviations is used (or 3.75 MAD). With the error being measured in MAD units, one can statistically determine the probability that the forecasts errors are attributable to random variation. Typically, a tracking signal that stays between 3 standard deviations is considered enough room for error. If the tracking signal is outside three standard deviations, the probability of random variation is low. For the purposes of this tutorial, limits for the tracking signal will be set at -4 and 4.

15. Example 1
Company XYZ has implemented a linear regression method to forecast sales. Actual sales for the months of January 2005 through January 2006 are given in Table 1 along with their corresponding forecasts.
This hypothetical example will illustrate a tracking signal using the step by step method described earlier in the tutorial.

16. Table 1 SALES (in thousands) MONTH PERIOD DEMAND FORECAST January-05 1
SALES (in thousands)
MONTH
PERIOD
DEMAND
FORECAST
January-05 1
$37
37.35
February-05 2
$40
38.97
March-05 3
$41
40.60
April-05 4
42.23
May-05 5
$45
43.85
June-05 6
$50
45.48
July-05 7
$43
47.10
August-05 8
$47
48.73
September-05 9
$56
50.36
October-05 10
$52
51.98
November-05 11
$55
53.61
December-05 12
$54
55.23
January-06 13
56.86
The forecasts in Table 1 were found using a linear regression line with the equation y= x. The units for the sales values are in thousands of dollars.

17. Example 1
Company XYZ would like to employ a tracking signal to measure the performance of its forecasting method.
This example assumes that a tracking signal was not put in place by Company XYZ while actual data was being collected. A tracking signal is being added later in order to better illustrate how a tracking signal is calculated.

18. Table 2 TRACKING SIGNAL -1.00 0.99 1.82 -2.37 -1.84 0.72 -1.08 -1.86
0.50
0.56
1.18
0.68
-0.16
MONTH
PERIOD
DEMAND
FORECAST
ERROR
ABS DVN
RUNNING SUM
MAD
TRACKING SIGNAL
January-05 1 37
37.35
-0.35
0.35
-1.00
February-05 2 40
38.97
1.03
0.68
0.69
0.99
March-05 3 41
40.60
0.40
1.08
0.59
1.82
April-05 4
42.23
-5.23
5.23
-4.15
1.75
-2.37
May-05 5 45
43.85
1.15
-3.00
1.63
-1.84
June-05 6 50
45.48
4.52
1.52
2.11
0.72
July-05 7 43
47.10
-4.10
4.10
-2.58
2.40
-1.08
August-05 8 47
48.73
-1.73
1.73
-4.31
2.31
-1.86
September-05 9 56
50.36
5.64
1.33
2.68
0.50
October-05 10 52
51.98
0.02
1.35
2.42
0.56
November-05 11 55
53.61
1.39
2.74
2.32
1.18
December-05 12 54
55.23
-1.23
1.23
1.51
2.23
January-06 13
56.86
1.86
2.20
-0.16
All of the computations that were necessary in finding the values in Table 2 were performed in Microsoft Excel. As you can see, the tracking signal for this example stays well within the limits of -4 and 4 and should indicate that the forecasting method being used by Company XYZ is following demand patterns closely enough for the time being.

19. Exercise
Your employer, Jones & Associates, has been using a linear regression method to forecast sales for After nine months have passed and actual sales data have been collected, your boss asks you to develop a tracking signal to measure the accuracy of the forecasts. The data for actual sales and forecasted sales is in Table 3.
This exercise is meant to be performed by the reader. If feasible, the reader should perform necessary calculations in a Microsoft Excel worksheet. The solution to the exercise is given in Appendix A at the end of the tutorial.

20. Table 3 SALES MONTH PERIOD DEMAND FORECAST January-06 1 $3,769 3664.18
SALES
MONTH
PERIOD
DEMAND
FORECAST
January-06 1
$3,769
February-06 2
$3,912
March-06 3
$4,212
April-06 4
$4,861
May-06 5
$4,672
June-06 6
$4,937
July-06 7
$5,346
August-06 8
$5,783
September-06 9
$6,021
The data in Table 3 is meant to be used to establish a tracking signal following the steps outlined earlier in the tutorial.

21. Summary
A tracking signal statistically determines if a forecasting method is out-of-control.
As long as tracking signal stays within 3 standard deviations, probability of forecast error caused by random variation is high
Used by companies to track changes in demand patterns
Calculated by dividing the most recent sum of forecast errors by the most recent estimate of MAD
A tracking signal outside of established limits indicates that a forecasting method should be modified.
Compatible with any forecasting method

22. Readings List
Chase, R. B. et al. (2004). Operations Management for Competitive Advantage 10th edition. McGraw-Hill Higher Education.
Duncan, Robert M. (1992). Quality Forecasting Drives Quality Inventory at GE. Industrial Engineer, January edition.
Hanke, J.E. & Wichern, D. W. (2004). Business Forecasting. Prentice Hall.
Lawrence, F. B. (1999). Closing the logistics loop: A tutorial. Production & Inventory Management Journal, 40(1).

23. Useful Websites http://www.bestforecastingsoftware.com
Information on forecasting and tracking signal can be found in most textbooks dealing with operations management or supply chain management. Also, many websites offer forecasting software, such as and

24. Appendix A – Solution to Exercise
MONTH
PERIOD
DEMAND
FORECAST
ERROR
ABS DVN
RUNNING SUM
MAD
TRACKING SIGNAL
January-05 1 37
37.35
-0.35
0.35
-1.00
February-05 2 40
38.97
1.03
0.68
0.69
0.99
March-05 3 41
40.60
0.40
1.08
0.59
1.82
April-05 4
42.23
-5.23
5.23
-4.15
1.75
-2.37
May-05 5 45
43.85
1.15
-3.00
1.63
-1.84
June-05 6 50
45.48
4.52
1.52
2.11
0.72
July-05 7 43
47.10
-4.10
4.10
-2.58
2.40
-1.08
August-05 8 47
48.73
-1.73
1.73
-4.31
2.31
-1.86
September-05 9 56
50.36
5.64
1.33
2.68
0.50
October-05 10 52
51.98
0.02
1.35
2.42
0.56
November-05 11 55
53.61
1.39
2.74
2.32
1.18
December-05 12 54
55.23
-1.23
1.23
1.51
2.23
January-06 13
56.86
1.86
2.20
-0.16
The tracking signal is well within the limits of -4 and 4, signifying that the linear regression forecast model is correctly predicting Jones & Associates’ pattern of sales demand, within reason.