1. Ifo Institute for Economic Research
Effectiveness of the Visual Analog Scale for the Measurement of Business Expectations
Anna Stangl
Ifo Institute for Economic Research
Brussels, 12. October 2009

2. Visual Analog Scale (VAS) in Web Surveys
3-Category Scale.
Visual Analog Scale (VAS)
Present business situation
Business expectations
Ifo Economic Climate Index

3. Motivation: Measurement of Expectations
Qualitative measurement of economic expectations: category-rating scales
limited, coarse data
information loss in the neutral category
no information on dispersion
strong assumptions for modeling
easy to apply
reliable measurement
Quantitative measurement of economic expectations: point forecasts
costly and time-demanding
prone to inaccuracies
not available in business surveys
interval-scale measurement
information on dispersion and the shape of the distribution
Probabilistic expectations
theoretically appealing
information on dispersion and the shape of the distribution
information on uncertainty
applicable in surveys of professionals
general tendency of respondents to
be optimistic / further drawbacks
not available in business surveys

4. Categorical Scale vs. Visual Analog Scale
Symmetric properties of the indifference interval are assumed
Information loss due to the central tendency of responses
Score range: 1 to 3
Economic confidence
bad
good
satisfactory a b I.
II.
This country's overall economic situation at present?
This country's overall economic situation at present?
Economic confidence I.
bad
satisfactory
good
Score range: 1 to 100
Enables scores between categories
Shows direction of change and magnitude
Delivers dispersion and uncertainty measures

5. Categorical Scale vs. Visual Analog Scale
3-Category Scale
Visual Analog Scale
bad
satisfactory
good
Uncertainty

6. Data Internet Business survey in manufacturing, Germany
Number of responses

7. Observation period

8. Scale Reliability Tests
Parallel-form reliability
Test-retest reliability
Internal consistency
Inter-rater reliability
VAS
3-Cat.
bad
satisfactory
good 1. 2.
1. Step: Common factor of „econ.
situation“ items extracted
2. Step: Correlation of the common
factor with VAS / 3-Cat.

9. 3-Category and VAS Business Expectations and Production Index
3-Category Business Climate
VAS Business Climate

10. Measures of Uncertainty and Heterogeneity
Direct survey-based measures: probability distribution of a point forecast
uncertainty of the individual forecaster
responses are comparable
internal consistency and accuracy
surveys of professionals
not available in business & consumer surveys
Time-series based measure of uncertainty: forecast errors
accuracy
not available in business and consumer surveys
proxy for uncertainty
readily available
Dispersion as a proxy for uncertainty (Zarnowith and Labros, 1987)
readily available
proxy for uncertainty
important in its own right as measure of heterogeneity
quantitative point-forecasts
not available in business and consumer surveys

11. Measuring Uncertainty and Heterogeneity with VAS
“Epistemic uncertainty”
Bruine de Bruin et al., 2000
Dispersion of business expectations (Heterogeneity)
Zarnowith and Labros, 1987, Bomberger, 1996, Batchelor and Dua, 1996,
Giordani and Soderlind, 2003, Rich and Tracy, 2006, Mitchel et al., 2005,
Lahiri and Liu, 2006, Doepke and Fritsche, 2006, Boero et al., 2007…
3. Kurtosis
Doepke and Fritsche, 2006: A significant kurtosis above 3 indicates
that the forecasters are very close to each other.
4. Skewness
Doepke and Fritsche, 2006: If the distribution of economic expectations is
significantly skewed, consensus among forecasters is rejected

12. Growth fall / turning point
Dispersion of VAS Business Expectations and the Production Index
Growth fall / turning point
Correlation: -0.84

13. Measuring Uncertainty with VAS
Correlation: 0.80
Correlation: 0.72

14. Skewness of the VAS Present Business Situation

15. Skewness of the VAS Business Expectations

16. Skewness of the VAS Business Expectations and Production Index

17. Correlation with Production Index in Manufacturing
Correlation of the VAS Indicators
with the Production Index in the German Manufacturing Sector
Lead in months 1 2 3
VAS Climate
0.94
0.95
0.93
0.89
3-Cat. Climate
0.92
0.95
0.95
0.93
VAS Expectations
0.90
0.94
0.95
0.94
3-Cat. Expectations
0.74
0.84
0.90
0.93
VAS Situation
0.94
0.92
0.88
0.83
3-Cat. Situation
0.93
0.91
0.87
0.81
Skewness of VAS situation
0.93
0.93
0.91
0.88
Skewness of VAS expectations
0.65
0.72
0.77
0.82
Epistemic uncertainty
0.56
0.62
0.66
0.61
Standard deviation of VAS expectations
0.84
0.82
0.78
0.76
Kurtosis of VAS expectations
0.90
0.89
0.88
0.86

18. Summary: Effectiveness of the VAS
Dispersion and Kurtosis of business expectations
- Contra-cyclical
- Proxy for uncertainty and heterogeneity
“Epistemic uncertainty” (Bruine et al. definition)
- Increases around turning points
- Proxy for uncertainty
Skewness of the VAS distribution
- Cyclical
- More pronounced in VAS business situation

19. The Present Study Adds to the Literature:
Dispersion as proxy of uncertainty was previously calculated only from point forecasts
Presents a meaningful dispersion measure based on qualitative expectations
There was no direct uncertainty measure derived from qualitative responses
Presents a meaningful measure of “epistemic” uncertainty without
eliciting probabilistic forecasts
Business expectations were assumed to be normally distributed
Demonstrates systematically variable skewness of business
expectations over the business cycle

20. General Summary
VAS is easy to apply and does not require any quantitative information
VAS presents a direct, almost interval scale measure of economic expectations
VAS delivers reliable and valid information
VAS is highly efficient, delivering a variety of valid economic indicators
- Dispersion measures
- Uncertainty proxies
- Distributional shape
- Enables differentiation between uncertainty and heterogeneity
Assumptions of the three-category expectations are systematically violated
VAS is an effective instrument for the measurement of business expectations

21. Research outlook Gathering longer time-series
Improving the business cycle forecasts with the information on uncertainty and heterogeneity of expectations (out-of-sample forecasts)
VAS measurement of consumer expectations
Application of VAS batteries in surveys, analysis of drop-outs

22. THANK YOU!