1. Workshop on Price Index Compilation Issues February 23-27, 2015 Imputation of Missing Values, Seasonal Products and Quality Changes Gefinor Rotana Hotel, Beirut, Lebanon

2. Lecture Outline Introduction Imputation Techniques Treatment of Seasonal Commodities Imputation Techniques for QA

3. Causes of missing prices Samples not universal coverage Temporarily unavailable supply shortages Seasonally unavailable Permanently unavailable

4. Current practices and their implications Take no action: samples deteriorate bias may be introduced Ensure the observations are comparable (matched between periods) Adjust weights to account for missing data Carry forward the last price can result in bias particularly during inflationary periods Impute prices for missing observations

5. Imputation Techniques Estimate the price change of comparable observations in the item category Move the previous price forward

6. Example 1 Imputing a Missing Month’s Value using the Jevons method

7. Imputation Techniques Estimate a missing current price from the short-term change in average prices of other observations Mark the estimated price as “imputed” If price is missing for several months chain its imputed prices until a replacement is found. See Example 2

8. Example 2 Imputing Multiple Month’s Missing Prices

9. Seasonal Items and Prices “ Hard” Seasonality Items missing from markets during part of year At the start of a new season changes in quality introduction of new products At the end of season discounting “Soft” Seasonality Seasonal price movement of year-round items Varying seasonal patterns from year to year

10. Treatment of Seasonal Commodities The index must reflect the price swing from:the last price (of old season) to: the first price (of new season) Impute missing prices of out-of-season observations prevents systematic bias Particularly when new products are introduced at the beginning of season

11. Other techniques for seasonal items Variable weights May not give reliable results Requires detailed monthly data for several years on the quantity of each item sold Rolling average: extrapolate recent 6-months Seasonal factors Month-on-(previous) 12-month figure OK. Do NOT use carry forward

12. Seasonal Clothing Extreme price movement End of season discounts Multiple selling seasons: fall/winter spring/summer Different varieties for each selling season Do not replace variety once price discounting begins

13. Seasonal Clothing After selling seasons and variety is no longer available, return the price to the last “normal” price begin imputation in the following period When the selling season returns the next year same product may not appear again select the most comparable replacement product (requires maintaining detailed specifications)

14. Quality Adjustment (QA) Direct comparisons between old and new item Link (splice) new item into the index Drop observation from index calculation

15. Alternative QA Techniques Direct Quality Adjustment data collector or analyst knowledge, information from producers, or hedonic regression models Indirect Quality Adjustment with Imputation overlap price available overall mean imputation class mean imputation

16. Example 3—Direct Quality Adjustment

17. Example 4—Indirect QA: Overlap Pricing

18. Example 5—Indirect QA: Overall mean imputation

19. Example 6—Indirect Q A: Class mean imputation

20. Effect of QA Imputation on Price Change

21. Recommendation For Missing Price Observations Always impute prices of missing items Use imputed prices in the next pricing period (as the previous period prices) Even in periods for which there are no price observations in the item group at all.

22. Recommendation For Missing Price Observations (con’t.) The “carry forward” method Use only when it is fairly certain that the price remains the same. Can generate undue stability in the index.

23. Recommendation For Imputing Missing Price Observations For temporarily missing products use one of the imputation methods (they are self-correcting) For products permanently missing select a replacement If replacement is comparable, use directly If replacement is noncomparable use overlap price or indirect quality adjustment

24. Thank you