1. Analysing the reliability of forecast information provided by UNECE member states
Master’s thesis by Markus Stolze

2. Who am I? And what am I doing here? Name: Markus Stolze From Finland
Forestry student in University of Helsinki
M.Sc. in Forest economics 6/2019
I was working summer 2018 here with FPAMR
As part of my master’s thesis conducting research for UNECE about reliability of forest products forecast

3. United Nations Economic Commission for Europe
What is UNECE?
Organisation founded in 1947 and has currently 56 member states
The UNECE region covers more than 47 million square kilometres, around 17% of world’s population and more than 40% of world’s forests

4. Background of the research
And data used in the research
Data used in this research is provided by member states of UNECE
JFSQ data collected each year by Forestry and Timber section in Geneva
Forecast data of forest products have been collected in database since 2002
Estimates (year n) and forecasts (year n+1) are produced annually
Haven’t been used yet, as nobody knows how reliable it is
Past study (2005) covered one product
Research was done in co-operation with UNECE and Department of Forestry in University of Helsinki

5. Objective of research & Research Questions
Objective of this research is to understand how reliable these predictions are
As part of thesis I will also see, what are the elements that affect data quality
Are some products more reliable than others?
Goal of prediction is to be as similar as possible to JFSQ-value provided
More specifically, this study aims to answer following questions:
What are the main elements of forest product market data quality?
Are forecast data about forest product markets presented by UNECE reliable? And if so, how reliable?

6. What products are included
All together 12 products were chosen
8 are forest products and 4 are removals
Product
JFSQ-code
HS2012-code
What information is collected 
Coniferous sawlogs and veneer logs
1.2.1.C
Only harvest
Non-Coniferous sawlogs and veneer logs
1.2.1.NC
Coniferous pulpwood
1.2.2.C
Non-Coniferous Pulpwood
1.2.2.NC
Coniferous sawnwood
5.C
Exports, Imports and production
Non-Coniferous sawnwood
5.NC
/22/25/26/27/28/29/91/92/93/94/95/99
Plywood
6.2
/32/39/94/99
Particle board (including OSB)
6.3
44.10
OSB
6.3.1
Fibreboard
6.4
44.11
Wood Pulp 7
47.01/02/03/04/05
Paper and Paperboard 10
48.01/02/03/04/05/06/08/09/10, / /13

7. How is the analysis structured
And other information about research
Countries with response rate for estimates and forecasts of 66.6% are included in the research
Out of possible 1116 data points (12 products & 15 years) 744 data points have to be filled actual estimates or forecasts
This allows to observe real trends from year to year
27 countries are included in the research
Estimates and forecasts are compared to historical data to see how far off they are
A repeated estimate is created by repeating historical data from prior year (year n- 1) and comparing it to historical outcome (year n)
Analysis includes all products, flows (imports, exports, production, removals) and countries, that have a high enough response rate
Results presented are averages as %-difference from historical value of that year
E.g. forecast of 500 is 10% smaller than historical value of 550

8. Results Part 1: Products Product Product code Estimate Forecast
Repeated
Paper and paperboard 10
10.0%
15.4%
8.6%
Coniferous sawlogs and veneer logs
1.2.1.C
12.4%
16.8%
11.5%
Coniferous sawnwood
5.C
12.6%
21.3%
13.4%
Particle board (including OSB)
6.3
13.5%
21.1%
14.7%
Fibreboard
6.4
18.4%
37.2%
25.5%
Coniferous pulpwood
1.2.2.C
19.6%
25.1%
15.3%
Non-Coniferous pulpwood
1.2.2.NC
22.5%
29.8%
Plywood
6.2
39.8%
25.4%
Non-Coniferous sawlogs and veneer logs
1.2.1.NC
27.5%
36.0%
22.0%
Non-Coniferous sawnwood
5.NC
28.1%
46.9%
18.0%
Wood Pulp 7
53.4%
70.9%
71.0%
OSB
6.3.1
56.0%
69.1%
42.2%
Averages
24.9%
35.8%
23.8%

9. Results
Part 2
As seen, repeated data are better than estimates 7 out of 12 times
There are clear differences between different products
Products with bigger volumes are more accurate
If 5 countries with lowest accuracy are ignored:
Estimates: 24.9% -> 14.6%
Forecasts: 35.8% -> 20.6%
Repeated: 23.8% -> 12.9%

10. Rank (based on estimate)
Results
Part 3: Countries
As expected, there are big differences between countries
Below is a table with averages from all years, products and flows
Countries with large volumes are more likely to be more accurate
Rank (based on estimate)
Country
estimate
forecast
repeated 1
Austria
6.3 %
9.3%
7.2% 2
Poland
7.8%
11.4%
7.7% 3
France
9.5%
18.3%
7.1% 4
Germany
9.8%
11.9%
9.2% 5
United States
9.9%
13.9%
7.4% 6
Canada
13.3%
15.6% 7
Italy
14.0%
22.5% 8
Lithuania
21.8%
16.1% 9
Spain
14.8%
32.9%
18.1% 10
United Kingdom
15.4%
30.9%
16.8%

11. Results Part 4: Estimates or repeated data
When all countries, products and flows are added together for products, repeated data are the most accurate 172 times, while estimates are 138 and forecasts 11 times
Product
Product code
Estimate
Forecast
Repeated
Paper and paperboard 10 16 11
Coniferous sawlogs and veneer logs
1.2.1.C 2 14
Coniferous sawnwood
5.C 12
Particle board (including OSB)
6.3 6 1 18
Fibreboard
6.4 13
Coniferous pulpwood
1.2.2.C 9
Non-Coniferous pulpwood
1.2.2.NC 7 20
Plywood
6.2 4
Non-Coniferous sawlogs and veneer logs
1.2.1.NC 15
Non-Coniferous sawnwood
5.NC
Wood Pulp
OSB
6.3.1
Total
138
172

12. Results
Part 5: Flows
Out of 4 flows, production is the most accurate, followed by removals, imports and exports
Removals were the only flow, where estimates were the most accurate
Exports and imports are more likely to be affected by outside influences

13. Conclusion What to make of all this
53.6% of times repeated data are more accurate than others
As repeated data were the most accurate 172 times when all products and countries were added up together
If nothing unexpected happens, estimates are rarely better than repeated data
Forecasts suffer from unexpected events more than others
Financial crisis is clearly visible in the results
Large volumes indicate accurate forecasts, both with countries and products
There are some exceptions to this rule: E.g. Wood Pulp

14. How different countries produced forecasts
Finding similarities between countries
In addition to percentage values, also absolute values were calculated
Different flows were transformed into graphs and visually compared
All flows, exports, imports, production and removals
Countries were divided into 4 groups based on how graphs of different flows looked like and selected countries interviewed
Main methods included talking with outside experts and using data from first 2 quarters of each year
Usually 1 or 2 people are making the final decisions
No clear connection between different groups of countries and methods were identified

15. Limitations of the research
And suggestions for the future
Historical data could be edited afterwards, if new information was provided
This creates a possibility that repeated data looks better than it is…
…and estimates/forecasts are not compared to ”original version” of historical data
E.g. removals from the Russian Federation, where forecasts are 90% off from actual values
If research is re-done in the future, unedited historical data should be used
Products with small volumes often have big effect on overall results
Drop of 2 to 0.1 is 200% difference, but doesn’t matter in the reality
Use of time series analysis could offer new information
Deeper dive into reasons why some products are harder to predict

16. Acknowledgements
I would like to thank following parties for crucial help with this project
UNECE
FAO
University of Helsinki

17. Markus Stolze
University of Helsinki
26 March 2019, Geneva