1. A Prophet’s Guide to Forecasting
A guide to forecasting at scale – Rafi Adler and Dan Robinson

2. A little bit about us: We are Marketing Analysts at BookingGo.
We Specialise in Paid Search Marketing aka PPC.
Thanks to Chris Campbell, we are both certified R experts.

3. What is Paid Search Marketing?
𝐶𝑜𝑠𝑡𝑃𝑒𝑟𝐶𝑙𝑖𝑐𝑘= 𝑇𝑜𝑡𝑎𝑙 𝐴𝑑 𝑆𝑝𝑒𝑛𝑑 𝑇𝑜𝑡𝑎𝑙 𝐶𝑙𝑖𝑐𝑘𝑠
𝐶𝑙𝑖𝑐𝑘𝑉𝑎𝑙𝑢𝑒= 𝑇𝑜𝑡𝑎𝑙 𝑅𝑒𝑣𝑒𝑛𝑢𝑒 𝑇𝑜𝑡𝑎𝑙 𝐶𝑙𝑖𝑐𝑘𝑠
A show of hands for Paid Search Marketing?

4. How can we Forecast Click Value
TREND
𝒚(𝒕)
SEASONALITY
ERRORS
Who here uses Google?

5. What is Prophet? Developed by Facebook Became Open Source Early 2017
Initially developed to forecast Facebook users across many different countries and demographics.
One of its core goals was to improve forecasting at scale.

6. So what is the Forecasted Click Value
Model creates the model object
Make future data frame creates the list of dates it’s going to predict. Here it also learns the periods for seasonality.
Predict is where we create a dataframe called forecast. This is where we have columns for seasonality / Confidence intervals and all the things you’d want to for a forecast.

7. Benchmarking How do we know our forecast is any good?
Use Cross-Validation
Calculate the Error (MAPE = Mean Absolute Percentage Error)
Compare it to another ‘basic’ forecast (eg Naïve, Moving Average…)

8. Benchmarking Model Error Prophet 6.51% Naïve 8.46% TBATS 6.87%
How accurate is my forecast?
Prophet 6.51% - But is this actually good?
Without context we don’t really know.
TBATS industry standard so is a good barometer.
T for trigonometric regressors to model multiple-seasonalities
B for Box-Cox transformations
A for ARMA errors
T for trend
S for seasonality

9. Clash of the Titans! Vs. Under constant development
Rob J Hyndman:
Developed the forecast package
Author of the blog ‘Hyndsight’
Vs.
Under constant development
Under constant development
Runs a forecast in < 1 second
X Runs a forecast in ~2 minutes.
Ability to tweak parameters
Ability to tweak parameters
Deals well with missing data
X Fails to run with missing data
Easy to scale for a business
X Challenging to scale for a business

10. Forecasting at Scale
Forecasting an individual market is all well and good..
BUT….
What about when you have 100s of markets to manage??
Each market has it’s own trend and seasonality
We want to produce or model for each one.

11. Forecasting at Scale There’s a Hadley video for that!
“Managing Many Models in R”
Like anything worth knowing in R, there’s a Hadley video for that.
If you don’t know who this is, Hadley Wickham is the Chief Scientist at Rstudio.
He’s responsible for the tidyverse (ggplot, dplyr).
He also gives amazing talks like this one, which recommend you all watch.

12. Forecasting at Scale Two Ideas: Nested Data Functional Programming
The talk explains a really useful trick for managing lots of different models
2 ideas (each is coupled with a package).
Nested data – Make a data frame where one of the columns is a list of data frames.
Functional Programming – An alternative to doing For Loops.

13. Forecasting at Scale Our raw data has one row per observation
Market
Date
Click Value
Australia
£1.12
£1.26 …
France
£0.91
Germany
£1.90
Our raw data has one row per observation
It’s better if we have one row per market!

14. Forecasting at Scale nest( ) Market Date Click Value Australia
£1.12
£1.26 …
France
£0.91
Germany
£1.90
Market
Data
Australia
<df>
France
Germany ….
nest( )

15. Forecasting at Scale We now have a nested data frame!
Date
Click Value
£1.12
£1.26 …
We now have a nested data frame!
Every entry of the Data column is a data frame.
Market
Data
Australia
<df>
France
Germany …
Date
Click Value
£0.91
£1.01 …

16. Forecasting at Scale Now let’s add a column for our forecasted data.
Market
Data
Fitted Model
Australia
<df>
France
Germany ….
Now let’s add a column for our forecasted data.
Market
Data
Australia
<df>
France
Germany ….
map( )

17. Forecasting at Scale map( ) Date Click Value Fitted Value 2015-01-05
£1.12
£1.15
£1.26
£1.20 ….
Market
Data
Fitted Model
Australia
<df>
France
Germany ….
Market
Data
Australia
<df>
France
Germany ….
map( )
Date
Click Value
Fitted Value
£0.91
£0.95
£1.01
£0.97 ….

18. Forecasting at Scale map( ) Market Data Fitted Model Australia
<df>
France
Germany ….
Market
Data
Australia
<df>
France
Germany ….
map( )

19. Life isn’t always easy…

20. Additional Parameters - Saturating Caps
Saturating caps are useful when data has max / min achievable point.
EG Number of Facebook users
In our case we want a floor of 0
The trend component is then calculated using a logistic model instead of piecewise linear.

21. Additional Parameters - Saturating Caps
Saturating caps are useful when data has max / min achievable point.
EG maximum population size.
In our case we want a floor of 0
The trend component is then calculated using a logistic model instead of piecewise linear.

22. In Summary Predicting the future is hard
Always benchmark against other methods
Forecasting at scale is easy using nested data and purrr
When you have a poor forecast, there are many tweakable parameters (eg saturating caps, changepoint prior) and more!
Facebook are terrible with privacy but are brilliant at forecasting!

23. Questions?

24. Further Reading
Official Prophet ‘Quick Start’:
Forecasting at Scale:
Generalised Additive Model (GAM):