the amazing science and delicate art of forecasting

The Amazing Science and Delicate Art of Forecasting

Cyrille Betant, March 2015

Summary1. The notion of forecasting

2. The mechanical tools for forecasting

3. Scaling your forecast

4. Looking beyond the numbers

5. Communicating the forecast

6. Rolling forecast

7. The value of crowd forecasting

8. Important factors to remember


1. The Notion of Forecasting

What is a forecast?


Forecasting, an Overview and Brief Definition

If the plan is your destination,…

…the forecast is the road leading to it.

- Creating a forecast is literally engineering the road that is going to lead you to your target goal, your destination.

- That road, however, will have to take into account the landscape it goes through and may very well change as your project unfolds.

- To know what the road looks like, you need to know where you are and where you are going.

- Hence, the forecast is the financial expression of what you anticipate will happen in your operations on top of what you know already happened.

- This is a representation of what your financials ar e going to look like at the end of your forecasting horizon (whatever your budget is ), where the road leads.


In Short

From the dawn of humanity, a disease, with one of the highest rate of mortality ever, has been plaguing every human life and every human endeavor:

Wishful Thinking

Whether the cave man thinking that they can outrun that nasty saber-tooth tiger…

…or the financial trader thinking that the market will continue to go up forever,

all are suffering from the same devastating disease that only a good forecast can cure.


The 5 Key Questions

In essence, to create a forecast, you need to answer these five key questions:

1) Who

2) What

3) When

4) Where

e.g. through its agreement with pharmacy distributors

e.g. the consumer product division

e.g. will introduce a new toothpaste

e.g. next quarter

e.g. in all major North American markets

5) How


Rules of the RoadMany of your forecasters will not have a finance education or background and might be unfamiliar with what the forecast exercise is really about. It is always

good to remind them these basic tenets:

A Forecast IS A Forecast IS NOT

- A candid view of your operational results

- An opportunity to re-negotiate your budget

- A way to hold on to previously planned spending

- A prediction of what your financial results are likely to be

- An opportunity to assess actions needed to achieve your

budget goals

- A change in your budget goals

- A mechanism to voice as early as possible operational

risks in your plans

- A tool to massage your numbers for messaging

purpose


2. The Mechanical Tools for Forecasting

How do you create a forecast?


Extrapolation, Projection and ForecastDifferent operational or tactical needs call for different tools and processes to project future results with the accurate granularity.

An extrapolation is the prediction of a future state where every element known today to be relevant to that prediction will remain unchanged.

A projection is the prediction of a future state where elements known today to be relevant to that prediction will change.

A forecast is the prediction of a future state where elements known today to be relevant to that prediction, as well as new elements will change.

This is if nothing at all changed, usually not a li kely scenario

This is a useful scenario to reflect known future c hanges (e.g. the rent will go up)

This is the most complex scenario, it integrate cha nges in unknown future factors (e.g. a competitor will open shop ne xt door)


The Notion of ForcesLet’s represent the elements that affect our predictions as “forces”. These forces act on our results in a neutral, positive or negative way. In addition, there is an intensity to these competing forces that will influence the results (e.g. a mild negative force vs. an enormous positive force).

1 – A force can be:

2 – In addition, a force can be:

Positive (e.g. increase in customers)

Negative (e.g. increase in costs)

Or Neutral (e.g. change of landlord)

Small (e.g. a 0.3% increase in rent

Medium (e.g. a 3% change in web traffic

Or Big (e.g. a 10% decrease in revenue

3 – Lastly, a force can be:

… ..| … … … …Limited in time Permanent Cyclical


Aggregating the DataOnce you have defined the right tools/formulas and quantified the force factors influencing your forecast, you need to aggregate your data to see the “big picture”

Current Market

Future Market

InflationInflation

Distributor agreementsDistributor

agreements

Distributor agreementsDistributor

agreements


In SummaryWhen building your forecast, regardless of whether you represent it graphically or numerically, it is important to understand the influence of the following criteria:

1 – Determine all the existing forces that will influence your results and that you know about (e.g. changes in relationship, pricing, etc.).

3 – Determine how these forces will influence your results, when they start having an influence and how much they will influence your operations.

4 – The better you are at defining the big picture in y our current and future environment, the easier it is going to be to transl ate it into sensible numbers.As a consequence, you should always ask yourself, n ot whether your numbers are right, but whether your assumptions are .

2 – Determine all the new forces that you anticipate will emerge or existing ones that will stop being relevant (e.g. introduction of a new product by a competitor, obsolescence of your equipment or technology, etc.).


3. Scaling Your Forecast

What order of magnitude is relevant?


Time Sensitivity

When is the right time to do a forecast and what is the right time horizon to consider?Here are 4 criteria to keep in mind:

1 – Determine what you want to measure (e.g. revenue, attendance, inventories, employee attrition, customer turn-over, etc.)

3 –Determine the volatility of your operations, in other words, how often your measured data is likely to change, to determine how often you should forecast.

4 –Determine how far in advance you need to predict your operations (the end of the quarter, the end of the year, 12 months ahead, etc.) to determine the time-horizon of your forecast.

2 – Determine when the previously available data becomes obsolete to define when you should start your forecast (e.g. when your budget becomes less of a realistic plan, is it on month 1 or after the 1st quarter?).


Appropriate Level of DetailWhen building your forecast, one of the early questions you have to ask yourself is: What is the

right granularity, should I predict my results to the million or to the cent?

1 – Determine the level of details at which the numbers will be looked at (e.g. is it a summary income statement rounded at the million level, or a detailed group P&L rounded at the ten-thousand dollars level?

3 – Determine who in your constituency should submit their forecast (e.g. all units below the division, all entities that have a project manager, only the main units?).

4 – Always remember that the answer might be different for each month, unit and line item (e.g. unit A should always repor t, but unit B only if there is a change of more than 10%, new trips don’t need to be reported but new CAPEX purchase always do, etc.).

2 – Determine the level at which your constituents can make their own unplanned transactions without impacting your results (e.g. any travel below $1,000, any purchase approved in your operations below $10,000?).


4. Looking Beyond the Numbers

How do I ensure that my forecast is reasonable?


Keeping Everybody HonestNow you have produced a forecast, but is it accurate?Here are 5 factors that will help you ensure that your numbers make sense:

1) Proof read the numbers: Your forecast arithmetically works, but needs to be validated to ensure reasonableness. It can be as simple as checking against a run-rate, or as complex as validating against a predictive algorithm, but there needs to be a “mechanical” boundary against which benchmarking the forecast.

2) Check assumptions compatibility: Check that assumptions are not mutually exclusive (e.g. incremental marketing investment is going to grow my market, and I will save everywhere to fund the extra manufacturing capacity I need).

3) Validate forecast feasibility: Use common sense to validate your forecast as a whole (e.g. your market may very well double, but if you don’t have the capacity to address it, it is not going to move your numbers).

4) Probe operating plan: Probe your forecasters for the operational details of how they are going to deliver their forecast (e.g. what is the game-changer to triple sales?)

5) It is OK to be the villain: Everybody likes to tell a good story, and people want to be heroes, not villains. To counter that pressure and a natural tendency to optimism, give your forecaster a safe environment where they can be candid about their forecast and comfortable telling you the bad news.


Probability and ReasonablenessAfter you have established the forces at play in your forecast, determined the right time horizon, and set the right granularity, you have produced a forecast.

But is it really what is going to happen, does it make sense?

No matter how complex your modeling is, no matter how scientifically proven your method is, you should always look at your forecast with skeptical eyes (e.g. are costs really doubling, sales tripling, people spend 50% more at my shop, etc.).

Consider narrowing/eliminating ranges (e.g. coming to an average reasonable answer instead of a wide range, 1000 visitors/day, not between 512 and 1,634).It is very important to define values for your assumptions instead of ranges, to do so you can:

1 - Use the average/mean of the top and bottom value in your range,2 - Use the most likely value in your range,3 - Use the historical pattern in your range (seasonality, growth rate, etc.),4 - Use your best judgment to adjust values that seem extreme), etc.


5. Communicating your Forecast

What to say about your forecast?


Coming to An AnswerYou have formulated your forecast, but what to say about it and how to say it? These seem like trivial questions, but they are often the difference between a fruitful and a fruitless exercise. Here are important factors to keep in mind when deciding how to relay your findings:

1 - Build the forecast for your audience . Both content, granularity, and time horizon are important to adjust for your audience (e.g. summarized P&L for the year rounded to the million, or monthly detail by line-item rounded to the thousand?).

2 - Always give a number (however rounded), not a ran ge. It is impossible to measure accuracy and drive accountability if you forecast a range. A range is an incentive for the measured units to make it as broad as possible in order for the forecast to be accurate, thus defeating the original purpose of the forecast.

3 - The aim of the forecast is to measure how far yo ur are from your goal . It is imperative for the forecast to be presented candidly, even if (especially) it diverges from the plan. The forecast ensures that appropriate and timely actions are taken in order to either close the gap with the plan, or adjust the expectations set in the plan.


Coming to the Wrong Answer

Many a time, your forecast will give you an answer that is not what you wanted to see. It is of course important to be able to go back to your sources and validate all the assumptions made as well as the calculations. However, it is crucial to remember that the forecast is a reflection of where you are, not where we want to be. It is very important to resist the temptation to adjust the numbers to a scenario closer to our wishes, as doing so could be extremely damaging to your organization.

If your forecast (the road you have built) doesn’t lead you to your plan (the destination you have set), don’t try to re-build your model, look at what the gap is and use your forecast as an opportunity to take the actions necessary to close it.


6. Rolling Forecast

How to keep your forecast fresh?


Doing It All Over AgainAfter your forecast has been done and communicated, the next phase starts immediately: doing it again!

For a forecast to be effective, it needs to be:

1- Current (with the most recent assumptions)

To deliver this, you must constantly refresh your forecast. That way, as time passes, you know that you always have a current forecast to guide you.

It is important to do it on a recurring basis, as one of the roles of the forecast is to be an early indicator of variances compared to your plan. The more often you refresh your forecast, the more chances you have to catch a changing trend that is going to affect your results.

2- Sensible (with all assumptions validated by those close to your operations)

4- Answering the questions you asked in the first place (if you are worried about delivering your annual revenue commitment, there is no point centering your forecast around your payroll benefits)

3- Realistic (no matter what your forecast tells you, it needs to make sense)


Principle of a Rolling ForecastOne appealing option is to forecast to a constant time window (usually 12 to 18 months) so that, no matter where you are in your fiscal year, you always have the same horizon in front of you. A rolling forecast gives you the additional work of only planning for one more month and gives you a constant “time-buffer” in your planning. It also gives you a solid basis for your budget or long-term plan when the time comes.A rolling forecast also gives you more comparison points for the same period in time. Here are the basic mechanics of a rolling forecast:

1 – Every month, change the current month financials from forecast to actuals data.

2 – Refresh your previous forecast for all the other periods.

3 – Add one month of forecast at the end of the previously forecasted period.

Mth 1 Mth 2 Mth 3 … Mth 12



Forecast Period 1 Jul Act Aug Fcst Sep Fcst … Jun Fcst

Forecast Period 2 Aug Act Sep Fcst Oct Fcst … Jul Fcst

Forecast Period 3 Sep Act Oct Fcst Nov Fcst … Aug Fcst

…


7. The Value of Crowd Forecasting

IARPA case study


IARPA Case

Part of the Office of the Director of National Intelligence (ODNI) is the Intelligence Advanced Research Projects Activity (IARPA). Part of IARPA is the Office of Anticipating Surprise (OAS).

IARPA’s OAS wanted to know how to improve the predictive accuracy, precision, and timeliness of their forecast.

IARPA funded the Good Judgment Project as part of their ACE (Aggregative Contingent Estimation) project.

The project, co-led by the University of Pennsylvania and UC Berkeley, let academics and industry groups form teams of forecasters to predict the outcome of hundreds of international political, military, and economic scenarios with the objective of beating the accuracy of a control group over a 4-year period.

Thousands of volunteers joined into this project.


IARPA Results

After only 2 years of the 4-year project, the overwhelming evidence showed that:

Teams of average citizens consistently beat the forecast accuracy of the best analysts, intelligence officers, experts, and pundits.

The average predictions are more accurate even when the experts have access to classified information not available to the average team.

The forecasters have to be generally well informed, but don’t have to be subject matter experts.

If the group is large enough, the average prediction will beat the expert’s one by a significant margin.


8. Important Factors to Remember

How to make the best of your forecast?


Important Factors to RememberAs a conclusion, here are important factors to keep in mind when engaging in a forecasting exercise:

1) Forecasting is an art more than a science: You need to have your math right and a lot of sophisticated algorithms can help you tremendously, but in the end, it is your knowledge of the subject, the relationships with your operation managers and analysts, and your judgment that will make the difference.

2) Segregate the duties: Separate the tasks of input, modeling, analysis, and review to maintain as impartial a look as possible on everybody's contribution to your forecast. Do not let people review their own work as it is psychologically too difficult to probe or contradict your own assumptions.

3) Collaborate across disciplines: Use inputs and analysis from as diverse a pool as possible. Nobody has all the answers all the time, so your only insurance against wishful thinking is to make sure that as many current of thoughts are represented in your analysis before you decide what the forecast should be.


the amazing science and delicate art of forecasting

Documents

good forecast

forecast exercise

remembercyrille betant

operational results

financial results

budget goals

prediction wil

road leads