the reader a condensed description of the necessary steps to run forecasting projects in the real world. We begin with a high-level overview of the whole sequence as a generic flowchart. Each key step in the work process is described briefly with its corresponding substeps and specific deliverables.
2.2.1 Generic Flowchart
The generic flowchart of the work process for developing, deploying, and maintenance of a forecasting project based on data mining is shown in Figure 2.1. The proposed sequence of action items includes all of the steps necessary for successful real-world applications–from defining the business objectives to organizing a reliable maintenance program to performance tracking of the applied forecasting models.
Figure 2.1: A Generic flowchart of the proposed work process
The forecasting project begins with a project definition phase. It gives a well-defined framework for approving the forecasting effort based on well-described business needs, allocated resources, and approved funding. As most practitioners already know, the next block—data preparation—often takes most of the time and the lion's share of the cost. It usually requires data extraction from internal and external sources and a lot of tricks to transfer the initial disarray in the data into a time series database acceptable for modeling and forecasting. The appropriate tricks are discussed in detail in Chapters 5 and 6.
The block for variable reduction and selection captures the corresponding activities, such as various data mining and modeling methods, that are used to take the initial broad range of potential inputs (Xs) that drive the targeted forecasting variables (outputs, Ys) to a short list of the most statistically significant factors. The next block includes the various forecasting techniques that generate the models for use. Usually, it takes several iterations along these blocks until the appropriate forecasting models are selected, reliably validated, and presented to the final user. The last step requires an effective consensus building process with all stakeholders. This loop is called the model development cycle.
The last three blocks in the generic flowchart in Figure 2.1 represent the key activities when the selected forecasting models are transferred from a development environment to a production mode. This requires automating some steps in the model development sequence, including the monitoring of data quality and forecasting performance. Of critical importance is tracking the business performance metric as defined by its key performance indicators (KPIs), and tracking the model performance metric as defined by forecasting accuracy criteria. This loop is called the model deployment cycle in which the fate of the model depends on the rate of model performance degradation. In the worst-case scenario of consistent performance degradation, the whole model development sequence, including project definition, might be revised and executed again.
2.2.2 Key Steps
Each block of the work process is described by defining the related activities and detailed substeps. In addition, the expected deliverables are discussed and illustrated with examples when appropriate.
Project definition steps
The first key step in the work process—project definition—builds the basis for forecasting applications. It is the least formalized step in the sequence and requires proactive communication skills, effective teamwork, and accurate documentation. The key objectives are to define the business motivation for starting the forecasting project and to set up as much structure as possible in the problem by effective knowledge acquisition. This is to be done well before beginning the technical work. The corresponding substeps to accomplish this goal as well as the expected deliverables from project the definition phase are described below.
Project objectives definition
This is one of the most important and most often mishandled substeps in the work process. A key challenge is defining the economic impact from the improved forecasts through KPIs such as reduced cost, increased productivity, increased market share, and so on. In the case of demand-driven forecasting, it is all about getting the right product to the right customer at the right time for the right price. Thus, the value benefits can be defined as any of the following (Chase 2009):
a reduction in the instances when retailers run out of stock
a significant reduction in customer back orders
a reduction in the finished goods inventory carrying costs
consistently high levels of customer service across all products and services
It is strongly recommended to quantify each of these benefits (for example, “a 15% reduction in customer back orders on an annual basis relative to the accepted benchmark”).
An example of an appropriate business objective for a forecasting project follows:
More accurate forecasts will lead to proactive business decisions that will consistently increase annual profit by at least 10% for the next three years.
Another challenge is finding a forecasting performance metric that is measurable, can be tracked, and is appropriate for defining success. An example of an appropriate quantitative objective that satisfies these conditions is the following definition:
The technical objective of the project is to develop, deploy, and support, for at least three years, a quarterly forecasting model that projects the price of Product A for a two-year time horizon and that out-performs the accepted statistical benchmark (naïve forecasting in this case) by 20% based on the average of the last four consecutive quarterly forecasts.
The key challenge, however, is ensuring that the defined technical objective (improved forecasting) will lead to accomplishing the business goal (increased profitability).
Project scope definition
Defining the forecasting project scope also needs to be as specific as possible. It usually includes the business geography boundaries, business envelope, market segments covered, data history limits, forecasting frequency, and work process requirements. For example, the project scope might include boundaries such as the following: the developed forecasting model will predict the prices of Product A in Germany based on internal record of sales. The internal historical data to be used starts in January of 2001, uses quarterly data, and the project implementation has to be done in Six Sigma according to the standard requirements for model deployment and with the support of the Information Technologies department.
Project roles definition
Identifying appropriate stakeholders is another very important substep to take to ensure the success of forecasting projects. In the case of a typical large-scale business forecasting project, the following stakeholders are recommended as members of the project team:
the management sponsor who provides the project funding
the project owner who has the authority to allow changes in the existing business process
the project leader who coordinates all project activities
the model developers who develop, deploy, and maintain the models
the subject matter experts—SMEs—who know the business process and the data
