lack of internal resources. The only allocated internal resources are for project management and interaction with the external consultants. However, even in this case, some basic training for forecasting and statistics is recommended. It is preferable to have a well-prepared test case when you begin the working relationship with the external consultants. (Some suggestions on how to prepare an effective test case are given by Michael Gilliland in his book The Business Forecasting Deal.) The key advantage of this solution is the minimum cost. The key disadvantage is the total dependence on external resources.
Distributed developers
This organizational structure is appropriate in small or medium-size businesses when the demand for forecasting services is concentrated in several key users, such as marketing and sales, supply chain, and purchasing. Often they prefer to own the whole model development and deployment process and hire experts with forecasting knowledge. In many cases they do not invest in the high-end hardware and software infrastructure, such as SAS Forecast Studio. The key advantage of this solution is the availability to implement forecasting capabilities with internal resources in appropriate business functions at an affordable cost. The key disadvantage is the limited capacity for growth.
Centralized developers group
The best-case scenario for applying data mining in forecasting in larger organizations is by building a centralized group of developers. The group must have the capacity to respond fast to the growing demand of forecasting projects from various sections of a large corporation. The skill set of the developers' team must have a proper balance between system and data support expertise and modeling capabilities in the area of statistics, data mining, and forecasting. An example of key roles in a centralized group of data mining for forecasting is given below.
The system administrator maintains servers, upgrades software, handles security issues, and interacts with IT.
The data administrator maintains data integrity, identifies internal and external data sources, and collects and harmonizes data.
The modeler interacts with clients, identifies system structure and data, pre-processes the data, performs variable reduction and selection, develops, validates, implements, and maintains forecasting models.
The manager manages the group, delivers needed resources, and brings in projects.
The proper place of this group within a large organization is in the centralized corporate business services. This group serves all potential users so that the return of investment is maximized. The size of the group depends on projected demand. However, at least five to seven developers are needed to be efficient. It is assumed that a period of at least two to three years is needed for the group to establish itself by building infrastructure, hiring, learning, promoting to potential clients, and developing test projects. The funding during this period is centralized and gradually gives way to a self-support mode where projects are supported directly by their clients. The key issue that will determine the fate of this group is whether a sustainable project pipeline can be maintained.
3.5.2 Users Infrastructure
Forecasting users come from different parts of the organization. Typical clients for statistical forecasting services are the marketing, sales, financial, purchasing, and operations planning departments. Forecasting users can be classified in the following four categories, briefly discussed below: (1) forecasting reports users, (2) planners, (3) decision-makers, and (4) top level managers. (A similar user classification for demand-driven forecasting is described in detail in Charles Chase's book Demand-Driven Forecasting: A Structured Approach to Forecasting.)
Forecasting reports users
These are the users who passively use the delivered forecasts for information purposes only without making direct business decisions based on specific forecasting results or participating in judgmental forecasting or process planning. Most of the top managers are in this category. Recently many businesses have included forecasts in their regular performance tracking reports distributed to middle-to-top-level managers. The value of forecasting for this category of users is in giving them an awareness of the projected directions of the key performance indicators of interest.
Planners
In contrast to the previous category, planners actively use the delivered statistically based forecasting models in developing their sales, marketing, or operations plans. Very often they also have the right to override the statistical forecasts with their judgmental estimates. In the case of demand-driven forecasting, these are the users in marketing and sales who “shape the demand” based on analytics and domain knowledge. From all the categories of forecasting users, planners are the most educated and directly involved in the model development and deployment loops. They have the decisive role in introducing expert knowledge by defining events, evaluating model performance, and making the final forecasts adjustments. Planners also have the responsibility to recommend to the decision-makers which developed plans, based on the delivered and adjusted forecasts, get final approval.
Decision-makers
This category of forecasting users includes the middle-layer managers at the departmental level who are responsible for the results of the plans recommended by the planners. They also make the final decision for implementing the plans. Part of the decision-making process is balancing the recommended statistically-driven forecasts from the experts (planners and model developers) and the top management push. Often the decision-making process goes through several iterations until a consensus is reached. This category of users is critical for the success of specific forecasting projects and the overall forecasting activities in the business. Success for decision-makers is not based on model performance measured by forecasting accuracy but is based on the expected value measured by the key performance indicators (KPIs).
Top level managers
This category includes the top executives related to finances, IT, and operations. As users, they might have different roles. One critical role is to establish and support financially, for some period of time, the forecasting capabilities in the organization. Executives might request forecasting projects for developing a business strategy as well. It is expected that at any moment the top executives can access the forecasting reports at any level of the organization and keep track of the KPIs. And finally they can actively influence what decisions are made regarding the implementation of the action plans based on forecasting models.
3.5.3 Work Process Implementation
A key component in developing the organizational infrastructure is selecting and implementing an appropriate work process for data mining in forecasting. An example of such a work process is given in Chapter 2. It is also very important to integrate the selected work process with the existing corporate culture. The best-case scenario is to consolidate the data mining for forecasting work process with the existing standard work processes in the organization. If you can do so, the implementation cost and the time for integration into the corporate culture will be significantly reduced. An example of integration with the most popular work process in industry, Six Sigma, is described in Chapter 2. Another example of a popular work process in the case of demand-driven forecasting – Sales & Operation Planning (S&OP) is given in Chase 2009.
3.5.4 Integration with IT
An organizational issue of critical importance for the final success of applying data mining for forecasting is the smooth integration with corporate IT services. Unfortunately the integration process can be bumpy largely due to the different mode of operation of IT. The IT department is often focused primarily on implementing standard solutions across the business. The focus of data mining for forecasting
