I love my job. Who wouldn’t, I predict the future and it is very satisfying when I am right. Specifically, I design and develop forecasts for clients that produce, distribute, and use technology products and services. I assume that if you are reading this then you also build forecasts, manage people who do, or perhaps use forecasts that others produce.
This is a richly rewarding and intellectually satisfying occupation but it is not easy. I have been in this business for over 35 years and have faced and overcome many challenges. This article address what I believe are the five most common and critical challenges to the success of any technology market forecasting effort
- Making Design Choices
- Working with Limited Data
- Segmenting to Multiple Taxonomies
- Facilitating Collaboration
- Dealing with Constraints
These challenges are not mutually exclusive, nor are they exhaustive. Yet understanding them, and how to address them, will lead to producing forecasts that have superior normative, descriptive and predictive properties, while reducing time and budget investments.
1. Making Design Choices – The forecast design process always begins with making choices about taxonomies, variables, and forecasting methods. The options can be overwhelming, confusing, and can have serious consequences if poorly made. That is why it is necessary to first understand both context and constraints.
A forecast only has value within the context of the decisions the users of the forecast are facing. Most users of technology market forecasts are the technology suppliers themselves. Their decisions typically fall into three categories that all support identifying and exploiting opportunities.
- Asset Investment and Resource Allocation
- Product/Service Portfolios and Specifications
- Marketing Strategy – Prices, Channels, Media and Message
A decision to fund a new product research & development project will require a different forecast than one that is used to make a decision about acquiring a competitor, or launching a new marketing campaign targeting a specific region.
The other factor influencing design choices are the constraints you need to work within. This will always include budget, staff and management time, analysts experience and expertise, and most critical, the inventory of available data, information, and knowledge.
I have found it most efficient to start with choices about taxonomies, then variables, leaving methods for last. The most common taxonomies are:
- Time – how long and what periodicity
- Product/Service Categories, Specifications and Attributes – E.g., Speed, Size, Capacity, Services Provided
- User Categories and Demand Demographics – E.g., Company Size, Consumer Age, Usage Categories
- Distribution Channels
Taxonomy choices determine the scope, scale and complexity of the development effort since they are most often multiplicative. The objective is to avoid either over- or under-engineering the design. . The key question that needs to be addressed when making taxonomy and granularity choices is what are the natural segments for this market in terms of the client’s decision context. The market topology for any technology product or service can be described by identifying its natural segmentation, i.e. aggregations that display the same demand and/or usage behavior patterns
Once the taxonomies are selected and defined choices about variables can be made. There are three types of variables. Independent variables that are the input to the model, dependent variable that are the specified output of the model, and variables that are used to validate and/or calibrate the model. While choice of variables may seem obvious an effort should be made to assure that you are counting and measuring the right quantities to address the forecast objectives defined by the context.
The most common basic variables are:
- Units of Product or Service
- Value of Units of Product or Service
- Units of Buyers or Users
- Performance Units of Product or Service
In additional there are many derivative variables that can be assigned input, output or validation roles. These include:
- Functions of any of the basic variables such as Average Unit Value
- Annual Growth Rates
- Compound Annual Growth Rate (CAGR)
- Percent of Totals and Ratios to other variables
Mobile Phones offer a good example. Depending on the context, the objective could be to forecast, subscribers, units, subscriber revenue/spending, manufacturers revenues, or aftermarket services revenue/spending. It is should be clear that variable choice is integral to method choice in many cases
The most difficult and complex choice is forecast method or algorithm. Many times it is dictated by the available data, and more often, by the nature of the product or service that is being forecasted. There are three major design choices that have to be made.
- Directionality – The computational progression in most forecast model may be either Top-Down or Bottom-up. A Top-Down progression beings with creating a forecast of the highest aggregation, i.e., Total Market, and then proceeds to allocate to each specified taxonomy either in turn, or simultaneously. A Bottom-Up progression starts at the lowest level and then aggregates upward through all the taxonomies until the total aggregation is reached. More sophisticated models utilize both Top-Down and Bottom-Up computational paths.
Which direction you select is dependent on the amount and granularity of the available data, as well as the constraints. If you have collected sufficient data about actual shipments from the majority of the suppliers then a bottom-up approach could be used. Otherwise, a Top-Down, or hybrid approach is recommended.
- Order of Resolution – refers to the order in which the data is hierarchically allocated to each of the taxonomies by the selected segmenting methods in a Top-Down progression. There are several criteria that may be applied in setting the order. First, is there a known causal dependency among the taxonomies such that the allocation to a specific taxonomy is influenced by the allocation to another? In the absence of a causal dependency, the recommended order is to allocate to the taxonomies in the order of what is known best. A third is the availability of data used to derive allocation parameters. In some case these criteria will suggest different orderings and a trade-off decision will need to be made.
- C. Forecast Methodology – This is highly dependent on the amount and nature of available data, the level of market aggregation (brand, product, technology category), and an understanding of the market topology and dynamics.
I. Trending – is the direct or indirect extrapolation of historical data into the future by application of any of the trending methods including time-series analysis, regression, or curve fitting. There are four Logical Dependencies that can be applied to trending methods:
- Historical – The independent variable is the historical time-series for this product or service. This assumes that the majority of the required information necessary to forecast this time-series exists in the historical data. In other words, the process that will create the future is the same that created the past. An example is certain classes of printers in the business markets.
- Analogous – The independent variable is an historical analogous product or service. The assumption is that the time-series being forecasted will essentially follow the historical trend taken by the analogous product or service since it is being purchased by the same buyers to meet the same needs. An example is certain classes of storage devices.
- Precursor – this method is chosen when there is a known usage dependency between current demand for a product or service, and the demand for a previously purchased and used product or service. Examples are consumable products such as toners, as well as aftermarket services.
- Enabling – this method is appropriate when there is a known usage dependency between the product or service that is being forecasted and a different product or service that is being purchased concurrently. An example is mobile applications.
II. Adoption/Penetration – application of any of the methods based on Diffusion of Innovation such as Fisher-Pry or the Bass Diffusion Model. These methods can be applied to any of the logical dependencies. Adoption model methods are most often used to forecasting new products or services, or a new generation of a product or service. These models generate S-shaped cumulative and peaked periodic forecasts. The primary benefit of these methods is their ability to predict the timing of inflection points were the market transitions from one phase to another. However, all of these methods require that the Total Available Market be estimated by an independent method.
III. Casual – These methods include all of the single and multivariate econometric and demographic modeling approaches, as well as those designed to transform qualitative assessments of influences into quantitative terms. Casual methods also include methods to model demand driven by installed base retention, retirement, and replacement rates.
2. Working with Limited Data– In contrast to many other industries, the technology sector often lacks sufficient available historical sales data to support the forecasting methods common to some other industries. We do not have product and markets that remain essentially unchanged for decades. In fact, the defining characteristic of our industry is destructive innovation. In many cases we need to rely on data that cannot always meet rigorous statistical tests, and occasionally data that at best can be considered anecdotal. Yet, we cannot abdicate our responsibility to provide our clients with the best forecasts we can produce given these constraints. Furthermore, it is not enough to just produce forecasts with good descriptive properties. This can be accomplished by mathematics alone. We need to produce forecasts with high normative (how and why) and predictive properties. You must assume that at some point you will be called upon to defend your results. This requires that you document the rational for all design choices.
If statistical rigor is not attainable then next best criteria is assurance that you have made the best design choices you can make, and that no other forecast provider can produce a better forecast given the same constraints. The most acceptable way of validating your forecast is to employ multiple independent approaches. For example, if your model is based on primary research with a limited sample size you might also build an independent causal forecast based on econometric or demographic data. Building forecasts for the same product or services’ using different logical dependencies is also a solution. At the least you should use independent methods to establish upper and lower bounds by constructing ratios to known related variables.
3. Segmenting to Multiple Taxonomies – The design choices that you make includes the Order of Resolution setting the progression in which taxonomies are addressed, as well as the methods used to allocate to each of the items in the taxonomies. The challenge is to produce final results that are intelligently informed from any taxonomy point of view. While this may seem simple at first glance it may in fact present unexpected complexities.
Most client custom forecasts produced by technology market research firms are derivatives of an existing published forecast that will require some taxonomy transformation into the client’s taxonomies. If the forecast is new and original it may involve collaboration with analysts and other stakeholders representing different points of view. A request to change the distribution in one taxonomy may cause unacceptable results when views from an alternative taxonomy. Additionally the taxonomies themselves may need to be modified during the development phase, or in subsequent updates. Many forecast architects fail to anticipate this possibility even though it is quite common. Including a process that can facilitate negotiations and handle taxonomy transformation is essential to meeting the objective of producing the forecast within the time and budget constraints.
4. Facilitating Collaboration – While there may be cases in which the forecast is designed, developed and deployed by a single person, in most cases this is a collaborative effort requiring two sets of complementary core competencies, those of the Forecast Architect encompassing the mathematical, computational, and forecast methodologies, and those of the Knowledge Analysts encompassing qualitative assessment of the factors and trends influencing the products/services and markets being forecasted.
The very best Knowledge Analysts are extraordinary people, experts in their area, and as such, think at levels of abstraction that are not easily quantifiable. They often develop their understandings intuitively rather than computationally. The single most important task of the Forecast Architect is to build a bridge between the Knowledge Analysts qualitative assessments of the market and the quantitative requirements of the model.
There are a number of process elements that need to be established to facilitate efficient collaboration:
- Adapt to the Analysts Worldview – the forecast taxonomies, variables and methods must conform to the worldview of the analysts that will use or provide the input, and validate the output. You need to build to their vision rather then ask them to conform to arbitrarily designed model requirements.
- Build a Qualitative Inventory of Influences – We all find it easier to view the world in comparative rather than absolute terms. We intuitively know larger or smaller, faster or slower, closer or further, weaker or stronger. Assigning rank is easier than assigning value. Starting with a qualitative comparative view of the drivers and inhibitors influencing the market, and then moving in successive steps to estimating input parameter values is often the more efficient process.
- Provide Qualitative to Quantitative Transformation Tools – That are easy to use and intuitive such as a weights and scores approach. Include the users in the design process to assure buy-in and acceptance.
- Base and Modify – This is often an iterative process. Do not expect the first results to meet the analyst’s expectations. Often the sensitivity to changes in the initial conditions will not be fully understood until after the first pass. From that point on the analysts will be able to provide changes that will eventually produce acceptable results. This is a learning process where interim results of the model providing feedback that will alter the analyst’s understanding of the relationships between the influences and the output.
5. Dealing with Constraints –There will always be limited budgets and tight schedules. The solution is to establish a formal design and development process based on the following guide lines:
- Always begin by defining the client’s decision context and recognizing the constraints.
- Plan and document your design choices
- Involve the users/analysts in the design process
- Make conservative estimate of the time required to complete tasks by all of the collaborator in the process
- Allow for unexpected changes in the objectives and/or specifications
Each of these challenges is easily met by applying the following three principles:
- Context – Understand how the forecast will be used, the dynamics and topologies of the market, and the design and development constraints.
- Collaboration – develop and facilitate the most efficient environment for melding complimentary core competencies.
- Clarity – Recognize the requirements and limits of the methodologies you chose, as well as the architecture of your design.
There is a natural process that governs the forecasting process. Structure extracts information from data. Information becomes knowledge in the light of experience. And knowledge yields understanding through insight and introspection. While it is clear that a forecast delivers data, information, and knowledge, it should also deliver understanding. And that Understanding of the Future is derived from the forecasting process itself.
∞∫Δ Daniel Research Group © (2012)
Daniel Research Group is a market research firm specializing in the design, development and application of market models and forecasts for clients in the technology sector including supplier, investors, and other market research firms. For more information contact Steve@DanielRG.com or visit www.DanielResearchGroup.com.