Forecasting Technology Markets – The Top Five Challenges

I love my job.  Who wouldn’t, I predict the future and it is very satisfying when I am right.   Specifically, I design and develop forecasts for clients that produce, distribute, and use technology products and services.   I assume that if you are reading this then you also build forecasts, manage people who do, or perhaps use forecasts that others produce.

This is a richly rewarding and intellectually satisfying occupation but it is not easy.  I have been in this business for over 35 years and have faced and overcome many challenges.  This article address what I believe are the five most common and critical challenges to the success of any technology market forecasting effort

1. Making Design Choices
2. Working with Limited Data
3. Segmenting to Multiple Taxonomies
4. Facilitating Collaboration
5. Dealing with Constraints

These challenges are not mutually exclusive, nor are they exhaustive.   Yet understanding them, and how to address them, will lead to producing forecasts that have superior normative, descriptive and predictive properties, while reducing time and budget investments.

1. Making Design Choices – The forecast design process always begins with making choices about taxonomies, variables, and forecasting methods.   The options can be overwhelming, confusing, and can have serious consequences if poorly made.   That is why it is necessary to first understand both context and constraints.

A forecast only has value within the context of the decisions the users of the forecast are facing.    Most users of technology market forecasts are the technology suppliers themselves.  Their decisions typically fall into three categories that all support identifying and exploiting opportunities.

• Asset Investment and Resource Allocation
• Product/Service Portfolios and Specifications
• Marketing Strategy – Prices, Channels, Media and Message

A decision to fund a new product research & development project will require a different forecast than one that is used to make a decision about acquiring a competitor, or launching a new marketing campaign targeting a specific region.

The other factor influencing design choices are the constraints you need to work within.  This will always include budget, staff and management time, analysts experience and expertise, and most critical, the inventory of available data, information, and knowledge.

I have found it most efficient to start with choices about taxonomies, then variables, leaving methods for last.   The most common taxonomies are:

• Time – how long and what periodicity
• Product/Service Categories, Specifications and Attributes – E.g., Speed, Size, Capacity, Services Provided
• Geography
• Industries/Markets
• User Categories and Demand Demographics – E.g.,  Company Size, Consumer Age, Usage Categories
• Distribution Channels

Taxonomy choices determine the scope, scale and complexity of the development effort since they are most often multiplicative.  The objective is to avoid either over- or under-engineering the design. .   The key question that needs to be addressed when making taxonomy and granularity choices is what are the natural segments for this market in terms of the client’s decision context.    The market topology for any technology product or service can be described by identifying its natural segmentation, i.e. aggregations that display the same demand and/or usage behavior patterns

Once the taxonomies are selected and defined choices about variables can be made.  There are three types of variables.   Independent variables that are the input to the model,  dependent variable that are the specified output of the model, and variables that are used to validate and/or calibrate the model.  While choice of variables may seem obvious an effort should be made to assure that you are counting and measuring the right quantities to address the forecast objectives defined by the context.

The most common basic variables are:

• Units of Product or Service
• Value of Units of Product or Service
• Units of Buyers or Users
• Performance Units  of Product or Service

In additional there are many derivative variables that can be assigned input, output or validation roles.  These include:

• Functions of any of the basic variables such as Average Unit Value
• Annual Growth Rates
• Compound Annual Growth Rate (CAGR)
• Percent of Totals and Ratios to other variables

Mobile Phones offer a good example.  Depending on the context, the objective could be to forecast, subscribers, units, subscriber revenue/spending, manufacturers revenues, or aftermarket services revenue/spending.  It is should be clear that variable choice is integral to method choice in many cases

The most difficult and complex choice is forecast method or algorithm.  Many times it is dictated by the available data, and more often, by the nature of the product or service that is being forecasted.  There are three major design choices that have to be made.

1. Directionality – The computational progression in most forecast model may be either Top-Down or Bottom-up.    A Top-Down progression beings with creating a forecast of the highest aggregation, i.e., Total Market, and then proceeds to allocate to each specified taxonomy either in turn, or simultaneously.     A Bottom-Up progression starts at the lowest level and then aggregates upward through all the taxonomies until the total aggregation is reached.    More sophisticated models utilize both Top-Down and Bottom-Up computational paths.

Which direction you select is dependent on the amount and granularity of the available data, as well as the constraints.   If you have collected sufficient data about actual shipments from the majority of the suppliers then a bottom-up approach could be used.   Otherwise, a Top-Down, or hybrid approach is recommended.

1. Order of Resolution – refers to the order in which the data is hierarchically allocated to each of the taxonomies by the selected segmenting methods in a Top-Down progression.   There are several criteria that may be applied in setting the order.  First, is there a known causal dependency among the taxonomies such that the allocation to a specific taxonomy is influenced by the allocation to another?   In the absence of a causal dependency, the recommended order is to allocate to the taxonomies in the order of what is known best.   A third is the availability of data used to derive allocation parameters.  In some case these criteria will suggest different orderings and a trade-off decision will need to be made.

1. C.      Forecast Methodology – This is highly dependent on the amount and nature of available data, the level of market aggregation (brand, product, technology category), and an understanding of the market topology and dynamics.

I.              Trending – is the direct or indirect extrapolation of historical data into the future by application of any of the trending methods including time-series analysis, regression, or curve fitting.    There are four Logical Dependencies that can be applied to trending methods:

• Historical – The independent variable is the historical time-series for this product or service.  This assumes that the majority of the required information necessary to forecast this time-series exists in the historical data.   In other words, the process that will create the future is the same that created the past.  An example is certain classes of printers in the business markets.

• Analogous – The independent variable is an historical analogous product or service.  The assumption is that the time-series being forecasted will essentially follow the historical trend taken by the analogous product or service since it is being purchased by the same buyers to meet the same needs.   An example is certain classes of storage devices.

• Precursor – this method is chosen when there is a known usage dependency between current demand for a product or service, and the demand for a previously purchased and used product or service.  Examples are consumable products such as toners, as well as aftermarket services.

• Enabling – this method is appropriate when there is a known usage dependency between the product or service that is being forecasted and a different product or service that is being purchased concurrently.  An example is mobile applications.

II.              Adoption/Penetration – application of any of the methods based on Diffusion of Innovation such as Fisher-Pry or the Bass Diffusion Model.   These methods can be applied to any of the logical dependencies.   Adoption model methods are most often used to forecasting new products or services, or a new generation of a product or service. These models generate S-shaped cumulative and peaked periodic forecasts.  The primary benefit of these methods is their ability to predict the timing of inflection points were the market transitions from one phase to another.   However, all of these methods require that the Total Available Market be estimated by an independent method.

III.              Casual – These methods include all of the single and multivariate econometric and demographic modeling approaches, as well as those designed to transform qualitative assessments of influences into quantitative terms.    Casual methods also include methods to model demand driven by installed base retention, retirement, and replacement rates.

2. Working with Limited Data– In contrast to many other industries, the technology sector often lacks sufficient available historical sales data to support the forecasting methods common to some other industries.   We do not have product and markets that remain essentially unchanged for decades.  In fact, the defining characteristic of our industry is destructive innovation.   In many cases we need to rely on data that cannot always meet rigorous statistical tests, and occasionally data that at best can be considered anecdotal.  Yet, we cannot abdicate our responsibility to provide our clients with the best forecasts we can produce given these constraints.   Furthermore, it is not enough to just produce forecasts with good descriptive properties.  This can be accomplished by mathematics alone.  We need to produce forecasts with high normative (how and why) and predictive properties.  You must assume that at some point you will be called upon to defend your results.   This requires that you document the rational for all design choices.

If statistical rigor is not attainable then next best criteria is assurance that you have made the best design choices you can make, and that no other forecast provider can produce a better forecast given the same constraints.   The most acceptable way of validating your forecast is to employ multiple independent approaches.   For example, if your model is based on primary research with a limited sample size you might also build an independent causal forecast based on econometric or demographic data.   Building forecasts for the same product or services’ using different logical dependencies is also a solution.   At the least you should use independent methods to establish upper and lower bounds by constructing ratios to known related variables.

3. Segmenting to Multiple Taxonomies – The design choices that you make includes the Order of Resolution setting the progression in which taxonomies are addressed, as well as the methods used to allocate to each of the items in the taxonomies.   The challenge is to produce final results that are intelligently informed from any taxonomy point of view.   While this may seem simple at first glance it may in fact present unexpected complexities.

Most client custom forecasts produced by technology market research firms are derivatives of an existing published forecast that will require some taxonomy transformation into the client’s taxonomies.   If the forecast is new and original it may involve collaboration with analysts and other stakeholders representing different points of view.   A request to change the distribution in one taxonomy may cause unacceptable results when views from an alternative taxonomy.    Additionally the taxonomies themselves may need to be modified during the development phase, or in subsequent updates.   Many forecast architects fail to anticipate this possibility even though it is quite common.   Including a process that can facilitate negotiations and handle taxonomy transformation is essential to meeting the objective of producing the forecast within the time and budget constraints.

4. Facilitating Collaboration –  While there may be cases in which the forecast is designed, developed and deployed by a single person, in most cases this is a collaborative effort requiring two sets of complementary core competencies, those of the Forecast Architect encompassing the mathematical, computational, and forecast methodologies, and those of the Knowledge Analysts encompassing qualitative assessment of the factors and trends influencing the products/services and markets being forecasted.

The very best Knowledge Analysts are extraordinary people, experts in their area, and as such, think at levels of abstraction that are not easily quantifiable.  They often develop their understandings intuitively rather than computationally.  The single most important task of the Forecast Architect is to build a bridge between the Knowledge Analysts qualitative assessments of the market and the quantitative requirements of the model.

There are a number of process elements that need to be established to facilitate efficient collaboration:

• Adapt to the Analysts Worldview – the forecast taxonomies, variables and methods must conform to the worldview of the analysts that will use or provide the input, and validate the output.   You need to build to their vision rather then ask them to conform to arbitrarily designed model requirements.

• Build a Qualitative Inventory of Influences – We all find it easier to view the world in comparative rather than absolute terms.  We intuitively know larger or smaller, faster or slower, closer or further, weaker or stronger.   Assigning rank is easier than assigning value.  Starting with a qualitative comparative view of the drivers and inhibitors influencing the market, and then moving in successive steps to estimating input parameter values is often the more efficient process.

• Provide Qualitative to Quantitative Transformation Tools – That are easy to use and intuitive such as a weights and scores approach.   Include the users in the design process to assure buy-in and acceptance.

• Base and Modify – This is often an iterative process.  Do not expect the first results to meet the analyst’s expectations.  Often the sensitivity to changes in the initial conditions will not be fully understood until after the first pass.   From that point on the analysts will be able to provide changes that will eventually produce acceptable results.    This is a learning process where interim results of the model providing feedback that will alter the analyst’s understanding of the relationships between the influences and the output.

5. Dealing with Constraints –There will always be limited budgets and tight schedules.  The solution is to establish a formal design and development process based on the following guide lines:

• Always begin by defining the client’s decision context and recognizing the constraints.
• Plan and document  your design choices
• Involve the users/analysts in the design process
• Make conservative estimate of the time required to complete tasks by all of the collaborator in the process
• Allow for unexpected changes in the objectives and/or specifications

Each of these challenges is easily met by applying the following three principles:

• Context – Understand how the forecast will be used, the dynamics and topologies of the market, and the design and development constraints.
• Collaboration – develop and facilitate the most efficient environment for melding complimentary core competencies.
• Clarity – Recognize the requirements and limits of the methodologies you chose, as well as the architecture of your design.

There is a natural process that governs the forecasting process.  Structure extracts information from data.   Information becomes knowledge in the light of experience.  And knowledge yields understanding through insight and introspection.   While it is clear that a forecast delivers data, information, and knowledge, it should also deliver understanding.   And that Understanding of the Future is derived from the forecasting process itself.

∞∫Δ Daniel Research Group © (2012)

_______________________________________________________________________________

Daniel Research Group is a market research firm specializing in the design, development and application of market models and forecasts for clients in the technology sector including supplier, investors, and other market research firms.  For more information contact Steve@DanielRG.com or visit www.DanielResearchGroup.com.

The Times are a-Changin’

One of the most important management metrics for technology vendors is the time it takes for a new product or service to attain specific market penetration levels.   The most common metric is the take-up time, usually defined as the number of months and/or years to go from x% penetration of the market to y% penetration of the market. Forecasting take-up time with high confidence is required for many critical business decisions.

Take-up times are getting shorter and the rate of change is accelerating in both the enterprise and consumer markets.   This acceleration is creating a new set of development and marketing challenges for technology vendors and a need for new approaches to forecasting market sizes, trends and segmentations.

As this chart first published by the New York Times in 2008  clearly shows, the cumulative curves are getting steeper and the market lives are getting correspondingly shorter.   More important, this is a relatively recent change.  It took approximately 50 years for the architectural and construction technologies responsible for the building of Gothic Cathedrals to spread in Europe.  It took about 50 years for steam power to replace human and animal power; about 50 years for canals to spread in the United States; about 50 years for railroads to replace canals; and  about 50 years for electrical power to replace steam.   For over 1000 years, the time it took for category-level technological innovation to spread remained at approximately 50 years.

So what has changed?  Why are market penetration rates accelerating?  Most analysts will answer, “because for the first time since the invention of the printing press, changes to the communication process itself have accelerated the rate that information about innovations diffuse in the marketplace.”   While this is true, it does not explain the real drivers and factors that define today’s markets.   Before addressing the real causes it is necessary to understand the major problem that this change has created for the technology market forecaster.

The most common process that many forecasters use to construct their forecasts is to  fit a curve to historical periodic unit data and extrapolate that curve.  Most of the time the curve is some form of the exponential function such as a Compound Annual Growth Rate (CAGR).   For most purposes this approach has sufficed since, for a sufficiently short segment of a bell-shaped curve, the error will be within the required forecast precision.   The chart below shows a Fisher-Pry Life Cycle market model for an innovation that has a 50 year life with an adoption growth rate of 20%.¹

Data for shipments in the 7^th through 12^th years were used to compute a CAGR that was then used to forecast the 13^th through the 17^th years.  The fit is reasonable.  The error in the 13^th year is only 13% usually acceptable in the 5^th year of a five-year forecast.   However, even in this example we can see that the error will increase if the forecast period contains an inflection point, and most significantly at the peak.  There are two other inflection points.  The earliest is at the point where accelerating growth changes to decelerating growth, and the final one is at the point where accelerating decline changes to  decelerating decline.

However, if the life of the market is shorter, the error can become significant.  In the chart below the estimated life has been shortened to 30 years.

It would seem that the solution would be to apply the classic life cycle models instead of the traditional growth models.   Unfortunately, the classic life cycle models have limits as well.   These models, while serving adequately for the past 50 years, are not complete.  The models harbor flaws that were not recognized or were ignored when life cycles were longer.  As life cycles shortened, the flaws in the classic models become problematic.

In order to understand the true nature of today’s markets we need to look back in time.  Building on three centuries of scientific, sociological and economic research, the modern theory of how new products and services are bought by consumers was first proposed by Everett M. Rodgers in his 1962 seminal work, Diffusion of Innovations. Many are familiar with the classic graphical representation of this model with its S-shaped cumulative curve and bell-shaped periodic curve, as well as the classic division of the market into five phases.

Rodgers defined diffusion as “the process by which an innovation is communicated though channels over time among members of a social system”.   From the start, most practitioners who applied this model to forecasting technology markets failed to recognize two subtle nuances of what Rodgers was saying, or not saying.  First, this is only half a model. It describes how information is diffused through a social system, as well as the end result; adoption. However it does not include the adoption decision making process itself.  While Rodger’s correctly implied that markets are heterogeneous with regard to factors influencing adoption, his five labels have been misinterpreted to mean that there are five distinct populations that successively influence the next population.   Even if this assumption of five distinct sub-populations were true it most likely would not produce the smooth curves predicted by the Roger’s model.  Rather the actual data would most likely have localized saddles, surges and bumps.

Geoffrey Moore in his 1991 book Crossing the Chasm, Marketing and Selling High-Tech Products to Mainstream Customer, addressed many of these issues and challenged the assumption that communication between sub-populations drives the adoption process.   More to the point is questioning the assumption that one can deduce the number and nature of the heterogeneity in the market from direct observation of aggregate behavior.  What if for any particular market there are 9 sub-population, or 15?   And what is the adoption decision factor that distinguishes each population from the other?

Four years after Rodger’s book was published, Frank M. Bass published the first of his papers presenting the Bass Diffusion Model (BDM).  His initial model simplified the market by segmenting it into two sub-populations; adopters who adopted because the innovation is new (“innovators”), and adopters who adopted because others did so first (“imitators”).²

Unfortunately, in later work, the interpretation and labeling of the BDM was changed. The “innovators” became adopters who adopted due to mass media external influences and the “imitators” became adopters who adopted due to word-of mouth internal influences.  For the past four decades the BDM and its extensions have been the foundation of most advanced theoretical and practical technology market forecasting work.   Yet the BDM also suffers from the same flaws as the Rogers model:

1. The BDM focuses only on the communication aspect and fails to provide a mechanism that explains the adoption decision process, and
2. The BDM does not reflect complex heterogeneity in the market regarding any adoption decision factors at the individual level.

What is surprising about this is that the original Bass work pointed directly at the missing process – imitation!   Imitation implies observation and that is what is missing from these models.  Advocacy does influence potential adopters, both externally via mass media and internally via solicited or unsolicited word-of-mouth.  However the single most important event that can influence one person, an individual consumer or business decision maker,  in choosing to purchase a product or service is direct observation others buying or using that product or service.

Furthermore, observation of  adoption behavior is bounded by practical and contextual considerations.  Adopters certainly do not observe the entire market.  Reports and claims of penetration by vendors are simply mass media content and can be considered external advocacy.   We observe those we know, or seek out, whom we have prequalified as having attributes that certify their behavior as being worthy of imitation.  Daniel Research Group has named this observed group the Local Relevant Reference Group (LRRG).

For any innovative product or service, each individual defines both the composition of  the LRRG, and the minimum number of people in the LRRG that need to be observed as having adopted the innovation, as a prerequisite for adoption by that person.

In the Daniel Research Group Individual Adoption Model (DIAM), the number of people in the LRRG who need to be observed adopting is symbolized by ∆. This quantity is unique for each individual, for that product or service, at that moment; and is a function of the collective demographic, economic, cultural and sociological factors influencing that person.   The resulting model then simplifies to:

For each time period:

If number of people in the LRRG that have adopted > ∆ then adopt

If not, then do not adopt.

For any market there is a distribution of ∆s that represent the effective heterogeneity in the market.  Since these are integer values, they represent threshold values that will account for the saddles, surges, bumps and even market failures that the older models could not predict.

Finally, how does this model explain the accelerating rate of adoption?  While the size of the LRRG has increased due to significant changes in the communication process, such as social networking, the ∆s have not.  Simply put, it takes less time today than in the past for adopters to reach their ∆s because they are observing larger LRRGs.

Applying this model using agent-based methods such as cellular automata will create market models that have better descriptive, predictive and normative properties which may be applied successfully to today’s rapidly evolving technology markets.   Parameters may be estimated using simple survey or panel methodologies, and eventually, analogous methods.

——————————————————————————————-

1.               The growth/shape parameter of the Fisher-Pry model is the constant rate of change of the proportion of the market that has adopted to the proportion that has not.  This is not normally a quantity that the typical technology market research analysts is familiar with.  However it can be computed from, or the model calibrated to, more conventional management metrics such a CAGR, market shares or even absolute value of shipments or customers.

2.               The Bass Diffusion Model is a two parameter logistic curve.  As applied today, the parameters are q representing the strength of the external influence, and p representing the strength of the internal influence.   The values of p and q have been empirically derived for many consumer, commercial and technical products and services and are often applied to create models  using analogous approaches.

 

A New Diffusion Approach

Since its introduction in 1969 the Bass-Diffusion Model (BDM) has become the standard Life Cycle model providing the mathematical foundation for much of the research in the field, as well as the basis for many forecasting approaches and method.  Today, there are over 150 published versions of this model.   The conceptual framework underlying the BDM states that the number of individuals that adopt an innovation is determined by two parameters:

*     p is the coefficient of innovation and is a measure of the influence of information communicated to the potential adopters from external sources such as mass media.

*     q is the coefficient of imitation and is a measure of the influence of information communicated to the potential adopters from internal sources such as inter-personal networks.

While this conceptual framework, and its mathematical expression in the BDM, has proven to be extraordinarily accurate at the long-term category level, its descriptive, normative and predictive properties decrease for shorter time scales.   Often actual data exhibits significant anomalies such as saddles, chasms, long tails, and even market collapses that cannot be explained by any endogenous economic, demographic, or technological influence.  In retrospect, this should not be surprising since at its most fundamental level Diffusion of Innovation and the adoption process is a complex adaptive system producing variance from inside the process itself.

There are two reasons why the BDM may fail to accurately predict market results for shorter time scales:

*     The model is not explicitly an adoption decision model.

*     The model does not sufficiently account for population heterogeneity

First, the BDM is essentially a model of diffusion of information and makes no clear distinction between influence arising from advice and advocacy, and influence arising from observation of prior adoption.   The values of the parameters are not determined by analysis of causal influences but rather are computed from historical data by regression.  Many studies have been conducted over the past 40 years producing a large inventory of p and q values for different classes of product and services.   In many applications, the practitioner simply applies known parameters for an analogous product to a new model.

Secondly, the observed category level market results are the aggregation of thousands, if not millions, of individual adoption decision events that are binary.    The key question is what causes the phase state change from not-adopted to adopted.  If the population of potential adopters is truly homogeneous, as many of the models assume, then the chance of any one individual adopting at any moment in time will be determined by some probability function.  The resulting observed pattern of aggregate adoption will be analogous to pop corn popping in a microwave, and will exhibit the classic cumulative “S” and non-cumulative bell shaped curves.

If however there is heterogeneity in the population in terms of a variable or parameter that influences the individual probability of adoption, then local discontinuities and anomalies will occur due to the introduction of thresholds.  In this case, the aggregate adoption process for a portion of the population will be analogous to a sand pile that slowly builds as one grain at a time is added, and then suddenly collapses.   This will introduce discontinuities, anomalies and increase overall variance into the aggregate results.

For example, consider a sub-population comprised of individuals that are identical in all respects expect for sensitivity to price.   One half will buy if the price is lower than $100, the other half will only buy if the price is lower than $80.  The launch price is $99 and one-half of the population adopts very quickly.  The price is then lowered $1 each month.  When the price finally reached $79, 20 months later, the other half quickly adopts.    The effect of this sub-population within the total population causes an observed deviation from the smooth curve predicted by a Life Cycle model such as the BDM.

Many of the Life Cycle models do make an assumption of heterogeneity in the population.  Rodger identified five groups defined in terms of observed behaviors with some linkage to demographics and personality traits.   The BDM specifies two populations defined in terms of communication channels and behaviors.  However, these are example of descriptive segmentation with no causal relationships.   Some will argue that in the BDM and the Rodgers model, an individual that is an “innovator” will be a first adopter, i.e. innovativeness causes adoption.  An alternative view would be that individuals that are first adopters are “innovators”, i.e. there is no proof of causality, just the assignment of a label based on correlation.  This also begs the question, are innovativeness and imitativeness traits, or behaviors?

Individual level causality has yet to be rigorously established for the BDM and other Life Cycle models at the category level.   All of us possess traits that give rise to innovative behavior in some situations and imitative behavior in others.  Likewise, all of us are susceptible to influences from external and internal sources at different times.  Context determines the mix.  An adopter can be an innovator for one product and an imitator for another.   Likewise, an adopter can be influenced more by external messages for one product, and by internal messages and signals for another.   Context is dependent on the attributes and characteristics of the innovation, the worldview of the adopter, as well as role of the adopter (e.g., individual consumer, contributor in a social or business group, decision maker in a social business group)

www.DanielResearchGroup.com