Big data analytics in supply chain: Tackling the tidal wave

Technology is making it possible for supply chain organizations to gather enormous amounts of information from an expanding variety of sources. Billions of data points are pouring in from supply chain network nodes, multiple retail channels, the industrial Internet of Things ... the list goes on and continues to grow. The aim, of course, is to analyze that information to gain visibility into opportunities for innovation and improvement. But few companies are actually deriving sustainable value from the supply chain data they are accumulating. Instead, they are struggling with how to ensure the quality of their data, how to analyze it, and how to make practical use of what they learn from it.

New research conducted by CSCMP's Supply Chain Quarterly; Arizona State University; Colorado State University; Competitive Insights LLC, a provider of advanced supply chain analytics solutions; and the consulting firm lharrington group LLC investigated the current state of supply chain data analytics and the strategies organizations are adopting to harness the power of big data. Over time, this annual survey will track, in the aggregate, companies' progress in using big data analytics in supply chain management.

Article Figures

[Figure 1] Satisfaction with dataEnlarge this image

[Figure 2] Types of technologyEnlarge this image

[Figure 3] Types of analyticsEnlarge this image

[Figure 4] Impediments to implementationEnlarge this image

[Figure 5] Beneficial impact to dateEnlarge this image

[Figure 6] Near-term investmentEnlarge this image

In this article, we outline the project's principal findings. Among the topics we'll discuss are respondents' satisfaction level with their data; what technologies companies are using for data analysis; the challenges and benefits associated with managing growing volumes of supply chain data; and finally, where respondents stand now as well as their priorities for near-term investment in data analytics.

Satisfaction with data proving elusive
The survey was conducted in June 2017 via an e-mail invitation to readers of CSCMP's Supply Chain Quarterly and subscribers to a newsletter produced by Competitive Insights. A total of 126 fully completed, usable responses were compiled to obtain the survey results. (For more information about the research, see the sidebar.)

There's no question that most companies are seeing a significant increase in the amount of data they are collecting. When asked to characterize the increase over the past three years in the volume of supply chain data available to them, 36 percent said it was moderately high, while 38 percent said it was high or very high. But, as is often the case, quantity does not necessarily equate to quality.

How satisfied are supply chain managers with the data they currently have to run their supply chains? A majority of survey respondents report being at least moderately satisfied with their supply chain data in terms of availability, usability, integrity, and consistency. However, the combined "favorable" numbers (moderately high, high, or very high level of satisfaction) were not overwhelmingly higher than those for the correlating unfavorable scores (Figure 1).

Interestingly, only a very few respondents report being very satisfied in all four data attribute areas: availability of data (3 percent); usability (2 percent); integrity (6 percent); and consistency (4 percent).

"Data is the foundation the 'house' sits on," observes Richard Sharpe, chief executive officer (CEO), Competitive Insights. "The survey results clearly show that there are cracks in that foundation—cracks in companies' ability to bring data together, integrate it, have confidence in it, and believe that it is consistent. To take advantage of big data analytics, we have to do better in all four categories."

If companies are only partially satisfied with the data they're getting, the logical question to ask is, exactly what software solutions are they currently using to gather that data? Overwhelmingly, the tools in heaviest use are not advanced analytics or business intelligence solutions. Nor are they operational point applications like warehouse management systems. Despite the availability of an array of sophisticated analytics software, the most widely used tool for managing supply chain data today is still Excel spreadsheets (Figure 2).

Despite their dependency on spreadsheets, users aren't necessarily happy with it as a data management tool. "Our survey shows that Excel is very negatively associated with user satisfaction in terms of usability, integrity, and consistency of data," Dale Rogers, ON Semiconductor Professor of Business at Arizona State University, reports. "The problem with Excel is that everyone builds their own spreadsheets, so there's no consistency, no single version of the truth that's shared across departments. That makes it very difficult to trust the data sufficiently to make big decisions across departments."

The survey also found that, not unexpectedly, large companies rely on their enterprise resource planning (ERP) systems to run the financials of the business. But for supply chain professionals, ERP has shortcomings.

"Supply chain folks don't really like ERP that much," notes Zac Rogers, assistant professor of supply chain management at Colorado State University. "Many do not think the data that comes out of ERP systems is very useable for their purposes. They find it too rigid. They also lose the granular operational data they used to get with older supply chain point solutions. And as with spreadsheets, they don't necessarily trust the ERP data—at least not to manage their supply chains the way they need to."

When talking about big data analytics, supply chain organizations typically rely on five basic kinds of tools:

• Descriptive—tells you what is happening
• Diagnostic—tells you why it's happening
• Predictive—tells you what will happen
• Prescriptive—tells you what should/could be done
• Cognitive—uses machine learning to tell you what should be done

By far the most widely used of these five is descriptive analytics, according to the survey results. Sixty-one percent of respondents report using this type of analytics tool. Moreover, use of the four other types of analytics tools lags descriptive applications by a significant margin. According to the survey, companies that deploy these tools regularly, frequently, or heavily use them as follows: diagnostic, 42 percent; prescriptive, 36 percent; predictive, 31 percent; and cognitive, 18 percent (Figure 3).

Supply chain organizations that limit themselves to descriptive analytics are unlikely to make much progress. "Descriptive data tools are absolutely necessary," Sharpe says. "But they are only good for telling you what has already happened. To get greater insight, companies need to move into the other types of applications."

Adoption of these more advanced analytics tools takes time, however. To that point, how far have companies come in their use of big data analytics in their supply chains? How mature are they not just in implementing the technologies, but in realizing benefits?

The answer is "not very far," as the survey numbers indicate:

• 28 percent of companies are in the "developing" stage, with one or more big data analytics initiatives underway.
• 24 percent are in the "early" stage, conducting proof-of-concept testing to determine benefits and drawbacks.
• 20 percent have not adopted big data analytics in their supply chain.
• Only 2 percent rank themselves as mature; that is, in the transformational stage of adoption and benefits.

One interesting note on the maturity question: Different industry sectors are at varying stages of not just maturity, but also plans for adoption. On a maturity model scale of 1 to 6, no industry was a 6; in fact, none reached the top two tiers—"advanced" and "transformational." The technology sector ranked highest at 3.7, just short of "somewhat advanced," while the lowest was life sciences, at 2.3 solidly in the "early" stage. Machinery manufacturers ranked themselves just slightly ahead of life sciences, and third-party logistics companies (3PLs) and retailers fell about halfway between "early" and "developing." (Other industries were not represented in significant numbers.)

Commenting on these rankings, Sharpe observes that some industries are more cognizant of the value that can be derived from supply chain data analytics, while some show little interest in moving beyond what they traditionally have done. For example, although life sciences (which also includes health care and pharmaceuticals) scored lowest in maturity, respondents in that industry put very high or moderately high priority on investing in big data analytics. "They understand they need to advance quickly, because of how fast their industry is changing, so they're making these investments," he says.

Roadblocks and benefits
As noted earlier, most companies are still in the early stages of implementing big data analytics in their supply chains. For many, it's not for lack of trying; respondents identified a number of impediments. Some have to do with technical issues, such as integration with siloed or data warehousing initiatives, deemed a significant or very significant impediment by 47 percent of respondents. Others that were high on respondents' list of concerns included the need to invest in software and hardware; analytical tools' level of difficulty for users; and security and other risks (Figure 4).

Other impediments deemed to be significant or very significant by a large number of respondents are business management, as opposed to technical, issues. One is the acquisition of talent and expertise (41 percent), which may correlate to respondents' comments about the difficulty of using analytical tools. Another is the level of management commitment and support (44 percent). Almost the same number (43 percent) named uncertainty about return on investment or value as a major impediment. "Usually things like this are big sweeping initiatives that come down from the top, but generally [big data analytics initiatives] originate at mid-level with people who are actually using data, and they have to sell it upwards," says Zac Rogers. "We're seeing in the results the difficulty of upselling the concept of big data to managers who don't understand why they should invest time and money in it."

Those that have implemented big data analytics say they're encountering some roadblocks to getting significant value from those efforts. Just under two-thirds (64 percent) agreed to some extent (somewhat agree, agree, or strongly agree) that overcoming insufficient data-capturing capabilities in their legacy systems was hindering their ability to harness value from supply chain data. "I think they're expecting value will come out of this, but they don't necessarily see a clear way to get there because of their legacy systems and frustration of working with their ERP system," Dale Rogers says.

In many cases, these roadblocks stem from the difficulty of integrating data from external suppliers as well as other internal departments, and ensuring the quality and consistency of that data. For example, 67 percent said difficulty breaking through internal data silos to access information that currently is not integrated or shared is hampering their ability to gain value from supply chain analytics, and 71 percent said the same about the difficulty of achieving consistent data quality and integrity.

Despite all those challenges, respondents are seeing some payback for their efforts. When asked how much beneficial impact their companies have already realized from big data analytics in their supply chains, their answers varied widely across eight critical areas (Figure 5). The big winner was profitability: 89 percent of respondents said big data analytics has already provided at least some positive impact in that area, with 44 percent reporting a significant or very significant beneficial impact on profits. And 6 percent of respondents went so far as to say big data analytics has had a transformative, supply chain-wide impact on their company's profitability.

Customer service and inventory management also scored well, with 47 percent and 42 percent, respectively, reporting a significant or very significant impact. Areas that have seen less beneficial impact so far include risk and resiliency management, end-to-end supply chain collaboration, and visibility to total cost-to-serve.

The research found that there was a strong correlation between the benefits already achieved and the type of analytics tools respondents were using. "What we found is descriptive analytics had negative associations with almost all benefits—meaning it didn't move the dial much," Zac Rogers says. "Conversely, prescriptive and diagnostic analytics had positive relationships with almost all benefits. For example, the companies using these more advanced applications report having better supply chain visibility, planning models, risk management, and customer service." The survey also found very strong positive correlations between predictive analytics and achieved benefits in end-to-end supply chain communication, supply chain visibility, risk management, demand planning, and cost-to-serve visibility.

Future plans and priorities
The final group of questions queried respondents on their plans and priorities for the future. For example, the survey asked about their companies' priorities over the next 12 months in regard to big data analytics projects. For many, getting the fundamentals right will be a high or extremely high/critical priority. That includes improving data accuracy (47 percent), data accessibility (46 percent), data availability (45 percent), and data consistency (43 percent).

Although descriptive analytics has little impact on benefits, more than one-third of respondents plan to make a moderate investment and 23 percent plan to make a large or very large investment in that technology. That's not a bad thing, says Sharpe, but descriptive analytics only provides "a look in the rearview mirror; the more important question for respondents is, what are we going to do about it?" Many respondents are indeed looking to answer that question: 29 percent plan to make a moderate investment and 22 percent will make a large or very large investment in diagnostic analytics, while 26 percent plan a moderate investment and 25 percent will make a large or very large investment in prescriptive analytics (Figure 6.)

Regardless of where they planned to invest, the great majority of respondents said they expect to see at least some benefits in all of the areas shown in Figure 6 over the next 12 months. While few foresee a transformative, supply chain-wide impact for any of those areas—percentages ranged from a low of 7 percent for profitability to a high of 13 percent for end-to-end supply chain collaboration—respondents clearly believe their data analytics efforts will pay off in the near term. More than half of respondents said they expect fairly significant, significant, or very significant beneficial impacts from applications of supply chain data analytics. In descending order, they included customer service (62 percent), profitability (60 percent), visibility to cost-to-serve (59 percent), inventory management (59 percent), risk and resiliency management (52 percent), demand planning (52 percent), and end-to-end supply chain collaboration (51 percent).

Overcoming barriers to success
One of the practical takeaways from the survey responses, says Dale Rogers, is that supply chain organizations need to devote more resources to overcoming the technical, organizational, and business impediments to big data analytics. "People want it, but there are a lot of problems and barriers, and they don't really know how to implement it," he says.

As noted earlier, one of the biggest barriers is the difficulty of getting consistent, clean, trustworthy, and appropriate data. That's a complex problem with many different causes, but one important aspect, Sharpe says, is to first determine exactly what data will be needed, and then establish a process for governing that data, including identifying the subject-matter experts who can validate that data.

One key to getting support and funding is to clearly understand a big data analytics project's purpose and objectives. That means supply chain organizations must convincingly demonstrate the business benefits of such initiatives to company leadership, Sharpe says. "You have to show that what you're ultimately trying to do with supply chain data analytics is to make the enterprise more successful and profitable."

Want to participate in the study? We're looking for more supply chain professionals to participate in future surveys. Information will only be used in the aggregate. For more information, contact Dr. Zac Rogers or Tami Kitajima.