Big Companies Are Embracing Analytics, But Most Still Don’t Have a Data-Driven Culture

For six consecutive years NewVantage Partners has conducted an annual survey on how executives in large corporations view data. This year, results show nearly every firm is investing in some form of analytics, and most are seeing value. But only one-third say they have succeeded in creating a data-driven culture. And many fear disruption by firms better equipped to use AI. Almost four in five respondents said they feared disruption or displacement from firms like those in the fintech sector or firms specializing in big data. The technology judged most disruptive is AI — by far. Seventy-two percent chose it as the disruptive technology with the most impact — far more than cloud computing (13%) or blockchain (7%).

For six consecutive years NewVantage Partners has conducted an annual survey on how executives in large corporations view data. Each year the response rate increases, and the reported urgency of making effective use of data increases as well. This year the results are both more encouraging and more worrisome than in the past.

Six years ago, the primary focus of questions and answers in the survey was big data, which was relatively new on the business scene. In the 2018 survey, the primary attention has moved to artificial intelligence. AI is now a well-established focus at these large, sophisticated firms. There is both a stronger feeling that big data and AI projects deliver value and a greater concern that established firms will be disrupted by startups.

The survey includes senior executives from 57 large corporations. The industry group with the most firms represented in the survey is one of the most data-intensive: financial services. Companies from the life sciences, manufacturing, telecom, and online industries also participated. The actual respondents are changing somewhat from the first surveys: It has always involved a high proportion of C-level executives responsible for data, but this year chief data officers are 56% of the respondents, up from 32% last year. Only 12% of firms in the 2012 survey had even appointed a chief data officer.

While AI gets the headlines here and elsewhere in the world, the survey addresses both big data and AI. Terminology comes and goes, but the constant is a data explosion and the need to make sense of it. Big data and AI projects have become virtually indistinguishable, particularly given that machine learning is one of the most popular techniques for dealing with large volumes of fast-moving data. It’s also the case that statistical approaches to AI — deep learning, for example — are increasingly popular. Therefore, we view traditional data analytics, big data, and AI as being on a continuum. Virtually all of the respondents (97%) say they are investing in these types of projects.

Perhaps the best news in this survey is that companies continue to believe they are getting value from their big data and AI projects. 73% of respondents said they have already received measurable value from these initiatives. That number is half again higher than in the 2017 survey, which suggests that more value is being achieved as companies grow familiar with the technologies.

The types of value received are perhaps consistent with other previous types of technology. Consistent with our view that big data and AI are extensions of analytical capabilities, the most common objectives — and those most likely to achieve success — are “advanced analytics/better decisions.” Thirty-six percent had that as their top priority, and 69% of those had already achieved success with the objective. Better customer service and expense reduction are also common objectives. Just over one-quarter of firms (27%) are pursuing some combination of innovation and disruption, speed to market, or data monetization initiatives. Data monetization programs had the lowest priority and the lowest percentage of success (27%).

One of the greatest issues for concern in the survey for large enterprises is the risk of disruption from new entrants. Almost four in five respondents said they feared disruption or displacement from firms like those in the fintech sector or firms specializing in big data. The technology judged most disruptive is AI — by far. Seventy-two percent chose it as the disruptive technology with the most impact — far more than cloud computing (13%) or blockchain (7%).

Another important and continuing issue is the slow speed with which these established firms make the shift to a data-driven culture. Virtually all respondents (99%) say their firms are trying to move in that direction, but only about one-third have succeeded at this objective. This gap appears every year in the surveys, and the level of success hasn’t improved much over time. Clearly firms need more-concerted programs to achieve data-related cultural change. Many startups have created data-driven cultures from their beginning, which is a key reason why large, established firms fear disruption from them.

One of the approaches that firms have established to deal with data-driven disruption and change is to establish new management roles. However, there is still a lack of clarity about how different data-oriented roles (chief information officer, chief data officer, chief digital officer, chief analytics officer, etc.) relate to each other.

With respect to the chief data officer role, there is substantial disagreement about the major responsibilities of the role and what types of backgrounds are appropriate for CDO jobs. Thirty-nine percent say their CDO has primary responsibility for data strategy and results, but 37% assign that responsibility to other C-level executives, and 24% say there is no single point of accountability for it. In terms of backgrounds, 34% of respondents believe the CDO should be a change agent from outside the company, while 32% believe the person should be a company veteran from inside the firm. Role clarity in senior data-related roles is critical for both leading AI/big data projects and accomplishing cultural change. And while all respondents believed it important, the majority of firms still lack an enterprise data strategy.

This continuing rise in the importance and challenges of big data is one of the most important features of contemporary economy and society. The survey results over time provide interesting and useful documentation of this revolution. The rise of AI is only exacerbating this trend. The keys to success are to determine how your firm should respond, assign clear responsibilities for data strategy and results, and then move ahead to execute the needed changes in a systematic and effective fashion.