Challenges and Best Practices in Data Analytics - Datamation

• Read original article here

It’s a study in contrast: on one hand, we hear that the power of data analytics is nearly miraculous; the cool, metric-based insight from an our analytics software will propel us to business success. On the other hand is the reality of data analytics in real world organizations: confusion, poorly designed systems, and executives operating by gut instinct rather than data-driven insight.

Why is it so hard for organizations to optimize their data analytics?

To understand the challenges in data analytics – and suggest some best practices – I spoke with four top experts:

Suer: It really depends. MIT CISR did some research a while back, and one of the things they discovered, that I thought was fascinating, was that only 28% of companies were really ready to transform, 51% were still in silos, so the way Tom thinks about it, they’re doing departmental analytics, and 21% were doing things that were duct tape and band-aids. So it’s really interesting. In terms of the use of AI and ML, it’s quite interesting. There are companies like Stitch Fix who are doing kind of amazing things for consumers, but there’s even people like Nordstrom who’ve managed to connect their supply chain, and their purchase data, and actually predict what you want, and I don’t even have that on Amazon. So, there are companies that are succeeding,

Davenport: The International Institute for Analytics does benchmarking of analytics maturity across organizations. And so God had decreed that all analytics maturity… All maturity models should have five levels, and so I complied with that, and level one is really screwed-up, and level five is really sophisticated. Sadly, the average across all companies assessed, and I think there are over 225 so far, is with two-digit, double-digit precision, 2.25.

Iansiti: There’s a bunch of different organizations that actually have done a lot, and my sense also depends a fair amount on the industry which you are in. Right? So I totally second Tom’s sort of analytics on AWS. Amazon is amazing in what they do and how they have done it, both across AWS and retail in fact. I think that they’re incredibly advanced. But even some more traditional companies, like if you go out… If you look at someone like AMex, for example, they’ve done a lot, or if you’re look at…Or some of the credit card companies have sort of have had to get their act together in some ways, and they tend to be fairly advanced in different stages. People in financial services, if they don’t actually invest in this, they know that they’re essentially out of business pretty quickly. More traditional CPG companies just like the Kraft Heinz or someone like that is not gonna be… You’re not gonna find them as this far along. But again, one thing that’s really interesting is that it’s accelerated [by the pandemic]

Hinchcliffe: I do a lot of surveys of CIOs, and one of the top issues, and for a couple years now, analytics has been high in the top five of priorities in terms of fueling it. I just did another CIO survey, it’s number two. Right? Digital transformation’s number one, analytics is number two in terms of priorities. So it’s gotten just a little bit higher.

But it’s like all powerful technologies, analytics separates the leaders from the laggers. And we see it, most organizations are just in the developing phase.

Iansiti: I think right now, we’re in the mode that to do things well, you’re gonna do things at scale, and you do things across a whole variety of different processes. So it’s not about building sort of one cool algorithm to do some prediction in marketing. It’s fundamentally about doing hundreds of these algorithms. I was talking to somebody at Fidelity, for example. They have something like 120 different project leaders that are dedicated to deploying, essentially, some digitally-enabled processes at scale.

Hinchcliffe: I once had a CIO tell me, “My dream is to be able to take everything that we know and make decisions better and faster than our competitors.” Right? “If can deliver that to the table, I can ride that forever.”

Suer: What’s happened in the legacy software world is we’ve required the companies to build their platform themselves. They bought this product over here, and this product over here, and then they had the job of assembling it. And most stumbled along the way and projects became narrower and narrower. So I think the big issue is, is that we need the data scientists, but before we need the data scientists as Marco has implied, we need the data engineers, or we need to somehow acquire something that allows you to go from the initial stage of the data pipeline to clean and pristine data.

Davenport: I think we certainly need data platforms, but we also need kind of workflow and decisioning platforms because, I don’t know, asking people to have a separate step for making their work intelligent, doesn’t seem to be successful. You’ve gotta just really make it easy. So if I’m a salesperson and I’m trying to decide, “Well, who do I call on today to sell my products and services?” You can just pick somebody at random, or you can say, “Oh, okay, we’re using Salesforce, and my boss has kindly bought this Einstein product that gives me a predictive lead scoring model. And gee, why wouldn’t I choose the most likely company to buy my product on a list that’s been prepared for me.” It’s just too easy to ignore, and I think in more and more cases, we’re gonna have to embed analytics and AI into these transactional and decisioning platforms if we’re gonna get them to be used successfully.

Hinchcliffe: I think most organizations are still working on it, and they are… They are in an emerging state, and they’re still having big challenges getting the data from wherever it is to wherever it needs to be, alright. So we have a lot of sclerosis in most organizations, and data ownership is the problem. Data access is an issue. It’s still too siloed. Our data foundations are not in good shape, but I think we’re now seeing the rise of things like customer data platforms, and other solutions that are allowing organizations to systematize, to make data consistent, to make it shareable, because we’re seeing a lot of under-utilization of one of the most valuable and irreplaceable assets in our organizations, which is data, right. And so I think we’ll see basic progress, so that we have stronger open data foundations, and that we’ll also have more skill level in our organization.

We’re gonna have much more sophisticated and workers in five years. They will have that Cloud experience with these now open data platforms with analytics tools. They’re not gonna be doing everything in Excel anymore. They’re gonna be using the next generation of analytics platforms like Snowflake. And so we’ll see… We’ll probably see that human dimension addressed.

Davenport: In terms of what’s happening in the future, I think more use of external data. We have for decades been primarily focused on internal data. If we want to know what’s happening in the world and with people who aren’t our customers yet and so on, we’ve gotta get more external. I think there will be much more availability of external data. You already see some change there. Ironically, maybe a move back to smaller data and smaller models than we have now. There is, I think in this latest AI system called GPT-3 for language creation, 175 billion neuron nodes in this deep-learning model. It’s kind of gone a little too far, one might say. [chuckle]

Iansiti: So they were going to be developing some of these tool sets to organize the data in a way that where the access is much more nuanced than it’s been in the past. And I feel that, also as traditional companies come up to speed on this as they are doing, they’re much more thoughtful and conservative in many ways than digital native companies, they were tiny things just a decade ago. And so from that perspective, I think that what I’m hoping we’ll see in 2025 is a lot more responsible data platform architecture and design. Not that right now everyone’s being irresponsible, but we certainly have some room to grow, I think in that domain I would say, so hopefully.

Suer: But one of the things that’s really interesting, I don’t know if Tom and Marco saw this, but we went from the year I was born and I think Tom was roughly born, from 55 years for the life of an average public company to 20 a few years ago, and last year it dropped to 10-and-a-half years for a public company. So I think the winners are really good at doing analytics and data and things like that, and so the legacy organizations have to figure out quickly how they’re gonna respond or they may become irrelevant to the market