What is the most important question for Data Science (and Digital Transformation) - KDnuggets

• Read original article here

With so many buzzwords surrounding AI and machine learning, understanding which can bring business value and which are best left in the lab to mature is difficult. While machine learning offers significant power in driving digital transformations, a business must start with the right questions and leave the math to the development teams.

By Polly Mitchell-Guthrie, VP of Industry Outreach and Thought Leadership, Kinaxis.

Deep learning, active learning, transfer learning, reinforcement learning, name-your-preferred-flavor-learning, AutoML, heuristics, stochastic gradient descent – does this list send a shiver of excitement up your spine? Have you just completed a boot camp or graduated with your degree in data science, computer science, machine learning, etc., so you’re armed and ready to sling some code and build one of these models?

Consider for a moment a different perspective, that of someone far up your leadership chain, the corporate executive. You may feel that they don’t understand what you do. You’re probably right. Because for most of them, these lists of what is trending in AI/ML and data science make them feel beaten downplaying buzz word bingo on a constantly changing board. Just when they were ramping up on machine learning, suddenly everyone is referring to AI, and they can’t sort out exactly how the two are related, let alone what to do about it.

Because as leaders, their challenge is to decide which new buzzwords bring business value and which are best left in the lab to mature. Feeling competitive pressure, many leap ahead and adopt corporate initiatives around data science or artificial intelligence/machine learning (AI/ML), even as they are still trying to figure out their meaning. To keep it simple, I often bundle these buzzwords and call them “fancy math,” and as passionate as I am about their power to make a positive impact on business, I also believe that starting with math misses the point.

Digital transformation is a strategic imperative for business today, but math-driven technology alone will not drive transformative change, which also requires a strong business vision and strategy. The most strategic step is to set the vision and identify the highest priority problems to solve, which helps people understand the “why.” The most successful initiatives clearly communicate what McKinsey calls a “change story.” Once the business problems are well-framed, I encourage executives to leave the math under the hood for data scientists, because which math method to use is an important but tactical decision. Leading with math amounts to letting the tail wag the dog.

But McKinsey also found that those organizations with successful digital transformations also are more likely to use fancier math. This probably explains why LinkedIn’s 2020 Emerging Jobs Report cites AI Specialist as the #1 growing job title, with 74% annual growth (followed by Robotics Engineer at #2 and Data Scientist at #3). Math can indeed move the world, but it is imperative to give it a chance to succeed. You, newly-minted data scientist, are in hot demand, but it will take a lot more than just hiring you to actually impact the business. Because as these McKinsey consultants write in the Harvard Business Review, Building the AI-Powered Organization requires many other core practices to ensure the adoption of your work. And their research shows that only 8% of organizations are doing what it takes to make that happen.

Einstein is said to have quipped:

If the venerable math genius prioritized problem formulation over math, shouldn’t the rest of us mere mortals?

First set the vision and frame the business problems and desired capabilities, which is the hardest step, because business is prone to describe symptoms (e.g., we can’t get our orders fulfilled in time, we spend too much time waiting for supplies) but struggles to synthesize them into a clearly-defined business problem (we need to shorten our planning cycles and improve responsiveness). Setting the vision and framing the problem gives the organization the clarity to proceed, and helps adoption when people understand the “why” of their work and changes to it.

Why does business problem definition matter so much? Too many expensive corporate AI initiatives fail because a team of data scientists is hired in a vacuum, with no vision for why they exist or strategy to deploy them. You scribble equations, sling code, and build a complex model that solves the wrong business problem. It is too easy to trip along common paths to failure: ignoring the input of end users, forgetting to engage IT on deployment, or building a black box model neither executives nor regulators understand. Getting clear on what problem you’re charged with solving, how to define a better decision through the benefits you want to achieve, and what assumptions and criteria impact that solution are all part of framing the problem and preventing large data science investments from failing to deliver their promised value.

Defining the problem also entails defining the solution – what constitutes a better decision? It may seem that the objective of data science is a more accurate model, but can you tie this objective to tangible business benefits? Ultimately, the C-suite cares about objectives like maximizing revenue, minimizing cost, or improving customer satisfaction. Getting clear on metrics is an important step in ensuring that the problem is framed correctly and that you can measure the success of a better business decision against the AI vision.

Another pitfall involves data, the bane of your existence. You know it is a critical and very time-consuming step, but sometimes leadership may need a deeper understanding of the importance of access to the data you need and the time needed to prepare it for analysis. Paying attention to data is not very sexy, but because AI/ML models are predicated on consuming large amounts of data, they will need to understand that they will fail without being properly fed.

Too many businesses try to jumpstart their problem-solving journey by looking at a fancy math menu: “Let’s find a vendor who does AI/ML to help us.” But without being clear on the vision, what problem the firm needs to solve, whether math can help solve it, and whether the available data supports that approach, the fancy math cart is being put before the problem horse.

While vendors are criticized for falling into the trap of being a hammer in search of a nail (“we build great optimization software”), buyers can fall into that same trap. If your company has a corporate AI/ML initiative and compares vendors based on a list of algorithms, then they run the risk of letting a tactical choice of a math method drive a strategic decision of what problems are being solved. Naturally, as a data scientist, you want access to the latest, greatest toys, but many other factors should drive what software is the best choice.

It’s easy to be seduced by arguments that a vendor (or fancy math menu item) will provide the “best” model. But has “best” been defined for your company for this problem? For example, is the best model the one with the most predictive accuracy? Consider the Netflix Prize, which offered a $1 million dollar prize to the team who could improve the accuracy of their movie recommendation model by 10%. In 2009, a winning team walked away with the cash, but Netflix later admitted that they never deployed the models. Their increased accuracy "did not seem to justify the engineering effort needed to bring them into a production environment.” They made a million-dollar mistake in assuming that the only criteria that would matter was accuracy.

Deep learning can indeed dazzle with its predictive accuracy and is making great progress every year. But have you considered the cost of that accuracy, and whether leadership is willing to pay those costs for this business problem? “Brittle” models requiring a lot of maintenance, that are complex for IT to deploy, black box, and computationally slow to run may not give the expected return on investment.

I may sound like I have “algorithmic aversion,” but nothing could be further from the truth. I have spent the last 20 years working in analytics and am deeply invested in advanced analytics methods like machine learning and optimization. When properly applied, these methods have remarkable abilities to render better business decisions (see this article I wrote on “How machine learning can heal a supply chain”).

I’m drawn not only to more accurate models, but also pattern-finding and decision-making far beyond the cognitive capacity of the human mind. While we must hold our models accountable to the human biases they can reflect if properly built they also have great potential to reduce human bias and yield more consistent, objective decisions. And math can automate many decisions, taking off our plate routine decisions and saving time for the exceptions that most need human skill and judgment.

But the key is the application of math to solve business problems, not the math as an end in itself. There is no universal standard for “best” – no one math method is best for every industry, every company, even every situation in the same company.  Organizations must first decide what problems they are trying to solve, along with criteria and metrics to measure the results against the larger vision for transformation.

Occam’s Razor holds that the simplest solution to a problem is likely the best. It is important to stay accountable to both the criteria and metrics and this principle, because fancier math may seem more fun. Depending on the business problem, the savings from a more accurate answer may justify greater complexity, in spite of longer compute times or a black box method. But for other problems, a good-enough answer reached quickly and widely and easily explainable may be the best choice.

If your executives feel stuck in buzzword bingo, how can you help them win? And why should you care, if they can’t even understand the words on the board?

Do you understand your company’s business, the competition, trends disrupting the industry, and what challenges your executives face? Leading large organizations and setting strategy is not easy work. Make yourself invaluable by learning the business and translate how fancy math can transform it. Winning means changing the rules of the game – it’s not about the buzzwords, it’s about delivering business value.

I believe in the power of fancy math to deliver true business value and enable smarter organizations, but the best way to deliver this value is for leadership to start with the strategic decision of setting a vision and defining the business problems and criteria, and then (and only then) choose the math. Human intelligence and vision, combined with fancy math (and data scientists like you), can truly transform a company. When you’ve all taken this hard journey together and delivered results, then you can truly call “bingo.”

Bio: Polly Mitchell-Guthrie (@PollyMGuthrie) is the VP of Industry Outreach and Thought Leadership at Kinaxis, the leader in empowering people to make confident supply chain decisions. Previously she served as director of Analytical Consulting Services at the University of North Carolina Health Care System and in several roles at SAS. She’s been very involved in INFORMS, the leading professional society for analytics and operations research.