Today, data science is a crucial component for an organization's growth. Given how important data science has grown, it’s important to think about what data scientists add to an organization, how they fit in, and how to hire and build effective data science teams.
Data science resounds throughout every industry and has reached the mainstream media. I no longer have to explain what I do for a living as long as I call it AI — we are at the peak of data science hype!
As a consequence, more and more companies are looking towards data science with big expectations, ready to invest into a team of their own. Unfortunately, the realities of data science in the enterprise are far from a success story.
NewVantage published a survey in January 2019 which found that 77% of businesses report challenges with business adaptation. This translates into ¾ of all data projects collecting dust rather than providing a return on the investment. Gartner has always been very critical of the data science success and they haven’t gotten more cheerful as of late: According to Gartner January 2019, even analytics insights will not deliver business outcomes through 2022, what’s the hope then for data science? It’s apparent that for some reasons making data science a success is really hard!
Regardless of whether you manage an existing data science team or are about to start a new greenfield project in big data or AI, it’s important to acknowledge the inevitable: the Hype Cycle.
The increasing visibility of data science and AI comes hand in hand with a peak of inflated expectations. In combination with the current success rate of such projects and teams we are headed straight for the cliff edge towards the trough of disillusionment.
Christopher Conroy summarised it perfectly in a recent interview for Information Age: the renewed hype around AI simply gives a false impression of progress from where businesses were years ago with big data and data science. Did we just find an even higher cliff edge?
Thankfully, it’s not all bad news. Some teams, projects and businesses are indeed successful (around 30% according to the surveys). We simply need a new focus on the requirements for success.
The first important fact is that there is not one priority or one thing which characterises successful companies in the data science space.
There are generally five wider themes for data science success. Some themes are well understood and widely discussed like the importance of culture. Other themes highlight misconceptions which causes for example the underestimated importance of technology in the retention of data science teams. The importance of technology is a key factor to why most successful companies in the data science space are tech-companies. Not because only tech-companies have the means to solve the technology requirements but because they have a better understanding of the challenges and their suitable solutions. But don’t despair! Thankfully, with the growing maturity of cloud solutions and PaaS this can be achieved in any industry.
The first differentiator for commercial success with data science is the business’s motivation to start a team in the first place. The motivation makes all the difference between the endeavour becoming a vanity project or being aligned with business strategy.
From my experience, data science requires a business motivated by a vision rather than short term goals. This is founded in the complexity and timelines of data science and data innovation. Data science is still a synonym for innovation which is difficult to deliver against a quickly changing priority of short term goals. Companies who look at data science to achieve their vision also demonstrate a better understanding of what data science actually is and how it adds value.
The motivation will also be the stand or fall of any culture or business change efforts which come with a business’s transformation to become data driven.