The promise and pitfalls of AI and deep learning

Artificial intelligence (AI) is no longer the stuff of science fiction. The majority of businesses today are using AI in some form, and those that aren’t have plans to in the near future. Deep learning, a technique that’s largely responsible for the widespread adoption of AI, has gained particular momentum as of late, with leading companies like Google, Microsoft and Amazon introducing deep learning across their services and replacing their existing machine learning systems with deep learning-based models.   

A recent O’Reilly study on how businesses are putting AI to work through deep learning, found that 54 percent of businesses predict deep learning will play a large or essential role in their future projects. Another 38 percent expect to use some amount of deep learning, and only eight percent of businesses said deep learning wouldn’t play a role in their future projects. Applications of AI and deep learning are already being applied across a variety of industries including art, finance, biology, healthcare and robotics, with many organizations leveraging the technology to make sense of structured or semi-structured data or text. 

Along with data science applications, deep learning is also increasingly being applied to power chatbots and other audio and speech applications. Additionally, the technology is frequently being used to enhance existing analytics and machine learning systems already in use, such as recommendation systems, search engines and business forecasting tools. In fact, these deep learning applications are already so ubiquitous that almost every consumer has no doubt made use of the technology (and most likely, they were completely unaware of it). 

While AI and deep learning have undoubtedly permeated both the consumer and business landscapes, a concerning pattern has emerged that’s curbing further industry innovation: More and more organizations are discovering how shallow the AI talent pool is.   

Because deep learning remains a relatively new technique – one that hasn’t been part of the typical suite of algorithms employed by industrial data scientists – we shouldn’t be too surprised that the main factor holding companies back from incorporating AI and deep learning is a skills gap. Furthermore, the skills gap is nowhere near as significant as it was several years ago. According to the Global AI Talent Report 2018, LinkedIn data shows that there are currently 22,000 PhD-educated AI and deep learning researchers, and a report from Tencent claims that there are over 200,000 active developers in the industry, along with another 100,000 students and academic researchers.

Still, our research indicates that 20 percent of companies find a lack of skilled AI and deep learning professionals to be the prime bottleneck of AI adoption, and when asked if they’ve hired specifically for deep learning applications, only 11 percent responded affirmatively. As a result, many businesses are compensating by hiring developers with solid software skills and hoping they can simply learn on the job. In fact, a majority of businesses in our survey (75%) reported that their company is using some form of in-house and external AI and deep learning training program. Thirty-five percent of businesses indicated their company went a step further and used either formal training from a third-party or from individual training consultants or contractors.   

To overcome the AI skills gap, and continue to advance widespread application of the technology, it behooves leaders in the industry to make deep learning more accessible to developers who may not have a PhD in the subject and/or domain experts from other disciplines. In fact, our survey data, coupled with the aforementioned LinkedIn and Tencent data, clearly demonstrates that democratization of AI and deep learning is essential, and that it may already be occurring. Although 20 percent of our research participants indicated that a lack of skilled AI and deep learning professionals was a bottleneck, the gap between 22,000 PhDs and the millions of developers needed suggests that hiring should be a much bigger problem than it’s currently being reported.   

Additionally, tools for using deep learning have come a long way in recent years, and their increasing simplicity and accessibility can make it easier for non-industry experts to embrace AI and deep learning technology. Open source deep learning tools like TensorFlow, Keras and PyTorch have proved particularly helpful, with 73 percent of respondents in our research indicating they’ve begun experimenting with deep learning software. Advances in cloud technology have also made building deep learning applications far more feasible for a wider demographic, with 70 percent of businesses in the O’Reilly survey citing cloud services as an essential component of their AI initiatives. 

Lastly, to perpetuate the democratization of AI and deep learning, established organizations should continue providing formal, in-house training programs for software developers interested in incorporating deep learning into existing products. In fact, there’s a growing market in China of startups that provide multi-week training programs to turn select recruits into deep learning engineers, which could prove especially valuable to emerging companies with fewer in-house resources.   

In his paper "Deep Learning: A Critical Appraisal," Gary Marcus argues that if AI is going to make progress, it must be supplemented by other techniques. In conjunction with democratizing AI and deep learning, today’s organizations and businesses leaders need to keep in mind that current and future AI systems will likely draw from many other methods, some of which have yet to be discovered. This could include Bayesian methods and deep learning, and neuroevolution and gradient-based learning. In fact, reinforcement learning is emerging as an especially exciting machine learning method, particularly for sectors like industrial automation, with 45 percent of businesses in our survey reporting that they’re already using or evaluating reinforcement learning. 

Perhaps most importantly, businesses and their leaders need to acknowledge that even as AI and deep learning progress and democratization of the technology continues, the shortage of trained engineers will likely persist, so training at every level is paramount. Colleges and universities need to continue offering more courses and degree programs, and companies should prioritize the development of deep learning skills within their own staff. In doing so, everyone stands to benefit, as the applications and resulting benefits of the technology are truly infinite.