We have gobs of data, nearly limitless cloud compute, and ever-improving machine learning algorithms, so what on earth is holding companies back from succeeding with big data? “Talent, talent, talent,” says Dr. Kirk Borne. “The limiting factor is talent.”
To be sure, Borne has done more than most when it comes to fostering data science talent. Fourteen years ago, before his recent stint at Booz Allen Hamilton or his new gig at DataPrime, Borne helped create the nation’s first data science degree program at George Mason University.
That proved to be a pivotal point for data science in academia, and today, there are thousands of Bachelors, Masters, and PhD-level data science programs all over the country, not to mention an untold number of bootcamps and certificate programs.
With all that effort put into minting new data scientists, the world should be awash with the unicorns now. Yet the data science gap still persists. According to Borne, it all comes down to insatiable demand.
“There’s almost exponential growth in the talent pool,” Borne tells Datanami. “But unfortunately, so to speak, for the business world, the number of job opportunities that businesses are creating is also exponentially growing, but at a faster pace than the talent production. The difference between two exponentials is effectively still an exponential, and so there’s still this rapidly growing talent gap.”
We’ve come a long way from the early days of big data, Borne says.
“The technologies we’re using today are not what we used eight years ago,” he says. “Remember, Hadoop was all the rage, and everyone had to learn Hadoop and hardly no one even mentions that word in a sentence anymore.” (Well, almost no one!)
The Hadoop experiment was painful for some, to be sure. It’s not exactly clear whether we had to go through it (a good case can be made that we did). In any case, the important thing now is that big data technology is much better and more usable today than it was 10 years ago or even five years ago, and that’s a huge benefit to organizations that want to work with big data.
“[Technology] is not a hindrance anymore. It’s the enabler,” Borne says. “What’s happening now is we’re in this platform revolution phase, where with cloud, you can basically have almost infinite scalability. You don’t have to buy your own supercomputer–you just rent it for minutes or hours or days you need it, and then you give it back.”
Today, organizations have a vast array of compelling big data tools and AI technologies to choose from, most of which is running in the cloud. The three bigs–AWS, Microsoft Azure, and Google Cloud–not to mention upstarts like Snowflake and Databricks and the hundreds of other companies in this vibrant ecosystem, are all participating in the rise of a “function as a service,” which has dramatically opened up access to big data tech.
“For example, you need to build a recommender engine, or you need a chat bot?” Borne says. “You just basically call this function that someone else has already built. Like, why build it yourself?”
The accumulation of pre-built functions and ready-to-use data science platforms on the cloud is opening up all kinds of new business opportunities. Two kids in a garage spinning up huge jobs to crunch data with SQL analytics, or train a machine learning model with the freshest data, can now control it by API call from a single console. They’re now competing with multi-billion-dollar multinationals. It’s lowered the bar–and raised the stakes for everybody.
“The platform revolution has enabled plug and play of all kinds of different applications and tools and services, Borne says. “You just put them together the right way to serve a business community and you’re off to the races, basically.
Alas, amid the compelling riches of cloud-based big data tech, the limiting factor is that persistent talent gap.
Signs of the talent gap show up all over the place. It shows up in more than 11,000 listings for data scientist at Glassdoor, and more than 15,000 at Indeed. It shows up in the want ads labeled “urgent,” in increasingly desperate recruiters, in the median salary for a mid-level data scientist of $130,000 (as reported last year by Burtch Works), and in data scientists changing jobs every 2.5 to 2.8 years (also per Burtch Works circa 2019).
All of which points to a continued seller’s market for data scientists–or whatever you want to call the technical folks who get their hands dirty with data and machine learning algorithms (LinkedIn’s number one job last year was “AI specialist” while others prefer machine learning engineers or research scientist).
This is, of course, great news if you happen to be a data scientist. In that case, everybody wants you! You’re a rock star! But if you happen to fall on the demand side of that equation–well, tough luck, buddy.
“If a business is just stitching together technologies and not understanding the science of A/B testing or the science of customer behavior modeling and all those kinds of things, then you’re just really going to fall flat,” Borne says. “You still need people who understand the benefits, risks, and appropriate business applications of this stuff, not just the technologies and the coding skills. And so the talent gap is really the fundamental roadblock right now.”
When Borne left BAH earlier this year, he considered being a retired person. But that didn’t last long, as he signed on as Chief Science Officer with new AI startup DataPrime.
DataPrime’s goal is help connect data science professionals with potential jobs using–you guessed it–data science techniques. That’s why they brought Borne in–to help design and implement that platform.
“It’s basically a recommender engine,” Borne says of DataPrime. “We’re basically recommending employee job candidates to employers.”
DataPrime is now accepting profiles of potential candidates. Individuals, in any of the data professions, who are looking for jobs can upload their resumes, including skills, interests, experience, education, desires, preferences, and requirements. Similarly, companies that are looking for data scientists can sign up, and enter their requirements and job descriptions. The data science magic that Borne is helping to create will then seek to match the work experiences and skills of candidates with what companies are looking for. DataPrime aims to deliver hyper-personalization in talent search, both to data professionals and to recruiters in those professions.
However, not all data scientists are the same, and not all data science positions are the same. The interesting bit about DataPrime will be its capability to understand the nuances of a particular job opening, and to find a candidate who will be a good fit.
The best candidate may not always be obvious, Borne says. “I was on a panel and one of the members of the panel was the director of the cancer research for a university on the West Coast,” he recalls. “She said the best hire she ever made for her research lab was not a cancer research scientist or a lab technician. The best hire she ever made, she said, was an artist.”
The creativity of the artist helped bring new ideas and experiences to the technical folks who were designing the cancer treatments and cancer programs for people. That’s not to say that artists will be recommended for data science jobs. But it does give you insight into the non-intuitive ways that the company will be looking to read resumes, job experiences, and life experiences.
Borne encourages anybody who’s done anything even remotely connected with data to check out DataPrime, “whether it’s a cloud engineer, machine learning researcher, business intelligence, dashboard creator, data storyteller, database engineer–anything with the word data analytics or AI anywhere in their job title–we want your job profile on our platform, and so will the recruiters,” he says.
Why Data Science Is Still a Top Job