Logo

The Data Daily

31 Data Science and Analytics Predictions from 24 Experts for 2021

31 Data Science and Analytics Predictions from 24 Experts for 2021

We polled 24 experts and received 31 data science and analytics predictions for 2021.

As part of the second annual Solutions Review #BIInsightJam, we called for the industry’s best and brightest to share their data science and analytics predictions for 2021. The experts featured here represent the top data science and analytics solution providers with experience in this niche. Data science and analytics predictions have been vetted for relevance and ability to add business value as well. These are the 31 best predictions from the dozens we received. We believe these are actionable and may impact a number of verticals, regions, and organization sizes.

“We’ll see more data-driven companies leverage open source for analytics and AI in 2021. Open source analytics technologies like Presto and Apache Spark power AI platforms and are much more flexible and cost-effective than their traditional enterprise data warehouse counterparts that rely on consolidating data in one place–a time-consuming and costly endeavor that usually requires vendor lock-in. Next year will see a rise in usage of analytic engines like Presto for AI applications because of its open nature – open source license, open format, open interfaces, and open cloud.”

“More technology companies will adopt an open source approach for analytics compared to the proprietary formats and technology lock-in that came with the traditional data warehousing approach. This open analytics stack uses open source Presto as the core engine; open formats such as JSON, Apache ORC, Apache Parquet and others; open interfaces such as standard JDBC / ODBC drivers to connect to any reporting / dashboarding / notebook tool and ANSI SQL compatibility; and is open cloud. Since Presto runs as a query engine without storage, it natively aligns with containers and can be run on any cloud.”

The next evolution of analytics brings a federated, disaggregated stack

“A federated, disaggregated stack that addresses the new realities of data is displacing the traditional data warehouse with its tightly coupled database. The next evolution of analytics foresees that a single database can no longer be the solution to support a wide range of analytics as data will be stored in both data lakes and a range of other databases. SQL analytics will be needed for querying both the data lake and other databases. We’ll see this new disaggregated stack become the dominant standard for analytics with SQL-based technologies like the Presto SQL query engine at the core, surrounded by notebooks like Jupyter and Zeppelin and BI tools like Tableau, PowerBI, and Looker.”

Data scientists will fiinally be free to uncover “the next best decision”

“Data science teams will move to the forefront of all enterprises and become the driving factor in an organization’s analysis and insights for the next best decision. This means supporting these data science teams with the right data, at the right time will become a priority task for internal IT organizations in 2021 – as data continues to be viewed as their untapped oil. However, if improperly managed, data can easily become an enterprises’ greatest liability. Unleashing data for data teams in a quick, efficient, and legally responsible way will pave the future for competitive differentiation and consumer confidence.”

How to interpret, trust, and explain your models and how to detect and mitigate bias (both the legal and factual kinds) will, rightfully, become threshold issues for anyone using data about humans in their modeling work. Unpack the black box.

The uncertainty introduced by COVID-19 will continue to expose the importance of monitoring models in production — a critical component of an effective MLOps process. Just as (to my knowledge) no model predicted the vast and varied implications that COVID has wrought on the world, no model will predict its ongoing effects for global business and consumer behavior. Manual monitoring solutions will not cut it. MLOps — specifically the capability for automated model management and monitoring — will become more widely acknowledged as the best solution.”

“Will rise from the devastation of the COVID crisis. When the pandemic turned the world economy upside down, organizations were forced to invest rapidly in business intelligence and data catalog software just to understand what the heck was going on and make basic business decisions. As we enter a new normal in 2021, they’ll be able to leverage those reactive investments to do proactive business process optimization.”

“In 2019, most people found math, stats, and data to be boring, intimidating, or irrelevant. But after a year of scrutinizing margins of error in election polling, watching exponential COVID case curves and learning about “R-naught,” those topics certainly seem important and impactful, and more accessible too.”

Data accessibility will be the difference between business success and struggle

“Visualizations can be great business tools, but if the data they’re sourcing from isn’t complete or accurate, it’s easy to overlook business-critical insights. Employees need to have confidence in their business decision, especially during an important and ever-evolving business climate, and that all starts with data. When employees have easy access to the data they need, they are able to embed intelligence into everything.”

“CDOs and CAOs will increasingly view their datasets and analytics beyond the boundaries of cloud and data platforms. While the expense of data movement will motivate data teams to leave data where it was born, many will pursue ways to engineer their analytics pipelines to source data from multiple public and private cloud platforms, and across cloud storage, data warehouses, and data lakes.”

“While some have cast doubt on the idea of Citizen Analysts/Data Scientists, there is an increasing desire by executives to push down machine enhanced decision making to a much broader population of information workers. The resulting flow of curated data and actionable intelligence will create a large pool of accidental analysts who are able to benefit from data-driven insights without unsustainable retraining requirements.”

Analytic Centers of Excellence are critical to success in 2021

“Agility, standardization, reuse and ROI make Analytic Centers of Excellence (ACE) a key strategic initiative for 2021. Business intelligence and analytics have proven their impact on data-driven businesses. Maturing that strategy often requires organizational maturity as well. Committing to an ACE structure makes teams faster and more collaborative, and brings diverse skill sets to a single group promoting innovation. A well-organized ACE allows for additional standardization of process, models and data while avoiding constant rebuilding by siloed teams, resulting in significant cost savings. “

Data analytics will help improve the supply chain

“We saw a classic case of a broken supply-and-demand chain at the onset of the COVID-19 outbreak in March. Demand for specific products surged, while supply plummeted due to manufacturing factory shut-downs, causing consumer panic and disruption and delays in the supply chain. The manufacturing and supply chain industries have been ripe for digital disruption for years, but now, accelerated by the pandemic, supply chains must operate in a distributed and data-driven manner ahead of the holiday season and in 2021.

Manufacturers can start by tightening their sales and operations planning (S&OP) processes to provide a transparent view of inventory, factory production schedules and delivery times for improved decision making at a centralized level. Leveraging data analytics to get a glimpse into real-time data for existing supply chain processes, distribution networks, and transportation solutions can also help find pain points and opportunities, which in turn can proactively improve the supply chain before issues arise. Looking at current and predicted delivery lead-times, logistics expenses, and inventory assets are also good data points to find a path to efficiency and effectiveness and make data-driven decisions. The more transparency and collaboration between the various levels and players involved in the supply chain, the better organizations can meet demand.”

“Containers” everywhere for analytics and AI

“Containerized application deployments and Kubernetes have started to gain traction with enterprises increasingly moving away from traditional Hadoop based data lakes. While moving away, enterprises are realizing the benefit of abstracting the physical infrastructure while also adopting public clouds for agility. Vendor lock in is a concern but at the same time a uniform toolset across environments is a must to reduce spending on the expertise required to operate across environments, such as hybrid and multi-cloud. Container based deployments for compute abstraction alongside new abstraction services for storage anywhere, will be the solution of choice for enterprises moving off Hadoop.”

AI and analytics provided by the same platform (team)

“AI and Analytics capabilities were provided by different platforms / teams in the past. Over the years, we are seeing the platform is converging and the AI team is more focused on the algorithmic side, while AI & Analytics platform teams merged to provide the software infrastructure for both analytics and AI use cases.”

“Emotion is a key factor affecting customer behavior and has a strong influence on brand loyalty. Therefore, it is increasingly useful for companies to find a way to measure emotions of customers during their decision-making processes. Emotional analytics focuses on studying and recognizing the full gamut of human emotions that includes mood, attitude and personality. It employs predictive models and AI/ML to analyze human movements, word choices, voice tones, and facial expressions.

The author will discuss how emotional analytics can help companies build a more holistic customer profile, understand how to influence emotions and develop customized product and services tailored to individuals. Sentiment analysis about products and services, across geographies, social networks, and review web sites enables companies to better understand and improve their customer satisfaction level. Using emotional analytics, companies can better understand how their marketing and services influence emotion in order to provide more positively engaging customer experiences.”

Greater focus on data and analytics is here to stay

“COVID-19 forced us to address an enormous amount of challenges and in doing so, drove an intense focus on data and analytics to provide much-needed needed insight. From genome sequencing to understand the virus and its mutations, to the spatial analytics helping us track its spread, to the prediction of its peak states, to anticipating the needs of healthcare workers to managing the rollout of trillions of dollars of government aid. Because the virus has also served to place a spotlight on other areas where society is failing its most vulnerable, we are now more acutely aware of the need to be more curious about the problems the world faces; homelessness, addiction, healthcare, security, social justice and more. Post pandemic, data science best practices also have the potential to provide efficient and innovative ways to also help tackle some of these fundamental real-world problems.”

“A study from Junaid showed a decrease in spending for data and analytics in 2020.This is a misstep as now more than ever it is important to get the greatest possible productivity out of employees. To compensate, analytics professionals will focus on consuming IT capabilities from companies that they know and trust, in particular they’ll look to larger enterprises that are known entities vs emerging players. With such tight budgets, there is no room to risk working with an unproven partner.”

Data democratization will become the new norm

“It’s the job of the CDO to ensure expansion of growth across the entire business. This can be achieved by providing structured data that people can actually use. A successful CDO should democratize data so that it’s accessible and understandable by people. A good CTO will complement the CDO by creating the necessary tooling to find the required data. This means giving users a set of visualization tools and reporting tools that allow them to get after the data to run insights. As we move into 2021, we’ll continue to see further and tighter collaboration between these two roles, driven by necessity. If you have tools with bad data, you’re exacerbating the data challenge. If you have limited tools, only a small subset can do anything with the data.”

The “analytic divide” is going to get worse

“Like the much-publicized “digital divide” we’re also seeing the emergence of an “analytic divide.” Many companies were driven to invest in analytics due to the pandemic, while others have been forced to cut anything they didn’t view as critical to keep the lights on – and a proper investment in analytics was, for these organizations, analytics was on the chopping block. This means that the analytic divide will further widen in 2021, and this trend will continue for many years to come.  Without a doubt, winners and losers in every industry will continue to be defined by those that are leveraging analytics and those that are not.”

“Businesses are already starting to democratize data across the organization, arming more employees with real-time insights. I see this accelerating with both a cultural shift and a technology shift.  This trend will result in data gurus and citizen data scientists with deep domain knowledge increasingly joining forces as part of a more holistic and effective problem-solving process.”

In the future, employee engagement will depend heavily on bringing more employees to the data analytics fold

“There have been many drawbacks to the new work environment created by COVID, but in the remote work set up, speed of insight has been increased as employees feel empowered and are able to accomplish more while getting to insights faster than before. In other words, data democratization has and will continue to spur better questions and new thinking as employees are uplifted. The more access to data that workers have with the right analytic skills and technology, the better questions they will be able to ask, the more complex problems they will be able to solve  – and not have to work so hard to do so.”

Citizen analysts will increasingly upskill to become data scientists

“The growing complexity of most industries and companies also means that once we see self-reliance in terms of developing IT processes or using analytics, there will quickly be a huge push to expand that skillset further. With the market erratically changing from month to month there will be a much greater emphasis placed on data science than ever before. This, in turn, will drive more citizen analysts to upskill to become data scientists.”

Basic data skills will start to disappear from resumes – and be gone by 2030

“We’ve entered a third wave of analytics, and with it the expectation that business users can interact with data without the help of an expert. Very soon, if you’re unable to marry hard data with business context to define and execute a strategy, you’re going to struggle in the workplace. The ideal candidate for businesses in 2021 and beyond will be a person who can both understand and speak data — because in a few short years, data literacy will be something employers demand and expect. Those who want to get ahead are acquiring these talents now.”

Business skills will become more critical than ever for data scientists

“Data scientists will need to speak the language of business in order to translate data insight and predictive modeling into actionable insight for business impact. Technology owners will also have to simplify access to the technology, so that technical and business owners can work together. The emphasis for data scientists will be not just on how quickly they can build things, but on how well they can collaborate with the rest of the business.”

The need to combine data integration with clean data visualizations shapes the decision to purchase business intelligence software

“The data explosion, automated data analysis, and cloud computing adoption have accelerated the demand for highly customized BI solutions. It’s time for self-service BI applications for visual data discovery and quick insights on the go, which do not sacrifice the quality for being smart. The trend predicts the need for forward-looking, data-driven, automated decision-making data analysis via visualization to have a strong foothold in the competitive business environment. Data integration with BI applications and the multi-cloud data strategy with upscaled security will bring significant changes in a short time and renovate the BI landscape.”

“Augmented analytics is the next step in BI evolution, the power to truly democratize data analytics. We went from raw data displays to visual graphical displays to interactive dashboards. The next step is natural language conversational analytics: ask your data a question and get back an answer, either in generated language, graphic images, actions, or some combination of all of the above. We’re already seeing companies like TesiSquare and Lumenore empowering people to use their own language to understand their data. Conversational analytics requires a potent analytical engine underneath like Vertica to give human comfortable responses to complex analytical queries, but it’s happening now. And it’s going to become more common over time.”

“Interactive business intelligence platforms that analyze as well as visualize are going to become more ubiquitous and more powerful. Instead of just asking you to plug in an analytical database, they’re going to start including one, or embedding one in the case of analytics as a service. We’re already seeing the first wave in Vertica-powered analytics-as-a-service offerings like Domo, Anritsu and GoodData. As the big three’s cloud stacks become more top to bottom, smart organizations looking to avoid cloud platform lock-in need business intelligence tools that work cross-platform or cross-deployment, but still offer the same analytical power.”

Knowledge graphs will be the foundation for automation

“The humble knowledge graph will be recognized as foundational to automation efforts throughout the enterprise. Knowledge graphs will move from simply housing knowledge to actively generating knowledge, including machine-readable data. For example, by using link prediction algorithms (predicting future behavior or an unobserved fact) or using graph embeddings (using the structure and patterns in the graph as information in itself), knowledge graphs will have broader applications — from drug repurposing to behavior-based criminal investigations, to ‘anti-recommendations’.”

Data-driven cultures ditch the dashboard for more dynamic feeds

“Over the past few years, companies have experienced an explosion of technology transforming the modern data stack. Cloud data warehouses, data pipelines, and machine learning platforms that accelerate the availability and accessibility of data. 

But, despite these advances in data systems, most companies still struggle with the most strategic goal of a digital transformation: building a more data-driven culture. So far, digital and data transformation efforts have mostly focused on the effective capture and management of raw data, but ignore the evolving needs of decision-makers. Unfortunately, this means that users are left with only two choices: learning to use highly advanced data science tools that require deep data literacy, or static dashboards that can only examine a few factors at a time.

As data volumes and complexity increase exponentially and the drive to make decisions based on data accelerates, companies will look past these static, manually curated dashboards. Business owners will demand new ways to consume data and insights, and we’ll see rapid experimentation with and adoption of platforms that provide more dynamic views of data, mirroring the experience of continual news feeds and personalized recommendations.”

“Business intelligence has no value unless a company translates improved visibility into improved action. The BI industry is still working to deliver on this charter, and it will come to fruition with the convergence of BI tools and transactional middleware. We have experienced a revolution in business applications, with solutions for everything from sales and accounting to HR and contracts, and an accompanying rise in middleware to allow interaction with the application data. Running in parallel are BI tools, which function like read-only middleware to splice data across the various business applications. The next frontier is the convergence of transactional middleware and BI middleware, enabling closure of an action loop where business users can analyze their data in a BI dashboard, identify the action they need to take, and then initiate that action without ever leaving the BI environment.”

The biggest change to the data analytics industry in 2021 will be a new focus on scenario analysis and simulation technologies

“Additionally, more solution builders will emerge as core AI stacks, like Nvidia’s and Intel’s AI stack, are utilized on top of public cloud. The jockeying for position in the hype cycle will continue, but just as the blush is wearing off on full-on AI, Quantum Computing will illegitimately emerge as the next hyped technology, which will not be ready for prime time. Other exascale-ready technologies can fill this gap needed for applications to solve STEM problems.”

Images Powered by Shutterstock