The Rise Of Data Culture

• Read original article here

Companies use data. This resounding truism is of course not news in and of itself, this has always been the case; even before the dawn of computers, we know that bookkeepers used ledgers and files to record business transactions.

Ever since commercial business has had access to IT backbone power from mainframe and personal computers (an era that started somewhere between the 1950s and 1960s), companies have been using data to store, backup and scrutinize business operations.

But computers are not just for tape backup procedures anymore, today we use data in a huge variety of different ways for analytics, business decision making and as an ingestion stream to build our new approach to intelligence. Given our progress thus far then, are we reaching a point of information management penetration where data intelligence maturity is directly correlated to higher levels of commercial business success?

Data intelligence companyCollibra has a vested interest in answering that question with a resounding yes and so, logically, has invested its press-facing expenditure in research aligned to attempt to validate the point. Given that surveys are (arguably) never undertaken (and certainly not publicised even when independent) if they fail to deliver the payload that the sponsoring organisation desires, should we give any credence to Collibra’s latest analysis in this space carried out with IDC?

Collibra’s IDCstudy does indeed ‘find’ (let’s say suggest) that data intelligence is crucial to making informed business decisions and driving better business outcomes. But data intelligence company surveys that tell us that data intelligence is important are not worth much. But what is (arguably) more interesting is the suggestion that nearly two-thirds of respondents say that they have challenges with ‘identifying & and controlling’ data sources in their organization.

If we live in a world where almost no companies (5% in this study) say that they have zero organizational data governance challenges, then we have to stop and ask what is happening in all the other firms – do they grab a bit of data ingestion, data orchestration, data deduplication and data management on a sort of piecemeal basis now and again? Is it a bit like staff bonding days and end-of-year parties i.e. nice to have, but not life-changing if they get missed?

The study in question here says that gathering data intelligence is important, but how organizations activate that intelligence is what sets them apart. Organizations with higher levels of maturity across data intelligence management, data cataloging, data governance, data quality are likely to realize higher business benefits and performance versus peers.

The most mature organizations achieve three times the business benefits of their peers, with the most significant improvements across their ability to innovate, adhere to regulatory compliance requirements and reduce their time to market for new products and services. These organisations also see an increase in trust around data thanks to improvements in data quality, leading to the faster delivery of actionable insights to accelerate decision-making.

The study also suggests that the most data-culture-mature organizations have clearly defined roles and responsibilities for data and analytics, take a holistic approach to data intelligence, always expect intelligence about data to be available when making data-driven decisions, measure results and have four times the adoption compared with less mature organizations.

Data culture is critically important for mature organizations and those organizations that make data culture a priority across the organization are better able to drive data-driven decisions and outcomes.

The study found that 86% of respondents had a dedicated team responsible for data culture that was focused on building data literacy. The least mature organizations were six times more likely to have no dedicated team responsible for data culture. Additionally, the most mature organizations had a higher representation of groups outside of IT participating in data intelligence initiatives and were more likely to have a chief data officer leading overall data strategy.

“The findings from IDC show that while organizations recognize the need for data intelligence, adoption and cultural barriers remain the biggest challenge to maximizing value and sustained business success,” said Stijn ‘Stan’ Christiaens, co-founder and chief data citizen at Collibra. “When organizations have executive leadership prioritizing data culture and are actively engaging a variety of stakeholders in data intelligence, digital transformation efforts succeed. Making an investment in data culture is the most significant move an organization can make to ensure efficiency, productivity and a strong competitive advantage.”

To help organizations understand their data intelligence maturity and make better decisions about how to invest in successful data cataloging, data governance, data quality, and data culture initiatives, Collibra partnered with IDC to develop the Data Intelligence Assessment Tool.

The assessment enables organizations to get a window into their data intelligence maturity relative to their peers and provides a customized report with recommendations for next steps to optimize their data intelligence journey.

Technology analyst house IDC tells is that in order for data to generate value, it must be considered in context. IDC data intelligence specialist Stewart Bond says that the most mature organizations understand that making investments in data catalog, lineage, governance, quality, and overall data intelligence structures will deliver higher quality and improved business outcomes.

“The greatest challenge and opportunity for every organization today is improving their data intelligence maturity,” said Bond.

Dom Couldwell is head of field engineering at real-time data platform company DataStax with its specialist approach to enterprise-grade Apache Cassandra database implementations and deployments. Talking about his experience working with real world data stack orchestration and management experiences, he suggests that for so many companies today, the most common complaint from [data service] product owners and individual software application developers is that access to data is holding them back.

“Too much data is locked in the data swamp, where users can’t actually use it. We have understood for years the importance of data and the benefits it can provide when used correctly, but the emphasis has been on storage rather than leverage,” explained DataStax’s Couldwell.

“An effective data strategy has to take into account the roles of a police officer (for your most sensitive data), that of a teacher (to enable the business to understand, interpret and leverage the data they are being provided) and soccer field referee (to quickly resolve issues when - not if - they arise) to prevent any potential problems from derailing projects. Alongside putting the right guide rails in place, an organization should take advantage of a best-in-class data stack, no or low code tooling… and an API-first approach to unlock data as much as possible,” he added.

Concurring with Couldwell’s position and keen to extend this discussion further outwards is data science guru Nick Jewell in his position as a technology evangelist for Incorta, a firm known for its self-service unified data analytics platform. Jewell suggests that even when a business does quite diligently collect data, it often struggles to make decisions with it across its entire operations base.

“In the real world, while a firm might connect (and so collect data) with the view of being data-driven, teams often still rely on gut instinct or anecdotal evidence in place of hard facts,” explained Jewell. “The reason for this is often latency. If data moves too slowly from business applications to analytics platforms to be of use in making decisions, then people won’t wait. This can apply to critical strategic decisions and to tactical or operational decisions alike. If your business teams find it hard to get data results quickly, then they won’t pull that information or use it effectively.”

Addressing this data latency and closing the gap between getting raw data into providing analytic insight or recommending an action, becomes a source of competitive advantage in a fast-moving, digital-first world.

So then, what processes have to change in order for companies to use their data more effectively - and what should they change? Often, the greatest bottlenecks to meaningful change are the legacy data pipelines that were built to extract, transform and load data from systems of record into systems of insight. These pipelines were fragile, over-complex technical ‘spaghetti’ that were costly to implement and even harder to maintain as business processes evolved over time.

"Technology innovations such as in-memory analytics, open source data formats and analytics query accelerators have made these pipelines largely redundant. By streamlining the processes needed to acquire and analyse data, you can make it easier to get data through to those that need it,” clarified Jewell.

The suggestion here is that organizations have to invest in a modern analytics user experience, ensuring that decision-makers can access all their data without waiting weeks for IT teams to raise service requests. This involves getting rid of any legacy processes and looking specifically at how to get analytic insights to users quickly.

“This will lead to a change – if a business can run queries in seconds, rather than taking all day to process, it means that teams can ask significantly more valuable questions throughout their working day - and if they can, they will! Companies must challenge the decades-old assumption that business application data can only arrive into analytics platforms on a nightly basis. When you compete with other companies based on your analytics, the speed you work at makes a huge difference. If you can develop insights based on data arriving in near real-time, your team will be armed with a game-changing advantage over your slower-moving competitors,” clarified Incorta’s Jewell.

Are firms really on the point of understanding that they need to develop, engender, proliferate and propagate a so-called data culture within their organization? Perhaps, in some cases, maybe and often no.

We are unlikely to see data culture officer (DCO) job postings cropping up, this role is the responsibility of the chief data officer (CDO) where they exist and the chief information officer (CIO) elsewhere. That being said, our spokesperson above is listed as a chief data citizen, so go figure.

If there is one resounding message we can extract from this, perhaps it is that firms do know they have data, they know they need to work with data and they know that data analytics and orchestration will be fundamental to their future success - they just haven’t learnt to drive this type of vehicle yet, often don’t know how to change gear and in some cases they don’t even know where the door handle is.