Big data: The most business critical tool for successful digital transformation

• Read original article here

In the continuing wake of the COVID-19 pandemic, both businesses and individuals face significant changes to their daily lives and operations.

Economic pressures, alongside increasingly digital and remote ways of working, are requiring companies from a variety of sectors to radically change their approach to customer contact, services, and product creation.

Many companies have had to quickly shift their operations online, leaving companies without the ability to meet their customers face-to-face. In response, a metamorphosis is underway, with companies shifting to make use of consumers’ digital footprints to develop products and services, and to continue to connect with their markets. The result is that many organisations are now dealing with significantly more information than ever before: a move towards (really) Big Data. The issue is that even while this trend is accelerating, companies are increasingly coming up against major blockers: tainted data, lack of consent to use data, processing difficulties, as well as accuracy concerns. All of these issues create obstacles, in the midst of an accelerating and necessary shift towards building innovation and revenue growth through digital strategies. Data is now a business-critical tool for achieving strategic goals, and companies must adopt new ways of handling data to continue to create value.

The shift towards digital is unquestionable. Interest in Big Data, analytics, AI, and machine learning has exploded, and numerous research institutes, think-tanks, and advisory firms a tracking and analysing the fundamental change that is taking place. McKinsey noted that even by May, US companies had “vaulted five years forward in consumer and business digital adoption in a matter of around eight weeks.” However, KPMG are quick to emphasise that “embracing new technologies is no longer just about doing things better, faster, and cheaper. It now has implications on survival and growth in a new business reality.” The increase in data coming from this digital adoption is enormous: smart cities, IoT, and consumer data collection through online platforms and applications is forming a much bigger Big Data landscape.

At the same time, companies are struggling. A Gartner survey from 2019, even pre-pandemic, found that “80% of marketers who have invested in personalization [plan to] abandon their efforts due to lack of ROI, the perils of customer data management or both.” Companies have been trying to manage much higher volumes of customer data, hoping to gain insights, and coming up against significant obstacles. These conflicting pressures and challenges can get in the way of companies going through a successful digital transformation process, and fundamentally different ways of approaching Big Data use and processing need to be considered.

For many organisations, the struggles they are facing relate to sourcing, gathering, and utilising data, as well as tainted data sets or data that cannot be used lawfully and ethically. Fundamentally, the problem arises from an impasse between business users, and compliance or legal teams. The ways that organisations try to resolve this issue generally follow two pathways: policy approaches, or technological approaches.

Policy approaches include applying frameworks of consent, forming contracts around data sharing and use, and standards such as ISO/IEC 27002. These types of policy approaches are also contained in many pieces of legislation, including the GDPR’s set of lawful bases (which include consent and contract), and the California Consumer Privacy Act (CCPA) in the US. However, policy approaches in many cases do not enable significant data utility to remain, and sometimes do not protect privacy either. The issue becomes that you have Big Data, but if you can’t gain consent, you can’t use it. Contracts may protect data within agreements, but fail to protect data in a breach; standards are useful, but do not actually ensure that you can gain maximum value and protect data simultaneously.

Furthermore, if businesses apply policy approaches alone, business disruption and potential liability can result. Many Big Data processing practices that were previously lawful pre-GDPR and CCPA, are now unlawful. Without new technical controls in place, Big Data processing can quickly become unlawful, jeopardising the entire business operation and limiting the essential information that’s needed for growth.

Technological approaches can provide different levels of protection, and may preserve more or less utility and accuracy in large data sets, depending on the technique applied. Many people are familiar with the application of anonymisation techniques, generalisation approaches, encryption, and so on. Many of these run into problems when it comes to re-identification risk, accuracy, or retaining data utility. However, there are also technological approaches that balance utility and protection, and are policy-backed: under the GDPR the approach is called “Pseudonymisation”, and under the CCPA it’s called advanced “De-Identification”. Both of these approaches rely on a concept called functional separation, which is the idea that if you can functionally separate the information value of the data from the individual to who the data relates, data utility can be preserved, while protecting privacy. This is the direction that organisations need to move in, to enable the Big Data applications they need to innovate.

When dealing with large volumes of data in Big Data applications, using the right kind of technological protection is paramount. This is because while smaller and isolated data sets can be protected on their own, as soon as data sets are combined (as in Big Data analysis), individuals can be identified. One study discovered that up to 87% of a population sample could be identified by combining ZIP code, gender, and date of birth alone. Technological approaches that allow distributed data sharing, data use, access, combining, and Big Data strategies are critical to allowing businesses to go through a successful digital transformation process.

Applying technical approaches based on functional separation concepts are some of the only ways that Big Data value can be unlocked for organisations, without running into issues of data privacy protection, low utility, or inaccuracy. Big Data has fast become the most crucial tool for business growth, customer acquisition and retention, and pushing the boundaries of just how quickly a business can evolve, and it’s critical that supportive technologies are applied. By making use of innovative data protection and enablement technologies, businesses can go through this digital transformation seamlessly and without set-backs

Gary LaFever is CEO and General Counsel at Anonos. Anonos simplifies data privacy and security by embedding controls that flow with the data to protect it at the time of use to simultaneously achieve both universal data protection and unrivaled data utility = Data Liquidity.