The Data Daily

Can we automate data quality to support machine learning? | 7wData

Last updated: 10-08-2019

Read original article here

Can we automate data quality to support machine learning? | 7wData

Can we automate data quality to support machine learning?
« Previous Article
Clint Hook, director of Data Governance at Experian , looks at how organisations can automate data quality to support artificial intelligence and machine learning
Over the last decade, companies have begun to grasp and unlock the potential that artificial intelligence (AI) and machine learning (ML) can bring. While still in its infancy, companies are starting to understand the significant impact this technology can bring, helping them make better, faster and more efficient decisions.
Of course, AI and ML is no silver bullet to help businesses embrace innovation. In fact, the success of these algorithms is only as good as their foundations — specifically, quality data.
Without it, businesses will see the very objective they’ve installed AI and ML to do fail, with the unforeseen consequences of bad data causing irreversible damage to the business both in terms of its efficiency and reputation.
But there’s another area of exploration which is ripe for development; namely, can data quality be improved and maintained by automation and machine learning itself?
The Importance of Innovation with Business Analytics
From movie streaming services, to chatbots, to helping inform how supermarkets arrange their shelves and guiding us through major transport hubs, ML influences our lives in ways that were unimaginable a decade ago.
But what happens if the algorithm is set to work on the foundation of poor data quality? The risks in the future could be far more severe than being served a film you don’t like.
If we begin trusting machine learning to improve the discovery and testing of pharmaceuticals, for example, what would happen if a drug were formulated but there were errors in the chemical compound data used to simulate testing? The implications could be grave.
An emerging application of ML which could also be impacted by poor base data is self-driving vehicles. From maps and addresses to how a vehicle reacts to a cyclist, the data used to teach the machine will be crucial to consumer and regulator adoption.
ML algorithms – those sets of rules and calculations that help solve defined problems — can either support the improvement of data quality or be thrown off by inaccurate data should the possibility of poor data not be considered in their construction.
Visual Business Intelligence – We Must Vet Our Data Visualization Advisers with Care
As with any digital transformation, moving from manual to automated and then ‘intelligent’ data quality management will require a long-term plan. Experian has identified four stages about the progression of data management, which we call the Data Management Maturity Curve. Unaware, Reactive, Proactive and Optimised & Governed reflect the four stages that span a full cycle of a data quality strategy.
The assessment has revealed a steady progression up the maturity curve, as organisations begin to release the potential of the data they hold and take it more seriously.
4 data-driven ways to digitize your business
Register with your social profile:
Your email address will not be published. Required fields are marked *
© 2015 7wData. All Rights Reserved | site by four eyes
/* ----------------------------------------- */ /* Content Template: Single Post with Sidebar v3 - start */ /* ----------------------------------------- */ /* remove the left overs from the inline related posts IRPP_ruby. It adds an extra div to add bottom space below the irrp */ #upcoming-events DIV[style="clear:both; margin-top:0em; margin-bottom:1em;"] {display: none;} /* ----------------------------------------- */ /* Content Template: Single Post with Sidebar v3 - end */ /* ----------------------------------------- */ /* ----------------------------------------- */ /* Content Template: Author Box - start */ /* ----------------------------------------- */ .ath-image{ margin: 0 auto; } .single-post #ath-box{ font-size: 0.75em; } .single-post #ath-box .IRPP_ruby { // display:none; } /*----- Genral Classes start ------*/ body { background: #2C303C; font-family: 'Source Sans Pro', sans-serif; font-size: 100%; margin: 0; padding: 0; } //h1 { text-align: center; color: #999; background: rgba(0, 0, 0, 0.36); margin: 0; padding: 5px; } //h2 { text-align: center; color: #999; margin: 0; padding: 5px; font-size: 1.8em; margin-bottom: 30px; } .wrapper,.copyright { padding: 20px; text-align: center; } hr{ margin: 30px 20px; border-top:2px solid #1C1E26 ; border-bottom:2px solid #38404D; } .list-unstyled { padding-left: 0; list-style: none; } .list-inline li { display: inline-block; padding-right: 0px; padding-left: 0px; margin-bottom: 10px; } /*---- Genral classes end -------*/ /*Change icons size here*/ .social-icons .fa { font-size: 1em; } /*Change icons circle size and color here*/ .social-icons .fa { width: 30px; height: 30px; line-height: 30px; text-align: center; color: #FFF; color: rgba(255, 255, 255, 0.8); -webkit-transition: all 0.3s ease-in-out; -moz-transition: all 0.3s ease-in-out; -ms-transition: all 0.3s ease-in-out; -o-transition: all 0.3s ease-in-out; transition: all 0.3s ease-in-out; } .social-icons.icon-circle .fa{ border-radius: 50%; } .social-icons.icon-rounded .fa{ border-radius:5px; } .social-icons.icon-flat .fa{ border-radius: 0; } .social-icons .fa:hover, .social-icons .fa:active { color: #FFF; -webkit-box-shadow: 1px 1px 3px #333; -moz-box-shadow: 1px 1px 3px #333; box-shadow: 1px 1px 3px #333; } .social-icons.icon-zoom .fa:hover, .social-icons.icon-zoom .fa:active { -webkit-transform: scale(1.1); -moz-transform: scale(1.1); -ms-transform: scale(1.1); -o-transform: scale(1.1); transform: scale(1.1); } .social-icons.icon-rotate .fa:hover, .social-icons.icon-rotate .fa:active { -webkit-transform: scale(1.1) rotate(360deg); -moz-transform: scale(1.1) rotate(360deg); -ms-transform: scale(1.1) rotate(360deg); -o-transform: scale(1.1) rotate(360deg); transform: scale(1.1) rotate(360deg); } .social-icons .fa-adn{background-color:#504e54;} .social-icons .fa-apple{background-color:#aeb5c5;} .social-icons .fa-android{background-color:#A5C63B;} .social-icons .fa-bitbucket,.social-icons .fa-bitbucket-square{background-color:#003366;} .social-icons .fa-bitcoin,.social-icons .fa-btc{background-color:#F7931A;} .social-icons .fa-css3{background-color:#1572B7;} .social-icons .fa-dribbble{background-color:#F46899;} .social-icons .fa-dropbox{background-color:#018BD3;} .social-icons .fa-facebook,.social-icons .fa-facebook-square{background-color:#3C599F;} .social-icons .fa-flickr{background-color:#FF0084;} .social-icons .fa-foursquare{background-color:#0086BE;} .social-icons .fa-github,.social-icons .fa-github-alt,.social-icons .fa-github-square{background-color:#070709;} .social-icons .fa-google-plus,.social-icons .fa-google-plus-square{background-color:#CF3D2E;} .social-icons .fa-html5{background-color:#E54D26;} .social-icons .fa-instagram{background-color:#A1755C;} .social-icons .fa-linkedin,.social-icons .fa-linkedin-square{background-color:#0085AE;} .social-icons .fa-linux{background-color:#FBC002;color:#333;} .social-icons .fa-maxcdn{background-color:#F6AE1C;} .social-icons .fa-pagelines{background-color:#241E20;color:#3984EA;} .social-icons .fa-pinterest,.social-icons .fa-pinterest-square{background-color:#CC2127;} .social-icons .fa-renren{background-color:#025DAC;} .social-icons .fa-skype{background-color:#01AEF2;} .social-icons .fa-stack-exchange{background-color:#245590;} .social-icons .fa-stack-overflow{background-color:#FF7300;} .social-icons .fa-trello{background-color:#265A7F;} .social-icons .fa-tumblr,.social-icons .fa-tumblr-square{background-color:#314E6C;} .social-icons .fa-twitter,.social-icons .fa-twitter-square{background-color:#32CCFE;} .social-icons .fa-vimeo-square{background-color:#229ACC;} .social-icons .fa-vk{background-color:#375474;} .social-icons .fa-weibo{background-color:#D72B2B;} .social-icons .fa-windows{background-color:#12B6F3;} .social-icons .fa-xing,.social-icons .fa-xing-square{background-color:#00555C;} .social-icons .fa-youtube,.social-icons .fa-youtube-play,.social-icons .fa-youtube-square{background-color:#C52F30;} /* ----------------------------------------- */ /* Content Template: Author Box - end */ /* ----------------------------------------- */ /* ----------------------------------------- */ /* Content Template: Loop item in related events v2 - start */ /* ----------------------------------------- */ .wp-caption { position: relative; //padding: 0; //margin: 0; } .wp-caption img { display: block; max-width: 100%; //height: auto; } .wp-caption:after { content: ""; position: absolute; display: block; left: 0; top: 0; width: 100%; height: 100%; background: rgba(0, 0, 0, 0) linear-gradient(to bottom, rgba(0, 0, 0, 0) 0px, rgba(0, 0, 0, 0.6) 100%) repeat 0 0; z-index: 1; } .wp-caption-text { display: block; position: absolute; width: 100%; color: #fff; left: 0; bottom: 0; padding: 1em; font-weight: 700; z-index: 2; -webkit-box-sizing: border-box; box-sizing: border-box; text-transform: none; padding-bottom: 10px; } .evnt-gal { -webkit-text-stroke: 1px grey; /*color: orange;*/ text-shadow: 1px 1px 0 #000, -1px -1px 0 #000, 1px -1px 0 #000, -1px 1px 0 #000, 1px 1px 0 #000; /*line-height: 1.5;*/ font-size: 40px; } .wp-caption>.wp-caption-text:last-child { font-size:30px; text-transform:none; } .evnt-gal-code { font-size:20px; font-weight:900; } .evnt-lbl{font-weight:800} .evnt-date { position: relative; width: 70px; /* font-family: Georgia, serif; */ /* color: #999; */ margin: 0 auto; } .evnt-day, .evnt-month, .evnt-year { position: absolute; } .evnt-day { font-size: 30px; top: 15px; } .evnt-month { top: 0; left: 0; font-size: 18px; } .evnt-year { top: 19px; right: 0; font-size: 20px; rotation: -90deg !important; /* ** Hacks ** */ -webkit-transform: rotate(-90deg); -moz-transform: rotate(-90deg); } .hangingindent { padding-left: 22px ; text-indent: -22px ; } /* ----------------------------------------- */ /* Content Template: Loop item in related events v2 - end */ /* ----------------------------------------- */ /* ----------------------------------------- */ /* Content Template: Loop item in related events v2 – bottom - start */ /* ----------------------------------------- */ .row.fh-2em { min-height: 4em; } /* ----------------------------------------- */ /* Content Template: Loop item in related events v2 – bottom - end */ /* ----------------------------------------- */

Read the rest of this article here