Using AI to Improve Data Quality

This IDC Perspective assesses the emerging market for AI-enabled data quality tools and offers recommendations for how IT leaders should respond. Data quality is vital for any IT project, especially so for AI-based projects, which could go dramatically off the rails due to poor data quality. Leveraging ML and GenAI, new tools can significantly improve the quality of data and give all projects, including those implementing AI, a better chance of success. This document prescribes three essential tasks for IT leaders: understand your data quality needs and how AI-based tools can improve data quality while closely evaluating vendor claims, launch a pilot project in which AI-based data quality tools are deployed and implemented by a team with AI skills, and plan for a future in which AI-based tools play a significant role in your data quality strategy. “The data quality software market is undergoing a transformation thanks to AI, as legacy rules-based tools are replaced by AI-enabled tools. ML brings to bear powerful new capabilities for detecting anomalies and should significantly improve data quality,” says Stewart Bond, vice president of IDC’s Data Intelligence and Integration Software Service. “GenAI should be able to fill in the data gaps that previously held back many initiatives. And we can expect these tools to work autonomically, which should be particularly valuable for agentic AI. In fact, if progress continues, an AI-enabled expert advisor could actually take over a data quality initiative, from identification to resolution.” “With AI permeating a vast spectrum of endeavors, it’s no surprise that AI is now being put to work to improve data quality, whether for conventional IT projects or those leveraging AI. IT leaders are likely to hear many claims for these tools but should remain skeptical due to the hyperbole around products that claim to incorporate AI. Used according to their capabilities, however, a new generation of AI-based data quality tools can deliver real value,” says Stanley B. Gibson, adjunct research advisor for IDC’s IT Executive Programs (IEP).