Home > Ask the Data Management Experts > Data governance and quality Questions & Answers > Data quality management for data warehouses
Ask The Data Management Expert: Questions & Answers
EMAIL THIS

Data quality management for data warehouses

David Loshin EXPERT RESPONSE FROM: David Loshin

Pose a Question
Other Data Management Categories
Meet all Data Management Experts
Become an Expert for this site


Enterprise IT tips and expert advice
Digg This!    StumbleUpon Toolbar StumbleUpon    Bookmark with Delicious Del.icio.us    Add to Google


>
QUESTION POSED ON: 10 June 2008
We are discussing the option of populating our data warehouse tables with less than quality data. We need to make recommendations to the (business project) sponsors. We need to balance the risk of credibility of having bad data with gaining knowledge of how bad things are by having people access the information. What would you recommend -- populate with bad data or not populate at all?

>
EXPERT RESPONSE
This is a little bit of a conundrum. If I understand the question correctly, you have two challenges. The first is that your team is not ready to populate the data warehouse because the quality of the data is not acceptable for the defined business objectives. The second is that you are unable to effectively articulate to the business users the degree to which the data does not meet data quality thresholds. You also have positioned the challenge in terms of "risk" -- risking credibility (presumably of the data warehouse).

By looking at these aspects, though, I suspect that there are some deeper issues involved. For example, by whose criteria are you determining the quality of the data? If the data warehouse team is making that assessment with respect to the needs of the business project sponsors, that implies that they have been engaged in defining the business data expectations, requirements, critical dimensions of data quality, measures and metrics for monitoring the quality of the data. In that case, there whould be no risk of credibility, since (1) the data loaded into the warehouse is not "owned" by the data warehouse team, but is just collected from other sources upstream and then determined to not meet the objectively defined standards, and (2), the business sponsors would be the ones who have contributed their data quality expectations, and those rules are not "owned" by the data warehouse team either. And, in that case, you do not have to answer the question about whether the data warehouse should be populated -- that should be up to the business project sponsors, since the levels of quality would already be reported to them based on their own set of criteria.

However, this is clearly not the fact, which means a few things. First, it is not clear what the definition is in this context for data quality acceptability, but it is not derived or owned by the business users. Second, it appears that there may be an opportunity to improve the data requirements analysis process in a way to capture the data quality dimensions, rules and metrics in a way that allows you to characterize the candidate data sources for suitability prior to data warehouse population time. Third, there is an opportunity to better communicate the value of high quality data deflecting the "risk" away from the data warehousing team and back to the business clients: are they willing to accept the risk of making bad decisions or not having accurate reporting due to poor upstream data quality?

More data quality management resources

More data warehouse resources


Digg This!    StumbleUpon Toolbar StumbleUpon    Bookmark with Delicious Del.icio.us    Add to Google


RELATED CONTENT
Data quality mgmt. best practices
Information assurance: Dependability and security of networked information systems
What is high quality information?
Data quality and governance management quiz
Thirteen causes of enterprise data quality problems
How to maintain data quality and provide high quality information management and analysis
Data quality management pitfalls: Three common mistakes to avoid
How to develop and maintain an enterprise data quality management strategy, with Larry English
Five steps for weaving data quality management into your enterprise data integration processes
Data quality management: Problems and horror stories
Data quality management: Follow the doctor's orders

Data governance and quality
What is high quality information?
Data quality management tools: Where to get unbiased information
Roles and salaries of data quality and governance analysts
The role of data governance in unstructured data
How to maintain data quality and provide high quality information management and analysis
Data quality management pitfalls: Three common mistakes to avoid
Data governance: Tips for techies and managers
Data cleansing: The business impact of dirty data
Data quality management begins with data governance
Troubleshooting: Performing an insert with DB2 z/OS

Data quality best practices
Best practices for designing and implementing sustainable, long-term data quality programs
Effective data quality program management: Tips and advice
Creating successful data stewardship programs, with Jill Dyché
Gartner's data quality management software rankings show convergence with data integration
Master data management must start with data governance
Data quality assessment helps identify, fix data quality problems
Integration competency centers centralize data integration projects
Informatica to buy identity resolution software maker for $85 million
Information assurance: Dependability and security of networked information systems
What is high quality information?

RELATED GLOSSARY TERMS
Terms from Whatis.com − the technology online dictionary
data  (SearchDataManagement.com)
data governance  (SearchDataManagement.com)
data quality  (SearchDataManagement.com)
data scrubbing  (SearchDataManagement.com)
fixed data  (SearchDataManagement.com)
raw data  (SearchDataManagement.com)

RELATED RESOURCES
2020software.com, trial software downloads for accounting software, ERP software, CRM software and business software systems
Search Bitpipe.com for the latest white papers and business webcasts
Whatis.com, the online computer dictionary



Search and Browse the Expert Answer Center
Search and browse more than 25,000 question and answer pairs from more than 250 TechTarget industry experts.
Browse our Expert Advice

About Us  |  Contact Us  |  For Advertisers  |  For Business Partners  |  Site Index  |  RSS
SEARCH 
TechTarget provides enterprise IT professionals with the information they need to perform their jobs - from developing strategy, to making cost-effective IT purchase decisions and managing their organizations' IT projects - with its network of technology-specific Web sites, events and magazines.

TechTarget Corporate Web Site  |  Media Kits  |  Reprints  |  Site Map




All Rights Reserved, Copyright 2005 - 2008, TechTarget | Read our Privacy Policy
  TechTarget - The IT Media ROI Experts