describe data cleaning and data dredging

By looking at a huge pool of data such as lottery winners without defining what you’re looking for, you’ll draw a whole lot of data points. Tests for statistical significance only work if you’ve defined your hypothesis up front. Data dredging is the failure to acknowledge that the correlation was, in fact, the result of chance. Slice your data in enough different ways and you’ll observe some correlations purely as a result of chance. It’s searching deep for answers in data regardless of whether or not there is really something to be found.

In simpler terms, it’s beginning to analyze data without saying what point you’re trying to prove and whether or not that point is actually valid.

We’ve put together a series of quick lessons to help you spot fallacies in data or call out dodgy visualizations so you can use data with confidence and make better decisions both in work and in life.ĭata dredging is the use of data mining to uncover patterns in data that can be presented as statistically significant without first devising a specific hypothesis as to the underlying causality. At Geckoboard, we’re on a mission to help people use data more simply and effectively.

Describe data cleaning and data dredging