Most machine learning guides cover data cleaning briefly or skip it entirely. However, many data scientists and analysts spend most of their time on data cleaning and data quality tasks, and their effectiveness can make or break project success. In Data Cleaning for Effective Data Science, leading data science trainer David Mertz provides the most systematic guide to cleaning data for any project, using any library or toolset.


Mertz introduces many powerful techniques for analyzing, manipulating, and pre-processing data sources. He offers best practices for working with leading data formats such as JSON, CSV, SQL RDBMSes, HDF5, NoSQL databases, files in image formats, binary serialized data structures, and more.


Mertz also focuses on crucial issues within the data itself, including missing data, outliers, biasing trends, class imbalance, value imputation, over/under-sampling, normalization and/or randomization, and anomalies.


This guide is organized around downloadable datasets, each illuminating specific issues with data integrity or quality. Each chapter explores the best ways to diagnose, analyze, and remediate these issues, offering hands-on practice using tools such as Python, Pandas, sklearn.preprocessing, scipy.stats, R, and Tidyverse. While the examples are demonstrated with widely-used tools, Mertz's concepts are applicable with any toolset. Each chapter also links to additional datasets with more problems, exercises, and solutions.