I Turned My Repetitive CSV Checks Into a Python CLI
I kept running the same checks whenever I opened a new CSV or Excel file. Missing values. Duplicate rows. Mixed numeric and text values. Bad dates. Columns that never change. Numbers that look unusually high or low. None of those checks is difficult, but doing them manually every time gets old pretty quickly. So I turned that first pass into a small Python command-line tool called Data Quality…
Repeated checks for data quality in CSV or Excel files can become tiresome. Shahed Razavi created a Python command-line tool called Data Quality Detective (dqdetect) to simplify this process. The tool analyzes the dataset, identifies issues like missing values, duplicate rows, mixed data types, and outliers. It then provides a concise report with suggestions for further inspection.
Users can install dqdetect via pip and run it on their files to generate Markdown and HTML reports. The project focuses on being lightweight and simple, leaving data cleaning decisions to the analyst. The tool was developed to help analysts quickly evaluate new datasets, and it includes automated testing and CI/CD integration. Future updates may include additional features like machine-readable JSON output and schema rules for specific databases.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.