What Happens When the Import Fails Halfway?

On a Tuesday morning in March, the warehouse supervisor at a manufacturing customer called me. She had just completed a migration of eleven years' worth of inventory records, a painstaking two weeks of manual work. The platform displayed a seemingly successful import, but nine hundred rows were found to be in error. Dates had incorrectly changed formats, certain rows were skipped without explanation, and the same three suppliers appeared four times. This customer was understandably frustrated with the deceptive nature of the import feature.

I learned that import is not a feature, but rather a contract. Import appears as a checkbox in feature matrices, alongside other superficial functionalities. However, the reality is far more complex. The import process involves parsing, validating, transforming, matching against existing records, deduplicating, committing to the database, and reporting. Each stage can fail, and they can fail in distinct ways. The progress bar and successful message in the checkbox fail to communicate these intricate steps.

Most platforms, including mine, treat the middle of this pipeline as an implementation detail, which is counterproductive. The first design decision should focus on the failure semantics of the import process. Should the system commit everything, nothing, or only the rows that succeeded? All-or-nothing is straightforward but harsh.

Best-effort is more forgiving but risks burying failures in downloadable logs and quietly corrupting data models. The optimal approach is staged validation, showing the customer exactly what would happen before committing any changes.

The most valuable feature, however, is often overlooked: preview. A dry-run mode allows customers to see the exact impact of an import without actually executing it. This feature transforms the import process from a gamble into a negotiation, enabling customers to identify and address edge cases before they become permanent mistakes. Preview also clarifies matching issues, allowing customers to decide whether to consider variations in supplier names as the same entity.

As file sizes grow, imports become asynchronous jobs with their own lifecycle. Support tickets not only decrease but also shift from reactive troubleshooting to proactive problem-solving. Import becomes a document that customers can review, understand, and negotiate. The lesson is clear: batch operations that outlive an HTTP request require a job-like design, complete with status pages, progress tracking, cancellation capabilities, and idempotency.

Exports, often perceived as the gentle sibling of import, share similar challenges. They are merely a query with a download button, and numerous failure modes can arise. Large exports may time out, stream exports consume database connections, and filtered view exports compete with concurrent edits. The customer's exported file becomes a snapshot that can be stored in various unauthorized locations, posing risks to data integrity and security.

Encoding issues also plague exports, as customers may encounter unreadable characters due to incorrect code page settings in their office computers.

In conclusion, the true essence of import and export lies in meticulous planning, robust validation, transparent communication, and robust job management. Neglecting any of these aspects leads to incomplete contracts, hidden failures, and customer dissatisfaction. By embracing a staged validation approach, offering comprehensive previews, and designing imports and exports as self-contained jobs, platforms can significantly enhance the user experience and foster trust with their customers.