A Shapefile Is Four Files, and the Important One Is Optional
A user clicks "upload" and picks a file. In most systems that sentence is the whole story. In a geospatial system, the thing they picked is one of four files that only mean something together, the one carrying the most important piece of information is optional and frequently absent, and the geometry inside it almost certainly doesn't match your database schema. This is what I learned building…
A shapefile is made up of four files that only make sense when used together. The most important file, though, is optional and often missing. This becomes apparent when building a web GIS to handle large shapefile uploads.
Initially, Django's synchronous endpoint was used for uploads, but this quickly led to problems with large files. The endpoint would occupy a worker for minutes, causing timeouts and making it impossible to know if the import had succeeded. To solve this, the imports were moved to a separate FastAPI service. Even though FastAPI isn't inherently faster, the async model allowed for better concurrency control.
A Semaphore is used to limit the amount of concurrent geospatial work, preventing out-of-memory errors. The endpoint returns a job ID, and the client polls for progress. This job-plus-poll method is now used for raster mosaics and exports, providing a consistent user experience.
A shapefile consists of several components: .shp holds geometry, .shx is an index, and .dbf contains attributes. If any of these are missing, the parser will throw an error. To handle this, validation checks for the required components before parsing.
Most users upload a ZIP file, which the service extracts and validates. The error message specifies the missing extension, providing clear instructions to the user. The .prj file, which defines the coordinate system, is not required but frequently absent. This is because most surveying data is delivered in a coordinate system that users already understand.
An important consideration is the coordinate system. While most data has a defined CRS, it's not always declared in the shapefile. The service asks users to select a coordinate system, and handles the data accordingly. The database schema is also typed, with 3D geometry being the default. This ensures consistency and prevents errors when querying the data.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
