Detecting AI-Generated Web Content from Structure Alone: SlopShape's Fingerprinting Architecture
AI-generated marketing content is everywhere. The problem is not quality. The problem is homogeneity. When five frontier models rewrite 2,250 human blog posts, they produce pages that share a structural fingerprint so consistent that a classifier can identify them at 97.0% macro-F1 without reading a single word. SlopShape, a paper from Sitefire (YC W26), shows that AI-generated commercial web…
AI-generated marketing content is abundant, not due to quality, but rather its uniformity. Five state-of-the-art models rewriting 2,250 human blog posts produce pages that bear a consistent structural fingerprint. This allows a classifier to identify AI-generated content with 97.0% macro-F1 accuracy, solely based on structural signals such as section ordering, evidence presentation, and voice consistency.
Unlike word-level detectors, SlopShape, a paper from Sitefire (YC W26), focuses on shape detection, maintaining accuracy even when AI-generated posts are reworded by different models. This approach ignores words altogether, using 203 features, 176 of which are structural, to classify content. These features, extracted by an undisclosed LLM, are validated against human annotations with high agreement rates (kappa 0.939 and 0.951).
The training corpus consists of 2,250 pre-ChatGPT human blog posts from 268 company domains, mirrored into 11,250 AI-generated versions by five frontier models. Human posts edited by AI assistants blur the ground truth, but the high F1 score indicates the training set is sufficiently clean. The classifier employs a two-stage architecture: an LLM extracts structured features, which are then fed into a gradient-boosted tree (XGBoost or LightGBM, unspecified) for prediction.
This design offers interpretability, robustness against rewording, and easy versioning as AI output evolves. Running the LLM at crawl-time would introduce unacceptable latency, so the paper proposes batch processing for feature extraction and asynchronous classification.
Detection performance remains strong even when each AI post is reworded by its own model (F1 drops to 96.1%). The classifier assigns 68.6% of AI posts to the correct source model, attributing distinct structural signatures to each model, even if they generate the same content. Human posts exhibit more varied structure than AI models, which tend to converge on a consistent shape.
The paper does not address adversarial dynamics or model versioning, but the detection problem is an ongoing observability challenge due to the continuous evolution of AI output. A production deployment would involve a crawler fetching HTML, an LLM feature extractor, and a fast classifier, with feature caching to minimize latency.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.