I built an epistemic gate to stop LLM data poisoning during fine-tuning. Tested across 5 architectures, orchestrated on a 2006 Toshiba laptop for $0.
Hi everyone, For a long time, the AI industry has pushed the narrative that advanced safety research, data poisoning auditing, and model alignment require massive clusters and millions of dollars. I wanted to test if that's true. Over the last few months, using a 2006 Toshiba Satellite (2GB RAM) for orchestration and free Kaggle T4 GPUs ($0 cost), I built and ran a 16-experiment series…
Over the last few months, the author, Eduardo AyalaT, conducted a 16-experiment series (EXP08–EXP16) aimed at investigating epistemic collapse and data poisoning in language models (LLMs). Using a 2006 Toshiba Satellite laptop with only 2GB of RAM, AyalaT managed to orchestrate these experiments without spending any money, leveraging free resources from Kaggle.
The primary issue addressed by this research is malicious data poisoning during fine-tuning, which can be both subtle and difficult to detect. Standard metrics like perplexity fail to reveal these targeted attacks, allowing attackers to manipulate specific facts while keeping global metrics seemingly intact. This presents a significant threat, especially for small teams and startups that fine-tune models locally.
To counter this threat, the author developed Beatriz Epistemic Gate, a lightweight defensive proxy that sits between a generative source and a training student model. With minimal decision latency (~0.1ms), Beatriz acts as a non-invasive barrier, verifying generated text against an immutable anchor corpus using cosine similarity in embedding spaces.
It employs two loss functions: LceLce, which maintains linguistic fluency, and LcontrastivaLcontrastiva (Softplus), which applies a contrastive term to keep the model anchored to truth even after cross-entropy reaches its maximum. The combination of these two loss functions yields a composite loss function.
The author tested Beatriz Epistemic Gate across five different model architectures, including GPT-2 (124M), Qwen-2.5-0.5B, TinyLlama-1.1B, Pythia-1.4B, and Phi-3-mini-4k-instruct (3.8B). They also conducted an ablative study (EXP15), which showed that the pure filtering approach (GATE_ONLY) provided 65% of the defensive benefit, with the remaining 35% coming from the contrastive term.
In a scaled held-out validation (EXP16), Beatriz demonstrated a robust truth margin of +4.19 ± 0.08, with 93% precision and 80% recall on a 30-domain unattacked facts benchmark. The complete project, including code, notebooks, a technical whitepaper, and a corrective manual, is available on the author's GitHub repository.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
