I Benchmarked My Vanilla JS CBT Detector Against 5 NLP Libraries — Here Is When Rule-Based Wins
The Question I built a cognitive distortion detector in 200 lines of vanilla JavaScript. No ML. No NLP library. No API. No backend. Pattern matching on keyword arrays. People asked: "Is that actually better than just using a real NLP library?" So I benchmarked it. Here are the results. The Contenders Approach Dependencies Bundle Size Setup My vanilla JS detector 0 8 KB Drop in <script> compromise…
I created a cognitive distortion detector using just 200 lines of vanilla JavaScript code. This code does not rely on any machine learning algorithms, NLP libraries, APIs, or a backend. Instead, it uses simple pattern matching on predefined keyword arrays to identify cognitive distortions. People inquired whether this simple approach could outperform more sophisticated NLP libraries.
To find out, I benchmarked the vanilla JS detector against five other NLP libraries. The benchmark involved detecting cognitive distortions in 100 thought samples. The distortions include all-or-nothing thinking, overgeneralization, mind reading, labeling, emotional reasoning, and more.
The vanilla JS detector achieved a 94% accuracy rate, correctly identifying the distortions in the sample inputs. However, it struggled with novel phrasings and missed some distortions. Other approaches had varying accuracies: compromise (NLP) at 61%, natural language processing at 58%, TensorFlow.js at 72%, OpenAI API at 89%, and Hugging Face at 83%.
In terms of setup and bundle size, the vanilla JS detector was the simplest to implement, with a small bundle size of just 8 KB. The other libraries required more complex setup and larger bundle sizes. The vanilla JS detector also had the fastest response times, with a p50 of 0.2ms and p99 of 0.8ms, demonstrating minimal latency. The other libraries had significantly higher latency, with some taking up to 3500ms.
Cost-wise, the vanilla JS detector incurs no costs, while the other libraries did. OpenAI API cost $1.80 per 1000 analyses, Hugging Face $0.90, and TensorFlow.js was free. Privacy was also a factor. The vanilla JS detector processed data locally, ensuring no data left the browser. The other libraries sent data to their servers, potentially posing privacy risks.
In conclusion, the vanilla JS detector is most effective when the output space is small, privacy is paramount, latency is crucial, and the cost needs to be minimal. It's best suited for closed sets of cognitive distortions, instant feedback requirements, and cost-sensitive applications.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
