Urgent.News

One page, thousands of outlets. See who else covered it.

Editions

More in Tech

Rate Limiting a Crawler Across Node Cluster Workers

Code: Megapixel99/webCrawler This project is a small search engine I wrote to understand how one works: a crawler, an inverted index in MongoDB, and a BM25 ranker, all written by hand instead of…

  • Rate limiter initially lacked rate limiting, causing server overload.
  • Fixed by storing rate limiting state in shared MongoDB collection.
  • Implemented atomic findOneAndUpdate operation to prevent race conditions.

GNU poke 5.0 released

Version 5.0 of GNU Poke , a binary-data editor, has been released. This release includes a number of improvements to the Poke compiler, additions to the Poke language, as well as runtime and standard…

More from Monday 17 August →