My screener recommended every job it could not read
This is a submission for DEV's Summer Bug Smash: Clear the Lineup powered by Sentry . Project Overview Today I wrote a screener for a freelance job board. One category alone holds ~800 open postings, most of which I can rule out from the text: the ones that require a video interview, the ones restricted by age or gender, the ones that want your personal anecdotes, and — the one that actually…
This report details issues with a screener tool used for a freelance job board, powered by Sentry. The tool is divided into two stages: parsing the job listings and fetching the detailed pages for further testing. The primary issue lies in the second stage where failed fetches were incorrectly considered as endorsements, leading to an increased shortlist of unreadable job postings.
Bug 1: Failed fetch was an endorsement
The second stage of the tool fetched each job's detail page and tested the body text against nine disqualifier regexes. However, when a fetch failed, the resulting empty string was mistakenly considered as satisfying all the disqualifier regexes, causing the job to be recommended without proper evaluation. This issue led to a flaw in the shortlist generation, as failed fetches were silently converted into positive recommendations.
To fix this bug, the tool was modified to give a specific outcome for "cannot look" (HTTP errors, 404, expired sessions, rate-limited responses) and to check twice before recommending a job. The revised implementation now properly handles these cases and adds them to the "unreadable" list, ensuring that only properly evaluated and disqualifiable jobs are recommended.
Bug 2: Zero results, silently
Another issue arose when the tool fetched the search URLs directly instead of navigating to the job listings. While the fetch returned an HTTP 200 and well-formed HTML, the parsed HTML contained no job postings. This resulted in a misleadingly high count of zero results for various keywords, which could mislead the user into believing that there were no relevant job postings.
To address this issue, the tool was modified to verify that the parser successfully extracted job postings from the fetched HTML. If the parser was unable to find any job postings, the tool now marks the job as "unreadable" instead of silently including it in the shortlist. This change ensures that only jobs that can be properly evaluated are recommended, improving the overall quality of the shortlist.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
