altcheck: draft alt text and contrast fixes with a local open vision model

altcheck: Creating Draft Alt Text and Contrast Fixes with a Local Open Vision Model

altcheck is a command-line tool designed for freelance web developers. When given a URL, it generates a local report.html that addresses images with missing, filename-style or generic alt text, as well as text that fails the color-contrast check. The tool provides drafted alt text and suggested color fixes, but it does not edit the site or perform full accessibility checks.

The tool is aimed at freelancers or small agencies who want to quickly improve the accessibility of client pages. Running it over a webpage and fixing obvious gaps can take mere minutes, making it a potential paid service for quick accessibility upgrades. The developer has not yet shown the tool to clients or paying users.

The demo video, available in the repository, showcases the tool's performance on a deliberately inaccessible W3C demo page. Three images with no alt text were examined, and the drafted alt texts matched the visible content. The page also contained two contrast failures.

To achieve its functionality, altcheck uses Playwright to load the page, axe-core to run the color-contrast rule, and a Python function to adjust text colors until they meet the requirement. For image processing, the SmolVLM-500M-Instruct model (released under Apache 2.0) runs locally via transformers. It generates a short sentence for each image and determines if the image appears decorative. Images with missing or empty alt attributes are handled differently, as an empty alt is valid for purely decorative purposes.

The tool's development process involved modifying the initial approach, which sent the model the surrounding page text and link details. This caused the 500M model to copy text or invent content, so the current implementation only provides the nearest heading and adds the link destination through code. Due to memory constraints on a 2-core CPU with 2 GB of RAM, the tool uses bfloat16 instead of float32, resulting in approximately 20 seconds of processing time per image. The tool has not been tested on GPUs, Apple Silicon, Windows, other models, or various real sites.

Six unit tests cover image selection, contrast math, and report escaping. Open innovation matters in this context because client pages are often unreleased or private. Using an open-weight model on the freelancer's own machine eliminates the need for image uploads to an API and removes per-image fees. The model's prompts are transparent, allowing anyone to swap in another open-licensed model using the --model command.

Additionally, the drafts generated by the tool are of small-model quality, and the report advises a review before using the suggestions. The chosen SmolVLM model is licensed for commercial use, making it suitable for a paid tool that requires no image upload fees.