I put seven LLMs on a USB stick. Here is everything that broke.
The idea fits in one sentence: plug a stick into any computer, double-click one file, and a real language model runs on that machine. No account, no internet, no install, and nothing left behind when you unplug it. That sentence took weeks to make true. Not because the AI part is hard, since llamafile and llama.cpp do the heavy lifting. It took weeks because every operating system has its own way…
Seven large language models were placed onto a USB stick, causing various issues. The problems encountered are:
1. Missing Microsoft runtime DLLs on Windows led to a successful start but no output from the program.
2. Windows shells commands failed to run, resulting in exit code -1, which could be resolved by using a separate engine for Windows.
3. Linux systems lack a double-click mechanism and may not launch .desktop files or executable scripts from outside the standard app folders.
4. FAT32 file systems only mark specific file extensions as executable, causing issues with Linux launchers.
5. The copying process sometimes reads files from the operating system's write cache instead of the actual USB stick.
6. Launchers that attempt to open a chat page in a browser may freeze on computers without a default browser.
7. The speed of the USB stick affects the loading time of the models, with faster drives providing quicker start times.
8. Models with reasoning enabled may take longer to respond, as seen with Qwen3 which spent 441 tokens deliberating before providing an incorrect response.
9. macOS may experience crashes during fork() operations when running shell commands, causing the server to restart automatically.
10. Small issues like duplicate lines in memory files can also cause problems, requiring additional troubleshooting time.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
