Vast.ai CLI 101: Finding a GPU Offer You Can Actually Use
TL;DR: Start with the workload’s GPU and memory needs, then search for offers with reliability at least 0.97, free inbound traffic, enough disk, and a driver compatible with your container. Sort by hourly price, inspect the raw offer data, and choose the cheapest offer that passes every check. A low hourly rate is useful only if the instance can run the job and receive its model weights without a…
To rent a GPU through Vast.ai for AI work, start by defining the requirements of your job. This includes the GPU memory needed for the model and settings, a compatible container image, enough disk space, and acceptable reliability and network charges. Next, use the Vast.ai CLI to search for offers with the command `vastai search offers reliability = 0.97 inet_down_cost = 0 disk_space = 150 -o dph+ --raw`.
This filters offers by reliability, excludes those with inbound traffic charges, and limits disk space to 150 GB. The `-o dph+` option sorts results by hourly price, and `--raw` returns data for inspection. Treat search results as candidates to inspect rather than permanent recommendations, as offers can change. Be mindful of inbound traffic costs, as they can significantly impact the total expense.
Hardware compatibility is crucial, so check that the GPU name, memory, driver version, and CUDA requirements match your container image. Use the fields `driver_version`, `cuda_vers`, and `cuda_max_good` to evaluate compatibility, and verify that the chosen image's requirements align with the offer's specifications. Direct ports and geolocation can also impact your setup, so consider `direct_port_count` and `geolocation` fields if they apply to your workflow.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
