Running Qwen Code on a Custom OpenAI-Compatible API: modelProviders, a 41,000-Character First Request and Retries That Keep Going for Minutes

Qwen Code is a terminal coding agent created by the Qwen team. It is compatible with any OpenAI-compatible server, such as vLLM, Ollama, LiteLLM, or a hosted gateway. To run Qwen Code on a custom OpenAI-compatible API, follow these steps:

1. Install Qwen Code globally using "npm install -g @qwen-code/qwen-code".

2. Set environment variables:

- OPENAI_API_KEY: your API key.

- OPENAI_BASE_URL: the base URL of your custom gateway (e.g., https://apiclaw.biz/v1).

- OPENAI_MODEL: the exact ID of your model from the provider's model list.

3. Run Qwen Code with the command "qwen -p" and your query, such as "qwen -p Explain what this repo's Makefile does".

Key aspects of the setup include:

- Qwen Code uses the official OpenAI Node SDK, which appends "/chat/completions" to your provided base URL. Ensure that you keep "/v1" in the base URL, as it is necessary for the gateway to respond correctly.

- The default max_tokens value is 32000, but you can adjust it in the generationConfig object within your settings.json file.

- When using a custom gateway, you can configure modelProviders in the settings.json file, specifying the model ID, name, API key, base URL, and generationConfig. This runtime model makes it easier to manage settings for daily use.

One thing to note is that when running Qwen Code for the first time, it sends two requests: one containing the answer and another containing metadata about the available skills and memory paths. This results in a total of around 41,500 characters being sent before your actual prompt. Additionally, the managed auto-memory feature is enabled by default, which may impact your token count and request count, depending on your pricing plan.