jina-embeddings-v4 as an OpenAI-Compatible Embeddings Server
jina-embeddings-v4 is a self-hosted server for the jina-embeddings-v4 embedding model with an OpenAI-compatible /v1/embeddings endpoint. It runs on a single NVIDIA GPU. An application that calls OpenAI for embeddings can call this server instead. The request and response bodies are the same, so setting the client's base URL is the only change needed. When this is useful Text that cannot leave…
jina-embeddings-v4 is an open-source, self-hosted server that enables running the jina-embeddings-v4 embedding model for OpenAI-compatible applications. This server can be deployed on a single NVIDIA GPU, and it offers the same request and response format as OpenAI's embeddings service, making it easy to swap in place of OpenAI's service.
The server is designed for cases where embedding text must remain within the network, such as in environments with strict data privacy policies. It also allows for embedding large datasets without incurring per-request charges, as there is no cost associated with the embedding process.
The embedding vectors generated by this server are 2048-dimensional and have no connection to OpenAI's vectors, which means that any text previously embedded with OpenAI's model would need to be re-embedded using this server. Additionally, the server supports multilingual and code retrieval tasks, enabling a single model to generate embeddings for various languages and source code.
The image of jina-embeddings-v4 is based on FastAPI and sentence-transformers, with a model architecture consisting of 3.8 billion parameters. The server requires an NVIDIA GPU with at least 10 GB of VRAM and a CUDA 12.4 compatible driver. To set up the server, a docker-compose.yml file needs to be created, specifying the necessary volumes, environment variables, and resource allocations. Once the server is up and running, it can be tested by sending a health check request, which should return a status of "ok."
Embedding a single text input can be done by sending a POST request to the /v1/embeddings endpoint, providing the required authorization token, content type, and input text in the request body. The server will return a 2048-dimensional embedding vector for the input text, formatted in a list containing a single object with the embedding array and its index. This response format is compatible with OpenAI's, allowing for seamless integration into applications using the OpenAI client.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
