Your self-hosted AI stack probably needs one process, not six
Open the compose file for almost any self-hosted AI assistant. You will find an app container, Redis for the queue, Postgres for state, a worker, usually a vector database, often a reverse proxy. Six services. For a household of five. Then ask what that architecture is actually buying you at that scale. The queue exists so work survives a restart and so you can scale workers horizontally. You are…
Most self-hosted AI assistant setups involve six separate services: an app container, Redis for the queue, Postgres for state, a worker, a vector database, and a reverse proxy. However, the complexity of this architecture may be unnecessary for smaller-scale deployments. The queue is used to ensure work persists during a restart and to enable horizontal scaling of workers, but such scaling is not required for a household of five users.
In fact, these six services provide six potential failure points, six sets of logs, six version upgrades, and a harder debugging process. TencentCloud's Octop offers an alternative approach by consolidating all functionalities into a single process. This includes serving the web dashboard, the CLI backend, every chat channel, and the cron scheduler.
There is no broker to handle events, simplifying the architecture and reducing the number of components that can fail. The runtime state is rebuilt from a SQLite control-plane database on boot, ensuring that the system can recover without relying on a broker. This design eliminates the need for a separate queue and a complex restart safety mechanism.
Restart safety is achieved by having the process hold no authoritative state at all, allowing it to be killed and rebuilt from the database. Octop also presents an option for Postgres instead of SQLite, but this is not the default recommendation. The choice of SQLite in WAL mode is considered appropriate for a low-concurrent-writer workload on a single machine, as it eliminates the need for connection pools, second daemons, tuning, and backup management.
The simplicity and robustness of this approach make it particularly suitable for households or teams of five users, where operational simplicity is valuable. However, the design has limitations. One process represents a single failure domain, meaning that if the process crashes, all functionalities will be affected. Additionally, there is a user count beyond which this architecture may become a bottleneck.
For a household or a team of five users, this scenario is considered theoretical, and the operational simplicity provided by Octop is highly beneficial. For setups with fifty concurrent users, the architecture may not be the optimal choice. Before deploying a self-hosted AI assistant with remote access, it is essential to consider the available upload bandwidth.
Bandwidth becomes the limiting factor in determining the usability of the system, and a quick speed test can provide a clearer answer than a long analysis of complex architectures. In summary, Octop presents an alternative approach to self-hosting AI assistants by simplifying the architecture and reducing the number of components that can fail.
While this design offers operational simplicity and ease of management for smaller-scale deployments, it may not be suitable for larger deployments with higher concurrency requirements.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.