Queryable Executables

My article 'Your executable is a SQLite database' seems to have captured the interest of many readers. SELF is a format where the program itself is a SQLite database. By using binfmt_misc, we can trigger a custom interpreter that maps the rows in the segments table and jumps to the entry point. This enables a whole class of binary tooling to become part of SQL.

The idea that the executable can function as a database, which can be written to, led to the realization that the running program can use the database to store its state as well. This concept allows for collapsing all application state into a single file, eliminating the need for separate filesystem locations like /var/, /tmp/, or /home/.

The SELF webserver is a proof-of-concept webserver that operates in this manner, containing the program, the website, the routes, and all visitor logs within a single SQLite file. The web-server was deployed at https://selfdb.exe.xyz.11 I admire the work of Justine Tunney, whose prior art redbean: a webserver in a single file, inspired the SELF idea.

SELF is a more streamlined approach, relying on simpler tools to achieve a similar goal. It demonstrates how much can be collapsed into a single domain: SQL, whereas redbean requires additional components like an archive format (ZIP) and Lua hooks. For this format to run, we need a way to access itself, and now with the VFS Linux maintainer's recent support for transparent binfmt_misc in the kernel, /proc/self/exe will point to the original file.

This allows the program to query its own file, including its segment table or a new table alongside it. Writes persist across invocations. The web-server for the SELF example consists of three tables: routes, visits, and presses. The page at https://selfdb.exe.xyz displays additional information, such as segments, symbols, and relocations, which are not pre-baked but queried from the file while running.

The transactional nature of SQLite enables the webserver to edit its own content while running, with updates committed to the same file. The ROLLBACK feature allows for undoing changes if needed. SQLite's extensive tooling can be leveraged, such as using sqldiff to track changes between two versions of the same program. Full-text search can also be implemented using FTS5, enabling a webserver to index its own pages within the same SQLite database.

The simplicity of this approach, as seen in products like exe.dev, is reminiscent of the "good old days" of scp and ssh for deploying a single file, but with the added benefit of being an Actually Queryable Executable. The SELF format allows for a self-contained system or application closure to be shipped as a single file, making deployment easier and more robust.