Text to SQL Without Sending Data to OpenAI (Local DuckDB & Ollama Setup)

Developers creating Natural Language to SQL features for enterprise applications often find themselves sending raw schema and table samples to OpenAI or cloud LLMs. However, for security-conscious data teams dealing with sensitive data, this practice can lead to compliance issues. In this guide, we explore how to perform Text-to-SQL analytics locally without exposing raw data to third-party APIs.

The proposed architecture involves separating Schema Context from Data Compute. Instead of transmitting entire CSV or database contents to a cloud API, we only supply table metadata, such as column names and data types, to the LLM to generate standard SQL queries. The generated SQL query then runs entirely on the local machine using DuckDB in-memory compute, ensuring zero raw-data transmission.

The process begins with a Natural Language Question, which is sent to a Local LLM (Ollama) or a Bring Your Own Key (BYOK) provider. The LLM generates a SELECT query, which is then passed through an AST Security Sanitizer. This filter uses the node-sql-parser library to validate the query, ensuring it is a SELECT statement and only allowing read-only queries. If the query passes validation, it is executed locally using the Local DuckDB Engine in-memory.

Finally, the rendered results are displayed on the client interface using a Local React Data Grid & Chart. This zero-raw-data architecture ensures compliance with HIPAA, GDPR, and other data protection regulations while still providing powerful Natural Language to SQL capabilities. To try it out, visit the VeilAnalytics live demo at veil-analytics.onrender.com or learn more on the official website at veilanalytics.netlify.app.