I thought Text-to-SQL was just a translation layer. I was completely wrong. 🤯

When a user asks a question about movies, the system doesn't immediately generate a SQL query. Instead, it goes through a multi-step process to understand the intent, identify entities, define relationships, apply filters, and finally create the SQL query.

A key part of this process, called Schema Linking, involves modeling the entire problem as a network graph. Every word in the user's question becomes a node, and every table, column, and relational operator in the database schema also becomes a node. Connections (edges) are drawn between them, weighted based on the probability of the semantic relationship.

For example, the word "Nolan" might connect to "directors.name" with a low cost, but to "actors.name" or "movies.title" with a high cost. The system then uses a shortest-path algorithm to find the cheapest cumulative path through the graph, which becomes the chosen interpretation.

This process is fundamentally different from simply translating human language into code, like Google Translate. It involves complex cognitive steps like NLP, NLU, and NLQ, which are often treated as a single translation layer.

With the rise of Large Language Models (LLMs), this architecture has been disrupted. Now, you can feed an LLM your database schema and a raw human question, and it will output working SQL instantly. However, this speed comes with trade-offs. There's a higher risk of Schema Hallucination, where the LLM invents columns that don't exist in the database.

Production systems often use a hybrid approach, combining the deterministic pipeline with LLM generation. This allows them to take advantage of the strengths of both systems.