What an AI Engineer job actually tests

An AI Engineer job listing almost never asks you to train a model. Instead, it focuses on Python skills, backend services, APIs, and deployment. For a mid-career professional, encountering terms like LLM-based retrieval systems in such a job posting may make them believe the role is beyond their reach. However, upon closer inspection, the majority of AI Engineer requirements often revolve around constructing a service that integrates a model through an API, storing vectors, and delivering a functional product.

A careful comparison of multiple AI Engineer job listings reveals that the ratio of tasks related to model training is significantly lower than those involving service integration and deployment. This discrepancy between the job title and its requirements is not a misleading tactic employed by companies; it is a clear illustration of what the position entails.

Model training, while a critical aspect of machine learning, is typically limited to specialized firms equipped with substantial budgets and extensive data resources. Conversely, the bulk of AI Engineer roles involve product engineering where a model is incorporated into an existing system. The core components of this composition include a Service that handles user requests, an External Service housing the model accessed via API calls, a Vector Database for ground-up document retrieval, and various storage and queue systems to maintain system performance during model responses.

Thus, the essence of an AI Engineer's work is less about model training and more about assembling these components into a robust, real-world application. This composition may look similar to other engineering tasks, such as building a checkout flow or notification pipeline, but with the added complexity of model integration. Contrary to popular belief, the technical mastery of gradients or the completion of an ML program do not define an AI Engineer's qualifications.

The hiring process primarily assesses the candidate's ability to combine a Service, External Service, Vector Database, and related systems into a functional and resilient solution. As such, the skills to compose these elements are not novel but mirror the competencies needed for any production-level engineering project. In a course taught at UC Berkeley this fall, students engaged in building real applications incorporating agentic coding, which were then evaluated based on their design decisions.

This approach highlights the significance of composition—the ability to select the appropriate primitive for a given task and integrate it correctly—over deep technical knowledge of model training. The Vector Database, in particular, has emerged as a crucial component made possible by the AI era, acting as the "retrieval half" of almost every AI feature encountered in daily life.