Your AI Team Manifest Passed Validation. It May Still Be Wrong.

A manifest can meet JSON Schema validation criteria and still describe an AI team that is unsafe to operate. Even though all required fields are present, the permissions field may be incorrectly formatted, a task may reference an unknown owner, and some actions may lack proper approval gates. This does not signify a failure of JSON Schema itself; the issue lies in assuming structural validation guarantees the entire model is correct.

It is crucial to maintain a clear distinction between structural validation and overall model correctness. The current NexFlow tool validates AI team configuration through YAML manifests, which cover various aspects such as project structure, actor definitions, task assignments, workflows, and more. Each supported manifest kind is mapped to a corresponding JSON Schema for validation.

The repository validator parses YAML, converts it to a JSON-compatible format, selects the appropriate schema based on the manifest kind, and reports errors with file and instance paths. At a repository checkpoint, 113 manifests are validated against 17 schemas. Structural validation is effective at catching errors that don't require knowledge of other files.

For instance, an invalid permission effect, missing required fields, unsupported specification version, unknown kind, incorrect data type, or invalid identifier format will be caught by the validator. However, when one manifest refers to another, semantic validation becomes necessary. This involves treating the manifest set as a connected model and resolving references across files.

NexFlow performs this check to verify task owners, workflow dependencies, handoff artifacts, permissions, capabilities, context sources, memory scopes, events, and extensions. It is important to note that a reference is only valid if it points to an actual target and adheres to the expected policies. Currently, NexFlow includes a bounded semantic reference smoke check for maintained examples, passing successfully for seven example projects, but this does not constitute complete semantic validation.

The phrase "one large green check" can be misleading, as it does not provide sufficient information about the validation process. Structural validation confirms that a document matches a published shape, while semantic validation only verifies the relationships and constraints explicitly listed by the tool. Runtime enforcement, which applies real permissions, credentials, approval gates, isolation, and audit rules, is handled separately and not performed by the structural or semantic validation layers.

A valid manifest does not guarantee safety; it only ensures the configuration adheres to the defined structure. JSON Schema does not isolate credentials, inspect actual access in external systems like GitHub or a filesystem, pause execution, or record audit logs. These responsibilities fall on a runtime layer that applies the declared policy during action execution.

NexFlow currently lacks a production runtime, CLI, and provider integrations, but it provides a specification, schemas, documentation, examples, and validation tooling to improve the model and its verifiable boundaries. However, claiming an AI team is executing based solely on this is insufficient. It is essential to clearly communicate the level of validation supported by any tool.

Simply stating "Supports NexFlow" is too broad. A useful validator should report detailed information, such as the file path, field path, validation layer, and specific error reason when a failure occurs. Omission of validation layers should also be clearly indicated. For example, if YAML parsing succeeded but JSON Schema validation failed or semantic reference resolution failed, the user should be able to distinguish these issues easily.

Both successful and unsuccessful validation outputs should provide precise details. Instead of a generic "valid," a tool should specify which schemas were passed or failed.