Latest Design Principles for Enterprise AI Systems

Start with a system boundary

A production AI service includes more than a model. It needs identity, authorized data access, workflow rules, evaluation, and a way to stop or recover when a dependency fails. This guide uses an illustrative internal policy assistant to make those responsibilities concrete.

Example: an assistant that answers policy questions

An employee asks which approval is required for a purchase. The assistant retrieves the current policy, cites the relevant section, and explains the next step. It cannot approve a purchase or change the policy. Those restrictions are part of the service contract, not instructions left solely to the model.

The request passes through six boundaries:

1. Identity: authenticate the employee and retrieve their current permissions.
2. Retrieval: query only policy sources the employee may access; retain document version and ownership metadata.
3. Generation: give the model the question and permitted evidence, with a defined answer format.
4. Policy checks: validate citations and enforce which tools may run. Approval rules remain outside the prompt.
5. Human escalation: ask for clarification or route to the policy owner when evidence is missing or contradictory.
6. Observability: record model and prompt versions, retrieval identifiers, latency, validation results, and escalation outcomes without unnecessarily logging sensitive content.

Design principles and concrete controls

Principle	Control in the example	How to validate it
Separate policy from generation	Application code controls tool permissions and approval limits	Ask the model to approve a purchase; no write operation should be possible
Enforce access during retrieval	Apply source permissions before passages reach the model	A user without access cannot retrieve or infer a restricted policy
Keep versions explicit	Version prompts, retrieval configuration, and model selection together	Reproduce a recorded evaluation after a change
Bound failure	Set timeouts, retry limits, and a fallback response	A retrieval outage produces an unavailable response, not an invented policy
Preserve intervention	Route conflicting evidence to an accountable policy owner	Conflicting documents trigger escalation
Minimize data exposure	Define log fields, retention, and access separately from prompts	Logs omit secrets and follow the agreed retention policy

Failure scenario: a revoked permission

Suppose a manager loses access to a confidential policy, but an embedding index still contains its text. Filtering only the initial login is insufficient. Retrieval must enforce current entitlements, and cached responses must not bypass that boundary.

Test revocation using two accounts and the same question. Revoke access, invalidate relevant cached results, and repeat the request. The restricted text must not reach the prompt, response, or unauthorized logs. If the system cannot confirm access, stop the retrieval and offer a safe next step.

Before expanding beyond a pilot

• Build an evaluation set containing ordinary, ambiguous, unauthorized, and adversarial requests.
• Agree acceptable answer quality and response time with the service owner.
• Track correction effort and escalation outcomes alongside model output quality.
• Test unavailable dependencies, retry exhaustion, and rollback to the previous configuration.
• Assign ownership for source updates, security incidents, and model changes.
• Document what the assistant may do and what still requires a human decision.

No single benchmark establishes production readiness. Review the controls together against the intended workflow and the consequences of failure.

Explore private AI architecture or discuss an implementation.