01
Begin with a decision or task.
Start by identifying the work the system should support, the people involved and the result they need. A narrow, testable use case creates a clearer basis for design and evaluation.
Insight
AI operating models.
A practical structure for putting intelligent systems into real organizational workflows.
← All insightsBy Quantilyx · Updated
The working idea.
Useful AI depends on more than a model. It needs a defined task, reliable information, human responsibility and a way to observe what happens in everyday use.
This perspective is intended as a practical starting point. The right structure depends on the organization, users, information and responsibilities involved.
What to consider
02
Keep responsibility visible.
Define where people review, approve or correct the system. The interface should make sources, uncertainty and exceptions understandable to the person responsible for the outcome.
03
Measure behavior in context.
Evaluation should reflect the real workflow: relevant examples, expected failure cases, response quality and the time or effort required from the team using it.
Write an acceptance set before choosing a model
Collect representative questions and record what an acceptable answer must contain. Include ambiguous questions, misspellings, outdated documents and questions that the source collection cannot answer. Keep some examples separate from development so that improving the prompt does not simply memorize the test. Review retrieval and answer quality separately: a good model cannot compensate for the wrong source being selected.
Trace one answer through the system
For a document assistant, follow the route from an approved document to extracted text, an indexed passage, a retrieved result and the final answer. Record where permissions are checked and where source references are attached. When an answer is wrong, this trace helps distinguish a stale document, a retrieval miss and an unsupported model statement. Each needs a different correction.
Define the fallback and the owner
A low-confidence result should offer a useful next step: refine the question, open a source document or contact the responsible team. Do not silently turn an uncertain response into a completed business action. Give a named role responsibility for reviewing examples, updating source material and deciding whether the system is ready for broader use.
Use a small release checklist
Before release, verify that restricted documents remain restricted, removed material leaves the index, unsupported questions receive a clear response and model failures leave the ordinary workflow available. Compare quality, response time and review effort with the existing process. A pilot is ready to expand when these checks meet agreed criteria, not merely when its demonstration sounds convincing.
Related capability
AI & Intelligence
AI assistants, document intelligence and decision-support tools connected to the work people actually do.
Explore AI & Intelligence ↗Discuss your context ↗