The decision
Choose the mechanism by the authority the answer requires. Retrieval-augmented generation can help explain material in documents. A deterministic query returns structured facts under explicit rules. Tool calling lets a model request a defined operation; your application must still validate that request and decide whether it may run.
A worked example
An illustrative service assistant receives three questions: “What is the returns policy?”, “Has my return arrived?” and “Please cancel the collection.” The first may use retrieved policy passages; the second should read the authorised record; the third changes state and needs a validated command with suitable confirmation.
Alternatives worth weighing
- You can use a normal form for the cancellation and conversational search for the policy. A single chat interface can combine these paths, but it should preserve their distinct permissions, evidence and error handling. Conversational convenience is not a reason to replace a reliable structured workflow.
Where the plan breaks
Do not answer live stock or balances from an old document index. Do not let the model invent a query with unrestricted database access. Keep supported operations narrow, validate parameters and handle missing or ambiguous identifiers before execution.
Route the request before composing the answer
A useful design exercise is to write the answer contract for each of the three return questions. A policy explanation must identify the policy version and supporting passage. A status lookup must identify the authorised return record and how recently it was read. A cancellation must identify the exact collection, the permitted transition and the result of the command. The wording can be conversational, but these obligations should remain separate in the application.
A single message may contain several intentions: explain a charge, show its status and cancel the collection if it has not started. Do not let a plausible explanation silently satisfy the condition for a write. Read the authoritative state, present the intended action and validate again when the user confirms. If the state has changed in the meantime, explain that change and ask for a fresh decision. Treat missing and ambiguous identifiers as requests for clarification, not permission to choose a convenient record.
Test the boundary between understanding and execution
For the illustrative assistant, prepare tests in which two collections share a customer name, a collection has already completed, the caller lacks access and the provider times out after receiving a cancellation. Define the acceptable outcome of each test with the business owner. A successful chat reply is not evidence that the cancellation happened. The interface should distinguish completed, rejected and unresolved operations and offer a reference that an operator can investigate.
Keep the command implementation usable without the model. This makes its permission checks, validation and repeat handling easier to test independently. Give source owners responsibility for policy updates, operations staff responsibility for unresolved commands and the product owner responsibility for which actions are offered. If the conversational route becomes unreliable, retain the conventional form or lookup as a usable fallback instead of disabling the whole workflow.
Before you commission the work
Is the user asking for explanation, a current fact or an action? Which system is authoritative? Can the user review a consequential operation before it runs? Those answers should drive the AI integration design.