Persuasive wording is not proof of correctness
AI can write fluently while using a wrong number, an outdated policy or an unsupported conclusion. A useful business answer must fulfil the specific task correctly.
Check whether it addresses the question, preserves limitations and gives the recipient an appropriate next step.
Ask five questions about each answer
Is it correct? Does the cited source support the claim? Is the source current? May this person access the information? Is missing knowledge acknowledged?
A link alone proves little. Open the relevant passage and compare the number, condition and scope. A source may mention the topic without supporting the particular claim.
Go beyond a successful demonstration
Test a current instruction, an outdated version, conflicting documents, a question without supporting information and a restricted source.
Include a conditional statement: a delivery commitment may depend on confirmed stock. Check that the condition survives in the answer. Use synthetic or approved anonymised fixtures.
Review framework: three fictional cases
An editorial proposal for your pilot, not results from a completed chifty customer test.
Classify errors by their impact
Awkward wording needs a different response from unauthorised disclosure. Define blocking failures before the pilot.
Record the question, permitted source, expected answer, actual result, cause and owner of the follow-up. This helps distinguish source, access and answer-preparation problems.
Keep people responsible for commitments
Prices, contract terms, deadlines and sensitive decisions require an authorised reviewer with sufficient context and time.
For email drafts, also check recipients, attachments, conversation context and confidential information. Sources support review rather than replacing responsibility.
Review quality during operation
New documents, changed access rights and different models can affect answers. Repeat relevant tests after changes and maintain a clear reporting route.
Discuss your intended knowledge and email workflow in a chifty demonstration. Bring representative questions and the criteria that matter to your team.
Useful next steps: