What to ask an AI vendor after the impressive demonstration
The demonstration runs smoothly. The answer appears in seconds, the interface looks familiar and the sales team has anticipated the obvious questions. It is a useful introduction to the product. The investment decision needs a different conversation.
Ask the provider to examine the work your organisation actually intends to perform. The difficult cases and the people checking the results belong in that discussion as much as the successful examples.
Bring your own test cases
Choose representative examples through an agreed process, using information you are authorised to share. Include incomplete inputs, ambiguous requests and cases the tool should decline or send for human review. Record the expected standard before seeing the results.
Where the provider cannot test on your environment or data, ask what that limitation means for the decision. A test on an easier workload may establish technical capability without establishing suitability for your process.
Document the complete workflow, including correction and review. Compare the effort with the current process and with reasonable alternatives. Ask the provider to explain failures as carefully as successful outputs.
Make the information arrangements explicit
Establish what information is sent to the provider, where it is processed and which other parties may receive it. Ask about retention, deletion, access permissions and use of customer information for model improvement. Technical and contractual specialists should check the answers against the proposed service and applicable requirements.
Identify the records available for investigating an error. Consider whether the organisation can trace a consequential output to the relevant inputs and system version, subject to appropriate information protection.
Treat a change in data access or action permissions as a change in the use being approved. A tool drafting text and a tool able to send it or change records require different controls.
Ask what changes after the purchase
Who informs you about a material model or service change? What testing can you perform before it affects production? Agree how performance concerns and incidents will be handled, including when support is needed outside normal hours.
Examine the full operating cost. Usage charges, integration, staff review, monitoring and exit work can affect the case. Ask what happens when volumes grow or the organisation needs to move to another provider.
The NIST AI Risk Management Framework is a voluntary reference for considering AI risks throughout design, development, use and evaluation. Procurement should feed the organisation's continuing oversight rather than end it. [1]
An approval paper should identify the tested use, the evidence obtained and the outstanding conditions. Keep those conditions attached to the purchase decision so that the operational team inherits more than a licence and a successful demonstration.
[1] NIST AI Risk Management Framework Read the source
Nasaq Partners works with a network of experts on governance, strategy and execution. Explore our advisory services.