Procurement teams are good at buying software. AI is not software in the way that matters for diligence. A traditional software review checks features, security, and price. An AI system can pass all three and still be the wrong choice, because the things that determine whether it works in your context, the data it was trained on and the data it will see in production, are not on the feature list.
Here are the seven categories every enterprise procurement scorecard for an AI system should cover.
1. Data handling and residency
Where is your data processed and stored, for how long, and who can access it. Does the vendor reserve any right to use your data for model training. For sensitive data, on-premise or in-region deployment may be non-negotiable. This is the category that most often gets skipped and most often causes the worst problems.
2. Model quality under your conditions
The vendor's accuracy number was measured on the vendor's data. What matters is performance on yours. For a system that will operate in Arabic and English, test it on bilingual, code-switched input. For one handling your document types, test it on your documents. The demo is curated. Insist on a pilot with your real data.
3. Integration and workflow fit
A model that produces output your team cannot use is worthless. How does it fit the existing workflow, what systems does it connect to, and what does a human have to do around it. Integration risk is where most AI procurements quietly fail after signing.
4. Evaluation and bias methodology
Ask for the model card and the bias testing methodology. A vendor that cannot produce either is telling you their evaluation is informal. This matters most where the system makes decisions about people.
5. Commercial and contractual terms
Beyond price: what happens on termination, do you get your data back and in what format, and if the vendor fine-tuned a model on your data, what happens to that model. The end of the contract deserves as much attention as the start.
6. Vendor capability and viability
Can this vendor maintain and improve the model, or are they reselling someone else's API with a thin layer on top. A young vendor is not disqualifying, but you should know what you are actually buying and who supports it when it breaks.
7. Regulatory and governance fit
Does the system, and the way you intend to use it, sit cleanly within the obligations you face. For a regulated entity in the GCC, that means mapping to the relevant CBUAE, DFSA, ADGM, or SDAIA expectations before you sign, not after.
Why the cheapest option is often the most expensive
We ran this checklist for a private counselling clinic choosing between three AI transcription vendors. The operations team had scored them on the standard criteria and was leaning toward the cheapest. Diligence showed that vendor would have sent recorded therapy sessions to offshore servers with ambiguous data rights, and that the second cheapest produced transcripts too unreliable for clinical use. The vendor we recommended cost 40 percent more and was the only safe choice. The full case study is here.
The point of procurement diligence is not to slow the purchase down. It is to make sure the number on the contract reflects the cost of the right system, not the cost of the cheapest demo.
If you are weighing an AI investment, acquisition, vendor selection, or training programme, our team is happy to start with a conversation about scope and approach.
The views and findings in this article are shared for general information only. They are high-level perspectives, not legal, financial, regulatory, or other professional advice, and should not be relied upon for any specific decision or circumstance. For guidance tailored to your situation, please consult a qualified adviser.