In short
Almost every supplier discovery tool now claims AI, and many claim agents. So "does it have AI?" no longer separates anything. In 2027 the useful questions are whether you can check what the AI tells you, whether you can see what its agents did, and whether the tool will survive the pilot.
This vendor-neutral checklist groups 27 criteria into eight categories, each with the question to ask the vendor and what a good answer looks like. Score every tool the same way, on your own briefs, and the result stops depending on whose demo was smoother.
Why the checklist changed for 2027
Generative AI in procurement hit its reality check. In July 2025 Gartner placed it in the trough of disillusionment, with "supplier recommendations" among the named use cases (Gartner). Analyst Kaitlynn Sommers named two problems a buyer can test for: "fragmented and low-quality data across procurement systems can hinder accurate outputs, and integrating stand-alone GenAI solutions with existing platforms is often complex."
Agents arrived, and so did agent washing. Gartner predicts over 40% of agentic AI projects across industries will be canceled by the end of 2027 "due to escalating costs, unclear business value or inadequate risk controls", and estimates only about 130 of the thousands of agentic AI vendors are real (Gartner).
Pilots are not turning into practice. The Hackett Group found 43% of procurement organisations actively pursuing AI deployment in 2026, but only 12% at large scale; 69% get their AI through platforms they already use (Hackett Group). Gartner's October 2026 survey found 63% of senior procurement leaders expect AI to significantly improve performance, yet only 25% have redesigned roles because of it (Gartner).
The need has not gone away: 74% of CPOs in Deloitte's 2025 survey named active alternative sources as a risk-mitigation measure (Deloitte). Finding them is what a discovery tool is for.
How to score
Score every criterion 0, 1 or 2:
0: no answer, or only a slide.
1: partly in place, or on the roadmap with a date.
2: shown live, on your own briefs or data.
Multiply each category's average by its weight in the scorecard below. Never give a 2 for something you have only seen in a scripted demo.
The checklist
1. Data coverage, freshness and provenance
Criterion | Ask the vendor | What good looks like |
|---|---|---|
Coverage on your briefs | "Run these five to ten past briefs, including our hardest region." | Qualified, reachable suppliers you did not already know. A headline profile count does not score. |
Per-field provenance | "Show me the source for this ISO certificate and when it was last checked." | Each claim links to a document you can open, with a last-checked date. |
Freshness | "How do you detect a supplier that has closed or changed owner?" | A stated refresh process and a date on every profile. |
Self-reported data | "Which fields come only from the supplier?" | Self-reported fields are labelled and kept apart from registry, certificate or filing data. |
2. Answer accuracy and citations
The DeepTRACE audit, presented at ICLR 2026, found that even deep-research AI systems "still exhibit large fractions of unsupported statements, with citation accuracy ranging from 40--80% across systems" (arXiv). No published study measures this for commercial supplier discovery tools, so measure it in your pilot.
Criterion | Ask the vendor | What good looks like |
|---|---|---|
Citations support the claim | "Show me the sentence in the source behind each statement in this summary." | Inline sources that actually say what the summary says. |
Invented results | "How often does a shortlist include a supplier that does not exist or does not make this product?" | The vendor measures it and lets you measure it. |
Shortlist precision | "How many of these 20 would pass our first qualification call?" | Scored against your criteria, not the vendor's. |
Model changes | "Which models run this, and how will you tell us when they change?" | Named versions and advance notice. Agent behaviour differs across models and shifts over time (EconEvals; Business Arena), so re-test after upgrades. |
3. Agentic AI: autonomy, agent washing and audit trails
"Many use cases positioned as agentic today don't require agentic implementations," said Anushree Verma of Gartner (Gartner). Gartner's May 2026 procurement session advised: "Press vendors for clear agent roadmaps" and "Guard against agent-washing" (Gartner).
Criterion | Ask the vendor | What good looks like |
|---|---|---|
What the agent does | "Which steps does the agent plan and run itself, and which are fixed workflows?" | A plain description you can check in the product. |
Approval gates | "What can the agent do without a human approving it?" | You set the limits; nothing reaches a supplier without the approvals you choose. |
Action log | "Show me everything the agent searched, opened and decided for this brief." | A complete, exportable log with timestamps and sources. |
Supplier-facing behaviour | "If the agent contacts suppliers, how does it disclose that it is AI?" | Disclosure by default. How a negotiation bot is prompted changes how much suppliers trust it (Herold et al., 2025). |
4. Enrichment and risk intelligence
Criterion | Ask the vendor | What good looks like |
|---|---|---|
Certificates | "Can I see the expiry date and the certificate itself?" | Expiry date plus a link to the document or issuing body. |
ESG data | "Is this from documents I can open, or a self-assessment?" | Source type shown for every data point. |
Risk signals | "Where does this risk score come from, and what triggers an alert?" | Named inputs and dated alerts, not a black-box score. |
5. Integration: APIs, MCP and your suite
Criterion | Ask the vendor | What good looks like |
|---|---|---|
Suite fit | "How does a shortlist get into our sourcing event or onboarding flow?" | A documented connector or API, shown working. |
Open agent interfaces | "Do you support MCP, and can our suite's agents call you?" | Support for open protocols such as the Model Context Protocol, now under the Linux Foundation's Agentic AI Foundation (Anthropic). |
Permissions | "Can we limit each connection to read-only or specific actions?" | Scoped, logged permissions per connection. |
6. Security, data protection and EU rules
This is not legal advice. Get the vendor's answers in writing and have counsel review them.
Criterion | Ask the vendor | What good looks like |
|---|---|---|
Residency and transfers | "Where is our data stored and processed, and on what transfer basis?" | Named regions and a legal basis. The EU General Court upheld the EU-US Data Privacy Framework on 3 September 2025, but the ruling can be appealed (IAPP). |
Training on your data | "Are our briefs or shortlists used to train shared models?" | A contractual no, or opt-in only. |
EU AI Act role | "What is your role under the AI Act, and how have you classified this system?" | A written answer. Article 50 transparency duties, including telling people they are dealing with AI, apply from 2 August 2026 (Addleshaw Goddard; Art. 50). Regulation (EU) 2026/1744 moved the Annex III high-risk deadline to 2 December 2027 (Mondaq). |
AI literacy | "What training do you give our users?" | Material that helps you meet the deployer duty to support AI literacy. |
7. Cost predictability and exit
Criterion | Ask the vendor | What good looks like |
|---|---|---|
Usage-based costs | "What drives cost per brief or agent run, and can we cap it?" | Clear units, caps and usage reports. Escalating cost is a top reason Gartner gives for canceled agentic projects. |
Exit | "How do we leave, and what do we get back?" | Full export in a standard format. Under the EU Data Act, cloud providers can generally no longer charge switching fees from 12 January 2027 (Alston & Bird). |
8. Pilot design and adoption
Criterion | Ask the vendor | What good looks like |
|---|---|---|
Defined job | "Which of our problems is this pilot meant to solve?" | One named job, such as second sources for one category. |
Real briefs and a baseline | "Will you run our briefs side by side with our current method?" | Four to eight weeks, metrics and exit criteria agreed before the start. |
Adoption and roles | "Who changes how they work, and how do you support that?" | Training, named owners, and a reviewer for agent output. As Gartner's Fareen Mehrzai put it, AI implementation "is fundamentally a role redesign challenge" (Gartner). |
Scorecard
Category | Suggested weight | Tool A | Tool B | Tool C |
|---|---|---|---|---|
1. Data coverage, freshness, provenance | 20% | |||
2. Answer accuracy and citations | 20% | |||
3. Agentic AI and audit trails | 15% | |||
4. Enrichment and risk intelligence | 10% | |||
5. Integration | 10% | |||
6. Security, data protection, EU rules | 10% | |||
7. Cost predictability and exit | 5% | |||
8. Pilot design and adoption | 10% |
Treat per-field provenance and citation support as pass/fail. A tool that scores 0 on either should not win on total points.
What practitioners say
"Make sure your tool is focused on really, really being good with the data and allowing for really good inputs, and there's no hallucinations," Karoline Dygas, VP and Chief Procurement Officer at Nordstrom, said at Manifest 2026 (Supply Chain Dive, 18 February 2026).
"I come back to, what's the business problem you're trying to solve?" said John Eustis, SVP of U.S. group procurement at Toray Industries (America), at ISM World 2026, where he described starting with a low-risk agent pilot. Chris Martus, VP of procurement at The Mosaic Company, agreed: "You have to have a clear business case for what you want to achieve, and clear objectives, and then you can identify whether the juice is worth the squeeze" (Supply Chain Dive, 21 May 2026).
Darshan Deshmukh, President at ProcureAbility, named the pattern behind failed pilots: "We keep on making that mistake again and again, but we lead with the technology instead of focusing on the outcome" (Art of Procurement, 13 September 2026).
A closing note
If you add Speya to your shortlist, score it like any other tool: describe the need in plain language; AI agents search millions of supplier profiles across 250+ data sources and return AI-vetted shortlists with sources beside every name.
Events to watch in 2027
ProcureCon West, 7–10 March 2027, Las Vegas (agenda).
Gartner Supply Chain Symposium/Xpo, 3–5 May 2027, Orlando, with "AI value realization" among its themes (Gartner).
World Procurement Congress, London: AI Forum 18 May 2027, Congress 19–20 May 2027 (Procurement Leaders).
Frequently Asked Questions
What should I test first in a supplier discovery tool?
The data, on your own briefs. Run five to ten past requests, including your hardest region, and check how many qualified suppliers you did not already know and whether each claim links to a source you can open.
How do I check whether an AI tool invents suppliers or sources?
Open every cited source in your pilot shortlists and record three rates: suppliers that do not exist or do not make the product, statements with no source, and citations that do not support the claim. Research found citation accuracy of 40% to 80% across AI research systems (DeepTRACE).
Does the EU AI Act apply to supplier discovery tools?
Supplier discovery is not an Annex III high-risk use case in itself, but Article 50 transparency duties apply from 2 August 2026, which matters if an agent contacts supplier staff. Ask each vendor to document its role and classification, and check with counsel.
Should I use the AI in my S2P suite or a specialist tool?
Most teams start with what is embedded; Hackett found 69% do. A specialist tool earns its place when it finds suppliers your suite cannot, shows its sources, and feeds results back into the suite. Score both on the same checklist.
Sources
All sources are linked inline. Key references: Gartner press releases (June 2025, July 2025, May 2026, October 2026); The Hackett Group 2026 Key Issues Study; Deloitte 2025 CPO Survey; DeepTRACE (arXiv 2509.04499, ICLR 2026); EconEvals (arXiv 2503.18825); Business Arena (arXiv 2608.08621); Herold et al. (2025); Regulation (EU) 2026/1744; EU AI Act Article 50; Alston & Bird on the EU Data Act; IAPP; Anthropic on MCP; Supply Chain Dive; Art of Procurement.