AI Vendor Risk: The Questions That Reveal How Your Data Is Used
Standard vendor questionnaires miss training use, retention of inputs and the model supply chain behind the product you are buying.
GuardsArm Team
Security Experts
Your existing vendor risk process asks about encryption, access control, certifications and breach notification. Those questions remain necessary and are no longer sufficient, because an AI product raises questions about what happens to your data in normal operation rather than only if something goes wrong.
The questions your current questionnaire misses
Is our data used to train or improve your models? The question that matters most, and the one standard questionnaires do not ask. Answers range from a clear contractual no, through opt-out defaults, to yes-in-aggregate. Get it in the contract rather than in a marketing page, because marketing pages change.
Push on "de-identified" or "aggregated" — those words do substantial work and their meaning varies. For free text, de-identification is considerably harder than for structured fields.
What happens to our inputs? Retention period, whether prompts and uploads are stored, whether staff can view them, and whether deletion can be requested and confirmed.
Whose model is it? Many AI products are an interface over someone else's foundation model. That means an additional party in your data path, possibly in another jurisdiction, with their own terms — and your vendor's data commitments are only as good as the ones they have from their provider.
What can it do autonomously? An AI feature that drafts something is a different risk from one that sends, purchases or modifies records without confirmation. See prompt injection.
How do you handle prompt injection? A vendor who does not understand the question is telling you something useful.
Mapping data flows properly
| Question | Why it matters |
|---|---|
| Where is inference performed? | Residency, and which law applies |
| Where are inputs stored, and for how long? | Your retention obligations |
| Which sub-processors are involved? | The full chain, not just your counterparty |
| Is data segregated between customers? | Shared context is a real failure mode |
| Can we require deletion, and is it confirmed? | Contractual and practical |
| What is logged, and can we see it? | Your accountability does not transfer |
For Canadian organisations, the residency answers connect directly to Quebec Law 25's transfer assessment and to accountability under PIPEDA. Processing in another province is a cross-border transfer under Law 25, which surprises buyers who were only thinking about US hosting.
Contract terms worth insisting on
- No training on our data, stated unambiguously
- Defined retention and deletion, with confirmation
- Sub-processor disclosure, with notice of changes and a right to object
- Data residency commitments where you have an obligation
- Breach notification fast enough to meet your own clock
- Audit or assurance rights, proportionate to sensitivity
- Model change notification, where the model materially affects outcomes
- Exit terms — return and deletion, confirmed in writing
The model change term is unusual and worth asking for where outcomes matter. A vendor silently switching the underlying model can change behaviour, accuracy and bias characteristics in a system you validated against the previous one.
Proportionality
Not every AI feature needs this treatment. Triage:
A summarisation feature operating on already-public content needs light touch. A system processing health information, influencing employment decisions, or acting autonomously on your behalf needs all of it.
Where to start
Take the AI product your organisation most recently adopted and ask the vendor two questions in writing: is our data used for training, and how long are our inputs retained? If those answers are not already in your contract, that is where your AI vendor risk programme begins.
Reassessment is not annual here
Conventional vendor risk runs on an annual cycle. AI products change faster than that, and the changes are material: an underlying model is swapped, a new capability is enabled by default, retention terms are revised, a sub-processor is added.
Trigger a reassessment on events rather than on the calendar:
- The vendor announces a model change or a significant new capability
- Terms of service or the data processing agreement are updated
- A new sub-processor appears in their disclosure
- Your own use expands into more sensitive data than originally assessed
- The feature gains the ability to take an action it previously only drafted
That last trigger is easy to miss. A product that summarised last quarter and now sends on your behalf is a different risk assessment, and vendors rarely frame a capability release as a change to your risk position.
Where existing certifications help, and where they stop
A vendor's SOC 2 or ISO 27001 certificate is worth having and does not answer the questions above. Those frameworks assess controls over the security of information, not whether your inputs train a model, how long prompts are kept, or what the system may do autonomously.
Read the certificate for scope — which services and which period — and then ask the AI-specific questions separately. A vendor who responds to them by resending the certificate has not answered, and that response is itself informative about how well they understand their own product. See compliance automation and its limits for the same distinction in a different setting.
GuardsArm assesses AI vendors and builds the assessment process. See third-party risk management or book a scoping call.
Written by GuardsArm Team
Our team of cybersecurity experts brings decades of combined experience in penetration testing, compliance auditing, and incident response. We're dedicated to helping organizations strengthen their security posture.
Take the next step on this topic
Talk to the GuardsArm team about how these services apply to your environment.
