Simple explanation
Two governance practices focused specifically on external AI tools and high-stakes AI decisions — this lesson covers both.
Technical explanation
- AI vendor risk assessment — evaluating a third-party AI tool or model's data handling practices, training data provenance, and security posture before adoption; specifically needs to ask AI-specific questions beyond a standard vendor security questionnaire, like whether customer data submitted to the tool is used to further train the vendor's underlying models (a question that has no traditional-software equivalent, but matters enormously for an AI tool).
- Human-in-the-loop / human approval — requiring a human to review and sign off before a high-stakes, AI-driven decision takes effect; common policy examples include requiring human review before an AI-driven hiring rejection, loan denial, or employee termination recommendation is finalized.
- Why AI vendor risk assessment differs from general third-party risk assessment: traditional vendor risk focuses heavily on data security and availability; AI vendor risk assessment adds an entirely new dimension around what happens to data used AS INPUT to the AI system itself — whether it's retained, whether it influences future model training, and whether that creates a data leakage risk to OTHER customers of the same vendor down the line.
- Ongoing monitoring, not just initial assessment — an AI vendor's data handling practices or model behavior can change over time (a policy update, a model version change), meaning vendor risk assessment for AI tools benefits from periodic re-review rather than a one-time approval that's assumed to remain valid indefinitely.
Synonyms / related terms
| Term | Means | |---|---| | Human-in-the-loop | Requiring human approval before a high-stakes AI decision takes effect | | AI vendor risk assessment | Third-party risk assessment adapted specifically for AI tools/vendors |
Concept Check
"A procurement team evaluates a new AI vendor using only their standard, generic third-party security questionnaire, with no AI-specific questions added." This misses the AI-specific dimension of vendor risk entirely — a generic questionnaire wouldn't ask the AI-specific questions that actually matter, like whether submitted data trains the vendor's models, which has no equivalent in a standard traditional-software vendor risk questionnaire.
Interview-style Q&A
Q: Why require human-in-the-loop specifically for high-stakes, IRREVERSIBLE decisions rather than for every AI-assisted decision an organization makes? A: "Practicality and proportionality. Requiring human sign-off on every single AI-assisted action, regardless of stakes, would eliminate most of the efficiency benefit AI is supposed to provide, and isn't actually where the real risk concentration is. Reserving mandatory human review specifically for high-stakes, hard-to-undo decisions — termination, loan denial, medical recommendations — focuses that governance control where an AI error would actually cause serious, difficult-to-reverse harm."
Memory trick
"Vet the Vendor's Data Handling, Approve the High-Stakes Decision" — two governance controls, each targeting a different point where AI risk concentrates: an external tool relationship, and an individual consequential decision.