A board does not need to understand transformer architecture to oversee AI. It needs a short set of questions that management answers the same way every quarter, with evidence. The fifteen below cover the areas where AI use creates new exposure: who holds the keys, where data goes, what stops spend, what happens when a provider fails or changes a model, which legal obligations apply and when, and whether the company's accounts could be abused. For each question there is a description of a good answer and the red flag that should prompt a follow-up.
A good answer has three parts: an owner by name or role, a record that shows the answer is true today, and a control that has been tested. An answer that relies on one person's memory is the most common red flag of all.
Access and credentials
1. Who holds our AI provider keys, and how many are there?
Provider API keys are spending authority and data access in one string. They live in applications, CI systems, laptops, third-party tools and platforms that hold them on the company's behalf.
- A good answer: "We hold this many active provider keys across these providers. Each is in the inventory with an owner, the system that uses it, where it is stored, its provider-side spend limit and the date it was last rotated. The inventory is reconciled monthly against each provider's key list."
- Red flag: "Engineering manages that." No count, no owner per key, or keys stored in application environments with no limit at the provider.
The columns of a working inventory are in AI API key inventory.
2. Who is allowed to create new keys?
If anyone with a provider account can create a key, the inventory is out of date the day after it is reconciled. Providers have added controls for this: OpenAI added organization and project controls for API key creation on 15 September 2026, and API key expiration controls on 10 September1.
- A good answer: "Only the platform team can create production keys; key creation is restricted at the organization level where the provider allows it, keys expire, and creation events go to the security team."
- Red flag: every developer can create keys in the production organization, and nobody is told when they do.
3. What happens when a key leaks?
- A good answer: a written runbook with a named on-call owner: revoke at the provider, replace, review the old key's usage from the exposure date, and an estimate of the maximum loss set by the key's spend limit. The runbook has been exercised.
- Red flag: the first step is "find out which system uses the key".
Data and providers
4. Which providers and models do we use, and what data goes to each?
- A good answer: a list of providers and models per product feature, the categories of data each receives (customer content, personal data, source code), the provider's data retention and training terms for that account, and the processing region.
- Red flag: the list covers the main product and not internal tools, support macros, analytics jobs or employee use of AI assistants.
Defaults move. Google's Gemini API changelog for 26 October 2026 states that Generate Content API storage is on by default for new paid-tier projects, with an opt-out per request or in settings2. A data-flow answer that was true last quarter may not be true for a project created this week.
5. Are prompts and responses logged, where, and for how long?
Logging is useful for debugging and evaluation and is also a store of sensitive data, often outside the systems the data protection team reviews.
- A good answer: "Prompt and response logging is off by default, switched on per system with a retention period agreed with legal; provider-side storage settings are recorded per project; personal data is redacted or excluded where it is not needed."
- Red flag: full prompts kept indefinitely in an observability tool nobody has assessed.
Spend
6. What did we spend on AI last month, and on what?
- A good answer: provider usage per workflow and per team, AI tool subscriptions as a separate line, estimates reconciled with invoices, and the share of spend that cannot be attributed. See the AI spend report leadership actually reads.
- Red flag: the number comes from the invoice a month later, with no breakdown.
7. What stops spend before it happens?
Alerts after the fact are not a control. A runaway agent or a leaked key can spend a month's budget in hours.
- A good answer: hard limits that refuse a request before the provider call, per organization, per team or project and per key, plus spend limits set at each provider account as the last line.
- Red flag: "We get an email when we pass the budget."
Incidents and change
8. What happens to our product when a provider goes down?
- A good answer: each critical workflow has a fallback model or provider that has been evaluated for it, failover has been exercised in a drill this quarter, and the product shows a clear message when every option fails. See AI provider concentration risk.
- Red flag: the fallback is "switch the model name and redeploy".
9. How do we learn about model retirements and changes, and who approves a model change?
Providers retire and change models on their own schedule. In October 2026 OpenAI deprecated its GPT-4o, GPT-4o mini, GPT-4.1 and GPT-4.1 mini snapshots for shutdown on 14 January 20273, and Google began routing API requests for gemini-3.7-flash and gemini-3.5-flash to newer models automatically2.
- A good answer: someone owns watching vendor deprecation pages; each production workflow records the model it uses; a model change goes through evaluation and approval like any other change; and an alert fires when responses start coming from a different model.
- Red flag: model names hard-coded across the codebase, and nobody notices a change until customers do.
10. Could we move off our main provider within a quarter?
- A good answer: an exit plan per critical workflow: the alternative, its evaluation result, the change needed and the time it takes, with the alternative's accounts kept active.
- Red flag: "We would have to rebuild."
Obligations and exposure
11. Which EU AI Act obligations apply to us, in which role, and when?
The European Commission's AI Act page sets out the timeline as amended by the AI Omnibus, which entered into force on 27 July 2026: prohibited practices and AI literacy obligations have applied since 2 February 2025; obligations for general-purpose AI models since 2 August 2025; the Act became applicable on 2 August 2026, with transparency rules from August 2026; rules for high-risk systems in the sensitive areas of Annex III apply from 2 December 2027; and rules for high-risk systems embedded in regulated products under Annex I from 2 August 20284.
- A good answer: an inventory of AI systems with the company's role for each (provider, deployer, importer, distributor), an assessment of which could be high-risk, and a plan with owners for the 2027 and 2028 dates. Legal counsel has reviewed it.
- Red flag: "The AI Act is about model makers." Deployers have obligations too. See the EU AI Act for AI platform teams.
12. Where is AI being used without approval?
Shadow AI is AI use outside the approved path: personal API keys on company cards, browser assistants handling customer data, a team's own provider account.
- A good answer: a quarterly search of expense reports, provider accounts, code repositories and network egress, an approved path that is faster than the unapproved one, and a policy that says what is allowed. See finding shadow AI.
- Red flag: "We have a policy" with no search behind it.
13. What do our AI vendors' terms say about data, retention, suspension and changes?
- A good answer: for each provider, a summary of data use and retention, model retirement and price change notice, the suspension process, and incident support, reviewed by legal and procurement. The AI vendor security review questions give a template.
- Red flag: click-through terms accepted by an individual engineer and never read.
14. If one of our customers abuses our AI feature, can the provider act against that customer alone?
A provider that sees abuse on an account may act against the account. Anthropic's release notes for 20 October 2026 introduce user profiles so that a platform or gateway can tell Anthropic which end customer each request is for, and "a safety enforcement action can then apply to one end customer instead of your whole organization"5.
- A good answer: every request carries an opaque end-customer identifier where the provider supports one, per-customer limits exist, and abuse by one customer can be stopped without stopping the product.
- Red flag: all customers share one provider account with no attribution, so one customer's abuse can suspend everyone.
15. Could our accounts or product be used for distillation or other abuse, and would we know?
Distillation means collecting a model's outputs at scale to train another model. The September 2026 joint advisory from CISA, NSA and FBI describes campaigns that reach frontier models through native APIs, cloud providers, third-party aggregators and proxy services, and asks AI companies to detect anomalous accounts and behaviour such as enterprise-scale throughput6. A company that exposes an AI feature to the public, or whose keys leak, can become one of those access paths.
- A good answer: per-customer and per-key limits, alerts on unusual volume or geography, a review of whether the free tier or trial of our own product could be used as an open model proxy, and a contact for the provider's trust and safety team.
- Red flag: an AI endpoint anyone can call without limits, and no one watching its volume.
The checklist
- Inventory of every AI provider key, with owner, storage, limit and rotation date
- Key creation restricted; keys expire; creation events reviewed
- Leaked-key runbook, exercised
- Providers, models and data categories listed per feature, including internal tools
- Logging and retention settings recorded, including provider-side storage defaults
- Monthly AI spend by workflow, reconciled with invoices
- Hard limits that refuse requests before the provider call, plus provider-side limits
- Evaluated fallback and a failover drill for each critical workflow
- Model change approval and an alert on unexpected model changes
- Written exit plan per critical workflow
- EU AI Act role and risk assessment, with owners for 2 December 2027 and 2 August 2028
- Quarterly shadow AI search
- Vendor terms summarised and reviewed
- End-customer attribution on requests and per-customer limits
- Abuse and distillation exposure reviewed for public AI features
Answering these questions with INFRO
INFRO is a control plane between your applications and the AI provider accounts you already hold. It produces records and controls for several of these questions, for the traffic that passes through it.
- Keys (1–3). Provider keys are connected once and stored encrypted; applications use gateway keys per team or service, which can be limited to chosen providers and IP ranges, rotated with an overlap window, given spend limits and turned off. INFRO emails you when a gateway key is used from six or more countries in a day or at twenty times its usual daily volume; nothing is blocked automatically. Key and policy changes are in the audit log.
- Data (4–5). Prompt and response logging is off by default and switched on per organization with a retention period; guardrails can block, redact or flag patterns such as email addresses and API keys, on every plan, and detection can miss things.
- Spend (6–7). Budgets per organization, project, member and key refuse a request before the provider call; usage is attributed by project, key and model, as an estimate of what your providers bill you.
- Change (8–9). Model allowlists restrict which models each project or key may call; routes fail over across your own connections; retiring models are flagged before their shutdown date.
- Customers (14). End-customer identifiers on requests drive per-customer caps.
What it does not do: it does not find keys or AI use in accounts you have not connected, it does not make an organization compliant with the AI Act, and it does not see traffic that goes straight to a provider. INFRO also holds your provider keys, so it belongs in your own vendor review.