# AI access review checklist: evidence to export for model keys

URL: https://infro.io/blog/ai-access-review-checklist-evidence-to-export-for-model-keys
Author: INFRO Editorial (https://infro.io/blog/editorial)
Publisher: INFRO
Published: 2026-11-03
Last modified: 2026-11-03
Last reviewed: 2026-11-03 (next review due 2027-02-01)
Topic: Security and governance
Format: Checklist

> An AI access review needs four exports each quarter: who can call which models, who owns each key, every policy change with its actor, and every exception with an expiry. Pull them as dated files from your gateway's APIs, check them against your own records, and keep them with a manifest.

Key takeaways:
- Review four things each quarter: who can call which models, who owns each key, what policy changed and who changed it, and which exceptions are still open.
- Export the evidence as dated, machine-readable files with a manifest, so the next quarter's review can diff against it.
- Check the retention window of every source before you promise a quarterly cadence; a 7-, 14- or 30-day request history needs more frequent exports.
- Gateway evidence covers traffic through the gateway; provider-side members and keys, and calls that bypass the gateway, need their own export.
- Test enforcement as well as configuration: keep one saved denied-request response per quarter as proof the allowlist refuses what it should.

## Scope the review before you collect anything

An AI access review answers four questions, and each one needs a file you can hand over:

1. Who, or which application, can call which models?
2. Who owns each key and each provider credential?
3. What policy changed during the period, who changed it, and why?
4. Which exceptions are still open, who approved them, and when do they expire?

Two public references help you phrase risks and tests. NIST's AI Risk Management Framework is intended for voluntary use, and NIST published a Generative AI Profile (NIST-AI-600-1) on July 26, 2024. The AI RMF 1.0 page also says the framework is being revised, so check the version you cite.[1] OWASP's LLM Top 10 now lives inside the wider OWASP GenAI Security Project, and the project page points to genai.owasp.org for the current list.[2] Neither is a SOC 2 or ISO 27001 control catalogue. Map each evidence file below to your own control IDs and treat those two as risk vocabulary.

This post is the repeatable part. For the questions a vendor questionnaire asks, see [what security reviewers ask about AI gateways](https://infro.io/blog/ai-vendor-security-review-questions); for building the key list itself, see [AI API key inventory](https://infro.io/blog/ai-credential-inventory).

## The evidence set at a glance

Each row below is one file with a question it answers, a source and a format an auditor can re-check.

| Evidence file | Question it answers | Where it comes from | Format |
| --- | --- | --- | --- |
| Member roster with roles | Who can change policy or keys | Console members screen or management API | JSON |
| Gateway key inventory | Who owns each key, its scope and expiry | Management API, keys resource | JSON |
| Allowlists per organization, project and key | Which models can be called | `model_allowlist` fields | JSON |
| Provider connections | Which provider accounts sit behind the gateway | Management API, connections | JSON |
| Audit events for the quarter | What changed, by whom, when | `GET /manage/v1/audit_events` | JSON |
| Request log and usage series | What was actually called | `GET /v1/requests/export`, `GET /v1/usage` | CSV |
| Denied-request sample | That enforcement works | One saved 403 response | JSON |
| Exceptions register | What is allowed outside the baseline | Your ticketing system | CSV |

Table: Quarterly evidence files for an AI access review.

The last row is yours, not the gateway's. A gateway records that an allowlist changed. It cannot know why you approved it.

## Who can call which models

Model access is set at three scopes, and the effective answer is the intersection. A model must satisfy every applicable organization, project and key policy. An empty list at one scope adds no restriction and does not override a narrower list elsewhere.[3] Entries are catalog model IDs or vendor wildcards such as `anthropic/*`, and an entry prefixed with `!` blocks what it names. A block always wins over an allow in the same list, and a list of only blocks allows every other model, including models added to the catalog later.[3]

That last behaviour is the one to look for in review. A block-only list is a permissive policy that looks restrictive. Flag any scope whose list has no allow entries and ask whether that was intended.

Export the lists from all three scopes. The `model_allowlist` field (up to 500 entries) is on organization settings, projects and gateway keys in the management API.[3] Then compute the effective set per key yourself and store it beside the raw lists, so the reviewer sees both the inputs and your reading of them.

Configuration is not enforcement, so add a test. Send one request for a model that a key must not reach and save the response. A denied model returns HTTP 403 with `error.type: "permission_denied"` and `error.code: "model_not_allowed"`, and the message names the list that refused it. A route whose targets are all outside the lists returns 503 `no_available_provider` instead.[3] Keep one of each per quarter. Also note the boundary: environment-scoped connections and key route restrictions apply separately from allowlists, so export those too.[3]

## Key owners and credentials

Applications should hold gateway keys, not provider credentials. A gateway key is created with a label, an optional spend limit, a rate-limit tier, and optionally an environment, a project and an expiry, and provider credentials stay sealed in the gateway.[4] For the review, that means three checks on the key export:

- Every key has an owner. The label is the only free-text field the docs describe, so put the owning team or person in a label convention (for example `payments-prod-<owner>`) and check it mechanically.
- Every production key has an expiry or a recorded reason it has none.
- Every key belongs to one environment and, where you use projects, one project.

Revoked and expired keys return 401, a disabled key returns 403 `key_disabled`, and a call from outside the key's allowed networks returns 403 `ip_not_allowed`.[4] Those codes are useful in a test: calling with a key you revoked this quarter and saving the 401 proves the revocation took effect.

Provider-side evidence is outside any gateway. Export the member list and key list from each provider account you connected, and reconcile them against the connections list. A provider key that the gateway does not know about is the finding a reviewer will care about most; see [detecting usage that bypassed your gateway](https://infro.io/blog/detect-gateway-bypass-provider-usage).

## People: roles, joiners and leavers

The roster matters because roles decide who can change a policy. In INFRO an organization has exactly one owner. Admins manage members, keys and policies, developers create and revoke gateway keys and call models but cannot change policies, members or billing, and analysts read usage, request history, policies and the audit log without creating keys or calling models.[5] A member limited to some projects cannot read the audit log.[6]

With SSO and SCIM (from the Business plan), the directory creates and deactivates memberships. An SSO assertion can grant admin or developer when a role attribute says so, everyone else joins as analyst, and it can never grant ownership.[7] Reconcile three lists each quarter: the roster export, your HR leavers list, and the audit events for members removed. A leaver who still appears in the roster, or a removal with no matching HR record, is a finding.

## Policy changes and exceptions

The audit log is append-only and records changes to gateway keys, provider connections, routes, projects, webhooks, policies, ownership, SSO and SCIM, plus membership changes. Each event carries actor, actor type, action, target type and ID, metadata, IP where recorded, and a timestamp.[6] Three limits shape how you use it:

- There is no CSV of the audit log. Read it as JSON from `GET /manage/v1/audit_events` with a token holding the `audit:read` scope; it takes `q`, `action`, `actor`, `from`, `to`, `limit` (up to 200) and `cursor`.[6]
- It can contain personal information, so share it with care.[6]
- No fixed retention is promised, so archive each quarter's pull yourself.[6]

Then do the join that makes it a review rather than a dump: for every policy, key or connection event in the quarter, find the change ticket. Events with no ticket go on the exceptions list or become findings.

An exception, for this review, is anything that departs from the baseline: an allowlist entry added for one project, a key with no expiry, a block removed. Each row in your register needs a requester, an approver, a scope, a start date and an expiry date. At quarter end, every expired exception must have a matching removal in the audit log, and every live exception must be re-approved.

> **Note: Provider-side changes**
> The gateway audit log describes changes made in the gateway. A key rotated or a member added directly in a provider's console will not appear in it. Pull each provider's own activity or audit export for the same period.

## Usage evidence, and how far back it goes

The request log records the model, the provider connection that served it, outcome, timings, usage and estimated provider cost. The docs call it operational evidence, not an invoice.[8] Two exports cover it: `GET /v1/requests/export` returns one request per row as CSV, up to 50,000 rows per file, and `GET /v1/usage` returns aggregates as JSON or CSV.[9]

Check the window before you promise a quarterly cadence. Queryable history is 7 days on Developer, 14 on Startup, 30 on Team, 90 on Business and 365 on Enterprise.[9] Only a 90-day or longer window covers a calendar quarter in one pull. On shorter windows, schedule exports monthly or weekly. Scheduled and emailed exports are not available, so the schedule is a job you run yourself.[9]

Also check what the key you export with can see. A gateway key's history access is `scoped` by default (its project, or only its own requests), `organization` (every request, grantable only by an owner or admin) or `none`.[8] Use an organization-scoped credential for the review, or the file will silently cover only one application. Prompt and completion capture is a separate opt-in, so the exports are metadata; review custom tags and user identifiers for personal information before sharing.[9]

## Pull it with a script

A read-only management token, scoped to the reads you need, keeps the pull reproducible. The management API serves every resource under `/manage/v1/<type>`, lists with GET, and pages with `{data, has_more, next_cursor}`.[10] Confirm the exact resource names and the token header format in the OpenAPI document and the management token docs before you rely on this sketch.

```python
import hashlib, json, os, pathlib, requests

BASE = "https://api.infro.io/manage/v1"
HEAD = {"Authorization": f"Bearer {os.environ['INFRO_MANAGE_TOKEN']}"}
OUT = pathlib.Path("evidence/2026-Q4")
OUT.mkdir(parents=True, exist_ok=True)

def pull(path, **params):
    rows, cursor = [], None
    while True:
        q = dict(params)
        if cursor:
            q["cursor"] = cursor
        r = requests.get(f"{BASE}/{path}", headers=HEAD, params=q, timeout=30)
        r.raise_for_status()
        body = r.json()
        rows += body["data"]
        if not body.get("has_more"):
            return rows
        cursor = body["next_cursor"]

files = {
    "gateway_keys.json": pull("gateway_keys"),
    "projects.json": pull("projects"),
    "org_settings.json": pull("org_settings"),
    "audit_events.json": pull("audit_events", limit=200,
                              **{"from": "2026-10-01", "to": "2026-12-31"}),
}
manifest = {}
for name, data in files.items():
    blob = json.dumps(data, indent=2, sort_keys=True).encode()
    (OUT / name).write_bytes(blob)
    manifest[name] = hashlib.sha256(blob).hexdigest()
(OUT / "MANIFEST.json").write_text(json.dumps(manifest, indent=2))
```

Add connections, routes and the request CSV the same way. The manifest of hashes lets you show that a file was not edited after the pull. Last quarter's `gateway_keys.json` diffed against this one is the fastest way to list keys created, changed or removed.

## The quarterly runbook

Run these in order. The review is done when every box has a file or a ticket behind it.

- [ ] Confirm the retention window of each source and schedule intermediate exports if the window is shorter than the quarter.
- [ ] Export the member roster with roles; reconcile it with the HR leavers list and the removal events in the audit log.
- [ ] Export gateway keys; check every key has an owner label, an environment, and an expiry or a recorded reason.
- [ ] Export organization, project and key allowlists; flag block-only lists; compute the effective model set per key.
- [ ] Export provider connections and the provider-side member and key lists; reconcile them against each other.
- [ ] Pull the quarter's audit events; match each policy, key and connection change to a ticket.
- [ ] Update the exceptions register; confirm each expired exception has a removal event and re-approve the live ones.
- [ ] Export the request log and usage series with an organization-scoped credential; check for models outside the baseline allowlist.
- [ ] Save one denied-request response (403 `model_not_allowed`) and one revoked-key response (401).
- [ ] Hash the files, write the manifest, and store them where your audit retention policy applies.
- [ ] Record the reviewer, the date and the findings, and open a ticket for each finding.

## What this evidence does not prove

Be explicit about the edges in the review notes, because a reviewer will find them anyway. Gateway evidence covers traffic through the gateway and changes made in it. It does not cover calls that go straight to a provider, changes made in a provider console, or the quality of model answers. It is also not an attestation about the vendor: a vendor's own audit reports are a separate request.

Some features that would shorten this work are on INFRO's roadmap rather than shipped, including a credential inventory screen, a security role and SIEM export. Do not build a control around them yet. For an owner-and-expiry check today, the key export and a label convention is the supported route. If you run INFRO, the console's audit log and the management API read the same events, and developer and analyst roles can read policies without being able to change them, which suits a reviewer account.[5]

## Questions

### Does the gateway audit log show which model each person called?

No. The audit log records administrative actions: key, connection, route, project, policy and membership changes. Inference activity is in the request log, so a review needs both, and they answer different questions.[6]

### How long should we keep the exported evidence?

Set it by your own audit period and retention policy. The gateway docs say retention and access are separate concerns and promise no fixed purge window, so the archive you keep in your own storage is the record you control.[6]

### Do NIST AI RMF or OWASP replace our SOC 2 or ISO control mapping?

No. The NIST framework is voluntary guidance, and the OWASP list is a catalogue of LLM application risks. Use them as vocabulary for risks and tests, and map each evidence file to your own control IDs.[1][2]

## Sources

1. AI Risk Management Framework — NIST. https://nist.gov/itl/ai-risk-management-framework (accessed 2026-11-03)
2. OWASP Top 10 for Large Language Model Applications — OWASP Foundation. https://owasp.org/www-project-top-10-for-large-language-model-applications/ (accessed 2026-11-03)
3. Model allowlists — INFRO Docs. https://infro.io/docs/org/model-allowlists (accessed 2026-11-03)
4. Authentication — INFRO Docs. https://infro.io/docs/authentication (accessed 2026-11-03)
5. Members — INFRO Docs. https://infro.io/docs/org/members (accessed 2026-11-03)
6. Audit log — INFRO Docs. https://infro.io/docs/org/audit-log (accessed 2026-11-03)
7. SSO & SCIM — INFRO Docs. https://infro.io/docs/org/sso (accessed 2026-11-03)
8. Request traces — INFRO Docs. https://infro.io/docs/observability/traces (accessed 2026-11-03)
9. Exports & OpenTelemetry — INFRO Docs. https://infro.io/docs/observability/exports (accessed 2026-11-03)
10. Management API resources — INFRO Docs. https://infro.io/docs/manage/resources (accessed 2026-11-03)