AI access review checklist: evidence to export for model keys

Scope the review before you collect anything

An AI access review answers four questions, and each one needs a file you can hand over:

1. Who, or which application, can call which models?
2. Who owns each key and each provider credential?
3. What policy changed during the period, who changed it, and why?
4. Which exceptions are still open, who approved them, and when do they expire?

Two public references help you phrase risks and tests. NIST's AI Risk Management Framework is intended for voluntary use, and NIST published a Generative AI Profile (NIST-AI-600-1) on July 26, 2024. The AI RMF 1.0 page also says the framework is being revised, so check the version you cite.¹ OWASP's LLM Top 10 now lives inside the wider OWASP GenAI Security Project, and the project page points to genai.owasp.org for the current list.² Neither is a SOC 2 or ISO 27001 control catalogue. Map each evidence file below to your own control IDs and treat those two as risk vocabulary.

This post is the repeatable part. For the questions a vendor questionnaire asks, see what security reviewers ask about AI gateways; for building the key list itself, see AI API key inventory.

The evidence set at a glance

Each row below is one file with a question it answers, a source and a format an auditor can re-check.

The last row is yours, not the gateway's. A gateway records that an allowlist changed. It cannot know why you approved it.

Who can call which models

Model access is set at three scopes, and the effective answer is the intersection. A model must satisfy every applicable organization, project and key policy. An empty list at one scope adds no restriction and does not override a narrower list elsewhere.³ Entries are catalog model IDs or vendor wildcards such as anthropic/*, and an entry prefixed with ! blocks what it names. A block always wins over an allow in the same list, and a list of only blocks allows every other model, including models added to the catalog later.³

That last behaviour is the one to look for in review. A block-only list is a permissive policy that looks restrictive. Flag any scope whose list has no allow entries and ask whether that was intended.

Export the lists from all three scopes. The model_allowlist field (up to 500 entries) is on organization settings, projects and gateway keys in the management API.³ Then compute the effective set per key yourself and store it beside the raw lists, so the reviewer sees both the inputs and your reading of them.

Configuration is not enforcement, so add a test. Send one request for a model that a key must not reach and save the response. A denied model returns HTTP 403 with error.type: "permission_denied" and error.code: "model_not_allowed", and the message names the list that refused it. A route whose targets are all outside the lists returns 503 no_available_provider instead.³ Keep one of each per quarter. Also note the boundary: environment-scoped connections and key route restrictions apply separately from allowlists, so export those too.³

Key owners and credentials

Applications should hold gateway keys, not provider credentials. A gateway key is created with a label, an optional spend limit, a rate-limit tier, and optionally an environment, a project and an expiry, and provider credentials stay sealed in the gateway.⁴ For the review, that means three checks on the key export:

• Every key has an owner. The label is the only free-text field the docs describe, so put the owning team or person in a label convention (for example payments-prod-<owner>) and check it mechanically.
• Every production key has an expiry or a recorded reason it has none.
• Every key belongs to one environment and, where you use projects, one project.

Revoked and expired keys return 401, a disabled key returns 403 key_disabled, and a call from outside the key's allowed networks returns 403 ip_not_allowed.⁴ Those codes are useful in a test: calling with a key you revoked this quarter and saving the 401 proves the revocation took effect.

Provider-side evidence is outside any gateway. Export the member list and key list from each provider account you connected, and reconcile them against the connections list. A provider key that the gateway does not know about is the finding a reviewer will care about most; see detecting usage that bypassed your gateway.

People: roles, joiners and leavers

The roster matters because roles decide who can change a policy. In INFRO an organization has exactly one owner. Admins manage members, keys and policies, developers create and revoke gateway keys and call models but cannot change policies, members or billing, and analysts read usage, request history, policies and the audit log without creating keys or calling models.⁵ A member limited to some projects cannot read the audit log.⁶

With SSO and SCIM (from the Business plan), the directory creates and deactivates memberships. An SSO assertion can grant admin or developer when a role attribute says so, everyone else joins as analyst, and it can never grant ownership.⁷ Reconcile three lists each quarter: the roster export, your HR leavers list, and the audit events for members removed. A leaver who still appears in the roster, or a removal with no matching HR record, is a finding.

Policy changes and exceptions

The audit log is append-only and records changes to gateway keys, provider connections, routes, projects, webhooks, policies, ownership, SSO and SCIM, plus membership changes. Each event carries actor, actor type, action, target type and ID, metadata, IP where recorded, and a timestamp.⁶ Three limits shape how you use it:

• There is no CSV of the audit log. Read it as JSON from GET /manage/v1/audit_events with a token holding the audit:read scope; it takes q, action, actor, from, to, limit (up to 200) and cursor.⁶
• It can contain personal information, so share it with care.⁶
• No fixed retention is promised, so archive each quarter's pull yourself.⁶

Then do the join that makes it a review rather than a dump: for every policy, key or connection event in the quarter, find the change ticket. Events with no ticket go on the exceptions list or become findings.

An exception, for this review, is anything that departs from the baseline: an allowlist entry added for one project, a key with no expiry, a block removed. Each row in your register needs a requester, an approver, a scope, a start date and an expiry date. At quarter end, every expired exception must have a matching removal in the audit log, and every live exception must be re-approved.

Usage evidence, and how far back it goes

The request log records the model, the provider connection that served it, outcome, timings, usage and estimated provider cost. The docs call it operational evidence, not an invoice.⁸ Two exports cover it: GET /v1/requests/export returns one request per row as CSV, up to 50,000 rows per file, and GET /v1/usage returns aggregates as JSON or CSV.⁹

Check the window before you promise a quarterly cadence. Queryable history is 7 days on Developer, 14 on Startup, 30 on Team, 90 on Business and 365 on Enterprise.⁹ Only a 90-day or longer window covers a calendar quarter in one pull. On shorter windows, schedule exports monthly or weekly. Scheduled and emailed exports are not available, so the schedule is a job you run yourself.⁹

Also check what the key you export with can see. A gateway key's history access is scoped by default (its project, or only its own requests), organization (every request, grantable only by an owner or admin) or none.⁸ Use an organization-scoped credential for the review, or the file will silently cover only one application. Prompt and completion capture is a separate opt-in, so the exports are metadata; review custom tags and user identifiers for personal information before sharing.⁹

Pull it with a script

A read-only management token, scoped to the reads you need, keeps the pull reproducible. The management API serves every resource under /manage/v1/<type>, lists with GET, and pages with {data, has_more, next_cursor}.¹⁰ Confirm the exact resource names and the token header format in the OpenAPI document and the management token docs before you rely on this sketch.

import hashlib, json, os, pathlib, requests

BASE = "https://api.infro.io/manage/v1"
HEAD = {"Authorization": f"Bearer {os.environ['INFRO_MANAGE_TOKEN']}"}
OUT = pathlib.Path("evidence/2026-Q4")
OUT.mkdir(parents=True, exist_ok=True)

def pull(path, **params):
    rows, cursor = [], None
    while True:
        q = dict(params)
        if cursor:
            q["cursor"] = cursor
        r = requests.get(f"{BASE}/{path}", headers=HEAD, params=q, timeout=30)
        r.raise_for_status()
        body = r.json()
        rows += body["data"]
        if not body.get("has_more"):
            return rows
        cursor = body["next_cursor"]

files = {
    "gateway_keys.json": pull("gateway_keys"),
    "projects.json": pull("projects"),
    "org_settings.json": pull("org_settings"),
    "audit_events.json": pull("audit_events", limit=200,
                              **{"from": "2026-10-01", "to": "2026-12-31"}),
}
manifest = {}
for name, data in files.items():
    blob = json.dumps(data, indent=2, sort_keys=True).encode()
    (OUT / name).write_bytes(blob)
    manifest[name] = hashlib.sha256(blob).hexdigest()
(OUT / "MANIFEST.json").write_text(json.dumps(manifest, indent=2))

Add connections, routes and the request CSV the same way. The manifest of hashes lets you show that a file was not edited after the pull. Last quarter's gateway_keys.json diffed against this one is the fastest way to list keys created, changed or removed.

The quarterly runbook

Run these in order. The review is done when every box has a file or a ticket behind it.

• Confirm the retention window of each source and schedule intermediate exports if the window is shorter than the quarter.
• Export the member roster with roles; reconcile it with the HR leavers list and the removal events in the audit log.
• Export gateway keys; check every key has an owner label, an environment, and an expiry or a recorded reason.
• Export organization, project and key allowlists; flag block-only lists; compute the effective model set per key.
• Export provider connections and the provider-side member and key lists; reconcile them against each other.
• Pull the quarter's audit events; match each policy, key and connection change to a ticket.
• Update the exceptions register; confirm each expired exception has a removal event and re-approve the live ones.
• Export the request log and usage series with an organization-scoped credential; check for models outside the baseline allowlist.
• Save one denied-request response (403 model_not_allowed) and one revoked-key response (401).
• Hash the files, write the manifest, and store them where your audit retention policy applies.
• Record the reviewer, the date and the findings, and open a ticket for each finding.

What this evidence does not prove

Be explicit about the edges in the review notes, because a reviewer will find them anyway. Gateway evidence covers traffic through the gateway and changes made in it. It does not cover calls that go straight to a provider, changes made in a provider console, or the quality of model answers. It is also not an attestation about the vendor: a vendor's own audit reports are a separate request.

Some features that would shorten this work are on INFRO's roadmap rather than shipped, including a credential inventory screen, a security role and SIEM export. Do not build a control around them yet. For an owner-and-expiry check today, the key export and a label convention is the supported route. If you run INFRO, the console's audit log and the management API read the same events, and developer and analyst roles can read policies without being able to change them, which suits a reviewer account.⁵