What the Act asks for, and what it leaves open
Two provisions carry most of the logging weight. Article 12 requires high-risk systems to be capable of automatic event logging that supports risk identification, post-market monitoring and the deployer's own monitoring. Article 26 then requires deployers of high-risk systems to keep the logs under their control for at least six months, unless other law such as data protection law says otherwise.1 The stand-alone (Annex III) high-risk obligations now apply from 2 December 2027, after the Digital Omnibus on AI moved them; product-embedded (Annex I) systems follow on 2 August 2028.1
Notice what the Act does not do: it does not publish a field list for an LLM gateway. It states purposes (find risks, monitor after market, let the deployer monitor) and leaves the schema to you. "Counts as evidence" therefore means something practical: a record that a reviewer can tie to a specific system, a specific time and a specific outcome, and that you can show was not edited afterwards.
A gateway log is not the system's log
A high-risk AI system is the whole thing that does the Annex III task: the application, its model calls, its pre- and post-processing, and the humans in the loop. A gateway sees only the model calls that pass through it. That makes its log a strong record of one layer and silent about the rest.
The practical split:
- Gateway can record: that a call happened, when, from which key and project, to which model on which connection, how it ended, how many tokens or units it used, and which policy acted on it.
- Application must record: which business decision the output fed into, what the output was turned into, and whether a person reviewed or overrode it.
- Provider of the system must supply: the capability to log, the instructions for use, and the technical documentation the deployer relies on.
- Model provider holds: its own side of the call, under your agreement with it. Turning off INFRO's content capture does not turn off storage at your AI provider.2
If you are the provider of the high-risk system as well as its deployer, you hold both the capability duty and the retention duty. Many platform teams do; the earlier overview explains how to find out which role you hold per system.1
Field-by-field: what a gateway can capture
INFRO's request log records the model, the selected provider connection, the outcome, timings, usage and estimated provider cost, and the docs call it operational evidence, not an invoice.3 Request records also carry fallback attempts and latency.3 The table maps those fields to what a reviewer would ask.
| Evidence question | Gateway field | Limit |
|---|---|---|
| When did the event occur, and how long did it take? | Request timestamps and timings | Gateway clock only, not the application's |
| Which model answered? | Model and selected connection | A retired or aliased name still needs your pin policy |
| Did a fallback change the answer path? | Fallback attempts on the request | Records the path, not whether quality differed |
| Who or what called? | Gateway key, project, environment, an end-user id you attach | Only as good as the id your app sends |
| Did a policy act? | Guardrail action and per-detector counts on the span; policy error for non-allowlisted models | Counts only, never the matched text4 |
| What was sent and returned? | Optional prompt and response capture | Off by default; plan-limited retention2 |
| Who changed the controls? | Audit log: actor, action, target, IP where recorded, timestamp5 | Administrative events only |
| Was the output reviewed by a person? | Nothing | Application or case-management system |
Two details matter for evidence quality. First, history access is per key: a key can be scoped, organization or none, and a request outside a key's scope answers exactly like an ID that does not exist.3 Give the archiving job an organization-scope key granted by an owner or admin, or it will quietly archive only part of the record. Second, captured content needs its own payload access and is never returned after its retention deadline, even if deletion has not yet run.3
The retention gap: six months against plan windows
Queryable request history by plan is 7 days on Developer, 14 on Startup, 30 on Team, 90 on Business and 365 on Enterprise.4 Optional payload retention is capped by plan too; on Business it is up to 90 days, and Enterprise follows the agreement.2 Six months is roughly 180 days. Only the 365-day window covers it, and the docs state that the request-history window is about access through the APIs and exports, not a promise to erase all underlying metering records on that date.2
So on any plan below Enterprise, you cannot rely on the gateway as the only store for a six-month obligation. Scheduled or emailed exports are not currently available, and there is no export job endpoint.4 You pull the data yourself. GET /v1/requests/export returns one request per row as CSV, up to 50,000 rows per file, and GET /v1/requests pages by cursor.4
OpenTelemetry export is not a substitute. Spans carry request ID, model, route, timing, token counts, estimated cost and configured request metadata, but never prompt or response captures. Delivery is best effort with one retry, persistent failure drops spans, and the docs advise against using the channel as the sole accounting record.4 Use it for live monitoring and join it to the archived request records by request ID.
Making the archive hold up as evidence
A log is better evidence when it is complete, attributable and hard to alter silently. A workable pattern:
- Issue one gateway key per AI system and attach project and environment, so each record maps to an inventory entry.
- Create an archive key with organization history access, and run a job that pages the request log on a fixed lag.
- Write each page to storage you control with object lock or an append-only policy, plus a manifest of window bounds and record counts.
- Record the retention window the API reports with each pull, so a gap is visible rather than assumed away.
- Archive the audit log separately through the management API with a token holding
audit:read; it is not available as a CSV.5
import json, os, requests
URL = "https://api.infro.io/v1/requests"
HEADERS = {"Authorization": f"Bearer {os.environ['INFRO_ARCHIVE_KEY']}"}
def archive(window_from: str, window_to: str, out_path: str) -> int:
"""Page the request log for a fixed window and append JSON lines."""
params = {"limit": 100, "from": window_from, "to": window_to}
count, cursor = 0, None
with open(out_path, "a", encoding="utf-8") as out:
while True:
if cursor:
params["after"] = cursor
page = requests.get(URL, headers=HEADERS, params=params, timeout=30).json()
for row in page["data"]:
out.write(json.dumps(row, sort_keys=True) + "\n")
count += 1
if not page.get("has_more"):
# keep window_start / retention_days from the last page in your manifest
return count
cursor = page["next_cursor"]This is a sketch: confirm the accepted timestamp format and filters on the traces page, and keep filters and time range stable during pagination as the docs require.4
What stays with the deployer and provider
Even a complete archive leaves gaps that no gateway can fill, because the information never crosses it.
- Oversight records. Article 26 asks deployers to assign oversight to competent people and to report risks and serious incidents.1 Who reviewed which output, and what they decided, belongs in your case or ticket system, joined to the request ID.
- Application decisions. The gateway sees the model's answer, not whether your code approved a loan, ranked a candidate or discarded the output.
- System version. Which build of your application, prompts and retrieval index produced a request is your release record, not a gateway field. Stamp a version in request metadata so the join exists.
- Provider-side records and documentation. Instructions for use and technical documentation come from the provider of the system; model-side processing sits under your provider agreement.
- Location of processing. INFRO does not currently provide guaranteed EU-only processing or data residency, and the docs tell you to review the location and transfer terms of every fallback target.6
- Certification. INFRO holds no SOC 2 or ISO 27001 certification, and the docs state that using INFRO does not by itself establish GDPR or AI Act compliance.6
For the gateway layer specifically, the controls that generate evidence rather than just records are the model allowlist, key scoping and the audit trail of who changed them. Request-level traces carry the model, the connection that served it and the route, grouped by users and sessions.7 Together those let you show that only approved models were callable and that the approval list did not change unnoticed.
Checklist before December 2027
- Each AI system has an inventory entry and its own gateway key and project.
- Classification and role (provider, deployer, both) recorded per system, confirmed with counsel.
- Archive job runs on a fixed lag, with an organization-scope key, into locked storage.
- Retention target set per system and compared with the plan's history window.
- Content capture decided per system, documented, with its own retention.
- Application version and end-user id sent with every request.
- Oversight reviews stored with request IDs.
- Audit log archived on its own schedule.
- Article 12 and Article 26 text re-read in the consolidated source on the day the design is frozen.
