Detect AI API usage that bypassed your gateway

How do you detect AI API usage that bypassed your gateway? Read the provider's own usage report grouped by API key, read what your gateway sent on each key, and compare them per UTC day. OpenAI and Anthropic both report token usage per API key through their admin APIs. Tokens on a key your gateway uses that your gateway never sent mean the key is being used somewhere else. Tokens on a key your gateway does not know mean another workload, shadow usage, or a leaked key nobody has told you about.

This works because of a basic asymmetry. A gateway can only count the requests it served; anything sent straight to the provider is invisible to it, however good its budgets and alerts. The provider sees every request on your account. Read the provider's report and the gaps become visible.

What bypass looks like

Traffic reaches your provider account without passing your gateway for ordinary reasons and for alarming ones:

• A copy of a gateway's provider key in a notebook, a script, a .env file on a laptop or a CI variable, used for "just one test".
• A team going direct because the gateway lacks a model or a parameter they need.
• A third-party tool holding its own key on your account: a coding agent, an evaluation service, a BYOK platform.
• Console and playground use by people with access to the provider console.
• A stolen key. Someone who copied a key from a repository, a log or a breached tool calls the provider from their own infrastructure. This is the case that turns into an invoice.

The first four need an owner and a policy; finding shadow AI API usage covers that side. The last one needs revocation today. The comparison below finds all five; telling them apart is the work after it.

What you need before you start

1. An admin credential per provider, held by the one system that runs the check. OpenAI's usage endpoints need an admin key¹. Anthropic's need an Admin API key or another admin credential; workspace API keys do not work². These are powerful keys; least privilege for provider admin keys covers how to hold them.
2. The provider's ID for every key your gateway uses. OpenAI identifies keys as key_… and Anthropic as apikey_… in usage reports. Record the ID when you create a key, in your credential inventory. Without that mapping you can see usage per key but cannot say which keys are yours.
3. Your gateway's tokens per provider key per UTC day. Most gateways record which connection served each request and the token counts the provider returned. Export them grouped by connection and day.

Reading OpenAI usage per key

The completions usage endpoint is GET https://api.openai.com/v1/organization/usage/completions. It takes start_time (Unix seconds, inclusive), an optional end_time (exclusive), bucket_width of 1m, 1h or 1d, and group_by with any combination of project_id, user_id, api_key_id, model, batch, service_tier and api_source. api_key_ids filters to particular keys. With bucket_width=1d, limit defaults to 7 buckets and goes up to 31; pages continue through next_page, passed back as page³.

Each result carries input_tokens (including cached and cache-write tokens), output_tokens, num_model_requests, and, when grouped by it, api_key_id³. Without a group_by, fields such as project_id and model come back as null¹.

START=$(date -u -d "yesterday 00:00" +%s)
END=$(date -u -d "today 00:00" +%s)
curl -s "https://api.openai.com/v1/organization/usage/completions?start_time=$START&end_time=$END&bucket_width=1d&group_by=api_key_id&group_by=project_id" \
  -H "Authorization: Bearer $OPENAI_ADMIN_KEY" |
  jq -r '.data[].results[] | [.api_key_id, .project_id, .input_tokens, .output_tokens, .num_model_requests] | @tsv'

To name the keys, list each project's keys with GET /v1/organization/projects/{project_id}/api_keys. Each key has id, name, created_at, last_used_at and its owner. Pass owner_project_access=any: without it, the endpoint applies membership-based visibility rules "which may exclude some enabled keys"⁴.

The completions endpoint covers completions traffic only. OpenAI exposes separate usage endpoints for other kinds of traffic, such as embeddings and images, in the same reference³; run the same check on each kind you use.

Reading Anthropic usage per key

Anthropic's endpoint is GET https://api.anthropic.com/v1/organizations/usage_report/messages. It takes starting_at and ending_at (RFC 3339), bucket_width of 1d, 1h or 1m, and group_by[] with any of api_key_id, workspace_id, account_id, service_account_id, model, service_tier, context_window, inference_geo and speed. Daily buckets default to 7 and go up to 31; pages continue while has_more is true⁵.

Each result carries uncached_input_tokens, cache_read_input_tokens, cache_creation (split into ephemeral_5m_input_tokens and ephemeral_1h_input_tokens) and output_tokens, plus api_key_id when grouped by it⁵. Three details matter for this check:

• Usage from the Console playground "is not associated with an API key, so api_key_id will be null" even when grouping by key².
• account_id is set for requests made with OAuth sign-in rather than an API key; group by it too, or that usage appears unattributed⁵.
• Usage typically appears within five minutes of a request completing².

START=$(date -u -d "yesterday 00:00" +%Y-%m-%dT%H:%M:%SZ)
END=$(date -u -d "today 00:00" +%Y-%m-%dT%H:%M:%SZ)
curl -s -G "https://api.anthropic.com/v1/organizations/usage_report/messages" \
  --data-urlencode "starting_at=$START" --data-urlencode "ending_at=$END" \
  --data-urlencode "bucket_width=1d" \
  --data-urlencode "group_by[]=api_key_id" --data-urlencode "group_by[]=workspace_id" \
  -H "anthropic-version: 2023-06-01" -H "x-api-key: $ANTHROPIC_ADMIN_KEY" |
  jq -r '.data[].results[] | [.api_key_id, .workspace_id,
    (.uncached_input_tokens + .cache_read_input_tokens
     + .cache_creation.ephemeral_5m_input_tokens + .cache_creation.ephemeral_1h_input_tokens),
    .output_tokens] | @tsv'

To name the keys, GET /v1/organizations/api_keys returns each key's id, name, status, created_at, created_by, expires_at, partial_key_hint and scope⁶. It has no last-used field, so the usage report grouped by key is also how you learn when an Anthropic key was last used.

A comparison script

The script below reads yesterday and the six days before it from both providers, reads your gateway's export, and classifies every key. It expects a CSV from your gateway with the columns provider,key_id,day,tokens, where key_id is the provider's ID for the key that served the request and tokens is input plus output tokens as the provider returned them.

import csv, datetime as dt, os, sys
from collections import defaultdict
import requests

UTC = dt.timezone.utc
today = dt.datetime.now(UTC).date()
start = today - dt.timedelta(days=7)           # seven complete days, today excluded

def openai_usage():
    url = "https://api.openai.com/v1/organization/usage/completions"
    headers = {"Authorization": f"Bearer {os.environ['OPENAI_ADMIN_KEY']}"}
    params = {"start_time": int(dt.datetime.combine(start, dt.time(), UTC).timestamp()),
              "end_time": int(dt.datetime.combine(today, dt.time(), UTC).timestamp()),
              "bucket_width": "1d", "group_by": ["api_key_id"], "limit": 31}
    out = defaultdict(int)
    while True:
        body = requests.get(url, headers=headers, params=params, timeout=30).json()
        for bucket in body.get("data", []):
            day = dt.datetime.fromtimestamp(bucket["start_time"], UTC).date().isoformat()
            for r in bucket.get("results", []):
                out[("openai", r.get("api_key_id"), day)] += r["input_tokens"] + r["output_tokens"]
        if not body.get("next_page"):
            return out
        params["page"] = body["next_page"]

def anthropic_usage():
    url = "https://api.anthropic.com/v1/organizations/usage_report/messages"
    headers = {"x-api-key": os.environ["ANTHROPIC_ADMIN_KEY"], "anthropic-version": "2023-06-01"}
    params = {"starting_at": f"{start}T00:00:00Z", "ending_at": f"{today}T00:00:00Z",
              "bucket_width": "1d", "group_by[]": ["api_key_id"], "limit": 31}
    out = defaultdict(int)
    while True:
        body = requests.get(url, headers=headers, params=params, timeout=30).json()
        for bucket in body.get("data", []):
            day = bucket["starting_at"][:10]
            for r in bucket.get("results", []):
                cc = r.get("cache_creation") or {}
                tokens = (r["uncached_input_tokens"] + r["cache_read_input_tokens"]
                          + cc.get("ephemeral_5m_input_tokens", 0) + cc.get("ephemeral_1h_input_tokens", 0)
                          + r["output_tokens"])
                out[("anthropic", r.get("api_key_id"), day)] += tokens
        if not body.get("has_more"):
            return out
        params["page"] = body["next_page"]

def gateway_usage(path):
    out = defaultdict(int)
    with open(path, newline="") as f:
        for row in csv.DictReader(f):
            out[(row["provider"], row["key_id"], row["day"])] += int(row["tokens"])
    return out

reported = {**openai_usage(), **anthropic_usage()}
sent = gateway_usage(sys.argv[1])
gateway_keys = {(p, k) for (p, k, _) in sent}

for (provider, key, day), tokens in sorted(reported.items(), key=lambda x: (x[0][0], str(x[0][1]), x[0][2])):
    if key is None:
        print(f"{day} {provider} UNATTRIBUTED  {tokens:>12,} tokens (console, playground or OAuth use)")
    elif (provider, key) not in gateway_keys:
        print(f"{day} {provider} UNKNOWN KEY   {key} {tokens:>12,} tokens: not a gateway key, find its owner")
    else:
        through = sent.get((provider, key, day), 0)
        extra = tokens - through
        if extra > max(10_000, 0.02 * tokens):  # illustrative tolerance
            print(f"{day} {provider} BYPASS        {key} {extra:>12,} tokens not sent by the gateway")

Run it daily with python bypass_check.py gateway_usage.csv. Every line it prints is a key that needs an explanation. The tolerance is illustrative: tune it to your traffic, and expect small gaps from requests that failed after the provider counted tokens, or from clocks that put a request on different sides of midnight.

Reading the result

Two habits make the findings faster to act on. Put each provider key in its own project or workspace, so a finding already names the workload. And when a bypass finding appears, switch to hourly buckets for that key: the hour the extra traffic started usually points at a deploy, a person or a leak.

Limits of this check

• It is after the fact. The fastest it can be is the provider's reporting delay; for Anthropic that is typically five minutes². It tells you a key is being misused; it does not stop it. Provider-side spend limits and revocation do that; see how to limit the damage from a stolen key.
• It covers what the endpoints cover. OpenAI splits usage by kind of traffic across several endpoints³, and Anthropic says code execution appears in its cost endpoint but not in the usage endpoint². Check each kind of traffic you use.
• The admin credential is itself a target. Keep it in one system, used read-only, and in your inventory.
• Other providers are not covered here. Google, Azure and AWS report usage through their own billing and monitoring tools. The same idea applies: find the most specific identity each one reports usage under, and compare it with what your gateway sent under that identity.

For the daily comparison of cost rather than tokens, see reconciling estimated LLM spend with provider invoices.

Bypass detection in INFRO

On the Business plan, INFRO's cost reconciliation reads OpenAI's and Anthropic's cost reports once a day with a separate billing credential. With usage access switched on for that credential (off by default), INFRO also reads token counts per day grouped by provider API key and model, matches the provider's key identifiers against the keys on your INFRO connections, and raises a finding when the provider reports more tokens than INFRO sent, listing the keys in the evidence. A key it cannot match is shown as unmatched. It reads no prompts or completions. Without usage access, bypass detection reports not checked, never "no bypass".

This check exists because INFRO's own controls, like any gateway's, govern only traffic that goes through INFRO. A provider key stolen from anywhere, including from a platform that stores customers' provider keys as INFRO does, can be used directly against the provider. Usage monitoring at the provider is how you find that out, and the provider's spend limits and revocation are how you stop it.