Skip to content

DocsObservability

Datasets

Inputs and expected outputs for experiments, built from logged requests in one click, uploaded as JSONL or CSV, or sent from code.


A dataset is a list of items, each an input (chat messages, in the OpenAI chat shape), an optional expected output and metadata, that experiments replay. Items are content you chose to keep: they are kept until you delete them, whatever your request log keeps, and deleting the dataset, its project or your organization deletes them.

From your traffic

In the console, open a request and choose Add to dataset, or use Add to dataset on Requests and Runs to add the requests the current filters show (up to 1,000). From code, send request_ids. The logged prompt is copied as the input (an Anthropic Messages or OpenAI Responses body is converted to chat messages), and with use_answer_as_expected the logged answer becomes the expected output. Only requests whose prompt was logged can be added: one without answers 400 request_not_logged, and in a list it is returned in not_logged. A request is added to a dataset once; adding it again counts it in duplicates.

Add two logged requests
curl -X POST "https://api.infro.io/v1/datasets/support-golden/items" \
  -H "Authorization: Bearer $INFRO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"request_ids": ["req_01JD…", "req_01JE…"], "use_answer_as_expected": true}'

Upload

JSONL, one item per line: input is {"messages": [...]} or a single user message as text, with optional expected and metadata. CSV needs a header row with an input column; system and expected columns are read, and every other column becomes metadata. A file is at most 5 MB, an item at most 256 KB, and a dataset holds at most 10,000 items; an organization has at most 200 datasets. A problem names the line or row.

Two JSONL items
{"input": {"messages": [{"role": "user", "content": "Where is my order?"}]}, "expected": "It ships tomorrow.", "metadata": {"customer": "c42"}}
{"input": "Can I change the delivery address?", "expected": null}

API

EndpointAccessWhat it does
GET /v1/datasets, GET /v1/datasets/{ref}A key with history accessList and read datasets (a key scoped to a project: that project's).
GET /v1/datasets/{ref}/itemsHistory and payload accessA page of items, oldest first (after, limit).
POST /v1/datasetsA key with history accessCreate a dataset (slug, name, description); a project's key creates it in its project.
POST /v1/datasets/{ref}/itemsHistory access; payload access for request_idsitems or request_ids, up to 1,000 a call.
/manage/v1/datasets…quality:read / quality:writeThe same, plus delete; MCP list_datasets.

Every change bumps the dataset's version; an experiment records the version it ran on.

↑ ↓ navigate↵ open/ anywhere in docs