DocsObservability
Datasets
Inputs and expected outputs for experiments, built from logged requests in one click, uploaded as JSONL or CSV, or sent from code.
A dataset is a list of items, each an input (chat messages, in the OpenAI chat shape), an optional expected output and metadata, that experiments replay. Items are content you chose to keep: they are kept until you delete them, whatever your request log keeps, and deleting the dataset, its project or your organization deletes them.
From your traffic
In the console, open a request and choose Add to dataset, or use Add to dataset on Requests and Runs to add the requests the current filters show (up to 1,000). From code, send request_ids. The logged prompt is copied as the input (an Anthropic Messages or OpenAI Responses body is converted to chat messages), and with use_answer_as_expected the logged answer becomes the expected output. Only requests whose prompt was logged can be added: one without answers 400 request_not_logged, and in a list it is returned in not_logged. A request is added to a dataset once; adding it again counts it in duplicates.
curl -X POST "https://api.infro.io/v1/datasets/support-golden/items" \
-H "Authorization: Bearer $INFRO_API_KEY" \
-H "Content-Type: application/json" \
-d '{"request_ids": ["req_01JD…", "req_01JE…"], "use_answer_as_expected": true}'Upload
JSONL, one item per line: input is {"messages": [...]} or a single user message as text, with optional expected and metadata. CSV needs a header row with an input column; system and expected columns are read, and every other column becomes metadata. A file is at most 5 MB, an item at most 256 KB, and a dataset holds at most 10,000 items; an organization has at most 200 datasets. A problem names the line or row.
{"input": {"messages": [{"role": "user", "content": "Where is my order?"}]}, "expected": "It ships tomorrow.", "metadata": {"customer": "c42"}}
{"input": "Can I change the delivery address?", "expected": null}API
| Endpoint | Access | What it does |
|---|---|---|
GET /v1/datasets, GET /v1/datasets/{ref} | A key with history access | List and read datasets (a key scoped to a project: that project's). |
GET /v1/datasets/{ref}/items | History and payload access | A page of items, oldest first (after, limit). |
POST /v1/datasets | A key with history access | Create a dataset (slug, name, description); a project's key creates it in its project. |
POST /v1/datasets/{ref}/items | History access; payload access for request_ids | items or request_ids, up to 1,000 a call. |
/manage/v1/datasets… | quality:read / quality:write | The same, plus delete; MCP list_datasets. |
Every change bumps the dataset's version; an experiment records the version it ran on.