> ## Documentation Index
> Fetch the complete documentation index at: https://docs.archetypeai.app/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Start with /introduction/getting-started. Use the Inference API's Direct Query endpoint (POST /api/inference/query) with the newton-c-2.6 or newton-fusion-1.0 model (text, image, and video reasoning) or the omega-embedding encoder (time-series embeddings). ATAI_API_ENDPOINT is the base URL https://platform.u1.archetypeai.app with no version path; every endpoint path begins with /api. Pages whose descriptions are marked (Archived) document the legacy Lens runtime — do not use them for new projects.

# Batch and Serving Modes

> Run an agent as a batch job that processes files and finishes, or as a serving agent that stays up to answer queries.

Requires [version 2.0.0](https://docs.archetypeai.app/release-notes/2.0.x.md#v2-0-0) or later of the Archetype platform.

Every agent runs in one of two modes:

* **Batch mode** processes a set of input files, writes its output, and finishes.
* **Serving mode** keeps the agent running so it can answer queries as they arrive, one payload at
  a time.

You choose the mode when you start the agent, not when you build it. The same
[package](https://docs.archetypeai.app/api-reference/agents/packages/create-package.md) can run as a batch job one time and as
a serving agent the next. The package fixes how the agent behaves (its blueprint, values, and
models). The mode only decides how the agent gets its input and returns its output.

## How the Modes Differ

| | Batch | Serving |
| - | - | - |
| **Input** | Files you've uploaded, bound in `connectors.source` when you start the run | The body of each query you send |
| **Output** | Files written when the run is done, read with [Get Agent Results](https://docs.archetypeai.app/api-reference/agents/instances/get-agent-results.md) | Each answer comes back in the response to its query |
| **`connectors`** | Required: at least one `file` ref in `source` | Not allowed: sending `connectors` returns `400` |
| **How it ends** | On its own, as `completed` or `failed` | When you cancel it, or when the platform releases it after it sits idle for too long |
| **Talking to it** | Poll the agent's status, then fetch its results | [Query Agent](https://docs.archetypeai.app/api-reference/agents/instances/query-agent.md) for one request and answer |
| **Pause and resume** | Batch runs can only be cancelled; pause and resume both return `501` errors | Not supported: a serving agent returns `409` |
| **Limits** | None specific to the mode | Your organization can run only a limited number of serving agents at a time |

## Choosing a Mode

Use **batch mode** when:

* Your data already exists as files, such as a day of sensor logs or a recorded video.
* You want a complete output file, such as per-window predictions for a whole CSV.
* Nothing is waiting on the answer, so it's fine for the run to take a while.

Use **serving mode** when:

* Data arrives continuously and you need an answer for each piece as it arrives, such as the
  newest readings from a live sensor.
* An application, dashboard, or another service needs to ask the agent questions and get answers
  back in the same request.
* You want to load the agent once and send it many small payloads, rather than starting a new run
  for each one.

## Before You Start

Both modes run a package that's `ready`. To build one, use [Create
Package](https://docs.archetypeai.app/api-reference/agents/packages/create-package.md), then poll [Get
Package](https://docs.archetypeai.app/api-reference/agents/packages/get-package.md) until its `status` is `ready`.

The examples on this page read your API key and the platform URL from [environment
variables](https://docs.archetypeai.app/libraries/environment-variables.md):

> [tab] Standard Account
>
> ```bash theme={"system"}
> export ATAI_API_KEY="your_api_key_here"
> export ATAI_API_ENDPOINT="https://platform.u1.archetypeai.app/api"
> export PACKAGE_ID="pkg_01jcb2x7m4qv8n3h5t6rk9dzs0"
> ```

> [tab] Free Trial Account
>
> ```bash theme={"system"}
> export ATAI_API_KEY="your_api_key_here"
> export ATAI_API_ENDPOINT="https://try.archetypeai.app/api"
> export PACKAGE_ID="pkg_01jcb2x7m4qv8n3h5t6rk9dzs0"
> ```

## Running an Agent in Batch Mode

### 1. Start the Run

Call [Create Agent](https://docs.archetypeai.app/api-reference/agents/instances/create-agent.md) with the package's ID and the
files to process. You can leave `mode` out, since `batch` is the default. Each entry in
`connectors.source` is a file you've already uploaded with the [Files
API](https://docs.archetypeai.app/core-concepts/files/files.md).

```bash cURL theme={"system"}
curl -X POST "$ATAI_API_ENDPOINT/agents/instances" \
  -H "Authorization: Bearer $ATAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "package_id": "'"$PACKAGE_ID"'",
    "mode": "batch",
    "connectors": {
      "source": [
        {"type": "file", "id": "file_abc123", "format": "csv"}
      ]
    }
  }'
```

```python Python theme={"system"}
import os

import requests

endpoint = os.environ["ATAI_API_ENDPOINT"]
headers = {"Authorization": f"Bearer {os.environ['ATAI_API_KEY']}"}

response = requests.post(
    f"{endpoint}/agents/instances",
    headers=headers,
    json={
        "package_id": os.environ["PACKAGE_ID"],
        "mode": "batch",
        "connectors": {
            "source": [{"type": "file", "id": "file_abc123", "format": "csv"}],
        },
    },
    timeout=60,
)
response.raise_for_status()

agent = response.json()
agent_id = agent["id"]
print(f"Started {agent_id} (status {agent['status']})")
```

The response is `202 Accepted` with the new agent, whose `status` is `running`. Save its `id`
(an `agt_` ID) for the next steps.

To process several files in one run, list them all in `source`. By default the agent writes one
output per input. To fold all output into a single file instead (for formats that can be
concatenated, such as CSV), name it in `connectors.sink`:

```json theme={"system"}
"connectors": {
  "source": [
    {"type": "file", "id": "file_abc123", "format": "csv"},
    {"type": "file", "id": "file_def456", "format": "csv"}
  ],
  "sink": {"type": "file", "id": "file_ghi789"}
}
```

### 2. Wait for the Run to Finish

Poll [Get Agent](https://docs.archetypeai.app/api-reference/agents/instances/get-agent.md) until `status` reaches `completed`,
`failed`, or `cancelled`. If the run failed, `error` says why.

```bash cURL theme={"system"}
curl "$ATAI_API_ENDPOINT/agents/instances/$AGENT_ID" \
  -H "Authorization: Bearer $ATAI_API_KEY"
```

```python Python theme={"system"}
import time

while True:
    response = requests.get(
        f"{endpoint}/agents/instances/{agent_id}", headers=headers, timeout=60
    )
    response.raise_for_status()
    agent = response.json()

    if agent["status"] in ("completed", "failed", "cancelled"):
        break
    time.sleep(10)

if agent["status"] != "completed":
    raise RuntimeError(f"Run ended {agent['status']}: {agent.get('error')}")
```

To follow progress while the run is going, read its
[events](https://docs.archetypeai.app/api-reference/agents/instances/list-agent-events.md) and
[logs](https://docs.archetypeai.app/api-reference/agents/instances/get-agent-logs.md).

### 3. Get the Results

Once the run is `completed`, page through its output files with [Get Agent
Results](https://docs.archetypeai.app/api-reference/agents/instances/get-agent-results.md). Each entry's `data.ref` points at a
stored output file.

```bash cURL theme={"system"}
curl "$ATAI_API_ENDPOINT/agents/instances/$AGENT_ID/results?limit=100" \
  -H "Authorization: Bearer $ATAI_API_KEY"
```

```python Python theme={"system"}
results_url = f"{endpoint}/agents/instances/{agent_id}/results"
cursor = None

while True:
    params = {"limit": 100}
    if cursor:
        params["after"] = cursor

    response = requests.get(results_url, headers=headers, params=params, timeout=60)
    response.raise_for_status()
    page = response.json()

    for result in page["data"]:
        print(f"{result['port_name']}: {result['data']['filename']} -> {result['data']['ref']}")

    if not page["has_more"]:
        break
    cursor = page["next_cursor"]
```

A batch run doesn't run again. To process more files, start a new run of the same package.

## Running an Agent in Serving Mode

### 1. Start the Agent

Call [Create Agent](https://docs.archetypeai.app/api-reference/agents/instances/create-agent.md) with `mode` set to `serving`.
Don't send `connectors`: a serving agent gets its input from each query, so a request that binds
files is rejected with `400`.

```bash cURL theme={"system"}
curl -X POST "$ATAI_API_ENDPOINT/agents/instances" \
  -H "Authorization: Bearer $ATAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "package_id": "'"$PACKAGE_ID"'",
    "mode": "serving"
  }'
```

```python Python theme={"system"}
import os

import requests

endpoint = os.environ["ATAI_API_ENDPOINT"]
headers = {"Authorization": f"Bearer {os.environ['ATAI_API_KEY']}"}

response = requests.post(
    f"{endpoint}/agents/instances",
    headers=headers,
    json={"package_id": os.environ["PACKAGE_ID"], "mode": "serving"},
    timeout=60,
)
response.raise_for_status()

agent = response.json()
agent_id = agent["id"]
print(f"Started {agent_id} (status {agent['status']})")
```

The response is `202 Accepted` with the new agent, whose `status` is `deploying`.

If the request returns `429` with code `serving_limit_reached`, your organization is already
running as many serving agents as it's allowed to. Cancel one you no longer need, then try again.
Not every blueprint can be served; one that can't returns `400` and no agent is created.

### 2. Wait for the Agent to Be Running

A serving agent can't answer queries until it has been placed on a worker. Poll [Get
Agent](https://docs.archetypeai.app/api-reference/agents/instances/get-agent.md) until `status` is `running`. If it ends up
`failed` instead, the agent couldn't be placed or started, and `error` says why; create another
agent to try again.

```bash cURL theme={"system"}
curl "$ATAI_API_ENDPOINT/agents/instances/$AGENT_ID" \
  -H "Authorization: Bearer $ATAI_API_KEY"
```

```python Python theme={"system"}
import time

while True:
    response = requests.get(
        f"{endpoint}/agents/instances/{agent_id}", headers=headers, timeout=60
    )
    response.raise_for_status()
    agent = response.json()

    if agent["status"] == "running":
        break
    if agent["status"] in ("completed", "failed", "cancelled"):
        raise RuntimeError(f"Agent ended {agent['status']}: {agent.get('error')}")
    time.sleep(5)
```

### 3. Send Queries

Send each payload to [Query Agent](https://docs.archetypeai.app/api-reference/agents/instances/query-agent.md). The request body
is whatever the blueprint's source connector accepts, and the agent's answer comes back in the
`response` field, shaped by the blueprint's sink connector. The `records` body below is only an
example of what a time-series blueprint might accept.

```bash cURL theme={"system"}
curl -X POST "$ATAI_API_ENDPOINT/agents/instances/$AGENT_ID/query" \
  -H "Authorization: Bearer $ATAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "records": [
      {"timestamp": "2026-09-18T11:24:03Z", "vibration": 0.41, "flow": 12.6}
    ]
  }'
```

```python Python theme={"system"}
def query_agent(payload, max_retries=3):
    """Send one payload to the serving agent, retrying when it's busy."""
    url = f"{endpoint}/agents/instances/{agent_id}/query"

    for attempt in range(max_retries + 1):
        response = requests.post(url, headers=headers, json=payload, timeout=120)

        # 429 (agent busy) and 503 (unavailable or at capacity) are worth retrying.
        if response.status_code in (429, 503) and attempt < max_retries:
            delay = int(response.headers.get("Retry-After", 2 ** attempt))
            time.sleep(delay)
            continue

        response.raise_for_status()
        return response.json()


answer = query_agent({
    "records": [
        {"timestamp": "2026-09-18T11:24:03Z", "vibration": 0.41, "flow": 12.6}
    ]
})
print(f"Answered in {answer['query_response_time_ms']} ms: {answer['response']}")
```

A successful query returns `200`:

```json theme={"system"}
{
  "status": "completed",
  "response": {
    "predicted_state": "running",
    "p_running": 0.94,
    "p_idle": 0.06
  },
  "query_response_time_ms": 38.4
}
```

Some errors are worth retrying and some aren't:

| Status | Meaning | What to do |
| - | - | - |
| `400` | The agent rejected the payload. | Fix the payload; resending it unchanged fails again. |
| `409` | The agent isn't a running serving agent. It was started in batch mode, or it has stopped. | Start a new serving agent. |
| `429` | The agent is already answering as many queries as it can at once. | Retry after the `Retry-After` delay. |
| `503` | The agent couldn't be reached, or the node is at capacity. | Retry with backoff, honoring `Retry-After` when it's sent. |
| `504` | The agent ran out of time before it had an answer. | Send less data per query, or fewer queries at once. |

### 4. Open a Session for Many Queries (Optional)

If you're sending a steady stream of payloads, [Connect to
Agent](https://docs.archetypeai.app/api-reference/agents/instances/connect-agent.md) opens one websocket session instead of
making a request for each payload. Each JSON text frame you send gets one answer frame back.
Every frame from the server has a `status` field: `connected` when the session opens,
`completed` for an answer, and `failed` for a frame the server couldn't act on.

cURL can't hold a websocket session open, so this example is in Python only. It uses the
[`websocket-client`](https://pypi.org/project/websocket-client/) package (`pip install
websocket-client`).

```python Python theme={"system"}
import json
import os

import websocket

endpoint = os.environ["ATAI_API_ENDPOINT"].replace("https://", "wss://", 1)

socket = websocket.create_connection(
    f"{endpoint}/agents/instances/{agent_id}/connect",
    header={"Authorization": f"Bearer {os.environ['ATAI_API_KEY']}"},
)

opened = json.loads(socket.recv())
print(f"{opened['status']}: {opened['agent_id']}")

readings = [
    {"timestamp": "2026-09-18T11:24:03Z", "vibration": 0.41, "flow": 12.6},
    {"timestamp": "2026-09-18T11:24:04Z", "vibration": 0.44, "flow": 12.4},
]

for reading in readings:
    socket.send(json.dumps({"records": [reading]}))
    frame = json.loads(socket.recv())

    if frame["status"] == "completed":
        print(frame["response"])
    else:
        print(f"Error: {frame['error']}")

socket.close()
```

A `failed` frame in answer to a bad payload doesn't end the session, so you can keep sending.
If the agent stops serving, the server sends a `failed` frame and then closes the session.

### 5. Stop the Agent

A serving agent keeps running, and counts against your organization's serving limit, until you
stop it. When you're done with it, cancel it with [Cancel
Agent](https://docs.archetypeai.app/api-reference/agents/instances/cancel-agent.md). Serving agents can't be paused, so cancel
is the only way to stop one yourself.

```bash cURL theme={"system"}
curl -X POST "$ATAI_API_ENDPOINT/agents/instances/$AGENT_ID/cancel" \
  -H "Authorization: Bearer $ATAI_API_KEY"
```

```python Python theme={"system"}
response = requests.post(
    f"{endpoint}/agents/instances/{agent_id}/cancel", headers=headers, timeout=60
)
response.raise_for_status()
print(f"Agent is now {response.json()['status']}")
```

The platform also releases a serving agent that goes without a query for longer than the
deployment's idle window. A released agent ends `completed`, and queries to it return `409` with a
message that starts `released after`. Released agents don't come back; start a new serving agent
from the same package to continue.

## Using the Python Client

The [Python client](https://docs.archetypeai.app/libraries/python.md#build-and-run-a-package) wraps these steps. It waits for a
batch run to finish with `wait_until_done`, waits for a serving agent to start with
`wait_until_running`, and retries `429` and `503` responses to `query` for you:

```python Python theme={"system"}
from archetypeai import ArchetypeAI

client = ArchetypeAI(api_key)

# Batch mode: process files and finish.
agent = client.agents.instances.create(package_id, source=["file_abc123"])
agent = client.agents.instances.wait_until_done(agent["id"])

# Serving mode: stay up and answer queries.
serving = client.agents.instances.create(package_id, mode="serving")
serving = client.agents.instances.wait_until_running(serving["id"])
answer = client.agents.instances.query(serving["id"], {"records": [...]})
```

> [!NOTE]
>
> `wait_until_done` raises a `ValueError` for a serving agent, since a serving agent never
> finishes on its own.

## Related Content

* [Using the Agents API](https://docs.archetypeai.app/core-concepts/agents/api.md)
* [Create Agent](https://docs.archetypeai.app/api-reference/agents/instances/create-agent.md)
* [Query Agent](https://docs.archetypeai.app/api-reference/agents/instances/query-agent.md)
* [Connect to Agent](https://docs.archetypeai.app/api-reference/agents/instances/connect-agent.md)
* [Python client](https://docs.archetypeai.app/libraries/python.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.