Skip to main content
Requires version 2.0.0 or later of the Archetype platform.
Every agent runs in one of two modes:
  • Batch mode processes a set of input files, writes its output, and finishes.
  • Serving mode keeps the agent running so it can answer queries as they arrive, one payload at a time.
You choose the mode when you start the agent, not when you build it. The same package can run as a batch job one time and as a serving agent the next. The package fixes how the agent behaves (its blueprint, values, and models). The mode only decides how the agent gets its input and returns its output.

How the Modes Differ

Choosing a Mode

Use batch mode when:
  • Your data already exists as files, such as a day of sensor logs or a recorded video.
  • You want a complete output file, such as per-window predictions for a whole CSV.
  • Nothing is waiting on the answer, so it’s fine for the run to take a while.
Use serving mode when:
  • Data arrives continuously and you need an answer for each piece as it arrives, such as the newest readings from a live sensor.
  • An application, dashboard, or another service needs to ask the agent questions and get answers back in the same request.
  • You want to load the agent once and send it many small payloads, rather than starting a new run for each one.

Before You Start

Both modes run a package that’s ready. To build one, use Create Package, then poll Get Package until its status is ready. The examples on this page read your API key and the platform URL from environment variables:

Running an Agent in Batch Mode

1. Start the Run

Call Create Agent with the package’s ID and the files to process. You can leave mode out, since batch is the default. Each entry in connectors.source is a file you’ve already uploaded with the Files API.
The response is 202 Accepted with the new agent, whose status is running. Save its id (an agt_ ID) for the next steps. To process several files in one run, list them all in source. By default the agent writes one output per input. To fold all output into a single file instead (for formats that can be concatenated, such as CSV), name it in connectors.sink:

2. Wait for the Run to Finish

Poll Get Agent until status reaches completed, failed, or cancelled. If the run failed, error says why.
To follow progress while the run is going, read its events and logs.

3. Get the Results

Once the run is completed, page through its output files with Get Agent Results. Each entry’s data.ref points at a stored output file.
A batch run doesn’t run again. To process more files, start a new run of the same package.

Running an Agent in Serving Mode

1. Start the Agent

Call Create Agent with mode set to serving. Don’t send connectors: a serving agent gets its input from each query, so a request that binds files is rejected with 400.
The response is 202 Accepted with the new agent, whose status is deploying. If the request returns 429 with code serving_limit_reached, your organization is already running as many serving agents as it’s allowed to. Cancel one you no longer need, then try again. Not every blueprint can be served; one that can’t returns 400 and no agent is created.

2. Wait for the Agent to Be Running

A serving agent can’t answer queries until it has been placed on a worker. Poll Get Agent until status is running. If it ends up failed instead, the agent couldn’t be placed or started, and error says why; create another agent to try again.

3. Send Queries

Send each payload to Query Agent. The request body is whatever the blueprint’s source connector accepts, and the agent’s answer comes back in the response field, shaped by the blueprint’s sink connector. The records body below is only an example of what a time-series blueprint might accept.
A successful query returns 200:
Some errors are worth retrying and some aren’t:

4. Open a Session for Many Queries (Optional)

If you’re sending a steady stream of payloads, Connect to Agent opens one websocket session instead of making a request for each payload. Each JSON text frame you send gets one answer frame back. Every frame from the server has a status field: connected when the session opens, completed for an answer, and failed for a frame the server couldn’t act on. cURL can’t hold a websocket session open, so this example is in Python only. It uses the websocket-client package (pip install websocket-client).
Python
A failed frame in answer to a bad payload doesn’t end the session, so you can keep sending. If the agent stops serving, the server sends a failed frame and then closes the session.

5. Stop the Agent

A serving agent keeps running, and counts against your organization’s serving limit, until you stop it. When you’re done with it, cancel it with Cancel Agent. Serving agents can’t be paused, so cancel is the only way to stop one yourself.
The platform also releases a serving agent that goes without a query for longer than the deployment’s idle window. A released agent ends completed, and queries to it return 409 with a message that starts released after. Released agents don’t come back; start a new serving agent from the same package to continue.

Using the Python Client

The Python client wraps these steps. It waits for a batch run to finish with wait_until_done, waits for a serving agent to start with wait_until_running, and retries 429 and 503 responses to query for you:
Python
wait_until_done raises a ValueError for a serving agent, since a serving agent never finishes on its own.