Skip to main content

Before you Begin

You’ll need Python 3.10+ and an API key.
  1. Go to the API Keys page (e.g. console.u1.archetypeai.app/profile/api-keys) to create an API key.
  2. Complete the Python Client Setup, or install directly as shown below.
Don’t have an API key yet? Request a free trial to get access and register your interest in a hands-on workshop.

1. Setup

Create a virtual environment, install the Archetype AI python client, and configure your credentials:
Note: Replace "insert_your_key" with the actual API secret key created using the Console. For more on environment variables, see Environment Variables.
The virtual environment matters: recent Python installs on macOS (Homebrew) and Debian/Ubuntu block system-wide installs, so a bare pip install fails with externally-managed-environment. If you see that error, activate the venv first. Conda environments work too — see Python Client Setup.
Both examples below use the Direct Query API: a single synchronous POST /query — no session lifecycle to manage. The model parameter selects which model runs against your inputs:
  • Newton Fusion (Newton::c2_6_...) — reasons over text, images, and video, and answers in natural language.
  • Newton Omega (OmegaEncoder::...) — encodes a window of time-series sensor readings into a fixed-size embedding vector per channel, for building classifiers and anomaly detectors.

2. Analyze a Video with the Newton Fusion Model

This example uploads a video and asks the Fusion model to describe what happens in it. The platform decodes the clip and uniformly samples frames server-side — no client-side video tooling needed. Save the following as fusion_example.py:
fusion_example.py
Download our sample delivery video to follow along:
Then run it:
To analyze a different video, pass the path as an argument. Try more clips from our starter video library, or use your own .mp4:

What you should see

After roughly 15 seconds, the model’s description of the video prints to your terminal:
Keep max_frames at 64 or below. Values above 64 fail silently — the request returns 200, but the video is dropped and the model answers from priors. For longer clips, allow a generous client timeout: video decoding can take several minutes.

3. Detect a Machine Fault with the Newton Omega Model

The Omega model turns a window of time-series sensor readings into a 768-dimensional embedding — no prompt needed. You send the raw numbers and get vectors back, ready for downstream classification or anomaly detection. This example runs the full cycle on real vibration data from a bearing run-to-failure experiment:
  1. Build an n-shot library — embed a handful of labeled windows (healthy and degraded).
  2. Classify held-out windows the library has never seen, with a k-nearest-neighbors vote over the embeddings.
  3. Score the predictions against ground-truth labels.
Download the sample data (a ~54 MB subset of the NASA IMS Bearing Dataset — see the attribution note below):
On Windows, use curl.exe — in PowerShell, plain curl is an alias for Invoke-WebRequest, which rejects these flags.
Save the following as omega_example.py:
omega_example.py
Run it:

What you should see

The 28 embedding calls fan out 8-way in parallel; the run takes about 30 seconds:
19 of 20 held-out windows classified correctly from just 8 labeled examples — no model training, no feature engineering. The one miss is a borderline window at the very start of the recording; a larger library, or embedding all four sensor channels jointly, resolves it. The atai-newton-omega-model skill runs this same pipeline at full scale — all ~1000 held-out windows, 4-channel joint embeddings, precision/recall reporting.
The encoder handles window lengths of 16 to 1024 timesteps natively. Shorter windows are padded and masked internally; inputs over 1024 are truncated to the last 1024 points.
Data attribution: the sample files are a curated subset of the NASA IMS Bearing Dataset (Set 2: a bearing run-to-failure experiment with four accelerometer channels), produced by the University of Cincinnati Center for Intelligent Maintenance Systems (IMS) and distributed via the NASA Prognostics Data Repository. If you use this data in a publication or redistribute it, include NASA’s requested citation: J. Lee, H. Qiu, G. Yu, J. Lin, and Rexnord Technical Services (2007). Bearing Data Set, IMS, University of Cincinnati. NASA Prognostics Data Repository, NASA Ames Research Center, Moffett Field, CA.

Try Variations

  • Edit the query and instruction prompt — ask the Fusion model for structured output by making the prompt the schema, for example: Respond with ONLY a JSON object: {"activity": "<what is happening>", "confidence": <0..1>}.
  • Adjust max_frames — compare 8 vs 32 on the same clip: more frames capture more temporal detail at the cost of latency.
  • Send images instead of video — pass an uploaded .png or .jpg in file_ids and ask the Fusion model to describe or extract data from it.
  • Vary the Omega window length — change WINDOW to 64 or 256 timesteps and see how accuracy holds up; pick the length that fits your signal’s dynamics (use the same length for the library and the test windows).
  • Evaluate more held-out windows — raise TEST_WINDOWS from 20 toward the ~1000 available; each window is one independent /query call.

Build with a Coding Agent

Do you have a dataset you want to try further? Use the Agent Skills with your favorite coding agent, or read the Direct Query API reference. The official Agent Skills teach coding agents like Claude Code, Cursor, and Codex these exact patterns — including the gotchas — so you can say “run this clip through Newton and tell me if the assembly passed” and let the agent write the code.

Troubleshooting

If you’re getting errors, see the Troubleshooting page for common issues and solutions, or check the Python Client documentation for installation and upgrade instructions. Error responses share a common shape — see Errors.

Learn More

Direct Query API

Full request and response reference for the /query endpoint.

Agent Skills

Teach your coding agent (Claude Code, Cursor, Codex) to build on Newton.

Core Concepts

Learn the fundamentals of the Newton AI platform.

Files

Manage your data uploads and supported file formats.

Fine-Tuning

Customize Newton models with your own data.

Example Projects

Working demos built on these patterns, from traffic analysis to wind-turbine fault detection.