Skip to main content
The Files API is used to upload data to the Archetype platform for tasks like data summarization or model fine tuning.

​
Organization & Access

All files are securely siloed within a developer’s organization. Any file uploaded by a developer is visible and accessible by any developer within the same organization, however, they are not visible or available to anyone outside the developer’s organization. Any developer in the same organization can upload files, download files, query information about a file, or modify a file for any file within the same org.

​
Files API

The Files API supports the following:
  • Upload new files
  • Download existing files
  • List information across all files
  • Retrieve metadata across all existing files
  • Modify metadata for existing files
  • Delete existing files

​
File IDs

Each file uploaded to the Archetype platform is assigned a file_id. The file_id can be used to reference the file across the Archetype platform, such as passing that file to a Lens for analysis. The file_id is returned as the response when each file is uploaded, and is typically the base name of the uploaded file.

​
Uploading Files

There are two ways to upload files to the Newton platform:
  • Upload through the Files API endpoints (Upload File and Upload Base64 File). These endpoints accept the file data and transfer it to the underlying AWS S3 data store automatically, but only accept files up to 512MB in size.
  • Direct-to-cloud upload lets you bypass the Newton platform, uploading the data directly to presigned URLs provided by AWS S3. Files uploaded this way may be up to 250GB.
Lenses and the Lens API do not support files larger than 255MB, regardless of how they were uploaded. Files over 255MB will cause the Lens to fail.

​
File Duplicates

The Files API will automatically update any file to the latest file instance if two files are uploaded to the Archetype platform with the same file name. As files are shared across multiple developers in the same organization, developers should be cautious when uploading common file names such as “test.png” or “image.png” as these could overwrite files from other developers in the same organization.

​
File Formats

At this time, the current file formats are supported:
  • image/jpeg
  • image/png
  • image/base64
  • video/mp4
  • CSV
  • JSON
  • JSONL