Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.archetypeai.app/llms.txt

Use this file to discover all available pages before exploring further.

While the basic file upload functions (Upload File and Upload Base64 File) are limited to files of 255MB and less, you can upload files up to 250GB using the Direct-to-Cloud upload workflow.

​
Step-by-Step

The direct-to-cloud workflow begins by using the Files API to request a list of presigned URLs: one for each chunk of the file to be uploaded. With those in hand, your code then performs an HTTPS PUT for each chunk, sending it to the corresponding presigned URL.

​
Initiate the Upload

To start a direct-to-cloud upload, call the Initiate Upload endpoint, providing the name of the file (filename), its MIME type (file_type), and its length in bytes (num_bytes). This returns an ID for the upload (upload_id) and an array of parts. Each part in the array specifies the part number (part_number), the presigned URL to which the part data should be uploaded (url), the offset into the file at which the part begins (offset), and the part’s length in bytes (length). The timestamp at which the presigned URL expires is also provided (expires_at).

​
Upload the parts

Upload each part using HTTPS PUT to its presigned URL. Capture the each response header’s ETag; this is the part’s part_token. Optionally, you can checkpoint completed parts with the Checkpoint Upload Parts endpoint. Doing this lets you skip them if the upload is paused and then later resumed. You can also optionally refresh any presigned URLs that are about to expire, using the Generate Upload Part URLs endpoint.

​
Complete the Upload

Complete the upload using the Complete Upload endpoint, submitting part numbers and corresponding part_token for each uploaded part. Parts which have been checkpointed do not need to be included in the part list when calling the Complete Upload endpoint.

​
Aborting an Upload

To abort an in-progress upload, use the Abort Upload endpoint, specifying the upload_id of the upload to abort. This releases the resources used on the server for the specified upload.

​
Example

For an example including the use of direct-to-cloud uploads, see our batch processing example repository, which includes a script that uploads files using the direct-to-cloud endpoints.