Skip to main content
The Developer Console supports processing large datasets asynchronously using the Batch Manager. The Batch Manager lets you create and manage batch processing jobs, as well as monitor their status. Selecting the Batch Manager in the console presents you with the Batch Manager dashboard, which shows existing jobs, along with filters to let you see only jobs in the desired state, job search functionality, and other tools to help you manage and monitor jobs easily.

​
Creating a Batch Processing Job

To create a batch processing job:
  1. Click the Create button on the Batch Manager dashboard page.
  2. Enter a name for the job.
  3. Choose the task type in which to run the job.
  4. Enter the configuration of the job using YAML. For example, for the Machine State Classification pipeline type, make sure the column names in the YAML configuration match the names of the columns in your data files.
  5. Select or upload the files to run the job against. You can select up to 50 files.
  6. For the Machine State Classification task type, select N-shot example files to train with. Be sure to add appropriate labels for each.
  7. When everything’s configured the way you want it, click Create Job.

​
Viewing the Status of a Batch Processing Job

To view status information about a job, such as its progress, its logs, and other job details, click on the job in the Batch Manager dashboard. The job status page displays information about the job, including:
  • The job’s status (PENDING, ADMITTED, RUNNING, COMPLETED, FAILED, PREEMPTED, or CANCELLED)
  • The amount of time the job has spent running
  • The total amount of time the job has spent in the queue
  • The job’s ID
  • The job’s task type
  • When the job was created
  • When the job started running
  • The YAML configuration of the job
  • Processing details about the job
The processing details pane offers three tabs:
  • Progress shows the execution process of the job, such as the timestamp at which training and inference began for each step of the job.
  • Logs shows detailed logs for the batch processing job.
  • Artifacts opens the Job Artifacts view, which lists all of the artifacts generated by the job.

​
Downloading Batch Artifacts

To download the artifacts generated by completed jobs, you have two options:
  • To download all of the artifacts for a batch at once, click the Download Artifacts button to the right of the job title. This will download a Zip archive containing all the artifact files.
  • To download the artifact file for a specific job, click the download icon at the right side of that artifact’s row in the Job Artifacts view.

​
Canceling a Batch Processing Job

To cancel a running job, click on the job in the Batch Manager dashboard, then click the Cancel button to the right of the job’s title.