Job configuration reference
Fields you set when you create a GPU Container Job or a Managed Inference Job.
You set these fields when you create a job, either in the web interface or with cosmicac jobs create. The job type determines which fields apply. In non-interactive mode, set each field with the flag in the CLI flag column. For the create flow, see Create a GPU Container Job and Create a Managed Inference Job.
Common fields
These fields apply to every job type.
| Field | Required | CLI flag | Description |
|---|---|---|---|
| Job type | Yes | --type | The kind of job to create, either GPU Container or Managed Inference. |
| Job name | Yes | --name | A name to identify the job. |
| Tags | Yes | --tags | One or more labels for the job. The CLI accepts a comma-separated list. |
| Location | Yes | --location | Where the job runs, for example IN. The CLI lists the locations your racks report. |
| Notifications | No | --notify | The job lifecycle events this job reports. Takes a comma-separated list of job_failed, job_degraded, job_recovered, and job_restart_storm, or all, or none. Defaults to all four. An event also needs the matching switch on in Settings → Notifications before it reaches your webhook. See What controls delivery. |
GPU configuration
These fields select the job's hardware.
| Field | Required | CLI flag | Description |
|---|---|---|---|
| GPU | Yes | --gpu-type | The GPU to use, for example GH100_H100_SXM5_80GB. The CLI lists the GPU types your racks report. |
| GPU count | Yes | --gpu-count | Number of GPUs. |
| CUDA / driver | Yes | --driver | GPU driver version. CosmicAC supports only CUDA 12.9. |
Set the GPU type and count in one flag with --gpu TYPE=COUNT, for example --gpu H100=2. This replaces --gpu-type and --gpu-count.
GPU Container parameters
These fields apply to a GPU Container Job.
| Field | Required | CLI flag | Description |
|---|---|---|---|
| Base OS image | Yes | --base-image | Base OS image for the container. CosmicAC supports only Ubuntu22.04/CUDA12.9. |
| Disk | Yes | --root-disk-size-gb | Root disk size in GB, for example 250. |
vLLM Managed Inference parameters
These fields apply to a vLLM Managed Inference Job.
| Field | Required | CLI flag | Description |
|---|---|---|---|
| Model | Yes | --model | Hugging Face model ID to serve, for example MiniMaxAI/MiniMax-M2.5. |
| Runtime image (CUDA) | Yes | --runtime-image | Serving runtime image as a Docker image reference. One of vllm/vllm-openai:v0.15.1 or vllm/vllm-openai:v0.17.1. |
| Data type | Yes | --data-type | Numeric precision the model runs at, for example BF16 or Auto. |
| Quantisation | Yes | --quantisation | Quantisation scheme, for example fp8 or awq. |
| Tensor parallel | No | --tensor-parallel | GPUs to split the model across. Must divide the GPU count exactly. Defaults to 1 in the CLI. |
| GPU memory utilization | Yes | --gpu-memory-utilization | Fraction of GPU memory to use, between 0 and 1. |
| Max concurrent sequences | Yes | --max-concurrent-sequences | Maximum requests handled at once. |
| Max model length | Yes | --max-model-length | Maximum model context length. |
| Reasoning parser | Yes | --reasoning-parser | Parser for the model's reasoning output. |
| Video & image input | Yes | --multimodal | Whether the model accepts multimodal input. true or false. |
| Root disk size | Yes | --root-disk-size-gb | VM root disk size in GB. Minimum is the model's disk_gb, listed as Root disk size in Recommended model parameters. |
| Environment variables | No | --env | Environment variables passed to the inference service. |
| Endpoint name | Yes | --endpoint-name | Name of the inference endpoint. Must be unique across active inference jobs. |
| Replicas | Yes | --replica | Number of endpoint replicas. |
| Require Authorization header | Yes | --require-auth-header / --no-auth-header | Whether callers must send an authorization header. true or false. |
Each model sets its own minimum, so no single size works everywhere. CosmicAC rejects a smaller root disk with root_disk_size_gb (X) is below model master disk_gb (Y).
The web interface doesn't show Tensor parallel. It sends the model master's value, so jobs you create there are already correct.
The CLI is different:
- Interactive mode:
cosmicac jobs createlists your model masters and pre-selects the value from the one you pick. With no model masters, it pre-selects1. - Flags: the CLI sends
1unless you pass--tensor-parallel.
Set tensor parallel to the model's per_replica_gpu_count, listed as GPUs per replica in Recommended model parameters.
If the value doesn't divide the job's GPU count exactly, CosmicAC rejects the job with tensor_parallel must evenly divide the GPU count per instance and a 400 status.
Parakeet Managed Inference parameters
These fields apply to a Parakeet Managed Inference Job.
| Field | Required | CLI flag | Description |
|---|---|---|---|
| Model | Yes | --model | Parakeet model to serve, nvidia/parakeet-tdt-0.6b-v3. |
| Endpoint name | Yes | --endpoint-name | Name of the transcription endpoint. |
| Chunk duration | Yes | --chunk-duration | Audio chunk length in seconds, for example 600. Minimum 10. |
| Chunk overlap | Yes | --chunk-overlap | Overlap between chunks in seconds, for example 5. Must be less than the chunk duration. Minimum 5. |
| Max file size | Yes | --max-file-size-mb | Maximum upload size in MB, for example 1024. Minimum 1024. |
| Require Authorization header | Yes | --require-auth-header / --no-auth-header | Whether callers must send an authorization header. true or false. |