Output mode

By default, HyperQueue CLI commands output information in a human-readable way, usually in the form of a table. If you want to use the CLI commands programmatically, HyperQueue offers two additional output modes that are designed to be machine-readable.

You can change the output type of any HyperQueue CLI command either by using the --output-mode flag or by setting the HQ_OUTPUT_MODE environment variable.

FlagEnvironment variable

$ hq --output-mode=json job list

$ HQ_OUTPUT_MODE=json hq job list

Currently, there are three output modes available. The default, human-readable cli mode, and then two machine-readable modes, JSON and Quiet.

Important

Each machine-readable mode supports a set of commands. You can also use commands that are not listed here, but their output might be unstable, or they might not output anything for a given output mode.

JSON#

The json output mode is intended to provide very detailed information in the form of a JSON value. With this mode, HyperQueue will always output exactly one JSON value, either an array or an object.

Warning

The JSON output is currently unstable and can change with every new HyperQueue version. It can also be incomplete and might not contain all information outputted by the human-readable cli mode. The documentation below can be inaccurate, please examine source code for more details.

Error handling#

When an error occurs during the execution of a command, the program will exit with exit code 1 and the program will output a JSON object with a single error key containing a human-readable description of the error.

Date formatting#

Time-based items are formatted in the following way:

Duration - formatted as a floating point number of seconds.
Datetime (timestamp) - formatted as a ISO8601 date in UTC

Supported commands#

Server info: hq server info

Example

{
  "host": "my-machine",
  "hq_port": 42189,
  "pid": 32586,
  "server_dir": "/foo/bar/.hq-server",
  "start_date": "2021-12-20T08:45:41.775753188Z",
  "version": "0.7.0",
  "worker_port": 38627
}

Worker list: hq worker list

Example

[{
  "configuration": {
    "heartbeat_interval": 8.0,
    "hostname": "my-machine",
    "idle_timeout": null,
    "listen_address": "my-machine:45611",
    "log_dir": "...",
    "resources": {
      "cpus": [[0, 1, 2, 3]],
      "generic": [{
        "kind": "sum",
        "name": "resource1",
        "params": {
          "size": 1000
        }
      }]
    },
    "time_limit": null,
    "work_dir": "..."
  },
  "ended": null,
  "id": 1
}]

Worker info: hq worker info

Example

{
  "configuration": {
    "heartbeat_interval": 8.0,
    "hostname": "my-machine",
    "idle_timeout": null,
    "listen_address": "my-machine:45611",
    "log_dir": "...",
    "resources": {
      "cpus": [[0, 1, 2, 3]],
      "generic": [{
        "kind": "sum",
        "name": "resource1",
        "params": {
          "size": 1000
        }
      }]
    },
    "time_limit": null,
    "work_dir": "..."
  },
  "ended": null,
  "id": 1
}

Submit a job: hq submit
Example
```
{
  "id": 1
}
```

Job list: hq job list

Example

[{
  "id": 1,
  "name": "ls",
  "resources": {
    "cpus": {
      "cpus": 1,
      "type": "compact"
    },
    "generic": [],
      "min_time": 0.0
    },
  "task_count": 1,
  "task_stats": {
    "canceled": 0,
    "failed": 0,
    "finished": 1,
    "running": 0,
    "waiting": 0
  }
}]

Job info: hq job info

Example

{
  "finished_at": "2021-12-20T08:56:16.438062340Z",
  "info": {
    "id": 1,
    "name": "ls",
    "resources": {
      "cpus": {
        "cpus": 1,
        "type": "compact"
      },
      "generic": [],
        "min_time": 0.0
      },
    "task_count": 1,
    "task_stats": {
      "canceled": 0,
      "failed": 0,
      "finished": 1,
      "running": 0,
      "waiting": 0
    }
  },
  "max_fails": null,
  "pin": null,
  "priority": 0,
  "program": {
    "args": [
      "ls"
    ],
    "cwd": "%{SUBMIT_DIR}",
    "env": {
      "FOO": "BAR"
    },
    "stderr": {
      "File": "job-%{JOB_ID}/%{TASK_ID}.stderr"
    },
    "stdout": {
      "File": "job-%{JOB_ID}/%{TASK_ID}.stdout"
    }
  },
  "started_at": "2021-12-20T08:45:53.458919345Z",
  "tasks": [{
    "finished_at": "2021-12-20T08:56:16.438062340Z",
    "id": 0,
    "started_at": "2021-12-20T08:56:16.437123396Z",
    "state": "finished",
    "worker": 1,
    "cwd": "/tmp/foo",
    "stderr": {
      "File": "job-1/0.stderr"
    },
    "stdout": {
      "File": "job-1/0.stdout"
    }
  }],
  "time_limit": null,
  "submit_dir": "/foo/bar/submit"
}

Automatic allocation queue list: hq alloc list

Example

[{
  "additional_args": [],
  "backlog": 4,
  "id": 1,
  "manager": "PBS",
  "max_worker_count": null,
  "name": null,
  "timelimit": 1800.0,
  "worker_cpu_args": null,
  "worker_resource_args": [],
  "workers_per_alloc": 1
}]

Automatic allocation queue info: hq alloc info

Example

[{
  "id": "pbs-1",
  "worker_count": 4,
  "queue_at": "2021-12-20T08:56:16.437123396Z",
  "started_at": "2021-12-20T08:58:25.538001256Z",
  "ended_at": null,
  "status": "running",
  "workdir": "/foo/bar"
}]

Quiet#

The quiet output mode will cause HyperQueue to output only the most important information that should be parseable without any complex parsing logic, e.g. using only Bash scripts.

Error handling#

When an error occurs during the execution of a command, the program will exit with exit code 1 and the error will be printed to the standard error output.

Supported commands#

Submit a job: hq submit

Schema

Outputs a single line containing the ID of the created job.
Example
```
$ hq --output-mode=quiet submit ls
1
```