Skip to content

Task Notifications#

HyperQueue contains a mechanism for task notifications that can be used to notify users that some user-defined event happens.

Properties of notifications:

  • A task may send an arbitrary number of notifications.
  • Each notification may contain a message with a size limit of 1KiB.
  • The system propagates notifications to listening clients while preserving send order within each task. However, no ordering is guaranteed across notifications from different tasks.
  • Notifications are not persisted. They are never stored in a journal; they are never replayed. They are just sent to listening clients (if there are any) and then immediately discarded.
  • Notifications can be observed in the event live stream, but not in event stream replayed from a journal.

CLI#

The task notification is sent via calling hq task notify <message> inside a task.

The task notification is handled via --on-notify=<COMMAND> option used in the submit command. The <COMMAND> is invoked for each incoming notification. You may also pass more arguments, e.g. "--on-notify=python3 handle.py".

Example#

Task producing notifications (task.sh):#

./compute-something
$HQ task notify "Phase 1 finished"
./compute-something2

Note: Variable $HQ contains a path to the hq executable that executed the task.

The script that reacts on incoming notifications (handle.sh):#

The client script that is invoked for each incoming notification.

echo "Notification from job: $HQ_JOB_ID: $0."

The message of the notification is passed as an argument to the script. There are also variables $HQ_JOB_ID, $HQ_TASK_ID, and $HQ_WORKER_ID that contain information about the task that sent the notification.

Submitting the job:#

hq submit --on-notify=handle.sh -- task.sh