Task Notifications#
HyperQueue contains a mechanism for task notifications that can be used to notify users that some user-defined event happens.
Properties of notifications:
- A task may send an arbitrary number of notifications.
- Each notification may contain a message with a size limit of 1KiB.
- The system propagates notifications to listening clients while preserving send order within each task. However, no ordering is guaranteed across notifications from different tasks.
- Notifications are not persisted. They are never stored in a journal; they are never replayed. They are just sent to listening clients (if there are any) and then immediately discarded.
- Notifications can be observed in the event live stream, but not in event stream replayed from a journal.
CLI#
The task notification is sent via calling hq task notify <message>
inside a task.
The task notification is handled via --on-notify=<COMMAND>
option used in the submit command. The <COMMAND>
is invoked for each incoming notification. You may also pass more arguments, e.g. "--on-notify=python3 handle.py"
.
Example#
Task producing notifications (task.sh
):#
./compute-something
$HQ task notify "Phase 1 finished"
./compute-something2
Note: Variable $HQ
contains a path to the hq
executable that executed the task.
The script that reacts on incoming notifications (handle.sh
):#
The client script that is invoked for each incoming notification.
echo "Notification from job: $HQ_JOB_ID: $0."
The message of the notification is passed as an argument to the script. There are also variables $HQ_JOB_ID
, $HQ_TASK_ID
, and $HQ_WORKER_ID
that contain information about the task that sent the notification.
Submitting the job:#
hq submit --on-notify=handle.sh -- task.sh