Skip to content
Vaicoa

Get told when a job fails

Job reports are part of Vaicoa Pro and Vaicoa Team. Each job belongs to a server that runs the agent.

A job report is the result of one of your own recurring jobs, such as a backup, a certificate renewal or a reachability test, sent to Vaicoa after each run. Vaicoa alerts you when the job fails, and when it has not succeeded for longer than you allow, which catches a backup that stopped running without an error.

Two alerts come with every job. They use the same push notifications and alert delivery as other alerts:

  • Failed: the last run exited with a code other than 0.
  • Stale: no successful run within the time the job declared. A job that declares no time only gets the failure alert. A job that has never succeeded is never stale; it shows as failed.

A job report uses the same path as the agent’s metrics and logs, so there is nothing extra to install or open:

How Vaicoa collects dataHetzner Cloud servers appear in the Hetzner Cloud API, which the app reads directly in free mode with a read-only token that never leaves the device. On Pro and Team, the Vaicoa agent — installed on cloud servers and on dedicated Robot servers, which the Cloud API does not cover — adds OS-level metrics and WARN-and-worse log lines, and carries the results of your own backup and cron jobs. Uptime checks run the other way round: Vaicoa probes your own endpoints from its own locations, from outside, and those endpoints need not be Hetzner machines at all — four check targets come with a free account. All of it arrives at Vaicoa and leaves again three ways: in the app as alerts, charts and job reports; on Pro and Team as alerts that reach you by push notification, email digest, ntfy topic or webhook; and through the read-only MCP connector, which a coding agent can ask about your fleet.Hetzner Cloud serversagentdisk · RAM · loadWARN+ log linesDedicated servers (Robot)agentnot in the Cloud APIBackup & cron jobscloudviewer-reportYour endpointsat Hetzner or anywhere elseapi.example.comOur own locationsuptime checks, Free toofrom outsideVaicoaHetzner Cloud APIinventory · CPU · networkconnected projectfree modeAlertsChartsJob reportsVaicoa appAlerts reach youPush notificationEmail digestntfy topicWebhookCoding agentMCP · read-onlyFree mode — your device reads the Hetzner CloudAPI directly; the read-only token never leaves it.Pro and Team — the agent adds OS metrics, WARN+ logs,job results and dedicated servers; alerts and MCP too.Uptime checks — probed from our own locations,from outside; four targets even on a free account.

Wrap the job

Section titled “Wrap the job”

The agent installs a wrapper, cloudviewer-report. It runs your command, measures how long it took, sends the report and returns the command’s exit code unchanged, so your cron line behaves as before.

  1. Find the line that runs the job, in a crontab or a systemd service.

  2. Put the wrapper in front of the command, with a job name and how old the last success may get:

    Terminal window
    cloudviewer-report --job backup-s3 --max-age 26h -- restic backup /data

  3. Let the job run. The first report creates the job; there is nothing to register in advance.

Choose --max-age a little longer than the schedule, so a run that starts late is not stale: 26h for a daily job. It takes seconds or a number with m, h or d. Job names are lowercase letters, digits, -, _ and ., up to 64 characters, starting with a letter or digit.

The wrapper reads the server’s token from /etc/cloudviewer-agent/agent.env, which ordinary users cannot read. Run the job as root, or give the wrapper a file that contains only the token: --token-file /path/to/token. If the token is missing or wrong, or Vaicoa is unreachable, the report is skipped; the job itself is never failed or delayed.

Report with curl instead

Section titled “Report with curl instead”

When a script knows its own result, or the wrapper is not installed, send the report with one HTTP call and the server’s agent token:

Terminal window
restic backup /data; ec=$?
curl -sS -X POST https://api.vaicoa.com/v1/ingest/job \
  -H "X-Agent-Token: $CV_AGENT_TOKEN" -H "Content-Type: application/json" \
  -d "{\"reports\":[{\"job_name\":\"backup-s3\",\"exit_code\":$ec,\"max_age_seconds\":93600}]}"

On a server with the agent installed, the token is in /etc/cloudviewer-agent/agent.env. All fields, limits and error codes are in the Job reports API.

Jobs that run elsewhere

Section titled “Jobs that run elsewhere”

A job that runs somewhere other than the server, such as GitHub Actions, a serverless function or a machine outside Hetzner, reports with the token of the server it relates to: the database it backs up, or the host behind the API it tests. The job then appears on that server’s screen. Keep the server’s agent token in your CI secret store and use --token-file or the curl call.

Where jobs appear

Section titled “Where jobs appear”

In the app, the server’s detail screen has a Jobs section between the charts and Log context, once the server has reported a job. Each row has a status dot, the name, Stale, Exit and the code, or Never succeeded when something is wrong, when the job last succeeded or ran, and how long it took.

Tap a row for the job’s own screen:

StatusSucceeded, Failed or Stale. Stale wins over a successful exit code, because a job that last succeeded a week ago is not working now
Last runhow long ago, with the exact time below
Last successthe same, or Never
Exit codethe number, in red when it is not 0
Durationwhen the report included one
Expectedfor example “every 24 hours”, from the declared time, or No schedule declared

Below that, Why it failed → Open log context leads to the server’s log. A job report carries no message, so the details are in the server’s own log around the time of the run. Let the job write them to journald at warning level or higher and they appear there:

Terminal window
logger -p user.warning -t backup "s3 sync failed: bucket not reachable"
Job reports APIEndpoint, fields, wrapper flags, limits and error codes.
When something needs attentionFrom the alert to the log lines that explain it.