Prometheus ZFS textfile collectors

This repository provides small Bash collectors for Prometheus node exporter's textfile collector. They are intended for hosts that use ZFS and the Debian/Ubuntu prometheus-node-exporter package.

The installer copies selected scripts to /usr/local/bin and runs each one as a root-owned, persistent systemd timer. Root is used because zpool and zfs normally require it; the generated .prom files remain readable by the prometheus user.

Prerequisites

• A Linux host with systemd, ZFS tools, and Bash. The Sanoid collector also requires sanoid to be installed and configured.
• Prometheus node exporter installed and using the prometheus user/group.
• The Debian/Ubuntu node-exporter configuration path: /etc/default/prometheus-node-exporter.

The collectors write to /var/lib/node_exporter/textfile.

Install

Fetch the repository and enter it:

git clone https://code.wenger.solutions/witops-public/prometheus-textfile-collector.git
cd prometheus-textfile-collector

zfs-keystatus queries backup/enc by default. Set ZFS_KEYSTATUS_DATASET in a systemd override if your encrypted dataset has a different name.

Install all available collectors:

sudo ./install.sh --all

Or choose individual collectors. --collector can be repeated:

sudo ./install.sh --collector zfs-pool
sudo ./install.sh --collector zfs-keystatus
sudo ./install.sh --collector sanoid-monitor-snapshots

The installer:

• creates /var/lib/node_exporter/textfile with owner and group prometheus;
• ensures the node-exporter ARGS setting includes --collector.textfile.directory=/var/lib/node_exporter/textfile, preserving its other existing options;
• installs executable copies of the selected scripts in /usr/local/bin;
• writes and enables a systemd .service and .timer for each collector;
• restarts prometheus-node-exporter.service if it is currently running.

The ZFS collectors run every minute; Sanoid snapshot monitoring runs every 15 minutes. Their persistent systemd scheduling runs a missed invocation when the timer becomes active. To skip the node-exporter restart, add --no-restart-node-exporter.

Installed collectors

Installer name	Installed command	Timer	Output
zfs-pool	/usr/local/bin/zfs-pool	prometheus-textfile-zfs-pool.timer	zfs.prom
zfs-keystatus	/usr/local/bin/zfs-keystatus	prometheus-textfile-zfs-keystatus.timer	zfs_keystatus.prom
sanoid-monitor-snapshots	/usr/local/bin/sanoid-monitor-snapshots	prometheus-textfile-sanoid-monitor-snapshots.timer	sanoid_monitor_snapshots.prom

Sanoid snapshot monitoring

sanoid-monitor-snapshots runs sanoid --monitor-snapshots every 15 minutes and translates its Nagios status into these metrics:

• sanoid_snapshot_monitor_status: 0 for OK, 1 for WARNING, 2 for CRITICAL, and 3 when Sanoid could not return a Nagios status.
• sanoid_snapshot_monitor_problem: one sample for every warning or critical snapshot tier, labelled with dataset, snapshot_type, severity, and reason (missing or stale).
• sanoid_snapshot_monitor_snapshot_age_seconds and sanoid_snapshot_monitor_snapshot_age_threshold_seconds: the parsed age and configured warning/critical threshold for stale snapshots.
• sanoid_snapshot_monitor_last_run_success and sanoid_snapshot_monitor_last_run_timestamp: collector execution health; sanoid_snapshot_monitor_exit_code preserves Sanoid's original exit code.

For example, Sanoid's CRIT: tank/data has no hourly snapshots at all! becomes sanoid_snapshot_monitor_problem{dataset="tank/data",snapshot_type="hourly",severity="CRIT",reason="missing"} 1.

Check installation and recent runs with:

systemctl list-timers 'prometheus-textfile-*'
systemctl status prometheus-textfile-zfs-pool.timer
journalctl -u prometheus-textfile-zfs-pool.service --since today
cat /var/lib/node_exporter/textfile/zfs.prom

Updating or adding collectors

Rerun the same install command after changing a script; it overwrites the installed command and unit definitions. Enabled timers stay enabled.

To add another collector later, add one entry to collector_info in install.sh (source filename, installed name, unit name, interval, and description). This single manifest drives command-line selection, installation, and systemd naming.

Removing a collector

Disable its timer, remove its units and installed executable, then reload systemd. For example, for the pool collector:

sudo systemctl disable --now prometheus-textfile-zfs-pool.timer
sudo rm /etc/systemd/system/prometheus-textfile-zfs-pool.{service,timer}
sudo rm /usr/local/bin/zfs-pool
sudo systemctl daemon-reload

You may also remove its .prom file from the textfile directory if it is no longer wanted.