audience: operators and AI coding agents first.

# system-monitor

Netdata hosts metrics, alarms, and dashboard. `agent-guard` handles event streams, desktop notifications, and action buttons.

## Install

Run as root yourself. Codex must not run this.

```sh
sudo ./install.sh
```

`install.sh` is idempotent. It installs netdata from signed apt repo, deploys configs/plugins/health alarms, installs `agent-guard`, ships slices/wrappers/tmpfiles policy, retires `~/.local/bin/mem-pressure-notify` plus service and `~/.local/bin/disk-fill-notify`.

## Uninstall

```sh
sudo ./uninstall.sh
sudo ./uninstall.sh --restore-retired
```

Default uninstall leaves retired old notifiers removed. Root oomd/caps/swappiness/zram prevention stays untouched.

## Check Map

| Check | Mechanism |
|---|---|
| memory runway | `netdata/plugins/mem_trajectory.plugin` emits `system_monitor.mem_runway`; `health.d/mem_trajectory.conf` alarms; `agentctl` re-scans culprit live |
| `/tmp` tmpfs fill | `tmpfs_guard.plugin` emits used/inodes/stale bytes; health alarm offers safe clean action |
| disk fill | stock prediction plus `disk_guard.plugin` block/inode/reserved-capacity metrics; `health.d/disk_fill.conf` |
| build disk admission | `run-build` rejects launch at >=85% used or <100 GiB available; override thresholds with `SM_DISK_MAX_USED_PCT` and `SM_DISK_MIN_FREE_GIB` |
| CPU runaway | `cpu_runaway.plugin`; `builds.slice` fact first, build allowlist fallback |
| process/fd/inotify pressure | `proc_fd.plugin`; `health.d/proc_fd.conf` |
| OOM/kernel/storage hang signals | `agent_guard.events.journald` follows journal stream |
| sensitive file changes | `agent_guard.events.fswatch` uses inotify on ssh/cron/systemd paths with cooldown |
| new listening ports | `agent_guard.events.ports` diffs `/proc/net/tcp*`; dev-port allowlist suppresses normal dev servers |
| desktop notification/actions | `agent_guard.notifier` uses `gdbus`; logs actions through journald fields |
| kill remediation | `agent_guard.culprit` re-scans live, checks cgroup and start time, refuses protect list |
| `/tmp` remediation | deletes only uid-1000 files older than configured age |
| self-survival | user unit has `MemoryMax=200M`, `ManagedOOMPreference=avoid`, `WatchdogSec=30`; daemon sends `sd_notify` pings |
| source fixes | `slices/bin/run-agent`, `slices/bin/run-build`, `fixes/tmpfiles.d/system-monitor-tmp.conf` |

## Commands

```sh
make test
make check
/usr/local/bin/run-build make
/usr/local/bin/run-agent codex
```

Wrapper tradeoff: wrapper slice is a cgroup fact; command allowlist is a fallback heuristic for unwrapped strays.

## Tier-2 Fault Injection

Run only after install, never in `make test`.

1. Memory: start a scratch cgroup, allocate memory until runway falls, confirm `mem_eta_critical` and desktop notification fire.
2. tmpfs: fill a scratch tmpfs, confirm `tmpfs_fill` and `Clean /tmp (safe)` action.
3. systemd unit: crashloop a dummy user unit, confirm netdata `systemdunits` alarm.
4. fswatch: write in a temporary watched directory, confirm notice.
5. End-to-end: assert notification appears through D-Bus, not only metric movement.