Report errors to Sentry

hal0 can report crashes to Sentry. It is off unless you turn it on, and turning it on takes two deliberate steps — installing an optional package and setting a DSN. A stock install has no Sentry package, no DSN, and sends nothing.

This sends data off your LAN

hal0 is otherwise a local-first appliance. Enabling Sentry means crash reports leave the box for a third-party service. Read What gets sent before you enable it on anything holding real data.

Enable it on the backend

1. Install the extra into the same environment hal0 runs from:

   /usr/lib/hal0/venv/bin/pip install 'hal0ai[sentry]'

2. Put the DSN somewhere the units read. A separate file keeps it out of api.env and makes “turn it off” a single rm:

   sudo tee /etc/hal0/sentry.env >/dev/null <<'EOF'
   HAL0_SENTRY_DSN=https://<key>@<org>.ingest.sentry.io/<project>
   HAL0_SENTRY_ENVIRONMENT=my-box
   HAL0_SENTRY_TRACES_SAMPLE_RATE=0.1
   EOF
   sudo chmod 0640 /etc/hal0/sentry.env
   sudo chgrp hal0 /etc/hal0/sentry.env

3. Point the units at it with drop-ins, then reload:

   for unit in hal0-api.service hal0-bench-worker.service hal0-agent@hermes.service; do
     sudo mkdir -p "/etc/systemd/system/$unit.d"
     printf '[Service]\nEnvironmentFile=-/etc/hal0/sentry.env\n' \
       | sudo tee "/etc/systemd/system/$unit.d/sentry.conf" >/dev/null
   done
   sudo systemctl daemon-reload
   sudo systemctl restart hal0-api.service

4. Confirm it took:

   /usr/lib/hal0/venv/bin/python -c \
     'from hal0.observability import sentry; print(sentry.init_sentry("api"))'

   True means a DSN is configured and the SDK initialised.

Settings

Variable	Default	Meaning
HAL0_SENTRY_DSN	(unset)	The DSN. Unset or empty means Sentry is off. Falls back to SENTRY_DSN.
HAL0_SENTRY_ENVIRONMENT	development	Sentry environment tag.
HAL0_SENTRY_SERVER_NAME	hostname	Overrides the reported host.
HAL0_SENTRY_TRACES_SAMPLE_RATE	0	Transaction sampling, 0–1.
HAL0_SENTRY_PROFILES_SAMPLE_RATE	0	Profile sampling, 0–1.
HAL0_SENTRY_DEBUG	off	Print the SDK’s own transport logging.

Tracing defaults to 0 on purpose: hal0 serves long-lived streaming requests, and sampling every one of them on an inference box is expensive.

Enable it on the dashboard

The dashboard reads its DSN at build time, so a dashboard you did not build with a DSN cannot start reporting later:

cd ui
VITE_SENTRY_DSN='https://<key>@<org>.ingest.sentry.io/<project>' \
VITE_SENTRY_ENVIRONMENT=my-box \
npm run build

Then deploy ui/dist as usual. Without VITE_SENTRY_DSN the SDK lands in an unreferenced chunk that the browser never fetches.

What gets covered

Surface	How
hal0-api.service	create_app() initialises the SDK; the catch-all exception handler reports every 500 explicitly (a registered handler makes the exception “handled”, so the Starlette integration would otherwise never see it).
hal0 CLI and hal0-bench-worker.service	The Typer root callback initialises before any subcommand; unhandled exceptions arrive via the SDK’s excepthook.
hal0-agent@<id>.service	The shim initialises inside main(), wrapped so a broken SDK can never stop an agent from starting.
Dashboard	Global browser errors, plus an explicit capture in the view-level error boundary (it swallows render throws to keep the chrome alive).

4xx responses are not reported — those are the API contract working as designed. Only 5xx and genuinely unhandled exceptions become events.

What gets sent

Every event passes a scrubber before it leaves the process (hal0.observability.sentry.scrub_event, mirrored in ui/src/sentry.ts):

• the user block is dropped — no id, IP or email from hal0 itself;
• request bodies, cookies, query strings and the WSGI environ are dropped, and URLs are truncated at ? (hal0 accepts ?api_key= on WebSocket and SSE upgrades, where browsers cannot set headers);
• Authorization and friends are masked;
• everything remaining is walked with hal0’s own redaction helpers (hal0.api._redact) — sensitive key names are masked by name, and free text is scanned for Bearer / *_KEY= / client_id= tokens. That last rule is what catches a credential echoed inside an upstream error message.

If the scrubber itself fails, the event is dropped rather than sent raw.

Two things the scrubber cannot do

Geo-IP. Sentry derives a coarse location from the IP that delivers the event. That happens on their side, after scrubbing. Turn on Prevent Storing of IP Addresses in the Sentry project’s security & privacy settings if you do not want it.

Prompt text inside exception messages. Prompts are never attached deliberately, but an exception whose message quotes user input will carry it. Do not add extra={...} payloads containing user text at capture sites.

Turn it off

Delete the DSN and restart — the code no-ops without one:

sudo rm /etc/hal0/sentry.env
sudo rm -rf /etc/systemd/system/hal0-*.service.d/sentry.conf
sudo systemctl daemon-reload
sudo systemctl restart hal0-api.service

To remove it entirely, also pip uninstall sentry-sdk and rebuild the dashboard without VITE_SENTRY_DSN.