Grandiras/Ponente

Fork 0

The updater recreates the app in a scope of its own, so it starts once #222

Merged

Grandiras merged 1 commit from fix-updater-conmon-scope into main

2026-09-22 23:08:04 +00:00

Grandiras commented

2026-09-22 23:05:39 +00:00

Owner

Every image refresh started the app twice and left it unmonitored. The same happens on dev after each merge, and on production after each release.

What happens (dev, the 22:54 UTC refresh on 2026-09-22)

ponente-update@ponente-dev (a oneshot unit) recreates the app. Podman forks the new container's conmon from the caller, so conmon lands in the updater unit's cgroup.
The script ends ("healthy… serving", 22:54:41), and systemd cleans up the finished unit's cgroup. The fresh container gets SIGTERM: the app logs a graceful shutdown ten seconds after starting, podman events show died with no stop or kill, and the restart policy starts it again (restart count 1).
Its conmon is still in the unit. The stop times out after 45s and systemd SIGABRTs it (Failed with result 'timeout'). The app keeps running unmonitored, with no restart on a crash and no exit code. The next recreate then reports conmon exited prematurely… conmon process killed.

Production follows the same updater. Its last update came through deploy.sh, so the next release would be the first timer-driven one, and would do all of the above.

Fix

update.sh runs podman-compose under systemd-run --user --scope, so conmon lives in a transient scope of its own, outside the updater unit, and a finished run has nothing left to kill. --collect tidies the scope away when its container goes. (podman.service avoids the same trap with KillMode=process. That would keep conmon inside the unit and log a leftover-process warning every minute.)

Probed on the server first: from a user service, a process detached inside such a scope outlives the service, in its own run-*.scope. bash -n passes. deploy/README.md documents the symptom next to the updater's troubleshooting commands.

Deploying it

update.sh reaches the server only through deploy.sh, which re-syncs it. So after merging: ./deploy/deploy.sh --env dev and ./deploy/deploy.sh, before tagging v2.7.0, so the release is the first one deployed by the fixed updater.

🤖 Generated with Claude Code

**Every image refresh started the app twice and left it unmonitored.** The same happens on dev after each merge, and on production after each release. ## What happens (dev, the 22:54 UTC refresh on 2026-09-22) - `ponente-update@ponente-dev` (a oneshot unit) recreates the app. Podman forks the new container's **conmon** from the caller, so conmon lands in the *updater unit's* cgroup. - The script ends ("healthy… serving", 22:54:41), and systemd cleans up the finished unit's cgroup. The fresh container gets SIGTERM: the app logs a graceful shutdown ten seconds after starting, podman events show `died` with no `stop` or `kill`, and the restart policy starts it again (restart count 1). - Its conmon is still in the unit. The stop times out after 45s and systemd SIGABRTs it (`Failed with result 'timeout'`). The app keeps running **unmonitored**, with no restart on a crash and no exit code. The next recreate then reports `conmon exited prematurely… conmon process killed`. Production follows the same updater. Its last update came through `deploy.sh`, so the next release would be the first timer-driven one, and would do all of the above. ## Fix `update.sh` runs `podman-compose` under `systemd-run --user --scope`, so conmon lives in a transient scope of its own, outside the updater unit, and a finished run has nothing left to kill. `--collect` tidies the scope away when its container goes. (`podman.service` avoids the same trap with `KillMode=process`. That would keep conmon inside the unit and log a leftover-process warning every minute.) Probed on the server first: from a user service, a process detached inside such a scope outlives the service, in its own `run-*.scope`. `bash -n` passes. `deploy/README.md` documents the symptom next to the updater's troubleshooting commands. ## Deploying it `update.sh` reaches the server only through `deploy.sh`, which re-syncs it. So after merging: `./deploy/deploy.sh --env dev` and `./deploy/deploy.sh`, **before** tagging v2.7.0, so the release is the first one deployed by the fixed updater. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Grandiras added 1 commit

2026-09-22 23:05:39 +00:00

The updater recreates the app in a scope of its own, so it starts once

CI / build (pull_request) Successful in 52s

Details

2d52de5f33

On every refresh the app started, came up healthy, shut down ten seconds
later and started again: podman forks the new container's conmon from the
caller, the caller is the oneshot ponente-update@ unit, and systemd cleans a
finished unit's cgroup up — SIGTERM to the fresh container (the restart
policy brought it back), then SIGABRT to its conmon when the stop timed out,
failing the run and leaving the app unmonitored. Production follows the same
updater, so the next release would have done it there.

podman-compose now runs under `systemd-run --user --scope`, keeping conmon
out of the unit. Probed on the server: a process detached inside such a scope
from a user service outlives the service, in its own run-*.scope.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

opencode-bot commented

2026-09-22 23:05:41 +00:00

Collaborator

⏳ Held on the OpenCode Go window — the 5-hour window is at 100% (the bot holds at 95%).
This continues automatically after the window resets (2026-09-23T01:12:57.059Z) — nothing to re-trigger.

⏳ **Held on the OpenCode Go window** — the 5-hour window is at 100% (the bot holds at 95%). This continues automatically after the window resets (2026-09-23T01:12:57.059Z) — nothing to re-trigger.

Grandiras commented

2026-09-22 23:06:10 +00:00

Author

Owner

@claude-reviewer

claude-bot commented

2026-09-22 23:06:15 +00:00

Collaborator

🤖 Reviewing this PR — follow along in the action run; the reply will link the full session transcript.

📜 Live transcript — updating every ~5 minutes while the task runs.

🤖 Reviewing this PR — follow along in [the action run](https://git.grandiras.net/Grandiras/claude-bot/actions/runs/571); the reply will link the full session transcript.  📜 [Live transcript](https://git.grandiras.net/Grandiras/claude-bot/src/branch/claude-logs/transcripts/Grandiras/Ponente/pr-222/run-571.md) — updating every ~5 minutes while the task runs.

claude-reviewer approved these changes

2026-09-22 23:07:35 +00:00

claude-reviewer left a comment

✅ Verdict: approve — no blocking issues found.

Correct, well-scoped fix. Wrapping the podman-compose up call in systemd-run --user --scope --quiet --collect -- moves conmon (and the container it monitors) into a transient scope outside the oneshot updater unit's cgroup, so systemd's cleanup of the finished unit no longer tears down the freshly recreated, healthy container. This matches a known and documented podman/systemd interaction (conmon inheriting the caller's cgroup unless explicitly isolated — see containers/podman issue tracking this exact class of bug), and --collect correctly lets the scope go away once conmon (and thus the container) exits, so nothing leaks across future recreates. systemd-run --scope (without --no-block) blocks and forwards the wrapped command's exit status, so the script's set -euo pipefail and later health-check logic are unaffected by the change. The README addition accurately describes the failure signature (graceful shutdown ~10s in, restart count 1, timeout+SIGABRT, conmon exited prematurely on the next recreate) for anyone diagnosing this later. No correctness or security issues found.

⏱ 1m 1s · 10 turns · tokens: 21.1k in / 4.3k out (+386.2k cache-read)
📜 Full session transcript

✅ **Verdict: approve** — no blocking issues found. Correct, well-scoped fix. Wrapping the `podman-compose up` call in `systemd-run --user --scope --quiet --collect --` moves conmon (and the container it monitors) into a transient scope outside the oneshot updater unit's cgroup, so systemd's cleanup of the finished unit no longer tears down the freshly recreated, healthy container. This matches a known and documented podman/systemd interaction (conmon inheriting the caller's cgroup unless explicitly isolated — see containers/podman issue tracking this exact class of bug), and `--collect` correctly lets the scope go away once conmon (and thus the container) exits, so nothing leaks across future recreates. `systemd-run --scope` (without `--no-block`) blocks and forwards the wrapped command's exit status, so the script's `set -euo pipefail` and later health-check logic are unaffected by the change. The README addition accurately describes the failure signature (graceful shutdown ~10s in, restart count 1, timeout+SIGABRT, `conmon exited prematurely` on the next recreate) for anyone diagnosing this later. No correctness or security issues found. --- ⏱ 1m 1s · 10 turns · tokens: 21.1k in / 4.3k out (+386.2k cache-read) 📜 [Full session transcript](https://git.grandiras.net/Grandiras/claude-bot/src/branch/claude-logs/transcripts/Grandiras/Ponente/pr-222/run-571.md)

Grandiras merged commit fd86933204 into main

2026-09-22 23:08:04 +00:00

Grandiras deleted branch fix-updater-conmon-scope

2026-09-22 23:08:04 +00:00

Grandiras referenced this pull request from a commit

2026-09-22 23:08:06 +00:00

Merge pull request 'The updater recreates the app in a scope of its own, so it starts once' (#222) from fix-updater-conmon-scope into main

Grandiras referenced this pull request

2026-09-23 10:06:23 +00:00

The e2e suite, 8.3 minutes to 6.8, with every assertion kept #221