Spaces: durable and multi-instance implementations of the space server stores

claude-bot commented

2026-08-20 21:51:38 +00:00

Collaborator

#91 landed the space server, but every store it defines ships with only an in-memory implementation:

ISpaceReplayStore → InMemorySpaceReplayStore
ISpaceAuthorityStore → InMemorySpaceAuthorityStore
ISimpleSpaceStore → InMemorySimpleSpaceStore

All three are in src/ATProtoNet.Server/Spaces/. That is fine for a single-instance service and for tests, and the XML docs say so, but two of the gaps are more than an inconvenience:

The replay store is a correctness gap across instances. It is what enforces single use on delegation tokens, client attestations, and DPoP proofs. Being per-process means a replay is caught only by the instance that saw the original, so two instances behind a load balancer accept the same delegation token twice. Anyone running more than one replica needs a shared implementation (Redis SET NX with the token's expiry as the TTL is the natural fit, since the store is already keyed on (iss, jti, exp) and needs no read-modify-write). This is worth calling out more loudly than a doc paragraph — possibly by making AddAtProtoSpaces() log a warning when the in-memory default is still registered at startup.

The simplespace member list cannot be rebuilt. Unlike the writer set — which is only what an authority claims, and which any repo host's next notifyWrite restores — a member list is never published to the network, so losing it on a restart loses the space's access control. A durable implementation is not optional for a real authority.

The repo already has the pattern to follow: src/ATProtoNet.Server/TokenStore/EntityFrameworkCore/ implements IAtProtoTokenStore over EF Core (namespace ATProtoNet.Server.EntityFrameworkCore, registered by AtProtoTokenStoreExtensions), and Microsoft.EntityFrameworkCore.Relational is already a dependency of ATProtoNet.Server. EF Core implementations of ISpaceAuthorityStore and ISimpleSpaceStore would sit alongside it with the same shape, including the pagination cursors the in-memory versions implement by ordinal DID ordering.

Worth splitting if it gets large: the replay store is its own concern (and probably wants Redis rather than EF Core), while the authority and simplespace stores are one EF Core change.

Filed by Claude while working on #91 (run).

#91 landed the space server, but every store it defines ships with only an in-memory implementation: - `ISpaceReplayStore` → `InMemorySpaceReplayStore` - `ISpaceAuthorityStore` → `InMemorySpaceAuthorityStore` - `ISimpleSpaceStore` → `InMemorySimpleSpaceStore` All three are in `src/ATProtoNet.Server/Spaces/`. That is fine for a single-instance service and for tests, and the XML docs say so, but two of the gaps are more than an inconvenience: **The replay store is a correctness gap across instances.** It is what enforces single use on delegation tokens, client attestations, and DPoP proofs. Being per-process means a replay is caught only by the instance that saw the original, so two instances behind a load balancer accept the same delegation token twice. Anyone running more than one replica needs a shared implementation (Redis `SET NX` with the token's expiry as the TTL is the natural fit, since the store is already keyed on `(iss, jti, exp)` and needs no read-modify-write). This is worth calling out more loudly than a doc paragraph — possibly by making `AddAtProtoSpaces()` log a warning when the in-memory default is still registered at startup. **The `simplespace` member list cannot be rebuilt.** Unlike the writer set — which is only what an authority claims, and which any repo host's next `notifyWrite` restores — a member list is never published to the network, so losing it on a restart loses the space's access control. A durable implementation is not optional for a real authority. The repo already has the pattern to follow: `src/ATProtoNet.Server/TokenStore/EntityFrameworkCore/` implements `IAtProtoTokenStore` over EF Core (namespace `ATProtoNet.Server.EntityFrameworkCore`, registered by `AtProtoTokenStoreExtensions`), and `Microsoft.EntityFrameworkCore.Relational` is already a dependency of `ATProtoNet.Server`. EF Core implementations of `ISpaceAuthorityStore` and `ISimpleSpaceStore` would sit alongside it with the same shape, including the pagination cursors the in-memory versions implement by ordinal DID ordering. Worth splitting if it gets large: the replay store is its own concern (and probably wants Redis rather than EF Core), while the authority and `simplespace` stores are one EF Core change. --- _Filed by Claude while working on #91 ([run](https://git.grandiras.net/Grandiras/claude-bot/actions/runs/318))._

claude-bot referenced this issue

2026-08-20 21:51:39 +00:00

Spaces: server-side support in ATProtoNet.Server (space authority and repo host) #91

claude-bot was assigned by Grandiras

2026-08-20 21:57:06 +00:00

claude-bot commented

2026-08-20 21:57:10 +00:00

Author

Collaborator

🤖 On it — follow along in the action run; the reply will link the full session transcript.

📜 Live transcript — updating every ~5 minutes while the task runs.

🤖 On it — follow along in [the action run](https://git.grandiras.net/Grandiras/claude-bot/actions/runs/339); the reply will link the full session transcript.  📜 [Live transcript](https://git.grandiras.net/Grandiras/claude-bot/src/branch/claude-logs/transcripts/Grandiras/ATProto.NET/issue-102/run-339.md) — updating every ~5 minutes while the task runs.

claude-bot commented

2026-08-21 01:47:36 +00:00

Author

Collaborator

🤖 On it — follow along in the action run; the reply will link the full session transcript.

📜 Live transcript — updating every ~5 minutes while the task runs.

🤖 On it — follow along in [the action run](https://git.grandiras.net/Grandiras/claude-bot/actions/runs/339); the reply will link the full session transcript.  📜 [Live transcript](https://git.grandiras.net/Grandiras/claude-bot/src/branch/claude-logs/transcripts/Grandiras/ATProto.NET/issue-102/run-339.md) — updating every ~5 minutes while the task runs.

claude-bot referenced this issue from a commit

2026-08-21 02:09:10 +00:00

feat: durable and multi-instance stores for the space server

claude-bot referenced this issue from a pull request that will close it,

2026-08-21 02:09:59 +00:00

Claude: changes for issue #102 #104

claude-bot referenced this issue

2026-08-21 02:10:00 +00:00

A space created through com.atproto.simplespace is never declared to the authority store, so listRepos/registerNotify/notifyWrite answer SpaceNotFound #105

claude-bot commented

2026-08-21 02:10:00 +00:00

Author

Collaborator

Pushed claude/issue-102 — durable, shared implementations for all three space server stores, plus the startup warning the issue asked for.

What landed

RedisSpaceReplayStore (Spaces/Redis/, namespace ATProtoNet.Server.Redis) — consuming a token is one SET key value NX EX ttl, keyed on (iss, jti, exp) exactly as the interface already is, so nothing reads before it writes. The entry's TTL is the token's own remaining lifetime, so nothing sweeps either. StackExchange.Redis 2.13.17 is a new dependency of ATProtoNet.Server: no abstraction in the shared framework offers an atomic set-if-absent, and IDistributedCache in particular does not.

Four EF Core stores (Spaces/EntityFrameworkCore/, alongside the existing token store in ATProtoNet.Server.EntityFrameworkCore): EfCoreSimpleSpaceStore<T>, EfCoreSpaceAuthorityStore<T>, EfCoreSpaceReplayStore<T>, over SpaceDbContext or any context calling ConfigureSpaceModel() (or one of the three narrower configurators). Registered with AddAtProtoEfCoreSimpleSpace<T>(), AddAtProtoEfCoreSpaceAuthority<T>(key), AddAtProtoEfCoreSpaceReplayStore<T>() / AddAtProtoRedisSpaceReplayStore(), all of which Replace the in-process default so call order relative to AddAtProtoSpaces() doesn't matter.

Details worth flagging: the replay store's primary key is the check, and a failed save is confirmed against the table before being reported as a replay — otherwise a value-too-long or a transient fault would silently refuse valid tokens. Policy unions are stored as their Lexicon JSON with discriminators, so a new variant needs no schema change. Pagination stays DID-ordered; the revision comparison in RecordWriteAsync stays ordinal rather than becoming whatever the column collation says. Subscriber expiries go through a Unix-millisecond value converter because SQLite cannot compare a DateTimeOffset column at all.

AddAtProtoSpaces() now reports its stores at startup — a warning for the replay store and the simplespace store while they are the in-process defaults, an informational line for the writer set (self-healing via the next notifyWrite). Suppressed with SpaceServerOptions.WarnOnInMemoryStores = false.

Docs: a new "The stores" section in docs/spaces.md, a cross-link from docs/server.md, CHANGELOG under [Unreleased] → Added.

Verification

Full unit suite: 1697 passed, 5 skipped (the pre-existing Aspire manifest skips), solution builds with 0 warnings. 40 of those tests are new, over SQLite rather than the EF in-memory provider — a key violation being how a replay is detected is not something the in-memory provider models.
PostgreSQL, via a scratch harness against the container's server: pagination across pages, out-of-order revisions, lapsed-subscriber filtering, policy round-trip, the opportunistic sweep, and 16 store instances presenting one token concurrently → exactly one success. Same result on SQLite.
Live Redis: 5 new integration tests behind a [RequiresRedisFact] / ATPROTO_REDIS_URL gate — cross-instance single use, a 16-way race, TTL actually set, and the identifier becoming free again after it expires. They pass against a real server and skip cleanly without the variable, so CI is unaffected.

I also pinned SQLitePCLRaw.bundle_e_sqlite3 3.0.5 in the test project — the version the Sqlite provider pulls carries GHSA-2m69-gcr7-jv3q and surfaced as a restore warning on every build.

One thing I did not do: simplespace and the authority store are still two separate pieces of state, and nothing declares a newly created space to the second one — I filed that separately rather than widening this change.

⏱ 22m 10s · 104 turns · tokens: 218.2k in / 100.5k out (+14912.3k cache-read)
📜 Full session transcript
→ Opened PR: #104
→ Filed follow-up issue #105: A space created through com.atproto.simplespace is never declared to the authority store, so listRepos/registerNotify/notifyWrite answer SpaceNotFound

Pushed **`claude/issue-102`** — durable, shared implementations for all three space server stores, plus the startup warning the issue asked for. ## What landed **`RedisSpaceReplayStore`** (`Spaces/Redis/`, namespace `ATProtoNet.Server.Redis`) — consuming a token is one `SET key value NX EX ttl`, keyed on `(iss, jti, exp)` exactly as the interface already is, so nothing reads before it writes. The entry's TTL is the token's own remaining lifetime, so nothing sweeps either. `StackExchange.Redis` 2.13.17 is a new dependency of `ATProtoNet.Server`: no abstraction in the shared framework offers an atomic set-if-absent, and `IDistributedCache` in particular does not. **Four EF Core stores** (`Spaces/EntityFrameworkCore/`, alongside the existing token store in `ATProtoNet.Server.EntityFrameworkCore`): `EfCoreSimpleSpaceStore<T>`, `EfCoreSpaceAuthorityStore<T>`, `EfCoreSpaceReplayStore<T>`, over `SpaceDbContext` or any context calling `ConfigureSpaceModel()` (or one of the three narrower configurators). Registered with `AddAtProtoEfCoreSimpleSpace<T>()`, `AddAtProtoEfCoreSpaceAuthority<T>(key)`, `AddAtProtoEfCoreSpaceReplayStore<T>()` / `AddAtProtoRedisSpaceReplayStore()`, all of which `Replace` the in-process default so call order relative to `AddAtProtoSpaces()` doesn't matter. Details worth flagging: the replay store's primary key **is** the check, and a failed save is confirmed against the table before being reported as a replay — otherwise a value-too-long or a transient fault would silently refuse valid tokens. Policy unions are stored as their Lexicon JSON with discriminators, so a new variant needs no schema change. Pagination stays DID-ordered; the revision comparison in `RecordWriteAsync` stays ordinal rather than becoming whatever the column collation says. Subscriber expiries go through a Unix-millisecond value converter because SQLite cannot compare a `DateTimeOffset` column at all. **`AddAtProtoSpaces()` now reports its stores at startup** — a warning for the replay store and the `simplespace` store while they are the in-process defaults, an informational line for the writer set (self-healing via the next `notifyWrite`). Suppressed with `SpaceServerOptions.WarnOnInMemoryStores = false`. Docs: a new "The stores" section in `docs/spaces.md`, a cross-link from `docs/server.md`, CHANGELOG under `[Unreleased] → Added`. ## Verification - Full unit suite: **1697 passed, 5 skipped** (the pre-existing Aspire manifest skips), solution builds with 0 warnings. 40 of those tests are new, over SQLite rather than the EF in-memory provider — a key violation being how a replay is detected is not something the in-memory provider models. - **PostgreSQL**, via a scratch harness against the container's server: pagination across pages, out-of-order revisions, lapsed-subscriber filtering, policy round-trip, the opportunistic sweep, and 16 store instances presenting one token concurrently → **exactly one success**. Same result on SQLite. - **Live Redis**: 5 new integration tests behind a `[RequiresRedisFact]` / `ATPROTO_REDIS_URL` gate — cross-instance single use, a 16-way race, TTL actually set, and the identifier becoming free again after it expires. They pass against a real server and skip cleanly without the variable, so CI is unaffected. I also pinned `SQLitePCLRaw.bundle_e_sqlite3` 3.0.5 in the test project — the version the Sqlite provider pulls carries GHSA-2m69-gcr7-jv3q and surfaced as a restore warning on every build. One thing I did **not** do: `simplespace` and the authority store are still two separate pieces of state, and nothing declares a newly created space to the second one — I filed that separately rather than widening this change. --- ⏱ 22m 10s · 104 turns · tokens: 218.2k in / 100.5k out (+14912.3k cache-read) 📜 [Full session transcript](https://git.grandiras.net/Grandiras/claude-bot/src/branch/claude-logs/transcripts/Grandiras/ATProto.NET/issue-102/run-339.md) → Opened PR: https://git.grandiras.net/Grandiras/ATProto.NET/pulls/104 → Filed follow-up issue [#105](https://git.grandiras.net/Grandiras/ATProto.NET/issues/105): A space created through com.atproto.simplespace is never declared to the authority store, so listRepos/registerNotify/notifyWrite answer SpaceNotFound