Pds: provide a persistent store package (ATProtoNet.Pds.EntityFrameworkCore) for IAccountStore/IRepoStore #38

New issue

Closed

opened 2026-06-10 22:46:51 +00:00 by Grandiras · 2 comments

Grandiras commented

2026-06-10 22:46:51 +00:00

Owner

ATProtoNet.Pds ships only InMemoryAccountStore / InMemoryRepoStore — fine for development, but every production consumer has to hand-roll persistent IAccountStore/IRepoStore implementations.

For Updraft we wrote Postgres-backed stores over EF Core (DbAccountStore, DbRepoStore in https://git.grandiras.net/Grandiras/Updraft, src/Updraft.Pds/Stores) — entities mirroring PdsAccount/RepoRecord/RepoBlob, cursor pagination on rkey, content-addressed blob dedup.

Suggestion: an ATProtoNet.Pds.EntityFrameworkCore package (parallel to the existing ATProtoNet.Server.EntityFrameworkCore token store): a PdsDbContext (or model-configuration hook for an existing context), EF-backed store implementations, and an AddAtProtoPdsEfCoreStores<TContext>() registration. Our implementation can serve as a starting point.

One design note from our build: if columns like handle/email are encrypted at rest with a non-deterministic scheme, equality lookups (GetByHandleAsync) must happen in memory — worth keeping the store contract documented as "implementations may load-and-filter".

`ATProtoNet.Pds` ships only `InMemoryAccountStore` / `InMemoryRepoStore` — fine for development, but every production consumer has to hand-roll persistent `IAccountStore`/`IRepoStore` implementations. For Updraft we wrote Postgres-backed stores over EF Core (`DbAccountStore`, `DbRepoStore` in https://git.grandiras.net/Grandiras/Updraft, src/Updraft.Pds/Stores) — entities mirroring `PdsAccount`/`RepoRecord`/`RepoBlob`, cursor pagination on rkey, content-addressed blob dedup. Suggestion: an `ATProtoNet.Pds.EntityFrameworkCore` package (parallel to the existing `ATProtoNet.Server.EntityFrameworkCore` token store): a `PdsDbContext` (or model-configuration hook for an existing context), EF-backed store implementations, and an `AddAtProtoPdsEfCoreStores<TContext>()` registration. Our implementation can serve as a starting point. One design note from our build: if columns like handle/email are encrypted at rest with a non-deterministic scheme, equality lookups (`GetByHandleAsync`) must happen in memory — worth keeping the store contract documented as "implementations may load-and-filter".

Grandiras referenced this issue

2026-06-11 23:16:17 +00:00

Pds: federation support (PLC registration, MST/signed commits/CAR, com.atproto.sync + firehose, handle resolution) #40

claude-bot was assigned by Grandiras

2026-07-25 14:36:39 +00:00

claude-bot commented

2026-07-25 14:36:44 +00:00

Collaborator

🤖 On it — follow along in the action run; the reply will link the full session transcript.

📜 Live transcript — updating every ~5 minutes while the task runs.

🤖 On it — follow along in [the action run](https://git.grandiras.net/Grandiras/claude-bot/actions/runs/51); the reply will link the full session transcript. 📜 [Live transcript](https://git.grandiras.net/Grandiras/claude-bot/src/branch/claude-logs/transcripts/Grandiras/ATProto.NET/issue-38/run-51.md) — updating every ~5 minutes while the task runs.

claude-bot referenced this issue from a commit

2026-07-25 14:47:12 +00:00

feat: EF Core-backed IAccountStore/IRepoStore for the PDS (closes #38)

claude-bot referenced this issue from a pull request that will close it,

2026-07-25 14:51:54 +00:00

Claude: changes for issue #38 #62

claude-bot commented

2026-07-25 14:55:20 +00:00

Collaborator

Implemented on branch claude/issue-38 (compare).

One deviation from the suggestion, deliberate

The issue asks for a separate ATProtoNet.Pds.EntityFrameworkCore package, parallel to ATProtoNet.Server.EntityFrameworkCore. That parallel no longer holds: issue #33 merged ATProtoNet.Server.EntityFrameworkCore (and ATProtoNet.Aspire) into ATProtoNet.Server, cutting the published set from 8 packages to 6. So the stores ship inside ATProtoNet.Pds, under the ATProtoNet.Pds.EntityFrameworkCore namespace and a Stores/EntityFrameworkCore/ folder — exactly the shape the token store has today. Consumers get the same API you'd have imported, just without a second <PackageReference>. ATProtoNet.Pds now depends on Microsoft.EntityFrameworkCore.Relational; you pick the provider package.

What's there

AddAtProtoPdsEfCoreStores<TContext>(Action<PdsEfCoreStoreOptions>?) — registers both stores over an IDbContextFactory<TContext> and replaces the in-memory defaults. Order-independent relative to AddAtProtoPds(): the extension removes any prior registration, and AddAtProtoPds() now uses TryAddSingleton for its in-memory fallbacks (small behaviour change, noted under Changed).
PdsDbContext — standalone context, plus PdsDbContext.ConfigurePdsModel(modelBuilder) as the model-configuration hook for folding the four entities into an existing application context.
EfCoreAccountStore<TContext> — case-insensitive handle/email lookups, matching InMemoryAccountStore; translated as LOWER(col) = @p.
EfCoreRepoStore<TContext> — keyset (seek) pagination on rkey with the same exclusive-cursor contract as InMemoryRepoStore, and content-addressed blob dedup: bytes in one PdsBlobs row per CID, a PdsBlobRefs row per uploading account carrying its own MIME type. Deleting drops the caller's reference; the shared content goes only when the last reference does, so one account can't destroy another's copy. DeleteAllAsync deletes by key stub, so record values and blob bytes are never loaded just to be deleted.
Your encryption note, acted on both ways — IAccountStore's XML docs now state that only GetByDidAsync is guaranteed keyed and that GetByHandleAsync/GetByEmailAsync/HandleExistsAsync may load and filter; and PdsEfCoreStoreOptions.ClientSideAccountLookup implements that path (streams accounts, compares post-decryption), with MaxClientSideLookupRows to bound the scan.
docs/pds.md gains a Persistent Storage (EF Core) section (setup, own-DbContext, encrypted columns); the old Custom Store Implementations sample was using a non-existent AccountInfo type, now corrected to PdsAccount. CHANGELOG updated under [Unreleased].

Verified

Solution builds clean (0 errors, no new warnings from the Pds project). 1095/1095 unit tests pass, including 62 new ones. The store suite is an abstract base run against two providers: the EF in-memory provider and SQLite — the SQLite run is what proves Handle.ToLower() and the Rkey.CompareTo(cursor) cursor predicate actually translate to SQL rather than being silently client-evaluated (added Microsoft.EntityFrameworkCore.Sqlite to the test project only). One test pages the EF store and InMemoryRepoStore side by side, forwards and reversed, asserting identical rkeys and cursors at every step.

Not done: no migrations are checked in (the schema depends on your provider), and PdsSample still uses the in-memory stores.

→ Opened PR: #62

⏱ 10m 53s · 56 turns · 📜 Full session transcript

(reply recovered from the transcript — the posting step in runs before v12.3 was broken by a shell-quoting bug)

Implemented on branch **`claude/issue-38`** ([compare](https://git.grandiras.net/Grandiras/ATProto.NET/compare/main...claude/issue-38)). ## One deviation from the suggestion, deliberate The issue asks for a separate `ATProtoNet.Pds.EntityFrameworkCore` **package**, parallel to `ATProtoNet.Server.EntityFrameworkCore`. That parallel no longer holds: issue #33 merged `ATProtoNet.Server.EntityFrameworkCore` (and `ATProtoNet.Aspire`) *into* `ATProtoNet.Server`, cutting the published set from 8 packages to 6. So the stores ship **inside `ATProtoNet.Pds`**, under the `ATProtoNet.Pds.EntityFrameworkCore` namespace and a `Stores/EntityFrameworkCore/` folder — exactly the shape the token store has today. Consumers get the same API you'd have imported, just without a second `<PackageReference>`. `ATProtoNet.Pds` now depends on `Microsoft.EntityFrameworkCore.Relational`; you pick the provider package. ## What's there - **`AddAtProtoPdsEfCoreStores<TContext>(Action<PdsEfCoreStoreOptions>?)`** — registers both stores over an `IDbContextFactory<TContext>` and replaces the in-memory defaults. Order-independent relative to `AddAtProtoPds()`: the extension removes any prior registration, and `AddAtProtoPds()` now uses `TryAddSingleton` for its in-memory fallbacks (small behaviour change, noted under *Changed*). - **`PdsDbContext`** — standalone context, plus `PdsDbContext.ConfigurePdsModel(modelBuilder)` as the model-configuration hook for folding the four entities into an existing application context. - **`EfCoreAccountStore<TContext>`** — case-insensitive handle/email lookups, matching `InMemoryAccountStore`; translated as `LOWER(col) = @p`. - **`EfCoreRepoStore<TContext>`** — keyset (seek) pagination on `rkey` with the same exclusive-cursor contract as `InMemoryRepoStore`, and **content-addressed blob dedup**: bytes in one `PdsBlobs` row per CID, a `PdsBlobRefs` row per uploading account carrying its own MIME type. Deleting drops the caller's reference; the shared content goes only when the last reference does, so one account can't destroy another's copy. `DeleteAllAsync` deletes by key stub, so record values and blob bytes are never loaded just to be deleted. - **Your encryption note, acted on both ways** — `IAccountStore`'s XML docs now state that only `GetByDidAsync` is guaranteed keyed and that `GetByHandleAsync`/`GetByEmailAsync`/`HandleExistsAsync` **may load and filter**; and `PdsEfCoreStoreOptions.ClientSideAccountLookup` implements that path (streams accounts, compares post-decryption), with `MaxClientSideLookupRows` to bound the scan. - `docs/pds.md` gains a *Persistent Storage (EF Core)* section (setup, own-DbContext, encrypted columns); the old *Custom Store Implementations* sample was using a non-existent `AccountInfo` type, now corrected to `PdsAccount`. CHANGELOG updated under `[Unreleased]`. ## Verified Solution builds clean (0 errors, no new warnings from the Pds project). **1095/1095 unit tests pass**, including 62 new ones. The store suite is an abstract base run against **two providers**: the EF in-memory provider *and* SQLite — the SQLite run is what proves `Handle.ToLower()` and the `Rkey.CompareTo(cursor)` cursor predicate actually translate to SQL rather than being silently client-evaluated (added `Microsoft.EntityFrameworkCore.Sqlite` to the test project only). One test pages the EF store and `InMemoryRepoStore` side by side, forwards and reversed, asserting identical rkeys and cursors at every step. Not done: no migrations are checked in (the schema depends on your provider), and `PdsSample` still uses the in-memory stores. → Opened PR: https://git.grandiras.net/Grandiras/ATProto.NET/pulls/62 --- ⏱ 10m 53s · 56 turns · 📜 [Full session transcript](https://git.grandiras.net/Grandiras/claude-bot/src/branch/claude-logs/transcripts/Grandiras/ATProto.NET/issue-38/run-51.md) _(reply recovered from the transcript — the posting step in runs before v12.3 was broken by a shell-quoting bug)_

Grandiras closed this issue

2026-07-25 16:13:57 +00:00

Grandiras referenced this issue from a commit

2026-07-25 16:13:59 +00:00

Claude: changes for issue #38 (#62)

claude-bot referenced this issue from a commit

2026-07-25 17:19:30 +00:00

merge: main into claude/issue-36

claude-bot referenced this issue

2026-07-25 17:19:45 +00:00

Claude: changes for issue #36 #64