refactor!: debloat shared primitives: one verify helper, one cache, one bounded reader #185
No reviewers
Labels
No labels
breaking-change
bug
documentation
duplicate
enhancement
good first issue
help wanted
performance
question
wontfix
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Grandiras/ATProto.NET!185
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "debloat-d-primitives"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Part of #180 (pass D: shared primitives).
One way to do each shared thing: verify a signature against a DID-published key, read a bounded body, cache, read optional JSON members, send an XRPC call. Behaviour is unchanged except where noted below; every regression, security and refresh test passes unchanged or retargeted.
What changed
V1 + B[7]: one verify-with-refresh path
DidResolverExtensions.VerifyWithRefreshAsync(did, selectKey, verify)(internal,Identity/) holds the refetch-once rule and its rationale, once:IDidResolver.RefreshAsync, whichCachingDidResolverrate-limits per DID;RepoSyncVerifierGetSigningKey()(#atproto)LabelVerifier#atproto_labelviaTryGetVerificationKey == FoundIssuerUnresolvedServiceAuthVerifierkidBadJwtIssSpaceDidResolution.VerifyTokenAsync(credentials, delegation tokens)kidfragment, absent or malformed refused at once; only a signature failure refetchesNotAuthorizedfor any resolver exception, as beforeSpaceSyncer#atproto, refused at once when unusableAtProtoCrypto.TryVerifySignature/TryVerifyJwtSignature(internal) replace the copied catch lists inRepoSyncVerifier,LabelSigning,ServiceAuthVerifier,JsonWebKeyVerifierandSpaceCommit.DidResolverExtensionsTestspin the rule on the helper. The callers' refresh tests are unchanged: rotated key verifies, forged key refused after exactly one refresh, valid token never refreshes, rate limit.K1
DidDocument.GetVerificationKey/GetSigningKeyreturnnullfor key material that does not decode, instead of throwingFormatException. The fourcatch (FormatException)wrappers go.H1 + B[12]: one bounded reader
BoundedContentmoves toHttp/and gains aStreamoverload.SyncClient,RepoFetchers, the Tap webhook andSpaceSyncer.CopyBoundedAsyncgo.RepoFetcherskeeps itsIOExceptionmapping.SpaceSyncerused 32 MiB,RepoFetchers1 MiB; I took the tighter bound.L1(core) + B[16] + L8: one LRU cache, one resolution-cache core
Caching/LruCache<TKey,TValue>(thread-safe) replaces the hand-rolled bounded maps of:CachingDidResolver,DidKeyCacheandSpaceCredentialVerifier, whose per-authority quota stays on top (RemoveLeastRecent);JsonWebKeyVerifier,SpaceClientAttestationVerifier,AtProtoClientFactory, the OAuth metadata cache andDPoPNonceCache.Caching/ResolutionCache<TKey,TValue,TError>holdsCachingDidResolver's logic:RetryAfterafter a failed refresh;CachingLexiconResolverruns on the same core, with differences only in:failureCapacity: null).Task.Run, as the Lexicon cache's did, and now the DID cache's too (review fix 1).LexiconCacheOptionsis kept rather than reusingDidCacheOptions. The defaults differ (1,000 entries and 5 minutes, against 10,000 and 1 hour), and reusing the type would change the public API for no gain.J1, X1, N1, A1
J1.
JsonElementExtensions(GetStringOrNull, newGetInt64OrNull/GetInt32OrNull/GetBooleanOrNull, andTryGetNumericDate) moves toSerialization/. The copies in these go:LexiconSchemaRecord.MainType;The helpers also answer
nullfor a non-object element, where the copies threwInvalidOperationException.X1.
QueryAsync,ProcedureAsync,UploadAsyncand the output-less calls share oneSendAsync<T>for the per-call deadline.BuildUriusesXrpcParams.ToQueryString.N1. Identity service URLs go through
AtProtoHttp.ValidateServiceUrl(…, allowLoopback: false), andIdentityNetworkPolicy.ValidateServiceUrlis removed. The repo fetchers' default client isIdentityNetworkPolicy.CreateClient(false, 5 min).A1.
AtProtoJsonDefaults.FormatTimestamp/NowTimestampare removed.C-Q3
The anonymous-object
QueryAsync<TOut>(Nsid, object, …)overload andXrpcParams.From's reflection path are removed. TheFrom_*formatting tests that still apply (DateTime, local time, identifier types) are retargeted toAdd.Pass A's leftover
SpaceRecordCborcatches throughDagCborDecoder.IsMalformed.Docs (Pass F's rule on this pass's files)
A separate, comment-only commit: plain-prose public summaries sit on one line, and docs on internal and private members become
//comments with the XML stripped. A script checked the diff: 0 non-comment lines changed. The refresh-once rationale now lives only on the helper.Breaking changes (CHANGELOG bullet +
docs/migrating-to-0.7.mdentry each)AtProtoJsonDefaults.FormatTimestamp,NowTimestamp(shipped in 0.6)AtDatetime.FromDateTime(dt).ToString(),AtDatetime.Now().ToString()DidDocument.GetSigningKey/GetVerificationKeyno longer throwFormatException(shipped in 0.6)TryGetVerificationKeyto tell malformed from absentQueryAsync<TOut>(Nsid, object, …)(0.7-only; the 0.6 anonymous-object form was already a breaking bullet)XrpcParams. The #122 bullet and the migration entry are amended.PublicAPI.Shipped.txt: 3 lines removed.Unshipped.txtis unchanged.Other bullets:
FormatTimestamp, one is reworded without the name, and the other, a fix to the removed method alone, is dropped.Behaviour changes worth knowing
SpaceCommitVerifier.Verifyreturnsfalseinstead of throwing for a key on a curve the platform lacks, which is the ruleRepoSyncVerifieralready had.SpaceSyncerreports a missing or broken#atprotokey as "publishes no usable AT Protocol signing key", one message instead of two.IdentityNetworkPolicy's HTTPS check now runs after the query/fragment check, so a URL with both problems reports the query.Verification
dotnet build -c Release: 0 warnings, 0 errors (full solution, doc snippets included).dotnet test tests/ATProtoNet.Tests/ -c Release: 5003 passed, 0 failed, 5 skipped, run twice. The count onorigin/mainwas 5000/0/5.XrpcParams.Fromand anonymous-objectQueryAsynctests, and the timestamp tests thatAtDatetimeTestsalready covers.SpaceSyncerTests), K1, andAddforDateTimeand identifiers.samples/ManagedPdsSample.AppHost, thenATPROTO_REQUIRE_INTEGRATION=1 … --filter AspireManifest: 6 passed.Line counts (
loc.py origin/main HEAD, origin/main =199b7d1)src/totalCaching/)//)src/ATProtoNet/Identitysrc/ATProtoNet/Httpsrc/ATProtoNet.Server/Spacestests/ATProtoNet.TestsReview fixes
Seven commits on top of
a03171b, rebased ontoorigin/main(d8a7404, #186). Each code fix has a regression test, and I checked that each test fails when run against the code before the fix.1518ff0).ResolutionCache.StartFetchran the fetch on the first caller's thread andSynchronizationContext. A resolver that awaits withoutConfigureAwait(false)then stalled for every caller that joined it if that context stopped running.Task.Runnow.CachingLexiconResolverdid that on main;CachingDidResolverdid not, so it gets a Fixed bullet.ResolutionCacheTests.GetAsync_FirstCallerOnAContextThatNeverRuns_DoesNotStallTheSharedFetch. It timed out without the fix.Content-Length(ccdf133).declaredLength: null; outbound responses keep the hint.ReadBodyAsync_DeclaredLength_ReservesNothingBeforeTheBytesArrive. It measures the backing array: 1,048,577 bytes before the fix, 16 KiB after.UploadAsync<T>threw synchronously on bad arguments (fbc8da9). It awaits again, so an invalid argument faults the returned task, as on main.UploadAsync_InvalidArguments_FaultTheTaskRatherThanThrowing.ea3f5d9).TryPeek, then wrote withSet, so a fetch landing in between could be overwritten by the stale marker.LruCache.AddOrUpdatedoes the update in one step under the cache's lock, asConcurrentDictionary.AddOrUpdatedid on main.7c1005c).{Did}and the "distributed DID document cache" warnings;{Nsid}.CachingDidResolverno longer logs failed refreshes.ResolverLoggingTestscheck the templates, the property names, and thatCachingDidResolverlogs nothing on a failed refresh.de584b5).SpaceTokenKeyIdTests: an absent or malformed#atproto_spacekey is refused withRefreshCount == 0.RepoSyncVerifier: "publishes no usable atproto signing key" after exactly one refetch, for an absent key and for an undecodable one.SpaceSyncer: refuses both of those without a refetch.16dd23f).LruCachereads aConcurrentDictionarywithout the lock. A hit moves its entry to the front only when the lock is free, so eviction is exact LRU single-threaded and approximate under contention.TryGetValueon a warm 1,024-entry cache, 12 logical CPUs, best of 5. The figures are aggregate wall-clock ns per get:ConcurrentDictionary(main)LruCacheConcurrentDictionary(one key), and 29 ns against 14 (random keys).DidKeyCache.Sharedgains lock-free hits too; it was lock-based on main.After the fixes:
Items 3–5 and 7 restore main's behaviour, so they have no CHANGELOG entry.
Deferred
C-Q6 (
PdsAdminClientthroughIXrpcSessionHandler) is not done, because it would change behaviour.XrpcClientasks the session handler to recover only onExpiredTokenor aWWW-Authenticate: … invalid_tokenchallenge.PdsAdminClientsigns in again on any 401. Tranquil answers a revoked or invalidated session with 401AuthenticationFailed, which the handler path would no longer recover from.Admin/Serverclients.Pass E's files are untouched. No Lexicon client test needed a call-site fix.
🤖 Generated with Claude Code
🤖 Reviewing this PR — the review will link the full session transcript.
📜 Live transcript — updating every ~5 minutes while the review runs.
⏳ Held on the OpenCode Go window — the 5-hour window is at 0% (the bot holds at 95%).
This continues automatically after the window resets (2026-09-27T22:13:23.362Z) — nothing to re-trigger.
@claude /review, please. This is debloat pass D of #180 (shared primitives), and it is security-relevant: the verify-with-refresh helper sits on every signature-verification path. The diff will be truncated, so please read these files in full from the branch:
DidResolverExtensions.VerifyWithRefreshAsync, and its use inRepoSyncVerifier,LabelVerifier,ServiceAuthVerifier,SpaceDidResolutionandSpaceSyncer. Check that each caller keeps its own key-id rule and resolution-failure mapping, refetches only on a missing key or failed signature, retries only on a changed key, and stays under the per-DID rate limit;AtProtoCrypto.TryVerifySignature/TryVerifyJwtSignature;Caching/LruCacheandResolutionCache, which now backCachingDidResolverandCachingLexiconResolver. Check stale-while-revalidate, the negative TTL, single-flight and detach-on-invalidate;SpaceCredentialVerifier's per-authority quota;DidDocument.GetSigningKey/GetVerificationKeyreturning null for a malformed key;XrpcClientsend path.The last commit,
e095384, only changes comments.🤖 Reviewing this PR — follow along in the action run; the reply will link the full session transcript.
📜 Live transcript — updating every ~5 minutes while the task runs.
💬 Verdict: comment — notes only, nothing blocking.
Large, well-organized refactor. The parts of the diff I can check are internally consistent:
SpaceDidResolution.VerifyTokenAsyncreproduces the old refetch-once/retry-only-if-key-changed semantics ofVerifyWithKeyRefreshAsyncthrough the new sharedDidResolverExtensions.VerifyWithRefreshAsynchelper,SpaceCredentialVerifier's LRU-backed cache and per-authority quota logic look equivalent to the old linked-list version, and the K1 change (GetVerificationKeyreturningnullinstead of throwingFormatException) is correctly threaded through call sites that used to catchFormatException(e.g.SpaceCredentialVerifier.IsKeyCurrentAsync,ServiceAuthVerifier). CHANGELOG and migration-guide entries are present as required by CLAUDE.md.Note: the PR diff was truncated before I could see the new core files themselves (
Caching/LruCache.cs,Caching/ResolutionCache.cs,Identity/DidResolverExtensions.cs,Http/BoundedContent.cs,Crypto/AtProtoCrypto.cs), so I could only verify caller-side consistency with the PR description's stated rules, not the helpers' internals.One minor concurrency regression worth a look (see inline finding) in
SpaceClientAttestationVerifier.FetchKeysAsync.⏱ 1m 17s · 1 turns · tokens: 76.4k in / 8.6k out (+23.9k cache-read)
📜 Full session transcript
This replaces the previous atomic
ConcurrentDictionary.AddOrUpdatewith a non-atomicTryPeek+Setpair. Two concurrent callers can both read the sameexistingvalue and then both callSet, so one thread'sAttemptedAtupdate can be silently lost (or a freshClientKeys(null, now, now)can overwrite a value that just got theAttemptedAtbump from another thread). The class doc forLruCachesays the cache itself is thread-safe, but that only makes each individualTryPeek/Setcall atomic, not this check-then-act sequence — the same class of raceAddOrUpdateexisted specifically to avoid. Impact is limited (worst case an extra JWKS refetch within the 30s window rather than a security issue), but it's a real behavior change from the atomic original that isn't called out in the PR description.a03171b7f63c48b0a42b@claude /review, please re-review. The branch is rebased onto main (#186, test-only, no conflicts). New commits fix an independent adversarial review and your note:
SynchronizationContextcan no longer deadlock them. There is a test with a context that is never pumped, and a Fixed bullet, becauseCachingDidResolverhad the same problem on main.Content-Length.UploadAsync<T>faults its task on bad arguments again.LruCache.AddOrUpdate. The test drives the race deterministically.I then added a concurrent stress test of the lock-free path. It caught
Countbriefly one over capacity during a write, and3c48b0afixes that by evicting before adding.Please read
src/ATProtoNet/Caching/LruCache.csin full, especially the lock-freeTryGetValue: thenode.Listrecheck under the lock, and the claim that published nodes are immutable. Please also read the fix commits.🤖 Reviewing this PR — follow along in the action run; the reply will link the full session transcript.
📜 Live transcript — updating every ~5 minutes while the task runs.
✅ Verdict: approve — no blocking issues found.
This is a re-review after new commits (the "review fixes" batch described in the PR body). My previous pass flagged one concrete issue:
SpaceClientAttestationVerifier.FetchKeysAsynchad replaced the old atomicConcurrentDictionary.AddOrUpdatewith a non-atomicTryPeek+Setpair, which could lose anAttemptedAtupdate under concurrent callers.That is now fixed (review fix 4, commit
ea3f5d9): the code now calls_clientKeys.AddOrUpdate(clientId, new ClientKeys(null, now, now), existing => existing with { AttemptedAt = now }), andLruCache.AddOrUpdateperforms the read-modify-write in one step under the cache's lock, matchingConcurrentDictionary.AddOrUpdate's atomicity. The PR description also notes this exact class of bug was caught independently inSpaceClientAttestationVerifier's sibling, the attestation key cache (ea3f5d9), with a deterministic-interleaving regression test — consistent with what I flagged.Spot-checking the rest of the new/changed logic visible in the diff:
SpaceDidResolution.VerifyTokenAsynccorrectly threads key selection (throwingSpaceVerificationExceptionfor absent/malformed key) and verification (catchingSpaceTokenExceptionwhenIsSignatureFailure) through the new sharedDidResolverExtensions.VerifyWithRefreshAsync, and only wrapsSpaceTokenExceptionfrom the outer call, lettingSpaceVerificationExceptionfrom key selection propagate unchanged — matches the oldVerifyWithKeyRefreshAsyncsemantics.AtProtoClientFactory'sLruCache-backed DPoP key cache correctly drops the manualLastUsed/EvictLeastRecentlyUsedbookkeeping now that the cache tracks recency itself, andForgetKey'sTryPeek+Remove(key, value)pair is safe since removal is conditioned on value equality.SpaceCredentialVerifier's cache now wraps an optionalLruCache(null when capacity ≤ 0), andLookup/Remember/Forgetare equivalent to the prior linked-list implementation, including the per-authority quota eviction viaRemoveLeastRecent.No new issues found in the visible diff (the full diff was truncated again past
SpaceServerExtensions.cs, as last time, so the newest-added core files aren't directly visible here, but the caller-side usage is consistent with the description). Approving.⏱ 0m 32s · 2 turns · tokens: 80.8k in / 2.5k out (+127.0k cache-read)
📜 Full session transcript
🤖 Reviewing this PR — the review will link the full session transcript.
📜 Live transcript — updating every ~5 minutes while the review runs.
⏳ Held on the OpenCode Go window — the 5-hour window is at 0% (the bot holds at 95%).
This continues automatically after the window resets (2026-09-27T22:53:49.031Z) — nothing to re-trigger.
Supplementary check of
src/ATProtoNet/Caching/LruCache.csat3c48b0a. The claude diff was truncated before this file, so I read it in full myself.Set,AddOrUpdate,GetOrAdd,Remove*,RemoveLeastRecent) runs under_lock. The map and the recency list are only ever changed together, under that lock.TryGetValuereads the node'sKeyValuePairfrom a node that is never changed after it is published: a new value always gets a new node. It only moves the node whenTryEntersucceeds, and re-checksnode.List is not nullfirst, so an entry that was evicted or replaced in the meantime is never re-inserted.ConcurrentUse_KeepsTheBoundAndLeavesACacheThatStillEvictsInOrderruns 16 threads × 20k mixed operations. It checks the bound on every step, then that a single-threaded refill evicts in exact order, which proves the map and list still agree. It found the add-then-evict transient that3c48b0afixes. It has passed 5 out of 5 runs since then.Merging on the approval of this head.