fix(lexicon)!: wire fidelity against upstream lexicons, enforced by a drift test

Grandiras commented

2026-09-24 12:39:06 +00:00

Owner

Part of the v0.7.0 update cycle (wave 4). The hand-written Lexicon models have drifted from upstream: wrong JSON names, an endpoint that never existed, and 7 invented permission sets. The root fix is a drift test against a pinned upstream snapshot, with everything it finds fixed.

Drift test

Vendor a pinned snapshot of the upstream lexicons into tests/: bluesky-social/atproto lexicons/ at a recorded commit, plus the site.standard.* schemas from the network. Add a script or README for refreshing it.
Tests assert:
- every NSID the SDK calls exists upstream with the right kind (query → GET, procedure → POST);
- every [JsonPropertyName] on a Lexicon model exists on its upstream def, with an explicit allow-list;
- every permission-set constant resolves to a published permission set;
- every knownValues constant class matches upstream.
One reflection convention test (camelCase [JsonPropertyName] on every public Lexicon property) replaces about 100 DTO-only serializer tests.

Fixes the drift test demands (known today)

GetConvoAvailabilityResponse.canConvo → canChat (+ convo); ListConvosAsync(readOnly) → readState; ProfileViewDetailed.associatedChat → the typed associated (profileAssociated); remove ConvoView.opened; AcceptConvoResponse is {rev} only.
tools.ozone.moderation.querySubjects never existed: implement queryStatuses (all params, subjectStatuses, boolean takendown/appealed).
tools.ozone.signature.searchAccounts is a GET query with values[]; AccountResult → admin.defs#accountView.
server.deactivateAccount and chat.bsky.convo.updateAllRead send their JSON bodies (deleteAfter, status); updateAllRead returns updatedCount. chat.bsky.actor.exportAccountData returns JSONL; don't deserialize it as JSON.
Deprecations:
- drop listNotifications seenAt, which "will cause an error" since 2026-09-21;
- [Obsolete] on priority, isFallback, sync.notifyOfUpdate and CommitEvent.TooBig/Rebase/Blobs;
- remove the firehose #handle / #tombstone events, removed upstream in 2025.
AtProtoScopes.PermissionSets: remove the 7 invented NSIDs and add authDeleteContent, authManageNotifications, chat.bsky.authFullChatClient, site.standard.authFull and site.standard.authSocial. Pin the list to the snapshot.
Well-known label values: add !hide, !warn and bot; [Obsolete] gore; fix NotAvailable.

Missing fields on existing models

Everything in the lexicon research §3 and §5, including:

posts: bookmarkCount, debug; viewer bookmarked, knownLikers; FeedViewPost.reqId and a typed reason union (#reasonRepost, #reasonPin); threadContext; a typed blockedAuthor; generator contentMode; searchPosts tag[]; typed threadgate/postgate rules; typed embed.record view variants; video presentation; the new external-embed fields.
profiles: views gain pronouns, associated, verification, status, debug, website, joinedViaStarterPack. Viewer gains mutedOnlyReposts/mutedOnlyQuoteposts/activitySubscription and typed mutedByList/blockingByList. Typed preferences (the 16 #*Pref types with an unknown fallback). recId/recIdStr.
notifications: starterPack, reasons[], all 13 reason constants, registerPush.ageRestricted.
graph: a typed relationships union; subjectOptedOut, referenceListOptOut; muteActor onlyReposts/onlyQuoteposts; followers/follows sort; getLists purposes[]; typed starter-pack feeds.
com.atproto and the rest:
- createSession.allowTakendown, describeServer.blobUploadLimit, createReport.modTool;
- labeler service fields, video job failureCode and the 4 new states;
- Ozone ModerationSubject convoRef/messageRef, modEventTag.durationInHours, emitEvent.reportAction;
- the site.standard record fields.

CHANGELOG Fixed (every wrong name, method or body), Breaking changes (renames and removals), Added (fields).

Part of the v0.7.0 update cycle (wave 4). The hand-written Lexicon models have drifted from upstream: wrong JSON names, an endpoint that never existed, and 7 invented permission sets. The root fix is a **drift test against a pinned upstream snapshot**, with everything it finds fixed. ## Drift test - [ ] Vendor a pinned snapshot of the upstream lexicons into `tests/`: `bluesky-social/atproto` `lexicons/` at a recorded commit, plus the `site.standard.*` schemas from the network. Add a script or README for refreshing it. - [ ] Tests assert: - every NSID the SDK calls exists upstream with the right kind (query → GET, procedure → POST); - every `[JsonPropertyName]` on a Lexicon model exists on its upstream def, with an explicit allow-list; - every permission-set constant resolves to a published permission set; - every knownValues constant class matches upstream. - [ ] One reflection convention test (camelCase `[JsonPropertyName]` on every public Lexicon property) replaces about 100 DTO-only serializer tests. ## Fixes the drift test demands (known today) - [ ] `GetConvoAvailabilityResponse.canConvo` → `canChat` (+ `convo`); `ListConvosAsync(readOnly)` → `readState`; `ProfileViewDetailed.associatedChat` → the typed `associated` (`profileAssociated`); remove `ConvoView.opened`; `AcceptConvoResponse` is `{rev}` only. - [ ] `tools.ozone.moderation.querySubjects` never existed: implement `queryStatuses` (all params, `subjectStatuses`, boolean `takendown`/`appealed`). - [ ] `tools.ozone.signature.searchAccounts` is a GET query with `values[]`; `AccountResult` → `admin.defs#accountView`. - [ ] `server.deactivateAccount` and `chat.bsky.convo.updateAllRead` send their JSON bodies (`deleteAfter`, `status`); `updateAllRead` returns `updatedCount`. `chat.bsky.actor.exportAccountData` returns JSONL; don't deserialize it as JSON. - [ ] Deprecations: - drop `listNotifications` `seenAt`, which "will cause an error" since 2026-09-21; - `[Obsolete]` on `priority`, `isFallback`, `sync.notifyOfUpdate` and `CommitEvent.TooBig`/`Rebase`/`Blobs`; - remove the firehose `#handle` / `#tombstone` events, removed upstream in 2025. - [ ] `AtProtoScopes.PermissionSets`: remove the 7 invented NSIDs and add `authDeleteContent`, `authManageNotifications`, `chat.bsky.authFullChatClient`, `site.standard.authFull` and `site.standard.authSocial`. Pin the list to the snapshot. - [ ] Well-known label values: add `!hide`, `!warn` and `bot`; `[Obsolete]` `gore`; fix `NotAvailable`. ## Missing fields on existing models Everything in the lexicon research §3 and §5, including: - **posts:** `bookmarkCount`, `debug`; viewer `bookmarked`, `knownLikers`; `FeedViewPost.reqId` and a **typed** `reason` union (`#reasonRepost`, `#reasonPin`); `threadContext`; a typed `blockedAuthor`; generator `contentMode`; `searchPosts` `tag[]`; typed threadgate/postgate rules; typed `embed.record` view variants; video `presentation`; the new external-embed fields. - **profiles:** views gain `pronouns`, `associated`, `verification`, `status`, `debug`, `website`, `joinedViaStarterPack`. Viewer gains `mutedOnlyReposts`/`mutedOnlyQuoteposts`/`activitySubscription` and typed `mutedByList`/`blockingByList`. Typed preferences (the 16 `#*Pref` types with an unknown fallback). `recId`/`recIdStr`. - **notifications:** `starterPack`, `reasons[]`, all 13 reason constants, `registerPush.ageRestricted`. - **graph:** a typed relationships union; `subjectOptedOut`, `referenceListOptOut`; `muteActor` `onlyReposts`/`onlyQuoteposts`; followers/follows `sort`; `getLists` `purposes[]`; typed starter-pack `feeds`. - **com.atproto and the rest:** - `createSession.allowTakendown`, `describeServer.blobUploadLimit`, `createReport.modTool`; - labeler service fields, video job `failureCode` and the 4 new states; - Ozone `ModerationSubject` `convoRef`/`messageRef`, `modEventTag.durationInHours`, `emitEvent.reportAction`; - the `site.standard` record fields. CHANGELOG `Fixed` (every wrong name, method or body), `Breaking changes` (renames and removals), `Added` (fields).

Grandiras referenced this issue

2026-09-24 12:39:33 +00:00

Tracking: v0.7.0 update cycle — protocol catch-up, optimization, debloat, API reshaping #141

Grandiras referenced this issue

2026-09-24 23:09:15 +00:00

feat!: typed identifiers, AtDatetime and enumerators for app.bsky (2/4 of #119) #155

Grandiras referenced this issue

2026-09-24 23:10:09 +00:00

feat!: typed identifiers, AtDatetime and enumerators for chat.bsky, tools.ozone and site.standard (3/4 of #119) #156

Grandiras commented

2026-09-24 23:10:10 +00:00

Author

Owner

Wire bugs found during the #119 type conversions (parts 2 and 3). JSON names and methods were deliberately left unchanged there, so add these to this issue's scope:

app.bsky (#155)

SearchPostsAsync tag is a single string, but the lexicon defines an array.
Missing params: listNotifications reasons[], getLists purposes[].
Missing fields:
- via on the like and repost records;
- 4 fields on the labeler service record;
- Relationship blocking / blockedBy / blockingByList / blockedByList.
StandardLabelValues.NotAvailable duplicates NoUnauthenticated.

chat.bsky / tools.ozone (#156)

Wrong response shapes, which fail against a real chat service:
- MuteConvoAsync, UnmuteConvoAsync and UpdateReadAsync read a ConvoView, but the server sends {convo: …}.
- AddReactionAsync / RemoveReactionAsync read a MessageView, but the server sends {message: …}.
RecordViewDetail.BlobCids should be blobs.
modEventLabel / modEventTag arrays that Ozone requires are optional here.
Missing: verifierDid, relatedProfiles, and several queryEvents / querySets / listMembers filters.

Wire bugs found during the #119 type conversions (parts 2 and 3). JSON names and methods were deliberately left unchanged there, so add these to this issue's scope: **app.bsky (#155)** - `SearchPostsAsync` `tag` is a single string, but the lexicon defines an array. - Missing params: `listNotifications` `reasons[]`, `getLists` `purposes[]`. - Missing fields: - `via` on the like and repost records; - 4 fields on the labeler service record; - `Relationship` `blocking` / `blockedBy` / `blockingByList` / `blockedByList`. - `StandardLabelValues.NotAvailable` duplicates `NoUnauthenticated`. **chat.bsky / tools.ozone (#156)** - **Wrong response shapes, which fail against a real chat service:** - `MuteConvoAsync`, `UnmuteConvoAsync` and `UpdateReadAsync` read a `ConvoView`, but the server sends `{convo: …}`. - `AddReactionAsync` / `RemoveReactionAsync` read a `MessageView`, but the server sends `{message: …}`. - `RecordViewDetail.BlobCids` should be `blobs`. - `modEventLabel` / `modEventTag` arrays that Ozone requires are optional here. - Missing: `verifierDid`, `relatedProfiles`, and several `queryEvents` / `querySets` / `listMembers` filters.