fix(identity)!: spec-conformant identifier types and a monotonic TID generator #146
No reviewers
Labels
No labels
breaking-change
bug
documentation
duplicate
enhancement
good first issue
help wanted
performance
question
wontfix
No milestone
No project
No assignees
5 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Grandiras/ATProto.NET!146
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "issue-114-identifiers"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Closes #114.
The identifier types are meant to guarantee validity, but several of them accepted input the atproto specs reject, and
Tid.Next()produced out-of-order and duplicate TIDs. This PR makes parsing spec-conformant (checked against the official interop fixtures), adds a monotonicTidGenerator, and replaces the hand-written value-type boilerplate with sealed records, one parse interface and one JSON converter.Bugs
Tid.Next()monotonic and collision-free. The newTidGenerator(int? clockId = null, TimeProvider? timeProvider = null)fixes one 10-bit clock id per generator (random by default,RandomNumberGenerator) and advances a microsecond timestamp with a lock-free CAS loop:next = max(now, last + 1). It never repeats and never goes backwards, even when the clock stalls or steps back. It throwsInvalidOperationExceptiononce the timestamp would overflow 53 bits.Tid.Next(),Tid.NextString()andRecordKey.NewTid()use one internal shared instance.bluesky-social/atproto-interop-testssyntax/is vendored verbatim at commit056e5741bb330757205d6b16db5266fffcae937b(CC0) intotests/ATProtoNet.Tests/Interop/syntax/, with a README recording the origin, commit and license. A nested.editorconfigstops editors from trimming the whitespace, which is part of the test data.SyntaxInteropTestsasserts every valid line parses, keeping its text, and every invalid line is rejected by bothTryParseandParse. It covers TID, AT URI, DID, NSID, handle, record key, AT identifier and the invalid CIDs. The parser fixes:Tid: the first character must be in234567abcdefghij(the high bit stays clear).RecordKey:^[A-Za-z0-9._:~-]{1,512}$, excluding.and...Did: the final character may no longer be%.Nsid: the spec's reference regex, which limits the name to 63 characters.AtUri: the restricted Lexicon syntax,at://AUTHORITY[/NSID[/RKEY]]. Each part is validated by its own type, and an authority with an@prefix is rejected. Trailing slashes, empty or extra segments, queries and fragments are rejected. The limit is 8 KiB.Cidvalidates. It accepts only a lower-case base32b…CIDv1 with codec0x71or0x55and a sha2-256 multihash with a 32-byte digest, in canonical encoding (the unused trailing bits must be zero). It exposesCidCodec Codec,ReadOnlyMemory<byte> Digestandbyte[] ToBytes()(a copy).Cid.Parse("hello")now throws.AtUrityped accessors. The type now hasAtIdentifier Repo,Nsid? CollectionandRecordKey? RecordKey, all computed once at parse time.Authority(a string) keeps the text as written, whileRepolower-cases a handle.Create(AtIdentifier repo, Nsid? collection = null, RecordKey? rkey = null)builds the URI without re-parsing it, and throws when an rkey is given without a collection.JsonExceptionwith a null message, so System.Text.Json writes its standard message withPath, line and position. The innerFormatExceptionnames the value and the type. Example:The JSON value could not be converted to ATProtoNet.Identity.AtUri. Path: $.Uri | ….API shape
Did,Handle,AtIdentifier,Nsid,Tid,RecordKey,Cid,AtUri,SpaceUri,SpaceRecordUri) are nowsealed records implementing an internalIIdentifier<TSelf>.static abstract TryCreate(ReadOnlySpan<char>, string?, out TSelf?)and suppliesIParsable<T>/ISpanParsable<T>as static virtual default implementations, so each type states its validation once.IComparable<T>(ordinal) and has an explicit conversion fromstring.ParseisTryParse ? x : throw. It still throwsArgumentException(ArgumentNullExceptionfor null). TheIParsablemembers throwFormatException, as that contract expects.IdentifierJsonConverter<T>in place of the eight copies, and the seven unused internalUnsafeCreatemethods are deleted.SpaceUri/SpaceRecordUrinow also serialize as JSON strings; before, they serialized as objects that could not be read back.Handlemoved fromOrdinalIgnoreCaseto ordinal, which is equivalent because handles are always lower-cased.@stripping.FromDid/FromHandle,Tid.NextString(),FromInt64/ToInt64.T → stringandDid/Handle → AtIdentifiernow mapnulltonull. Before, they threw aNullReferenceExceptionor built anAtIdentifierwith neither DID nor handle.Breaking changes and migration
Parseand from JSON deserialization. Migration: read legacy or foreign data withTryParseand handlefalse.Parse/TryParse/==/Equals/ToString/conversions). Migration: recompile.AtUri:Collectionis nowNsid?andRecordKeyis nowRecordKey?(both convert implicitly tostring).Create(AtIdentifier, string?, string?)is replaced byCreate(AtIdentifier, Nsid?, RecordKey?). Migration:AtUri.Create(repo, Nsid.Parse(c), RecordKey.Parse(k)), anduri.Collection?.Valuewherever astringmember was used.JsonExceptionrather than anArgumentException. This is listed under Fixed: code that caughtArgumentExceptionaround deserialization should catchJsonException.Verification
TidGeneratorTests:3jzfcijpj2z2ais reproduced from its instant and clock id 6.CidTestspins real CIDs: the empty MST nodebafyreie5737gdxlw5i64vzichcalba3z2v5n6icifvx5xytvske7mr3hpmand the raw empty-bytes CID. It rejects upper case, CIDv0, dag-pb, BLAKE3, non-canonical padding bits, padding and truncation.IdentifierContractTestsruns genericT.Parse/T.TryParse(string and span),CompareTo, conversions and null handling over all ten types.Deliberately deferred / notes
#fragment. The current fixtures and the spec's restricted syntax reject every fragment and query, soAtUrirejects them too. The general AT URI syntax (with query and fragment) is not modelled.cid_syntax_valid.txtis not asserted. Its lines are generic multiformats CIDs (base58, base16, dag-pb), none of which the atproto data model blesses. NoCidParseMode.Lenientwas added: nothing in the SDK needs one.datetime_*,language_*anduri_*. They are vendored but unused, because the SDK has noDatetime/Languagetypes yet (see the Interop README).bin/andartifacts/layouts.JetstreamCommitEvent.Urinow throws, lazily, on an event whose collection or rkey is not spec-valid. The event itself is still delivered.🤖 Generated with Claude Code
@claude /review
🤖 Reviewing this PR — follow along in the action run; the reply will link the full session transcript.
📜 Live transcript — updating every ~5 minutes while the task runs.
🤖 Reviewing this PR — the review will link the full session transcript.
📜 Live transcript — updating every ~5 minutes while the review runs.
0c283200605c7c779655✅ Verdict: approve — no blocking issues found.
This is a large, careful rewrite of the identifier types and I did not find correctness bugs after tracing through the trickiest parts.
Things I specifically checked and found correct:
AtUri's manual path splitting correctly rejects trailing slashes, extra segments, queries and fragments by relying on the fact that/?#are not in the Did/Handle/Nsid/RecordKey character classes, so a segment containing one of those characters fails its own type's regex rather than needing extra AtUri-level checks.AtIdentifier.TryCreateStrict(used only for AT URI authorities) correctly rejects a leading@, while the publicAtIdentifier.Parse/TryParseintentionally keeps the lenient@-stripping behavior viaHandle.TryCreate— two different entry points for two different contracts, which is a deliberate and consistent design rather than an inconsistency.Cid's base32 decoder uses a 12-bit (0xFFF) scratch buffer without ever explicitly clearing already-extracted bits; I worked through several iterations by hand and confirmed the modular arithmetic is still correct (a standard fixed-width bit-accumulator idiom: 12 bits is exactly7 pending + 5 new, so the mask always discards only already-consumed bits before they could corrupt a later extraction). The canonical-encoding check (buffer & ((1<<bits)-1) == 0) correctly rejects non-zero padding bits.Equals/GetHashCodeexplicitly (AtUri,Cid) avoid the pitfall of default record equality touching internal fields likeCid's byte array orAtUri's derivedRepo/Collection/RecordKey. Records that rely on default record equality (Did,Handle,Nsid,Tid,RecordKey,AtIdentifier) only haveValue(or, forAtIdentifier,Did/Handle) as their actual stored fields, so the compiler-generated equality ends up equivalent to the old hand-written ordinal comparison — includingHandle's move fromOrdinalIgnoreCaseto ordinal, which is safe precisely becauseValueis already lower-cased at construction.TidGenerator's CAS loop (next = max(now, last + 1)) is correct for monotonicity across threads and clock stalls/steps, and the 53-bit overflow check throwing after already committing the bad value to_lastTimestampis the right behavior (a real overflow is permanent, so failing permanently afterward is correct, not a bug).%/:, Nsid name capped at 63 chars, RecordKey's tightened character class, Tid's high-bit-clear first character) all match the PR's stated rules and are backed by regression tests.One very minor nit, not worth blocking on: the PR description states span-based parsing "allocates only on success," but
AtUri'sIIdentifier<AtUri>.TryCreatecan allocate a string viaspan.ToString()even when the body ultimately fails to parse (once the scheme/length pre-check passes), andSpaceUri/SpaceRecordUri'sTryCreatealways allocates on the span path regardless of outcome, since their engines are string-based. Purely a documentation precision issue, not a functional one.⏱ 6m 17s · 1 turns · tokens: 78.3k in / 36.2k out (+23.9k cache-read)
📜 Full session transcript
5c7c779655c3be6a8d56Rebased onto main twice to resolve CHANGELOG-only conflicts (#143, #148, #147 merged).
verify-rebase: the PR's patch outsideCHANGELOG.mdis byte-identical between the approved head0c283200and the new headc3be6a8d(3164 changed lines, hunk offsets ignored). CI is green on the new head, so the approval carries over (maintainer-approved policy for CHANGELOG-only rebases).✅ Verdict: approve — no blocking issues found.
Solid, well-tested change. I verified the implementation against the atproto reference patterns and the pinned upstream fixtures rather than taking the description on faith:
TidGenerator— the CAS loop is correct:next = max(now, previous + 1)withVolatile.Read+Interlocked.CompareExchangegives strictly increasing, distinct TIDs per generator even when the clock stalls or steps back, and per-callnowis hoisted correctly. Epoch conversion, clock-id range, and the 53-bit overflow path all check out; the spec example3jzfcijpj2z2adecodes to 2023-06-30T15:03:01.887…Z as claimed.Cid— the hand-rolled base32 decoder is correct (12-bit buffer holds the max 7 leftover + 5 new bits; 58 chars yield exactly 36 bytes; canonical trailing-bits check makes the string form unique), and every line ofcid_syntax_invalid.txtfails by construction.AtUri— splitting on/and validating each part genuinely rejects fragments, queries, trailing slashes and extra/empty segments, because the DID/handle/NSID/record-key patterns exclude/?#and whitespace;Create's no-re-parse argument holds (max component lengths sum to ~2.9 KiB).056e5741bb33…(the tree exists upstream;aturi_syntax_valid.txt,tid_syntax_*,recordkey_syntax_*,cid_syntax_invalid.txtmatch byte-for-byte, including upstream's own quirks). The source-tree walk works for bothbin/andartifacts/layouts.^did:[a-z]+:[a-zA-Z0-9._:%-]*[a-zA-Z0-9._-]$, NSID 63-char name limit, TID first char in the lower half of base32-sortable, record key[A-Za-z0-9._:~-]{1,512}minus./...Urithrows lazily as documented. The strict-CID change doesn't break the CAR/MST paths, which work with binary CIDs; the only affected path (RepoCommit.Cid) previously produced a silent garbage CID for non-blessed blocks and now throws, which is the intended fix.Minor remarks (non-blocking, no action required to merge):
TidGenerator.Next()commits the overflowing value to_lastTimestampbefore throwing, so once the 53-bit limit is hit the generator throws on every later call, even if the clock is corrected back within range. Unreachable until year 2255, and arguably intended for a broken generator — but checkingnext > MaxTimestampbefore the CAS would let a clock correction recover it.Tid.FromInt64documents "the high bit must be clear" but doesn't reject values ≥ 2^63; such a TID round-trips throughFromInt64/ToInt64yet is rejected byParse(pre-existing, unchanged by this PR).IdentifierJsonConverter<T>.Readcallsreader.GetString()without checking the token kind, so a non-string token (e.g. a number where a CID is expected) throwsInvalidOperationExceptionrather thanJsonException— pre-existing behavior, unchanged here; worth a follow-up if you want uniform failures.The breaking changes are properly documented in the CHANGELOG with migration notes, and the JSON converter change to
JsonExceptionis correctly classified as a fix.⏱ 15m 47s · 38 steps · $0.1728 · tokens: 247.8k in / 44.1k out (+3786.6k cache-read) · Go: 5h 2% · wk 96% · mo 81%
📜 Full session transcript