Trustvice è in beta limitata. Alla registrazione o all’accesso devi inserire la password da beta tester che hai ricevuto oppure richiederne una a preview@trustvice.com. Grazie.
Where canonical JSON breaks: edge cases that fail good signatures
Two correct RFC 8785 implementations can still disagree on key order, big numbers and unpaired surrogates. We walk through each edge case in JavaScript and Python, what our canonicalizers do, and how shared test vectors catch drift.
Di The Trustvice team··8 min lettura
Questo post è disponibile solo in inglese.
Inside the receipt format explains why we sign an RFC 8785 canonical form. This post is about the part that goes wrong. A signature verifies only if the verifier rebuilds the same bytes the signer hashed, and canonicalisation is where two reasonable programs quietly produce different bytes from the same JSON.
Everything below was reproduced against the signer's canonicalizer in runtime/shared/canonical-json.ts and an independent Python port of it. Two of the cases turned up real divergences between them while we wrote this. We fixed both, and say how.
Our canonicalizer is deliberately thin. It validates the value, walks each object with its keys sorted, and writes the output as text. Number formatting and string escaping come from JSON.stringify applied to each primitive, because RFC 8785 was written to match what the engine already does.
ts
for (const key of Object.keys(obj).sort()) {
const child = obj[key];
if (child === undefined) continue;
members.push(`${JSON.stringify(key)}:${serializeValue(child, path ? `${path}.${key}` : key)}`);
}
return`{${members.join(",")}}`;
That is correct for a JavaScript signer and a JavaScript verifier, because they share the engine. A Python verifier shares nothing, so it has to reproduce engine behaviour by hand. Every section below is a place where that reproduction can fail.
RFC 8785 says numbers serialise as ECMAScript Number.prototype.toString does for an IEEE 754 double. Three consequences matter.
The digits are shortest round-trip, but the layout is not. Python's repr and JavaScript's String agree on the digits of a double. They disagree on layout:
JavaScript prints 1e+21, 1e-7, 1 and 0 for the same four values. Python pads the exponent (1e-07), keeps .0 on integral floats and keeps the sign of negative zero. Any one of these breaks a signature. Our Python port (_es_number) takes the digits from repr and lays them out by the ECMAScript rules: decimal notation when the decimal exponent is within -6 to 21, exponential otherwise, and -0 prints as 0. jq fails the same test: it keeps number text as written, so 1e21 prints as 1E+21.
Integers past 2^53 are not integers. JSON has no integer type; JCS says everything is a double. Take the 20-digit literal 12345678901234567890:
Python's parser keeps the exact integer. If a verifier hashes what it parsed, it signs ...993 where the signer signed ...992, and nothing in the error says why. Our port converts any integer with magnitude of 2^53 or more through float before formatting, to match what the signer's parser would have held. It is a lossy step by design: it reproduces the loss rather than preserving the value.
Why identifiers are strings. The practical rule that falls out is that nothing you need to compare exactly should be a JSON number past 2^53. Receipt IDs are 26-character ULIDs, key IDs are base64url, digests are hex, and counts and sizes are small integers. A 64-bit database ID serialised as a number would survive transit through a Python or Go service and then fail verification in a browser, so we do not put one in a signed body. Treat any large numeric ID in your own label or purpose fields the same way: send it as a string.
JCS sorts keys by UTF-16 code units. That is not code point order, and it is not UTF-8 byte order. The two differ for characters above U+FFFF, which are stored as a surrogate pair in the range D800 to DFFF, below the BMP characters in the range E000 to FFFF.
Take the keys 😀 (U+1F600, UTF-16 D83D DE00) and ｚ (U+FF5A):
JavaScript's default sort() compares code units, so it is already right. Python's sorted() and Go's and Rust's byte-wise string comparison give the other order. Our Python port sorts on the utf-16-be encoding with surrogatepass, so that even an unpaired surrogate in a key sorts without an exception.
Our docs say receipts use ASCII keys, and the first-party bodies do. The reason the vectors still include an astral key and a BMP key is that a verifier written for receipts today gets reused for artifacts and bundles tomorrow, and a bug that needs an emoji key will wait for one.
A JavaScript string is a sequence of UTF-16 code units, so it can hold an unpaired surrogate such as "\ud800". That is not a valid Unicode scalar value and it has no UTF-8 encoding. Since ES2019, JSON.stringify does not emit it raw. It writes the escape in lowercase hex:
js
JSON.stringify("\ud800x") // "\ud800x" (six ASCII characters, then x)
RFC 8785 inherits this. A verifier that tries to encode the string as UTF-8 either throws, or replaces the character with U+FFFD and produces different bytes. Python has the mirror-image trap: a string built from two escaped surrogate halves, "😀", is two characters in Python but one in JavaScript. Our port first joins valid pairs through a UTF-16 round trip, then escapes any remaining lone surrogate as \udxxx, lowercase. Both behaviours are pinned by the vectors.
é can be U+00E9 (NFC) or e followed by U+0301 (NFD). They render the same and are different strings:
js
"é".length // 1"é".normalize("NFD").length // 2
JCS does not normalise, and neither do we. The two spellings canonicalise to different bytes, and they are different values as far as a signature goes. That is the right behaviour for a signature, because normalising would make two distinct strings verify as equal. It does shift work onto you. If your system compares identifiers or labels as text, normalise them once, before you submit them, and store what you submitted. A pipeline that normalises on the way out of your database but not on the way in will fail verification on exactly the strings where it matters.
{"a":1,"a":2} is valid to many parsers and is not valid to RFC 8785, which requires the input to satisfy I-JSON and so forbids duplicate names. JavaScript's JSON.parse and Python's json.loads both keep the last value and say nothing:
js
JSON.parse('{"a":1,"a":2}').a // 2
Because both keep the last value, our two verifiers agree on what they see. The risk is a third parser that keeps the first, or one that errors, in a system that sits between the signer and the verifier. Neither of our verifiers rejects duplicates today. If you parse receipts with a strict parser, turn on its duplicate-key error. A signed object with a repeated key has no single meaning.
JSON has no spelling for these, and JSON.stringify handles them inconsistently: undefined in an object disappears, NaN becomes null, and undefined in an array becomes null.
js
JSON.stringify({ a: undefined, b: NaN }) // {"b":null}
Two callers who mean the same thing would sign different bytes. So the signer rejects them. NaN, infinities, bigint, function, symbol, and non-plain objects such as Date or Map throw NonCanonicalizableError with the path of the offending value. An object member whose value is undefined is the one exception: it is omitted, as JSON.stringify would. Python has no undefined, so it has no equivalent, and its json.loads accepts the literals NaN and Infinity by default. A Python verifier should pass a parse_constant hook that refuses them, matching JSON.parse.
Reproducing the cases above turned up two places where the JavaScript canonicalizer and the Python one disagreed. Both came from an earlier version that rebuilt the output as an ordinary JavaScript object and let JSON.stringify serialise that.
Integer-like keys. JavaScript orders object properties with integer-like names first, ascending by value, whatever order you inserted them. So rebuilding an object with sorted keys does not preserve the sort:
js
const out = {};
for (const k of Object.keys({ b: 1, "10": 1, "9": 1 }).sort()) out[k] = 1;
JSON.stringify(out) // {"9":1,"10":1,"b":1}
JCS orders "10" before "9" (code unit 1 is less than 9). The Python port, which does not use an object for its output, emitted {"10":1,"9":1,"b":1}. Same input, different bytes, so a signature over the first failed in the second.
A __proto__ key.JSON.parse creates __proto__ as an ordinary own property, but assigning out["__proto__"] = value sets the object's prototype instead. The member vanished from the output. Python kept it.
The receipt bodies documented in our receipt format use fixed, named keys, so a receipt was unlikely to hit either case. Caller-supplied objects were the exposure. The fix is the one the cases point to: the canonicalizer now writes its output as a string, never as an object, so the engine's property ordering cannot touch it. The browser verifier at /verify builds its bytes the same way, so signer and verifier stay byte-identical. Both cases are now regression tests, and they belong in the shared vectors too, so every language is held to them.
The signature in that fixture only verifies if the verifier gets key order, 1e+21, 1e-7, the rounded 20-digit integer, 0.30000000000000004, the null member and the lone surrogate exactly right.
Run every verifier you maintain against the same fixture, so a drift in any canonicalizer fails a test in the language that drifted.
Test tampering alongside: change n in the awkward object to 2 and the check must return bad_signature, which shows the signature really covers the awkward fields and not just the easy ones.
For your own verifier, the test vector in Verify a receipt offline is the minimum. Add the cases that your language is most likely to get wrong: 1e21, 1e-7, an integral float, -0, a 20-digit integer, an astral key next to a U+FF5A key, a lone surrogate, and a member whose value is null. Generate expected bytes from the JavaScript reference, never from your own code.
Do not trust a canonicalizer you wrote until it has verified a signature produced by someone else's code on awkward input. Keep large numeric identifiers out of signed JSON, normalise text before you submit it rather than after, and treat any parse-and-reserialise step in your own pipeline as a place where bytes can change.
Then run your verifier against a real receipt, and compare its answer with /verify.
Incolla una qualsiasi ricevuta Trustvice nel verificatore. Controlla la firma nel tuo browser con le nostre chiavi pubblicate, senza bisogno di un account.
Trustvice 360