How it works
Where the 0.35 median comes from
The URL is taken apart and each piece is coded against what that piece usually looks like. Nothing has a fixed width — every question is priced against a probability.
1. A URL is a sequence of decisions, not of bytes
Scheme, the www. prefix, host, path segments, query parameters and fragment are written as separate decisions. The scheme is one question — nine times in ten the answer is https, so it costs a fraction of a bit rather than a whole one.
The host goes through a Huffman code built from a ranked domain dictionary. On a hit, the host costs a few bits instead of a dozen characters. On a miss we write a public-suffix code plus the rest as literal text.
2. The arithmetic coder prices every decision
Everything runs through a binary arithmetic coder against probabilities trained on the corpus and adapted as the payload goes on. A near-certain decision costs a fraction of a bit — which is why all the format's flags together are about 30 bits of a payload.
3. Text is predicted character by character
Two thirds of a typical payload is free text: a path slug, a query value. Each character is coded against the few before it — four for hosts and query text, five for paths. Path text has an order-0 entropy of 4.9 bits per character; this model gets it to about 3.3.
Modelled text ends with a terminator priced in context rather than with a length prefix — after the letters lmth, the end of a literal is almost certain and costs almost nothing.
A literal that looks like an opaque identifier gets a flat six-bit set instead. A context model asked to predict uniform data does worse than no model at all.
4. Path and query are modelled per site
The 256 most popular domains each get their own probabilities for the shape of a path. YouTube is nearly always /watch?v=…, and a news site nearly always starts with a year. The host is written before the path, so the decoder computes the same site class the encoder did.
5. Bits become text
The bit stream is converted over one of two alphabets: 68 paste-safe characters for links, or a QR-alphanumeric subset where two characters pack into 11 bits and the QR code stays small.
6. What it cannot do
None of this beats entropy. A link that is mostly a random identifier cannot shrink. When the structured encoder loses to raw bytes, the raw form is kept and the result is reported as passthrough.
7. Format stability
A payload is only meaningful together with the dictionaries and probabilities that produced it. Any change to a dictionary, to the context layout or to field order would change what existing links decode to, so it comes with a format-version bump and the old tables kept alongside the new ones. Frozen payloads in the test suite keep that a deliberate act rather than a side effect.