HTML
HTML character references
& and friends. How a page spells what it can't type.
- Overhead
- text only
- Padding
- none
- Standard
- WHATWG HTML, named and numeric character references
- Used by
- HTML, RSS, every web archive
Sample
- Out
- hello world!
- Back
- "hello world!"
Options
Access
- Import
import { html } from "@agntn/encodings/html" - CLI
encodings encode html 'hello world!' - Tryplayground with the sample above
A page can't just write < in its text. The parser would take it for a tag. So HTML spells it <, and the ampersand spells itself &. That's a character reference.
const read = (text: string) => new TextDecoder().decode(decode("html", text).bytes);
encode("html", 'Tom & "Jerry" <3'); // "Tom & "Jerry" <3"
read("Universität"); // "Universität"
Encoding escapes the five characters markup cares about: &, <, >, " and '. Everything else stays put. Want plain ASCII out? ascii: true writes the rest as hex references too.
encode("html", "Zażółć gęślą", { ascii: true });
// "Zażółć gęślą"
Ugly? Sure. It survives any charset, though.
Reading it like a browser
Decoding is the fun part. A browser never throws on a bad reference. It shrugs and keeps going. decode does the same, rule for rule from the WHATWG tokenizer.
- Every named reference in the spec's table, from
&to∳. - Decimal and hex ones,
Aand😀. - The old names that still work without a semicolon.
© 2001is© 2001. - Anything that names nothing stays text.
AT&Tcomes back asAT&T.
And ¬it;? That's ¬it;. not is one of the old names, notit isn't a name at all. Browsers really do this.
Why is ’ an apostrophe?
Because Windows said so. Code point 146 is an invisible control character. But pages from the nineties were typed in Windows-1252, where byte 146 is ’. Their authors wrote what they saw.
read("it’s – €5"); // "it’s – €5"
So browsers read 128 to 159 through Windows-1252, and this does too. Zero, surrogates and anything past U+10FFFF turn into �.
Spotting it in the wild
identify wants at least one reference that ends in ;. R&D is valid HTML too. It just never said anything. Same for a URL with ©=2 in its query.
identify("Universität, it’s 1999")[0]?.text; // "Universität, it’s 1999"
Base64 wrapped around a page? peel takes off both layers.
It's text in and text out, so there's no overhead on random bytes. Hand encode bytes that aren't UTF-8 and it refuses. A reference names a character, never a raw byte.