HTML entities

The five that matter

& becomes &, < becomes <, > becomes >, " becomes " and ' becomes '. Loaded above is a working anchor tag with an event handler, and the output shows all five at once, with nothing left for a parser to treat as structure.

The ampersand has to go first. An escaper that replaces < with < before handling & escapes the ampersand it just introduced, giving <, which renders as the literal text <.

' came from XML and reached HTML only with HTML5, so a parser built for HTML 4 prints it instead of resolving it. The numeric forms carry no such history, and OWASP’s cross-site scripting cheat sheet writes the apostrophe as '.

Where escaping stops being enough

The sink decides. Escaping these five covers a value dropped into element text or a quoted attribute.

In an unquoted attribute the five are beside the point, because a space ends the attribute. A value of x onmouseover=alert(1) needs no bracket and no quote to attach a handler. Quoting the attribute closes it.

Inside a <script> element the browser resolves no character references. Script content is raw text, so " reaches the JavaScript parser as six literal characters, breaking the code while protecting nothing. The characters that count there are the backslash, the line terminators, and the sequence </script>.

In a URL-valued attribute the scheme is the payload. href="javascript:alert(1)" contains none of the five and passes an HTML escaper untouched.

Double-escaping

Escape at input and again at output and readers see Tom & Jerry on the page. That is the visible half of the bug, and the half that gets reported.

The other half runs in the decode direction. Put a filter that resolves entities in front of something that resolves them again, and <script> walks past: the first pass produces <script>, which matches no tag pattern, and the second produces a tag. Decoding here is single-pass, so stacking two stages shows the payload unwrap one layer at a time.

Named entities beyond the five are left alone, so   and © come through unchanged: the full HTML5 table runs past two thousand names, and resolving it all is a browser’s job. Numeric references are read in both decimal and hex.

Also escape non-ASCII rewrites every character above U+007F as a decimal reference, one per code point, so an emoji becomes a single 😀 rather than a surrogate pair.

Related

URL encoding does this job for the request side, where a value must survive a path or query string. Unicode code points names the character behind a numeric reference you do not recognize.