Invisible characters

What is loaded above

That line reads as a transfer of 1,000 USD. Stored, it says 000,1: a direction override flips the digits for the reader and nothing else. Behind the account number sit two zero-width spaces, a zero-width non-joiner, a variation selector, and two characters from the Unicode tag block spelling hi.

Nothing above is a trick of this page. That is the form the text takes in a document, a message or a commit, and it is the form that reads as ordinary.

What it does

Some Unicode characters render as nothing at all. Zero-width spaces and joiners, word joiners, direction overrides, variation selectors and the Unicode tag block occupy no pixels, so text carrying them looks completely ordinary.

Reveal replaces each one with a visible label so you can see what is actually in the string. Strip removes them.

Why it matters

Text you read is not always the text a machine reads.

Hidden characters get used to smuggle instructions into content that appears clean, to defeat filters that match on visible characters only, and to watermark documents so a leak can be traced to a recipient. The Unicode tag block is particularly effective, because it can encode entire sentences that no viewer will display.

Direction overrides are the basis of the Trojan Source class of vulnerabilities, where source code renders in one order to a human reviewer and compiles in another. A pull request can look correct in the browser and do something else entirely.

When to reach for it

• Reviewing code or a document you did not write
• Checking text before pasting it somewhere that will act on it
• Working out why two strings that look identical are not equal
• Cleaning content of tracking marks before republishing it

Strip is one-way. Once removed, the original cannot be reconstructed, so keep a copy if the hidden data is itself the evidence.

Related

Unicode code points shows every character’s exact identity, which is the next step once you know something is there.