Base64 and UTF-8: encode accents and emoji correctly

Short answer

Base64 encodes bytes, not letters. To preserve accents, non-Latin scripts, and emoji, first turn the text into UTF-8 bytes and then encode those bytes as Base64. On the way back, recover the bytes and require valid UTF-8. Treating each JavaScript code unit as one byte causes exceptions, corrupted text, or mojibake.

The missing conversion step

The string España 🌍 is Unicode text. UTF-8 defines the bytes for each character, and Base64 then maps those bytes to ASCII characters suitable for transport. They are separate layers:

Unicode text → UTF-8 bytes → Base64

Decoding must reverse the same sequence:

Base64 → bytes → strict UTF-8 decoding

The Base64 and Base64URL converter follows these steps in the browser. If the resulting bytes are not valid UTF-8, it reports that fact instead of silently inserting replacement characters.

Why btoa() fails for some characters

The browser’s historical btoa() function treats each string unit as a single-byte value. It works directly for ASCII and part of the Latin-1 range, but an emoji and many other Unicode characters do not fit that model. The function may throw InvalidCharacterError; incomplete workarounds may instead produce unreadable output.

Modern JavaScript should use TextEncoder to obtain UTF-8 bytes and a fatal TextDecoder to validate the reverse conversion. Recipes based on encodeURIComponent plus hand-edited escapes are harder to audit and handle malformed input less clearly.

Tests that expose the bug

Hello uses the same basic values in ASCII and UTF-8, so it hides many implementation mistakes. Always test strings such as Málaga, niño, 東京, and 👩🏽‍💻. A useful round-trip test requires the decoded result to equal the original Unicode sequence, not merely look similar on screen.

For example, UTF-8 represents ñ with hexadecimal bytes C3 B1, whose Base64 form is w7E=. If those bytes are later read separately as Latin-1, the screen may show ñ. The Base64 layer was correct; the text decoder was not.

Valid Base64 may still contain non-text bytes

Base64 can transport any byte sequence, including an image, ZIP archive, or text in another encoding. A valid Base64 string does not promise UTF-8 content. A text-focused tool should therefore distinguish invalid Base64 syntax from valid bytes that are not UTF-8 text.

The WHATWG Encoding Standard defines UTF-8 decoding and error handling. If the source is known to use Windows-1252 or another legacy encoding, decode with that information before expecting correct Unicode.

Recommended JavaScript workflow

1. Call new TextEncoder().encode(text) to obtain bytes.
2. Convert those bytes to Base64 or Base64URL as required by the protocol.
3. On input, validate alphabet, padding, and canonical trailing bits first.
4. Decode Base64 into bytes.
5. Use new TextDecoder('utf-8', { fatal: true }) to reject invalid sequences.

Do not split emoji into visible symbols. Some are sequences of multiple Unicode code points joined together. UTF-8 should receive the complete string.

Preventing mojibake in an API

Document the character encoding alongside the field, not just “Base64.” State UTF-8 when the payload is text; for a file, preserve its media type and handle the decoded result as bytes. During debugging, compare the hexadecimal source and destination bytes first. That reveals whether corruption happened before Base64, during transport, or when the bytes were interpreted as text again.