JavaScript Basics: UTF-8, Hex, and Base64 with Buffers
A Node.js Buffer holds bytes. Use it when text needs an encoding or when data isn’t text at all. These examples were tested on Linux with Node.js v24.19.0 and use only built-in modules.
Set up a scratch directory
Run in Bash and keep the same terminal open:
You need Bash, a text editor, the mktemp, rm, and rmdir utilities, and Node.js on your PATH. Save source files as UTF-8; their .mjs extension selects ES modules.
start_dir="$PWD"
work_dir="$(mktemp -d)" || exit 1
cd -- "$work_dir" || exit 1Round-trip UTF-8 text
In your editor, save this as buffer-text.mjs in the scratch directory:
import assert from 'node:assert/strict';
import { Buffer } from 'node:buffer';
const text = 'café 🍵';
const bytes = Buffer.from(text, 'utf8');
const restored = bytes.toString('utf8');
assert.equal(text.length, 7);
assert.equal(bytes.length, 10);
assert.equal(Buffer.byteLength(text, 'utf8'), bytes.length);
assert.equal(restored, text);
assert.equal(Buffer.from('', 'utf8').length, 0);
console.log(`string length: ${text.length}`);
console.log(`UTF-8 bytes: ${bytes.length}`);
console.log(restored);Run:
node buffer-text.mjsOutput:
string length: 7
UTF-8 bytes: 10
café 🍵JavaScript string length counts UTF-16 code units. The emoji takes two of those units, so this is not a count of visible characters. Buffer length counts bytes; Buffer.byteLength() calculates the encoded size without creating a Buffer yourself.
Preserve binary bytes with hex or Base64
Save as buffer-binary.mjs:
import assert from 'node:assert/strict';
import { Buffer, isUtf8 } from 'node:buffer';
const bytes = Buffer.from([0x00, 0xff, 0x10, 0x80]);
const hex = bytes.toString('hex');
const base64 = bytes.toString('base64');
assert.equal(hex, '00ff1080');
assert.equal(base64, 'AP8QgA==');
assert.ok(Buffer.from(hex, 'hex').equals(bytes));
assert.ok(Buffer.from(base64, 'base64').equals(bytes));
assert.equal(isUtf8(bytes), false);
console.log(`hex: ${hex}`);
console.log(`Base64: ${base64}`);
console.log(`valid UTF-8: ${isUtf8(bytes)}`);Run:
node buffer-binary.mjsOutput:
hex: 00ff1080
Base64: AP8QgA==
valid UTF-8: falseHex and Base64 represent bytes as text. They are encodings, not encryption: anyone with the encoded value can decode it. Arbitrary bytes may be invalid UTF-8, so don’t use a UTF-8 string as an intermediate format for binary data. See Node’s encoding reference.
Don’t mistake decoding for validation
Save as buffer-input.mjs:
import assert from 'node:assert/strict';
import { Buffer } from 'node:buffer';
const truncated = Buffer.from('abcz', 'hex');
assert.equal(truncated.toString('hex'), 'ab');
assert.equal(Buffer.from('Y Q==\n', 'base64').toString('utf8'), 'a');
function decodeHex(input) {
if (typeof input !== 'string' || input.length % 2 !== 0 || /[^0-9a-f]/i.test(input)) {
throw new TypeError('Expected pairs of hexadecimal digits');
}
return Buffer.from(input, 'hex');
}
assert.equal(decodeHex('00FF').toString('hex'), '00ff');
assert.equal(decodeHex('').length, 0);
assert.throws(() => decodeHex('abc'), TypeError);
assert.throws(() => decodeHex('abcz'), TypeError);
assert.throws(() => decodeHex('ab\n'), TypeError);
assert.throws(() => decodeHex('ab\n\n'), TypeError);
assert.throws(() => decodeHex(null), TypeError);
console.log(`permissive hex result: ${truncated.toString('hex')}`);
console.log('strict hex checks passed');Run:
node buffer-input.mjsOutput:
permissive hex result: ab
strict hex checks passedNode can truncate malformed hex and ignores whitespace in Base64. Successful decoding doesn’t prove that input follows your format. This helper deliberately accepts empty hex and rejects odd lengths or non-hex characters. Base64 validation needs its own policy for alphabet, whitespace, and padding.
Clean up
Remove only the named tutorial files:
rm -- buffer-text.mjs buffer-binary.mjs buffer-input.mjs
cd -- "$start_dir" || exit 1
rmdir -- "$work_dir"
unset start_dir work_dirAcknowledgements
This post was created with assistance from AI.