Validating uploads: magic bytes vs extension vs MIME type

An upload reaches your server with three claims about what it is: the file name extension, the Content-Type header, and the bytes themselves. Only the bytes describe the content. The other two are labels, and they are often wrong by accident and sometimes on purpose.

The three signals

Extension. The user picked the file name, so it can be anything. Cameras write .JPG in uppercase, Windows saves some downloads as .jfif, and anyone can rename invoice.html to invoice.jpg.

Content-Type header. Browsers usually derive it from the extension, using the operating system’s type table, so it adds no new information. Scripts and API clients can send any value they like. Treat it as a hint and never as a check.

Magic bytes. Most binary formats start with a fixed signature, so reading the first few bytes identifies the format reliably, whatever the file is called. This is what file, libmagic and the libraries below do.

Common signatures

These come from the first bytes of our own sample files. The file signature reference has the full list.

To see a file’s first bytes yourself:

xxd -l 16 sample-640x480.png
file --mime-type -b html-named.jpg     # text/html, despite the .jpg name

You can also drop a file into the file inspector. It reads the signature in your browser and tells you whether the extension matches. Nothing is uploaded.

Detecting the type on the server

Python has two common options. python-magic wraps libmagic, the same database the file command uses, so it needs the system library installed. filetype is pure Python and smaller, but it recognizes fewer formats.

import magic      # pip install python-magic  (needs libmagic)
import filetype   # pip install filetype

with open(path, "rb") as f:
    head = f.read(4096)

print(magic.from_buffer(head, mime=True))   # e.g. "image/png"

kind = filetype.guess(head)                 # None for text and unknown formats
print(kind.mime if kind else "unknown")

Node has file-type, which is ESM-only in current versions:

import { fileTypeFromFile } from 'file-type';

const type = await fileTypeFromFile('png-named.jpg');
console.log(type); // { ext: 'png', mime: 'image/png' }, or undefined if unknown

Both filetype and file-type return nothing for text formats (CSV, JSON, HTML, SVG, plain text), because those have no signature. libmagic uses heuristics for text, so it can identify HTML or SVG, but treat a text result with more suspicion than a binary one.

Putting it together

1. Read the first few KB and detect the type. Reject anything that isn’t on your allowlist of detected types. Don’t run a blocklist of “dangerous” extensions.
2. Require the extension to agree with the detected type, or simply replace it. Then a PNG named .jpg becomes .png instead of confusing your thumbnailer.
3. Fully decode anything you’ll process. A matching signature only proves the first bytes are right. The PNG with a damaged signature fails detection, while the truncated PDF passes it and fails later. See corrupt files for testing.
4. Store under a generated name, never the uploaded one.
5. Serve with a fixed Content-Type taken from the detected type, plus X-Content-Type-Options: nosniff, and Content-Disposition: attachment for anything you don’t intend to display inline.

Our own file server works this way. The HTML file named .jpg is served as image/jpeg with nosniff, so browsers show a broken image instead of rendering the page.

Test files for each check

Wrong extension, harmless content. Your validator should either rename or reject these consistently:

• PNG named .jpg, JPG named .png, GIF named .png
• WebM named .mp4 and MP3 named .wav
• JSON named .csv, a text-format mix-up that magic bytes can’t catch

Text pretending to be an image. These must never reach a browser as HTML or SVG:

• HTML named .jpg
• SVG named .png. SVG is XML and can contain script, so treat it as active content.

Container lookalikes. DOCX, XLSX, PPTX, EPUB and JAR are all ZIP files, so a signature check alone only says “ZIP”:

• Plain ZIP named .docx: passes a ZIP check, but it isn’t a Word document
• DOCX named .xlsx: both are Office Open XML; only [Content_Types].xml inside reveals which
• PDF named .docx

For ZIP-based formats, open the archive and check its contents (for example word/document.xml for DOCX), or load it with the library you’ll use to process it.

Names that trip up parsers:

• Double extension sample.php.jpg: catches code that splits on the first dot
• Uppercase .JPG: catches case-sensitive allowlists
• PDF with no extension: detection must rely on content
• Text named .pdf: no PDF signature at all

Magic bytes are necessary, not sufficient

A file can carry a valid signature and still be dangerous or broken. Polyglot files are valid in two formats at once, and a correct JPEG header can be followed by anything. That’s why steps 3 and 5 matter: decoding (or re-encoding images) proves the content is what it claims to be, and safe serving headers limit the damage when it isn’t. The MIME type reference lists the Content-Type to send for each format.

All the files above are collected on the file-type validation use case.

Sample files for this guide

• Sample Mislabeled & odd names files
• Sample Corrupt & malformed files
• Sample PNG files
• Sample JPG files
• Sample PDF files
• Sample ZIP files