Find Duplicate Files on Linux: I Tested 4 Tools

Short answer: yes, and the fastest tool I tested found every duplicate in 107 files in 0.08 seconds. Duplicate files — the same photo saved in three folders, installers downloaded twice, "final_v2_REALLY_FINAL" copies — quietly eat gigabytes. I tested four duplicate finders on Linux (fdupes, jdupes, rdfind, and a no-install Python script) against a corpus with a known ground truth. jdupes was the fastest scanner, fdupes had the friendliest delete workflow, and rdfind freed 47.1 MB without deleting a single file. Here are the real numbers and the exact safe commands.

Why Duplicates Pile Up

Every backup you copy, every photo you import twice, every installer you re-download leaves a twin behind. They don't just waste disk space — they make backups slower and searching messier. The fix is a duplicate finder: a tool that compares files by content (size, then hash, then byte-by-byte), not by name, so renamed copies still get caught and same-size-but-different files don't get falsely flagged.

How I Tested This

I built a test corpus of 107 files (~96 MB) with a known ground truth, verified by an independent SHA-256 script:

• 31 duplicate groups (30 text groups of 2–3 identical 1 MB copies under different names and folders, plus one duplicated ~2 MB file) — 47.1 MB of redundant data
• 10 decoy files: same size (1 MB) but completely different random content — to catch false positives
• 15 unique files and 5 empty files

I installed fdupes 2.3.0, jdupes 1.27.3, and rdfind 1.6.0 from Ubuntu's repositories (sudo apt install fdupes jdupes rdfind) and ran each scan twice on 2026-10-23, on a 2-vCPU Ubuntu 24.04 machine. The corpus is synthetic and modest in size, so treat absolute times as illustrative — the relative differences and accuracy are the real findings.

The Results

Every tool found all 31 real duplicate groups. None was fooled by the 10 same-size decoys. The differences are in speed, safety features, and quirks.

jdupes — the fastest scanner

jdupes (a faster fork of fdupes) scanned the corpus in under a tenth of a second — roughly 4x faster than fdupes here — and matched the ground truth exactly. It quietly skipped the five empty files rather than listing them as duplicates, which is the sanest default of the three. Usage is identical to fdupes:

jdupes -r ~/Downloads

It also has an interactive delete mode (-d); I tested its scanning, which is its standout strength.

rdfind — the safest way to reclaim space

rdfind was the second-fastest scanner and the only tool with a genuinely different superpower: instead of deleting duplicates, it can replace them with hard links. Every duplicate path stays exactly where it is, but they all point to one copy on disk. I ran it on a copy of the corpus:

rdfind -makehardlinks true ~/Downloads

Result: 46 hard links created, disk usage dropped from 96 MB to 46.7 MB — the same 47.1 MB freed as deleting — while all 107 files remained in place. If an application expects a file at a specific path, nothing breaks. By default rdfind only reports (it writes a results.txt you can review); it deletes nothing unless you pass the explicitly dangerous -deleteduplicates true flag, which I deliberately did not run.

fdupes — the classic, friendliest delete flow

fdupes is the original tool the others descend from. It's the slowest of the three here (still a third of a second on this corpus) and has one quirk worth knowing: it groups zero-byte files together as duplicates, so my five empty files showed up as a 32nd "duplicate" set. Harmless, but don't be surprised. Where fdupes shines is deletion — its interactive mode walks you through each set:

fdupes -r -d ~/Downloads

For scripting, -N keeps the first file of each set without prompting. I ran fdupes -r -d -N on a corpus copy: it freed exactly 49,345,920 bytes (47.1 MB) — byte-for-byte matching the ground truth — a rescan found zero duplicates, and every file's content survived in one remaining copy.

The no-install Python script

If you can't or won't install anything, a ~30-line Python script using size-grouping plus SHA-256 matched the ground truth exactly in ~0.3 seconds. It skips empty files and needs nothing beyond Python 3:

#!/usr/bin/env python3
"""finddups.py - groups files by size, then SHA-256. Usage: python3 finddups.py DIR..."""
import os, sys, hashlib
from collections import defaultdict

def sha256(p):
    h = hashlib.sha256()
    with open(p, "rb") as f:
        for b in iter(lambda: f.read(1 << 20), b""):
            h.update(b)
    return h.hexdigest()

by_size = defaultdict(list)
for root in sys.argv[1:]:
    for dp, _, fs in os.walk(root):
        for f in fs:
            p = os.path.join(dp, f)
            try:
                by_size[os.path.getsize(p)].append(p)
            except OSError:
                pass

groups = 0
for size, paths in by_size.items():
    if len(paths) < 2 or size == 0:
        continue
    by_hash = defaultdict(list)
    for p in paths:
        try:
            by_hash[sha256(p)].append(p)
        except OSError:
            pass
    for h, ps in by_hash.items():
        if len(ps) > 1:
            groups += 1
            print("--- group %d (%d bytes each)" % (groups, size))
            print("\n".join(ps))
print("\n%d duplicate groups" % groups)

The Safe Workflow (What I'd Actually Do)

1. Scan first, touch nothing: jdupes -r ~/Downloads (or fdupes -r, or rdfind and read results.txt).
2. Eyeball the list. Make sure nothing surprising is flagged — especially before any delete.
3. Reclaim space the safe way: rdfind -makehardlinks true <dir> frees the space with zero deletions. Prefer this on folders other programs depend on.
4. Or delete with confirmation: fdupes -r -d <dir> asks you about each set. Only use -N (no prompt) in scripts, on folders you've already reviewed.
5. Verify: rescan — you should get zero groups — and spot-check that the files you care about still open.

One rule: never run any duplicate deleter on system directories (/usr, /etc, /bin). Those "duplicates" are often intentional.

Which Should You Use?

• Fastest scan, set-and-review: jdupes.
• Reclaim space without deleting anything: rdfind with -makehardlinks true.
• Interactive, guided deletion: fdupes with -d.
• No install / one-off check: the Python script above.
• Prefer a GUI? Czkawka is a popular graphical alternative I did not test here — worth knowing about, but the numbers above are only for the four methods I ran.

FAQ

Is deleting duplicates safe? Yes, if you scan first and keep one copy of each set. These tools compare file contents, not names, so a kept copy is byte-identical to the deleted ones. Hard-linking (rdfind) is the zero-risk option.

What if two files have the same name but different content? All four methods compare content — size first, then hash, then bytes. Same name + different content = not duplicates, correctly left alone. My 10 same-size decoys confirmed this: zero false positives.

Do these work on USB drives or network shares? Yes — point them at any mounted path. Speed depends on the drive; on a slow USB stick, jdupes' speed advantage matters more.

Should I delete empty (0-byte) files? They take no space, so there's nothing to gain. fdupes lists them as a group; jdupes and rdfind ignore them. Either way, harmless.

How much space will I actually get back? It depends on your habits. My test corpus was 49% redundant (47.1 MB of 96 MB) — deliberately duplicate-heavy. Real Downloads folders often yield hundreds of megabytes; photo libraries with repeated imports can yield gigabytes.

Conclusion

For finding duplicate files on Linux, you can't go wrong with any of the three dedicated tools — all matched my ground truth with zero false positives. jdupes is the speed king, fdupes has the most approachable delete flow, and rdfind's hard-link mode is the cleverest safe option: same space freed, nothing deleted. Scan first, review, then reclaim — in that order — and never point these at system folders.

How this article was made

Test corpus (107 files, 96 MB) built 2026-10-23 with a seeded generator; ground truth (31 duplicate groups, 47.1 MB redundant) verified by an independent SHA-256 script. fdupes 2.3.0, jdupes 1.27.3, and rdfind 1.6.0 installed via apt on Ubuntu 24.04 (2 vCPUs); each scan run twice, times and group counts reported as measured. Deletion and hard-link runs were performed on separate copies of the corpus, with byte counts verified before and after. The Python script above is the exact script run in the test. All four linked project pages were checked to return HTTP 200 on 2026-10-23. This article was drafted with AI assistance; every test, number, and command result is from the author's own runs on that date.