Before you retire a drive

How do I find duplicate photos across two external drives?

Short answer: you cannot work this out by looking. Folder names, file dates and directory structures diverged the moment you started copying between the drives. The only question that survives all of that is whether the contents match — and it has to be asked across both drives at once.

Scanning both drives and exporting the full report is free, with no account. A licence is only needed to move anything.

There is also an order of operations that makes this safe, and it isn't the obvious one. It's further down, under the safe order. Read that part even if you use something other than our software.

Why two drives is a harder problem than one

On a single drive, a duplicate is just wasted space. Across two drives, duplicate and backup are the same word describing the same file, and nothing in the file itself tells you which one you meant.

The history is nearly always the same. One drive filled up. You bought a second, larger one and copied some of the first across — probably the folders you were working on, probably not all of them. You kept shooting onto the new drive. Months later you copied a couple of old jobs back the other way because a client resurfaced. Somewhere in there a backup tool ran once and you can't remember what it was pointed at.

What you have now is three categories of file, thoroughly mixed together:

  • Files that exist on both drives — genuinely redundant, and safe to reduce to one copy
  • Files that exist only on the newer drive
  • Files that exist only on the older drive, the one you were planning to wipe

That third category is the reason this is dangerous. It is usually small, it is usually old, and it is the material you would never think to check for — a job from four years ago, a personal folder, the one shoot that never got copied forward because the drive disconnected halfway through. Deleting from the old drive what you believe is on the new one is how people lose the only copy of something.

Why the obvious methods don't answer it

Sorting both drives by name and comparing

Folder names diverge immediately. 2019_Smith_Wedding on one drive is Smith Wedding (June) on the other, because you reorganised at some point and only reorganised one of them. The photographs inside are identical. Sorted side by side, they don't line up at all — and two folders that do line up by name may hold different selects from the same shoot.

Comparing dates

Copying rewrites timestamps, and which timestamp it rewrites depends on how the copy was made. Dragging in File Explorer, robocopy with and without /COPY:DAT, and backup software all leave different date fingerprints on identical content. A file whose modified date is three years newer than its twin can still be the same photograph, byte for byte.

Comparing folder sizes

Size is an aggregate, so at best it tells you that something differs somewhere. It cannot name the file. And it produces false alarms of its own: Windows reports both Size and Size on disk, and the second depends on the cluster size of the filesystem. A drive formatted exFAT and a drive formatted NTFS will legitimately report different totals for identical content.

Asking Windows to do it

There's no facility in File Explorer for comparing two volumes. Nothing built into Windows answers the question "which files on E: also exist somewhere on F:" across a whole drive.

Using a duplicate finder that scans one location

This is the trap worth naming, because plenty of tools are built this way. They take a folder tree, find groups of identical files inside it, and offer to delete the extras. Point one at both drives at once and it may well be right about what's identical — but it has no concept of which drive you intend to keep, so its "keep one, delete the rest" logic can leave your only surviving copy on the drive you're about to wipe. Being correct about duplication is not the same as being correct about which copy survives.

What "identical" has to mean here

Only one comparison is trustworthy across two drives: reading both files and checking whether their contents match. The difficulty is that doing that naively means reading every byte of two archives that might total twenty terabytes, which is a scan measured in days.

DuplicateFinder Pro runs three tests in order, cheapest first, and a file only reaches the next test if it survived the one before.

The three tests

  • Size. Files of different sizes cannot be identical, so they are never read. This eliminates almost everything.
  • Partial fingerprint. The first and last megabyte of each survivor is hashed. Anything differing near head or tail is separated without reading the middle.
  • Full MD5. Whatever remains is read end to end. Only a full-content match is reported.

What is never used as evidence

  • A matching filename
  • A matching timestamp
  • A matching file size on its own
  • A matching folder structure

One deliberate exception, which the app labels rather than hides: RAW files sharing a filename but not byte-identical are still grouped so you can see them side by side. Those rows read stem+size in the Match column instead of md5, and if the sizes differ by more than 5% they are marked Review rather than flagged. Every row tells you which test found it.

The safe order of operations

This is the part most guides skip, and it matters more than which software you use.

The instinct is to compare the two drives in place, then delete the duplicates from the old one and wipe it. That works right up until the moment it doesn't, because it leaves the files that exist only on the old drive completely unexamined. A duplicate finder finds duplicates. Files with no counterpart aren't duplicates, so they never appear in the report — they are invisible precisely because they are the ones at risk.

If the keeper drive has room: consolidate first, then deduplicate

Copy everything from the old drive onto the new one. Don't merge folders, don't tidy — drop it in as a single dated folder and let it be untidy. Now every file you own is on one drive, including the ones that were only ever in one place. Then run the scan on that drive alone, and reduce the doubled files to one copy each. Nothing unique is ever exposed, because nothing unique was ever the only copy during the operation.

If it doesn't have room: compare in place, but don't wipe

Run the two-drive comparison, quarantine the redundant copies on the old drive, and let that free the space. Then copy what remains on the old drive — which is now, by definition, the material that exists nowhere else — onto the new one. Only after that has landed and been verified does the old drive get formatted.

In both routes the same rule holds: the drive you plan to retire is never the drive you delete from and then trust.

Comparing two drives, step by step

Scanning and the full CSV and HTML reports work without a licence, so you can run all of this and see the result before deciding anything.

  1. Connect both drives and let Windows finish mounting them If either drive spins down when idle, use a powered hub rather than a bus-powered port. A drive that sleeps mid-scan turns a clean result into an interrupted one.
  2. Decide which drive is the keeper before you open anything This decision is the whole answer, and the software will not make it for you. Choose the newer, larger or healthier drive — whichever you intend to still be using next year.
  3. Open the Folder Duplicates tab This is where comparison across two or more locations happens. It has its own folder list and its own scan button, separate from the single Scan Target field in the left-hand rail — that field is for scanning one location on its own.
  4. Add the keeper drive first, then the drive you're retiring Use + Add Folder / Drive… under Folders / Drives to Compare. You can add a whole drive root, not just a folder, and you can add more than two. No naming convention or folder structure is assumed — every file underneath each location is checked by content, at any depth, against every file under the others.
  5. Read the keeper rule before you scan The app states it directly above the scan button: the copy in the first folder listed is the one kept, and matching copies in the folders below it are the ones flagged. There is no drag to reorder — to change which copy wins, select a folder, click Remove Selected, and add it back in the order you want.
  6. Click Scan for Duplicate Files The button directly beneath the folder list — not Start Scan in the left rail, which runs the single-target scan instead. Progress is checkpointed, so a paused or interrupted scan resumes rather than starting over. If the window greys out, Windows is busy, not crashed.
  7. Read the results before you move anything The Duplicate Files Found table gives every row a Role — whether it is the copy being kept or a flagged copy — along with its size, its folder, and a Match column naming which test found it. Select any row to preview the file. A running Recoverable figure, group count and flagged count sit along the bottom. Use Export Report (CSV) before you touch anything: it is your record of what both drives looked like on the day you did this.
  8. Flag, then quarantine Marking rows with Flag for Deletion does not delete anything — it marks them, and Move Flagged to Quarantine is what actually moves them. You can also select rows and use Move Selected to Quarantine directly. Files move to a quarantine folder on their own drive; nothing crosses between drives and nothing is erased. The left rail says so in as many words: Quarantine never deletes.
  9. Use the 24 hours Open the archive in whatever you actually work in — Lightroom, Bridge, Explorer — and confirm it looks right before emptying quarantine. Restore is one click and stays available after the hold expires. The moment you regret a cleanup is usually the moment after it finishes.

The parts that only bite on multi-drive jobs

Most duplicate finders are written for one disk. These are the behaviours that only matter once a second drive is involved, and they're the reason this tool exists at all — it was written to consolidate a 25 TB archive spread across external drives.

The same filename on three drives

DSC_0001.NEF exists on all of them and means something different each time. Identical basenames from different drives are given unique names in quarantine, and each keeps its own recorded origin, so a multi-drive cleanup can be undone one file at a time without ambiguity.

Restoring when the origin drive isn't plugged in

If a file's original location is on a drive that isn't currently connected, the app names that location, leaves the file safely in quarantine, and offers to continue with the rest. It will not redirect anything to a plausible-looking substitute path.

Re-checking at the moment of the move

On cross-folder and cross-drive matches, each flagged file is hashed against the copy you're keeping right as you click quarantine — not only when the scan ran, which on a multi-drive job may have been hours earlier. If a file changed in between, it is skipped and named.

Scans that survive being interrupted

Checkpointing at phase boundaries means a paused or interrupted multi-hour scan resumes: everything already finished stays finished, and only the phase in progress runs again. Over two drives, a scan that has to restart from zero is a scan that never completes.

Before you format the old drive

Four things, in order. None of them takes long, and the drive isn't going anywhere.

  • The CSV report is exported and saved somewhere that isn't either drive
  • Everything that existed only on the old drive has been copied to the keeper, and you have opened a few of those files to confirm they arrived
  • Your catalog — Lightroom, Capture One, Bridge — has been opened against the keeper drive and doesn't report missing files. What a deduplicator does to a catalog is worth reading first if you work in Lightroom
  • You have lived with the result for a week

Formatting is the one step here with no undo. Everything before it is reversible, and there's no prize for reaching the end quickly.

Questions

Do both drives have to be connected at the same time?

Yes. The comparison reads files from both drives during the scan, so both need to be mounted and stay mounted. This is also why a drive that sleeps on idle is worth plugging into a powered hub first.

Does it matter that the drives are different sizes or formatted differently?

No. Content matching reads the files themselves, so cluster size, filesystem and capacity are irrelevant. A file copied from an exFAT drive to an NTFS one is the same file, and is recognised as such.

Which copy gets kept?

The one in the first folder listed in the Folder Duplicates tab. The app states this rule on screen above the scan button, and every result row carries a Role column naming whether it is the copy being kept or a flagged copy. To change which copy wins, remove a folder from the list and add it back in the order you want. The app never tries to guess which drive is your archive.

What about files that exist only on the drive I'm retiring?

They aren't duplicates, so they will not appear in the report — which is exactly why they are the risk. Copy the old drive's contents onto the keeper before deduplicating, or copy what remains after quarantine, but do not treat an empty report as evidence that a drive is safe to wipe.

Can it handle a 12 TB drive?

Yes. The staged tests mean most files are eliminated on size alone without being read, and scans are checkpointed so an interruption doesn't cost you the whole run. Scanning one file category at a time keeps individual passes shorter.

Is anything deleted?

No. Flagged files are moved to a quarantine folder on the same drive and recorded, with one-click restore to their exact original location. Emptying quarantine is a separate decision you make yourself, and restore and permanent delete are never locked behind the licence.

Do I have to buy it to find out what's duplicated?

No. Scanning, the Storage Map and the full CSV and HTML reports are free with no account. The $34 licence is only needed to move flagged files into quarantine.

Run it on your own two drives

The free scan tells you how much of the old drive is genuinely redundant before you commit to anything. If the answer turns out to be "less than I thought", that is a useful thing to learn while both drives still exist.

See what it does → Download and verify