My spreadsheet has thousands of rows with duplicates I need gone, but I want to be careful about what counts as a duplicate and not lose data I need.
What is the right way to remove duplicate rows safely?
My spreadsheet has thousands of rows with duplicates I need gone, but I want to be careful about what counts as a duplicate and not lose data I need.
What is the right way to remove duplicate rows safely?
The be careful instinct is exactly right, because remove duplicates is destructive and the what counts as a duplicate decision is where data gets lost, so the safe procedure:
Work on a copy first, always: duplicate the sheet or the file before removing anything, since Excel's duplicate removal deletes rows permanently with no per row undo once you move on, and a copy means a mistake costs nothing. This single habit separates a safe cleanup from a data loss thread.
The built in tool, Data, Remove Duplicates: select your range, Data, Remove Duplicates and the dialog lists your columns with checkboxes, which is the critical part, since the checked columns define what counts as a duplicate. All columns checked means only fully identical rows are duplicates, while checking just an email column means rows with the same email are duplicates regardless of other differences, keeping only the first. Choosing which columns define duplication is the whole decision, and getting it wrong either misses duplicates or deletes rows that only matched on some fields.
The see before you delete approach for safety: rather than deleting blind, first highlight duplicates to see them, Home, Conditional Formatting, Highlight Cells Rules, Duplicate Values, which colors duplicate values so you can inspect what will go before committing. Or add a helper column with COUNTIF to flag duplicates and review them, deleting only after seeing what qualified, the cautious path when the data matters.
The which one survives detail: Remove Duplicates keeps the first occurrence and deletes the rest, so if it matters which duplicate you keep, the one with the most complete data or the latest date, sort the data first so the keeper is on top, since Excel keeps the first and discards later ones. For thousands of rows where the surviving copy matters, that sort before removing is the difference between keeping good data and keeping arbitrary data.
Worked on a copy, used Conditional Formatting to see the duplicates first, then chose just the email column as the duplicate definition which was exactly my need. Sorted by date first so the latest record survived per your tip. Thousands of rows cleaned without losing anything. The see before delete approach saved me.