Remove Duplicate Lines
Paste a list of emails, names, keywords or links and get each line once. The first copy of every line stays where it was, so your order does not change.
What is the Remove Duplicate Lines?
Remove Duplicate Lines takes a list with one item per line and keeps each line only once, in the order it first appeared. You can ignore differences in upper and lower case, ignore spaces at the start and end of lines, and drop empty lines. It shows how many duplicates were removed. Marketers cleaning email and keyword lists, data entry staff tidying exports and developers cleaning logs use it on lists from a few lines to 100,000.
Who it is for: Marketers cleaning email and keyword lists, data entry staff and developers tidying logs.
How to use the Remove Duplicate Lines
- Paste your list, one item per line.
- Turn on Ignore upper and lower case if "Lagos" and "lagos" should count as the same.
- Copy the unique lines.
Why use it
- Original order keptThe first copy of each line stays where it was.
- Smart matchingIgnore case and stray spaces so near-identical lines are caught.
- See what changedA count shows how many unique lines remain and how many were removed.
- Big listsTested with 100,000 lines in the browser.
At a glance
- What you put in
- A list with one item per line
- What you get
- Each line once, in the original order, with a count of what was removed
- Limits
- Tested with 100,000 lines
- Privacy
- Runs in your browser. Nothing you enter is uploaded or stored.
- Price
- Free for everyone. No subscriptions, no payments, no watermark.
- Works offline
- Yes, once the page has loaded
- Last checked
- by AK Shine
Worked example
apple banana Apple banana cherry
apple banana cherry
With Ignore upper and lower case on, "Apple" counts as a repeat of "apple".
When to use something else
- To compare two different lists and see what is in one but not the other, a text compare tool is the better choice.
Questions people ask
Does it keep my original order?
Yes. The first time a line appears it stays in place, and later copies are dropped. Use Sort Lines Alphabetically afterwards if you want it in A to Z order.
Can it handle large lists?
Yes. Tens of thousands of lines take a fraction of a second, because it all runs in your browser.
Why are two lines that look the same not removed?
One probably has an extra space or a different capital letter. Keep Ignore spaces on, and turn on Ignore upper and lower case.
How do I remove duplicates in Excel instead?
Select the column, open the Data tab and choose Remove Duplicates. This tool is quicker for lists that are not in a spreadsheet yet.
Does it remove duplicate words inside a line?
No. It compares whole lines. Put one item per line first if you want to remove repeated words.
Are blank lines kept?
Remove empty lines is on by default, so blank lines are dropped. Turn it off to keep one blank line where it first appears; later blank lines count as duplicates.