How to clean up text before pasting it somewhere

The problem is that text remembers where it came from

Copy a paragraph out of a PDF and paste it into an email, and it arrives broken into lines about eleven words long. Copy a column out of a spreadsheet and it arrives with trailing tabs. Copy anything out of a word processor and it may arrive carrying characters that look like spaces and are not.

None of this is visible. That is what makes it annoying: the text looks right, and then a search fails, an import rejects half the rows, or a heading comes out in capitals in a place that does not use capitals.

The fixes are individually trivial. What matters is the order, because each step changes what the next one can see.

Do it in this order

First, the characters you cannot see. Non-breaking spaces, zero-width joiners, and the two different ways of writing an accented letter. A find-and-replace run before this step is searching for something that is not there, and a duplicate check run before it will treat two identical-looking lines as different.

Unicode calls the last of these normalisation, and it is worth knowing about even if you never think about it again: é can be stored as one code point or as e plus a combining accent, and the two are not equal. Text assembled from more than one source routinely contains both.

Then the line breaks. Remove line breaks joins lines back into paragraphs. The decision it has to make is which breaks were the author’s and which came from the layout, and the rule that works is that a blank line separates paragraphs while a line ending mid-sentence was the column width running out.

Do this before anything that works line by line, or you are sorting and deduplicating fragments of sentences.

Then duplicates, and sort first. Remove duplicate lines is exact: two lines are the same or they are not. Sorting first puts identical lines next to each other so you can see what is about to disappear. Deleting rows you have not looked at is how a list of five hundred customers quietly becomes four hundred and ninety.

Then substitutions. Find and replace after normalisation, not before, for the reason above. This is also where you strip the trailing punctuation, the stray tabs, and the [1] footnote markers that came along from the PDF.

Case last. Changing case is the one step that destroys information: once a name is lowercased, nothing can tell you it was McDonald rather than Mcdonald. Do it at the end, on text you have already checked, and only when the destination genuinely needs it.

Line breaks are the hard one, and here is why

A PDF does not store paragraphs. It stores glyphs at coordinates. What you read as a paragraph is a set of lines that happen to sit under one another, and any tool extracting the text sees exactly that: a list of lines.

So joining them back is inference. It has to be, and the honest thing is to say so rather than promise a perfect reconstruction. The rules that get it right most of the time:

• A line ending in a full stop, question mark or colon probably ends a sentence. Keep the break.
• A line ending in a hyphen was probably a word split across two lines. Join without a space.
• A blank line is almost always a real paragraph break. Keep both.
• Everything else is the column width running out. Join with a space.

Where it goes wrong is poetry, addresses, code, and bulleted lists - all of which use short lines on purpose. If your text is one of those, keep the breaks and fix something else.

Slugs are a different job with the same feel

Turning “Résumé - Final (v3)” into resume-final-v3 looks like cleaning and is actually encoding. A URL has a defined character set, and everything outside it either gets escaped into %C3%A9 or has to be replaced by something inside it.

Slugify does the replacing: accents folded to their base letters, punctuation dropped, spaces turned into hyphens, case flattened. The result is stable, readable, and, the point of it, the same every time, which is what a URL has to be.

The one thing to watch is that folding is lossy in some scripts in ways it is not in others. Latin text with accents comes through recognisably. Text in a script with no Latin equivalent does not, and a slug of empty hyphens is worse than a percent-encoded URL that at least round-trips.

None of this needs an upload

Every tool named here runs in your browser. The text you are cleaning is frequently the text you would least like to paste into a stranger’s server: a customer list, a contract, an internal draft, a set of email addresses.

That is not a claim about intent, it is a property of where the code runs. Nothing leaves the tab, which you can check yourself in your browser’s network panel while you use it.