7 Pandas One-Liners That Replace 20 Lines of Data Cleaning

Most data-cleaning code is longer than it needs to be. Not wrong, just long. Loops where a vectorized call would do, if chains where a mapping would do, three lines where one would read better.

Here are seven one-liners that each replace a small pile of code. All of them are vectorized, which means pandas pushes the work down into NumPy instead of stepping through your DataFrame row by row.

1. Drop columns that are mostly empty

You inherit a CSV with 60 columns and half of them are 90% null. Don't inspect them one at a time:

# Keep only columns with at least 50% non-null values
df = df.dropna(axis=1, thresh=int(len(df) * 0.5))

thresh is the minimum number of non-null values a column must have to survive. Set the ratio, let pandas do the counting.

2. Fill missing values by group

This is the one people write ten lines for. "Fill each missing salary with the median salary for that job title":

df["salary"] = df["salary"].fillna(
    df.groupby("title")["salary"].transform("median")
)

transform is the key. It returns a Series aligned to the original index: one median per row, matched to that row's group, so it slots straight into fillna.

3. Standardize messy text in one pass

Real-world text columns are a mess of stray whitespace and inconsistent casing, " lagos", "LAGOS", and "Lagos " all become three different categories:

df["city"] = df["city"].str.strip().str.title()

The .str accessor chains. Three cleanups, one line, no loop.

4. Deduplicate, keeping the most recent record

Not just "drop duplicates", drop duplicates and keep the right one:

df = df.sort_values("updated_at").drop_duplicates("id", keep="last")

Sort first, then keep="last" gives you the freshest row per id. Change to keep="first" for the original. Most people write a groupby-and-merge for this.

5. Cap outliers instead of deleting them

Deleting outliers throws away real customers. Clipping keeps the row and tames the value:

df["amount"] = df["amount"].clip(*df["amount"].quantile([0.01, 0.99]))

That squeezes everything into the 1st–99th percentile range. quantile returns two values, and * unpacks them straight into clip as the lower and upper bounds.

6. Coerce bad values instead of crashing

A single "N/A" in a numeric column will break your whole pipeline. Stop fighting it:

df["amount"] = pd.to_numeric(df["amount"], errors="coerce")
df["signup"] = pd.to_datetime(df["signup"], errors="coerce")

errors="coerce" turns anything unparseable into NaN instead of raising. Now it's a missing-value problem, and you already know how to handle those (see #2).

7. Replace an if chain with a mapping

The code everyone writes:

# Don't do this
def get_tier(plan):
    if plan == "enterprise":
        return "high"
    elif plan == "pro":
        return "mid"
    ...
df["tier"] = df["plan"].apply(get_tier)

The code you should write:

df["tier"] = df["plan"].map({
    "enterprise": "high",
    "pro": "mid",
    "free": "low",
}).fillna("unknown")

map is vectorized; apply with a Python function is a loop wearing a disguise. The fillna("unknown") catches anything not in your mapping, which is the case the if chain always forgets.

Quick reference

Task	One-liner
Drop mostly-empty columns	df.dropna(axis=1, thresh=int(len(df)*0.5))
Fill by group	df.groupby(k)[c].transform("median")
Clean text	df[c].str.strip().str.title()
Dedupe, keep newest	df.sort_values(t).drop_duplicates(k, keep="last")
Cap outliers	df[c].clip(*df[c].quantile([.01, .99]))
Coerce bad values	pd.to_numeric(df[c], errors="coerce")
Map instead of branch	df[c].map(mapping).fillna("unknown")

Key takeaways

1. iterrows is a smell. If you're looping, there's a vectorized version.
2. transform, not agg, for group-wise fills, it's the one that stays aligned.
3. errors="coerce" turns crashes into missing values, which are a solved problem.
4. map beats apply for lookups. apply is a loop in disguise.

Which of these have you been writing the long way? Or is there a one-liner you'd add to the list? 👇