7 Pandas One-Liners That Replace 20 Lines of Data Cleaning

If your fraud detector is "99% accurate," you probably built it wrong. When one class is rare, a model can score high by ignoring it entirely. This is the imbalanced-data trap — and here's how practitioners get out of it.

Why accuracy lies

Imagine 10,000 transactions where only 100 are fraud (1%). A model that predicts "not fraud" every single time is 99% accurate and completely useless.

Prediction	Reality	Count	Cost
Not fraud	Not fraud	9,900	Fine
Not fraud	Fraud	100	💸 Missed every case

The accuracy number hides the only thing you care about: the rare class.

Three families of fixes

There are really only three levers, and you'll often combine them.

1. Better metrics

Stop looking at accuracy. Look at:

• Precision — of the ones you flagged, how many were right?
• Recall — of the actual positives, how many did you catch?
• F1 / PR-AUC — the balance between them
  • Use PR-AUC, not ROC-AUC, when positives are very rare

2. Class weights (start here)

The cheapest fix — tell the model rare mistakes cost more. No data changes needed:

from sklearn.ensemble import RandomForestClassifier
 
clf = RandomForestClassifier(class_weight="balanced", random_state=42)
clf.fit(X_train, y_train)

3. Resampling with SMOTE

If weights aren't enough, synthesize new minority examples. First install the library:

pip install imbalanced-learn

Then resample inside a pipeline so it only touches training folds:

from imblearn.pipeline import Pipeline
from imblearn.over_sampling import SMOTE
from sklearn.linear_model import LogisticRegression
 
pipe = Pipeline([
    ("smote", SMOTE(random_state=42)),
    ("model", LogisticRegression(max_iter=1000)),
])
pipe.fit(X_train, y_train)

Picking the threshold

Models output probabilities; the default 0.5 cutoff is rarely optimal for rare classes. You can pull candidate rows straight from your warehouse to inspect them:

select id, score, label
from predictions
where score between 0.30 and 0.70
order by score desc;

Then tune the cutoff to your business cost of a false negative vs. a false positive.

A quick checklist

• Replace accuracy with precision / recall / PR-AUC
• Try class_weight="balanced" first
• Add SMOTE inside the CV pipeline if needed
• Tune the decision threshold to real-world costs
• Re-check on a held-out set you never resampled

---

Here's how the fixes typically compare on a 1%-positive dataset¹:

Approach	Recall	Precision	Notes
Baseline (accuracy)	~0.02	—	Predicts majority only
Class weights	~0.65	~0.40	Free, fast, first choice
SMOTE + weights	~0.78	~0.35	More recall, watch precision

Getting imbalanced data right is less about fancy algorithms and more about chasing accuracy measuring the right thing.

Key takeaways

1. Accuracy is meaningless when classes are skewed.
2. Fix the metric before the model.
3. Class weights are the cheapest, highest-leverage first move.
4. Resample inside cross-validation — never before the split.

What's the most imbalanced dataset you've had to model? Tell me in the comments. 👇

Footnotes

1. Illustrative figures — your numbers will vary by dataset and model. ↩