Showerthought: I spent 5 years training AI models on bad labeling

I used to just dump raw CSV files into my training pipelines without ever checking if the labels were consistent across batches, and I thought my 72% accuracy was just normal for the dataset. Last month I ran a quick audit on 2,000 samples and realized 30% of them had conflicting labels from different annotators who never talked to each other. Has anyone else hit a similar wall where the data quality was way worse than you assumed?

2 comments

2 Comments

sam_wood6017d ago

I read somewhere that bad labeling is way more common than people admit. Garbage in, garbage out.

blake_smith17d ago

30% bad labels? Rookie numbers, I once hit 60% on a cat vs dog dataset.