Freelancers often hear that deep learning requires massive datasets. That framing is partially accurate and frequently misapplied.
When Data Volume Actually Matters
Training a vision model from scratch on a narrow industrial defect category with 80 labeled images will likely underperform a classical SVM with hand-crafted features. Data volume matters most when the model needs to learn general representations without prior knowledge baked in.
What Changes with Pre-Trained Weights
Fine-tuning a model pre-trained on a large general corpus changes the equation. A BERT model fine-tuned for contract clause classification can perform well with 300 to 500 labeled examples if the domain vocabulary is not highly specialized. The pre-trained weights carry the general language understanding; the fine-tuning step adjusts for the specific task.
Data Quality vs. Data Volume
- Mislabeled training examples degrade performance faster than small dataset size
- Class imbalance is a more common failure point than insufficient total samples
- Augmentation can extend small datasets but does not substitute for label accuracy
Recommending a data collection sprint before evaluating what pre-trained options exist is a pattern that delays projects without necessarily improving outcomes.