A structured reference for understanding deep learning techniques — how different architectures compare, where each fits, and what the published research actually shows about their performance characteristics.
Selected figures from published research — not projections
Top-1 accuracy on ImageNet achieved by Vision Transformers (ViT-H/14) — a figure that took convolutional networks nearly a decade to approach through incremental architecture changes.
Approximate training time difference between a ResNet-50 baseline and a comparably performing transformer model on the same hardware — attention mechanisms are expensive before optimizations kick in.
GPT-3 parameter count — a figure that made large language models a serious engineering challenge, not just a research curiosity. Modern efficient variants achieve similar capability at a fraction of that scale.
Minimum labeled examples where Vision Transformers begin to outperform CNNs without pre-training — below that threshold, convolutional inductive biases still hold a clear practical advantage.
Each major deep learning architecture emerged to solve a specific structural problem. Understanding what problem each one targets makes their design choices obvious rather than arbitrary.
CNNs assume nearby pixels share information. That assumption is correct for most natural images, which is why they dominated computer vision for over a decade despite being wrong about long-range dependencies.
RNNs process sequences one step at a time, carrying a hidden state forward. LSTMs added gating to control what gets remembered — a practical fix that kept recurrent models viable for nearly fifteen years.
GANs pit a generator against a discriminator in a minimax game. The training instability is real and well-documented, but the output quality on image synthesis tasks made them the dominant generative approach before diffusion models emerged.
The attention mechanism computes relationships between every pair of positions simultaneously. Quadratic memory cost is the tradeoff — sparse attention, linear attention, and flash attention are all engineering responses to that same constraint.
Diffusion models learn to reverse a gradual noising process. Generation is slow compared to GANs — typically dozens to hundreds of forward passes — but the training stability and mode coverage are substantially better.
GNNs operate on arbitrary graph topology rather than fixed grids or sequences. Molecular property prediction, recommendation systems, and traffic forecasting all share the same underlying structure that makes GNNs a natural fit.
Choosing an architecture is a constraint-satisfaction problem. The table below maps common task categories to the architectures that have demonstrated consistent results — not theoretical potential.
| Task Category | Primary Architecture | Compute Demand | Data Requirement | Practical Ceiling |
|---|---|---|---|---|
| Image classification | ViT / EfficientNet | Medium–High | Large (> 1M samples) | 91% top-1 ImageNet |
| Object detection | DETR / YOLOv8 | Medium | Moderate (COCO-scale) | 63 mAP on COCO |
| Text generation | GPT / LLaMA family | Very High | Massive (web-scale) | Task-dependent BLEU |
| Text classification | BERT / RoBERTa | Low–Medium | Small viable with fine-tune | 96% on SST-2 |
| Time series forecasting | LSTM / Temporal Fusion | Low | Domain-specific sequences | Varies by horizon |
| Image synthesis | Stable Diffusion | High (inference: medium) | Large image-text pairs | FID < 5 on benchmarks |
| Molecular property prediction | GAT / MPNN | Low | Small (thousands of molecules) | MAE < 0.1 eV on QM9 |
| Speech recognition | Whisper / Conformer | Medium | Hundreds of hours minimum | 3–5% WER on LibriSpeech |
Figures above reflect published results under controlled benchmark conditions. Real-world performance depends heavily on data quality, fine-tuning strategy, and hardware configuration — treat these numbers as orientation points, not guarantees.
Major architecture families covered across vision, language, and structured data tasks
Year the original Transformer paper appeared — most architectures above derive from that single publication
Typical gap in inference speed between a fine-tuned BERT-base and a comparably accurate GPT-2 medium on classification
Standard patch size used in the original ViT paper — smaller patches improve accuracy but increase sequence length quadratically