Transformer-based models have largely displaced RNNs in research, but freelance NLP projects do not always follow research trends.
Where RNNs Still Show Up
Clients with legacy pipelines built on LSTMs or GRUs often need maintenance, fine-tuning, or incremental improvements rather than full rewrites. Rewriting a working LSTM sentiment classifier into a BERT-based system may double inference latency on their existing hardware while offering marginal accuracy gains on their specific dataset.
The Transformer Assumption Problem
Some freelancers now default to transformer architectures for any sequence task. On short, structured sequences like log parsing or time-series anomaly detection, a well-tuned LSTM can match transformer performance at a fraction of the computational cost. The architecture choice should follow the data characteristics, not the publication date of the paper.
Practical Factors in Architecture Selection
- Sequence length and whether attention overhead is justified
- Available training data volume
- Client infrastructure and deployment environment
Staying current with research is useful, but freelance decisions are constrained by client budgets and existing systems, not benchmark leaderboards.