Active Learning for Systematic Reviews
How machine learning prioritizes likely-relevant records, where workload savings can arise, and why recall must be validated for each review.
What Is Active Learning?
Active learning is a machine learning approach where the model learns from your screening decisions during screening. As you include or exclude papers, Lumina refreshes the semantic ranking; after enough new decisions, it retrains the classifier and blends that signal into the queue.
You Screen
Make a few include/exclude decisions
AI Learns
Ranking updates; classifier retrains periodically
Re-rank
Remaining papers are re-ordered
Evaluate
Validate before changing the stopping point
How It Works Under the Hood
Text Representation
Each paper's title and abstract are converted into a numerical representation (embedding) that captures its semantic meaning.
Relevance Ranking
Your inclusion criteria are converted to the same embedding space. Papers are then ranked by similarity — the more similar to your criteria, the higher they rank.
Uncertainty Sampling
Some active learning systems also prioritize papers the model is most uncertain about. These are the most informative for learning.
Stopping Criteria
Once the observed yield becomes low, stopping strategies can prompt a review of whether to continue. They do not guarantee that no relevant records remain.
What Does the Evidence Say?
What published evaluations support
van de Schoot et al. (2021) describe ASReview and show that active-learning performance can be evaluated retrospectively on known review datasets. Read the paper.
O'Mara-Eves et al. (2015) found promising uses for text mining, while also emphasizing heterogeneity in methods and evaluation. Read the systematic review.
Practical implication: results from one benchmark do not establish recall for a new review. Dataset size, prevalence, seed records, labeling quality, ranking model, and stopping rule all affect performance.
Active Learning vs. Manual Screening
| Manual Screening | Active Learning | |
|---|---|---|
| Order | Random or chronological | Most relevant first |
| Workload | Screen 100% of papers | Potentially fewer, after review-specific validation |
| Recall | ~100% (human error aside) | Dataset-dependent; estimate and report uncertainty |
| Adapts | No | Yes — learns from each decision |
How to Report AI-Assisted Screening
If you use active learning, transparently report:
- ✓ The tool used (e.g., Lumina, ASReview, Rayyan)
- ✓ The stopping criteria applied
- ✓ Total papers vs. papers actually screened
- ✓ Whether dual screening was used alongside AI
- ✓ Any sensitivity analysis or validation performed
Continue Learning
Experience Active Learning in Action
Try Lumina's AI screening with a demo dataset and see how active learning re-ranks papers after each decision.