When to Stop Screening in a Systematic Review
Evidence-based stopping strategies that save 30-70% time while finding 95% of relevant papers. Free guide for researchers.
Why Early Stopping?
In AI-assisted screening, relevant papers are prioritized. Often, the last 50% of the queue contains zero relevant papers. Screening them is "wasted effort."
Early stopping strategies provide statistical stopping points, potentially reducing workload by 30-70% while maintaining high recall (typically >95%).
Efficiency vs. Completeness. No method is perfect. There is always a small statistical risk of missing a relevant paper that appears very late in the queue.
Screen more to miss less. Early stopping allows you to search more broadly initially (e.g., 5,000 papers instead of 2,000) without being overwhelmed. Broadening your search actually lowers the risk of excluding relevant studies during the database search phase.
Consecutive Irrelevant
Best for simplicity & safety
Knee Detection
Best for large datasets
Slope Threshold
Best for continuous monitoring
Scenario Guide
| Scenario | Recommended Strategy | Why? |
|---|---|---|
|
🚀
Rapid / Scoping Review
|
Any (Aggressive Settings) | Time is critical. Missing one minor paper is acceptable. Use Consecutive (50) or Knee (1.2). |
|
📊
Standard Systematic Review
|
Consecutive Irrelevant | Offers the best balance of safety and ease of explanation for methodology sections. Set threshold to 100. |
|
📚
Large Dataset (>2000)
|
Knee Detection | Large datasets have clearer statistical distributions, making the "knee" (point of diminishing returns) very reliable. |
|
🏥
Clinical / Safety Critical
|
Screen All (One-Pass) | If missing even a single study has clinical safety implications, do not use early stopping. Screen 100%. |
Consecutive Irrelevant Rule
"Stop after seeing X irrelevant papers in a row."
This is the most intuitive strategy. Since AI ranks relevant papers to the top, relevant papers become rarer as you go. A long streak of nothing but irrelevant papers is a strong signal that you have exhausted the relevant set.
Parameter Selection
Knee Detection Method
"Find the mathematical point of diminishing returns."
Plots relevant papers found vs. total papers screened. The curve is initially steep (easy finds), then flattens. The "knee" is the sharp turn in this curve. We suggest stopping after passing this knee by a safety margin.
Parameter: Past Knee Factor
Screen 50% more papers after detected knee. Good balance.
Double the workload after knee. Safer for publications.
Slope/Gradient Threshold
"Stop when relevance rate drops below X%."
Continuously calculates the % of relevant papers in the last N papers (Window Size). If the current "yield" is lower than your Min Rate, it prompts to stop.
Parameter: Rate & Window
Stop if fewer than 2.5 relevant papers found in last 50.
Golden Rules
Always state in your methodology which strategy and parameters you used.
You can always check 50 more papers if unsure. You can't un-check what you skipped.
These strategies rely on the AI ranking working well. If the ranking feels random, don't use early stopping.