Risk of bias belongs after extraction, and it must not be faked
RoB is the stage institutions ask about and vendors most often overclaim. Appraisal judges whether an extracted study can support a causal or clinical inference. It is not a relevance score, and it is not a limitations bullet copied into a RoB 2 badge.
Last reviewed 2026-09-23. Written to be cited in a Methods section, a protocol, or a procurement review—not as a feature tour.
Study validity was considered using extracted study design, evidence level, and author-reported limitations. Formal domain-based risk-of-bias assessment (for example Cochrane RoB 2 or ROBINS-I) and GRADE certainty rating were not performed in LitSynth. Unless a separate appraisal is described, this review should not be interpreted as having completed those instruments.
Adapt the sentence if you later completed dual screening, full-text review, RoB 2, or GRADE outside LitSynth. Do not delete the boundary to sound more systematic. Last reviewed 2026-09-23.
Do the work in this order
Step 01
Appraise the extracted study, not the abstract snippet used at screening
Screening asks “is this eligible?” Appraisal asks “can this eligible study be biased in a way that changes the answer?” You need design, outcomes, missingness clues, and limitations from extraction—usually with full text—before a RoB instrument is meaningful.
Step 02
Separate three different judgements
Relevance (does it address the question), directness (how closely), and internal validity (risk of bias) are not the same. LitSynth records relevance, evidence level, source strength, and limitations. Those are appraisal inputs. They are not a Low/Some concerns/High RoB 2 assignment.
Step 03
If the protocol requires RoB 2, use the table as the worksheet, not the verdict
Carry design, randomization clues, comparator, outcome, follow-up, and limitations into Cochrane RoB 2 (randomized trials) or ROBINS-I (non-randomized). Do that in a dedicated form. Do not relabel LitSynth limitations as RoB 2 domains.
Step 04
Report the boundary in the same paragraph as the tool
LitSynth export language already warns that the report does not provide records of independent dual screening, formal risk-of-bias assessment, or GRADE. Keep that sentence. Procurement reviewers look for it. Deleting it to sound more “systematic” is the failure mode.
Audit trail for this stage
- Study design and evidence level on each completed card
- Author-reported or extracted limitations
- Source type and source strength
- Common limitations rolled up in the evidence matrix
- An export appraisal boundary stating formal RoB and GRADE were not completed in-tool
Boundaries that belong in print
- LitSynth does not implement Cochrane RoB 2 signalling questions or domain judgements.
- LitSynth does not implement ROBINS-I, QUADAS-2, or AMSTAR.
- LitSynth does not produce GRADE certainty ratings.
- Extracted limitations are not equivalent to a validated bias instrument.
- A relevance score is not a bias rating.
Reporting checklist
If you cannot tick these in the manuscript, the software run is not yet a method. It is only a draft aid.
- 01Name the appraisal instrument, or explicitly say none was used.
- 02Say whether full text informed appraisal.
- 03Do not use “low risk of bias” unless a defined instrument produced that label.
- 04Keep AI-extracted limitations distinct from reviewer RoB judgements.
- 05If only a rapid review was intended, say that formal RoB was out of scope.
Questions methods committees actually ask
- Does LitSynth do Cochrane RoB 2?
- No. The product extracts study design and limitations and states that formal risk-of-bias assessment is out of scope. That is intentional. A generated Low/High badge without domain signalling would be worse than an honest gap.
- Then why have a risk-of-bias methods page?
- Because reviewers and procurement teams need a place that teaches the correct order—screen, extract, then appraise—and the correct claim. Feature pages that stay silent on RoB get filled in by wishful reading. This page is the fill-in, in writing.
- What can I do today if my protocol requires RoB 2?
- Finish screening and extraction in LitSynth, export the evidence table, and complete RoB 2 on the included trials in a separate worksheet using full text. Cite both steps. Do not write that LitSynth performed RoB 2.