LibGuides: Knowledge Synthesis: Critical Appraisal

About

Garegnani, L. I. (2023). Bias, quality and reporting in health research: Differences and tools for appraisal. BMJ Evidence-Based Medicine. https://doi.org/10.1136/bmjebm-2023-112260

Critical appraisal tools

Critical appraisal is an integral part of evidence-based medicine. It can also be referred to as "quality assessment" or "risk of bias". The organizations listed below offer a variety of checklists to help with the process of critically appraising evidence

Risk of Bias Tools

RoB 2 (Risk of Bias in Randomized Trials)
The RoB 2 tool provides a framework for considering the risk of bias in the findings of any type of randomized trial. The assessment is specific to a single trial result that is an estimate of the relative effect of two interventions or intervention strategies on a particular outcome. We refer to the interventions as the experimental intervention and the comparator intervention, although we recognize that the result may sometimes refer to a comparison of two active interventions.
ROB-ME (Risk of Bias due to Missing Evidence)
The ROB-ME tool is designed for authors or users of systematic reviews to assess risk of bias due to missing evidence in pairwise meta-analyses of the effects of interventions, in any other type of
pairwise synthesis that yields a point estimate of an intervention effect (e.g. calculation of the median effect across studies when meta-analysis is not possible or appropriate), or in syntheses consisting of only one study with available results.
ROBINS-E tool (Risk of Bias In Non-randomized Studies - of Exposures)
The Risk Of Bias In Non-randomized Studies - of Exposure (ROBINS-E) tool provides a structured approach to assessing the risk of bias in observational epidemiological studies. ROBINS-E is designed primarily for use in the context of a systematic review. It should contribute to a thorough examination of the strength of evidence about the presence of, and/or nature of, a potential effect of an exposure on an outcome. A key feature of the ROBINS-E approach is the specification, for each study, of the causal effect estimated by the result under consideration.
ROBINS-I (Risk Of Bias In Non-randomized Studies of Interventions)
Evaluates the risk of bias (RoB) in the results of non-randomized studies of interventions (NRSI) that compare the health effects of two or more interventions. The types of NRSI that can be evaluated using this tool are quantitative studies estimating the effectiveness (harm or benefit) of an intervention, which did not use randomization to allocate units (individuals or clusters of individuals) to comparison groups. This includes studies where allocation occurs during the course of usual treatment decisions or peoples’ choices: such studies are often called “observational”.
robvis (visualization tool)
robvis is a web app designed to for visualizing risk-of-bias assessments performed as part of a systematic review. The figures are of publication quality. The tool creates:
1. “traffic light” plots of the domain-level judgements for each individual result
2. Weighted bar plots of the distribution of risk-of-bias judgments within each bias domain.

Downes, M. J., Brennan, M. L., Williams, H. C., & Dean, R. S. (2016). Development of a critical appraisal tool to assess the quality of cross-sectional studies (AXIS). BMJ Open, 6(12), e011458. https://doi.org/10.1136/bmjopen-2016-011458
Murad, M. H., Sultan, S., Haffar, S., & Bazerbachi, F. (2018). Methodological quality and synthesis of case series and case reports. BMJ Evidence-Based Medicine, 23(2), 60–63. https://doi.org/10.1136/bmjebm-2017-110853
Risk of bias tools—RoB 2 tool. (n.d.). Retrieved July 6, 2023, from https://www.riskofbias.info/welcome/rob-2-0-tool
Viswanathan, M., Patnode, C. D., Berkman, N. D., Bass, E. B., Chang, S., Hartling, L., Murad, M. H., Treadwell, J. R., & Kane, R. L. (2017). Assessing the Risk of Bias of Individual Studies in Systematic Reviews of Health Care Interventions. Agency for Healthcare Research and Quality. https://doi.org/10.23970/AHRQEPCMETHGUIDE2
Viswanathan, M., Patnode, C. D., Berkman, N. D., Bass, E. B., Chang, S., Hartling, L., Murad, M. H., Treadwell, J. R., & Kane, R. L. (2018). Recommendations for assessing the risk of bias in systematic reviews of health-care interventions. Journal of Clinical Epidemiology, 97, 26–34. https://doi.org/10.1016/j.jclinepi.2017.12.004

Knowledge Synthesis

About

Critical appraisal tools

Risk of Bias Tools

Read more