QUADAS-3: Updated Tool to Evaluate Risk of Bias and Applicability Concerns in Diagnostic Test Accuracy Studies

Penny Whiting; Eve Tomlinson; Bada Yang; Clare  Davenport; Mariska M. Leeflang; Sue Mallett; Anne W.S. Rutjes

doi:10.54103/2282-0930/29260

Authors

Penny Whiting Population Health Sciences, Bristol Medical School, University of Bristol
Eve Tomlinson Population Health Sciences, Bristol Medical School, University of Bristol
Bada Yang Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University
Clare Davenport Test and Prediction Group, Department of Applied Health Science, University of Birmingham ; NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
Mariska M. Leeflang Amsterdam University Medical Centers, University of Amsterdam
Sue Mallett Centre for Medical Imaging, University College London
Anne W.S. Rutjes UniCamillus - Saint Camillus International University of Health and Medical Sciences

DOI:

https://doi.org/10.54103/2282-0930/29260

Abstract

Introduction: The QUADAS‐2 tool, published in 2011, was designed to evaluate the risk of bias and applicability of diagnostic test accuracy (DTA) studies. The publication reporting QUADAS-2 has been cited over 12, 000 times and it is the recommended tool to assess risk of bias and applicability of studies for major HTA organizations. Although feedback on QUADAS-2 has generally been positive, some signaling questions have been identified as problematic and the tool could be improved based on features included in more recently developed tools.

Objectives: To update QUADAS-2 to develop the new QUADAS-3 tool.

Methods: We established a core-group of methodological experts to lead the development of the QUADAS-3 tool supported by a wider steering group.

We followed the following steps:

Summarised modifications made to QUADAS-2 for the Cochrane Handbook
Web-based survey of reviewers that have used QUADAS-2
Considered developments from more recent tools in terms of tool structure and implementation
Undertook a review of methodological studies that had evaluated QUADAS-2
Undertook a review of 50 Cochrane DTA reviews to highlight challenges with the assessment of applicability

We have produced a draft tool which has undergone piloting. The results of the piloting, which also included a comparison of the use of signalling questions with signalling statements, was used to inform the final version of the tool.

Results

The new tool follows a similar structure to the QUADAS-2 tool but with some major updates. Key changes include:

An option to define separate synthesis questions rather than just a single review question
A new section on defining the ideal test accuracy trial for each synthesis question
Assessment of risk of bias and applicability at the accuracy estimate level rather than the study level
A change in answers to signaling questions to include options of “probably yes” and “probably no” and to replace “unclear” with “no information”
Replacement of “Flow and Timing domain” with new “Analysis” domain
Changes to some signaling questions
Inclusion of a section for judging overall risk of bias and applicability (across domains)

Conclusions: The QUADAS-3 tool incorporates several changes compared to the previous version (QUADAS-2) which we hope will improve its validity, usability, and usefulness. QUADAS-3 will be introduced at the conference and the results of piloting discussed.