Assessing the robustness of negative vascular surgery randomized controlled trials using their reverse fragility index

Allen Li, Arshia P. Javidan, Eva Liu, Aryan Ahmadvand, Derrick Y. Tam, Faysal Naji, Thomas L. Forbes

Research output: Contribution to journalReview articlepeer-review

2 Scopus citations


Objective: The reverse fragility index (RFI) describes the number of event conversions needed to convert a statistically nonsignificant dichotomous outcome to a significant one. The objective of the present study was to assess the RFI of vascular surgery randomized controlled trials (RCTs) comparing endovascular vs open surgery for the treatment of abdominal aortic aneurysms (AAAs), carotid artery stenosis (CAS), and peripheral artery disease (PAD). Methods: MEDLINE and Embase were searched for RCTs that had investigated AAAs, CAS, or PAD with statistically nonsignificant binary primary outcomes. The primary outcome for the present study was the median RFI. Calculation of the RFI was performed by creating two-by-two contingency tables and subtracting events from the group with fewer events and adding nonevents to the same group until a two-tailed Fisher exact test had produced a statistically significant result (P ≤ .05). Results: Of 4187 reports, 49 studies reporting 103 different primary end points were included. The overall median RFI was 7 (interquartile range [IQR], 5-13). The specific RFIs for AAA, CAS, and PAD were 10 (IQR, 6-15.5), 6 (IQR, 5-9.5), and 7 (IQR, 5.5-10), respectively. Of the 103 end points, 42 (47%) had had a loss to follow-up greater than the RFI, of which 10 were AAA trials (24%), 23 were CAS trials (55%), and 9 were PAD trials (21%). The Pearson correlation demonstrated a significant positive relationship between a study's RFI and the impact factor of its publishing journal (r = 0.38; 95% confidence interval [CI], 0.20-0.54; P < .01), length of follow-up (r = 0.43; 95% CI, 0.26-0.58; P < .01), and sample size (r = 0.28; 95% CI, 0.09-0.45; P < .01). Conclusions: A small number of events (median, 7) was required to change the outcome of negative RCTs from statistically nonsignificant to significant, with 47% of the studies having missing data that could have reversed the finding of its primary outcome. Reporting of the RFI relative to the loss to follow-up could be of benefit in future trials and provide confidence regarding the robustness of the P value.

Original languageEnglish
Pages (from-to)253-259.e11
JournalJournal of Vascular Surgery
Issue number1
StatePublished - Jul 2023
Externally publishedYes


  • Abdominal aortic aneurysm
  • Carotid artery stenosis
  • Clinical trials
  • Fragility
  • Methodologic epidemiology
  • Peripheral artery disease
  • Reverse fragility


Dive into the research topics of 'Assessing the robustness of negative vascular surgery randomized controlled trials using their reverse fragility index'. Together they form a unique fingerprint.

Cite this