Thompson E. et al., 2020: A Longitudinal Assessment of the Reporting Quality of Randomized Controlled Trials for Surgical Interventions to Treat Nephrolithiasis Over 16 Years (2002 to 2017)

Thursday, 04 June 2020

3925 Hits

Thompson E, Lai A, Morrey L, Borofsky MS, Dahm P.
Department of Urology, University of Minnesota, Minneapolis, Minnesota, USA.
Urology Section, Minneapolis VA Health Care System, Minneapolis, Minnesota, USA.
Department of Urology, University of Illinois, Chicago, Illinois, USA.

Abstract

PURPOSE: Transparently reported, high-quality randomized controlled trials (RCTs) play a critical role in guiding evidence based clinical practice and informing evidence-based guidelines in patients with nephrolithiasis. Prior studies have found reporting quality to be low. We performed this study to assess whether the reporting of RCTs has improved over time.

MATERIALS AND METHODS: This study was governed by an a priori protocol. We performed a systematic literature search for RCTs analyzing nephrolithiasis treatment. Selection of eligible studies and data abstraction were performed by two of three reviewers independently and in duplicate. We developed and pilot tested a data extraction checklist based on the Consolidated Standards of Reporting Trials (CONSORT) criteria on a scale of 0 to 25. Our primary outcome measure was the mean CONSORT score. We performed statistical hypothesis testing to compare scores between 2002-2006, 2007-2011, and 2012-2017.

RESULTS: A total of 203 studies (2002-06: 38; 2007-11: 64; 2012-17: 101) met inclusion criteria. The most common procedure types studied were percutaneous nephrolithotomy (35.1%), shockwave lithotripsy (25.4%), and ureteroscopy (22.9%). Asia contributed an increasing proportion of studies (25.6%, 44.6%, and 74.3%, respectively) in these three time periods. The main journals of publication were the Journal of Endourology (23.9%), the Journal of Urology (19.5%), and Urology (8.3%). The mean ± standard error of the CONSORT summary scores was 11.4 ± 0.4, (2002 to 2006), 12.1 ± 0.3, (2007 to 2011), and 13.3 ± 0.4 (p = 0.003) reflecting an increase by 1.92 (95% confidence interval: 0.86 - 2.98).

CONCLUSIONS: The number of RCTs investigating the use of urologic devices to treat stone disease has substantially increased overtime. There has been a small improvement in reporting quality; however, this remains suboptimal overall. Increased efforts to promote the transparent reporting of RCTs in endourology are warranted.
J Endourol. 2020 Apr;34(4):502-508. doi: 10.1089/end.2019.0649. Epub 2020 Mar 24.

Comments 1

Hans-Göran Tiselius on Thursday, 04 June 2020 10:30

Treatment recommendations in urology are currently based on conclusions drawn from randomized controlled trials (RCTs) This is so for treatment guidelines formulated by both EAU and AUA. The bottom-line is that when one or more RCTs have been published on for instance different methods for stone removal, the level of evidence (LE) is set to 1b and when a meta-analysis has been published LE comes up to 1a.
The clinical decisions accordingly are highly dependent on the quality of the RTC. To assure a good quality standard of data reporting, a specific checklist was invented: CONSORT (Consolidate standards of reporting data from trials) [1]. The latest update on nonpharmacological trials was published in 2017 [2]. The latter document is indeed extensive and comprises many items with the aim of increasing transparency of the report.

The authors of the report above compared RCTs on studies with focus on methods for stone removal (mainly PCNL, URS and SWL) published during three different periods: 2002-2006, 2007-2011 and 2012-2017. A CONSORT score based on items from the checklist was invented with a scale between 0 and 25 and this score was used to evaluate the different RTCs. Although the mean score increased slightly over time, the conclusion was that the reporting quality remains suboptimal. It is of note that CONSORT scores derived for the different periods only came up to 46%, 48% and 53%, respectively.

It is indeed important to keep these shortcomings in mind when RCTs are used as basis for guidelines and treatment recommendations. There are RCTs of both high and low quality and only the fact that a publication is classified as RCT does not guarantee high quality.

References
1. Schulz KF, Altman DG, Moher D; CONSORT Group.
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomized trials.
Open Med. 2010;4(1):e60-8. Epub 2010 Mar 24. No abstract available.

2. Boutron I, Altman DG, Moher D, Schulz KF, Ravaud P; CONSORT NPT Group.
CONSORT Statement for Randomized Trials of Nonpharmacologic Treatments: A 2017 Update and a CONSORT Extension for Nonpharmacologic Trial Abstracts.
Ann Intern Med. 2017 Jul 4;167(1):40-47. doi: 10.7326/M17-0046. Epub 2017 Jun 20.

The authors of the report above compared RCTs on studies with focus on methods for stone removal (mainly PCNL, URS and SWL) published during three different periods: 2002-2006, 2007-2011 and 2012-2017. A CONSORT score based on items from the checklist was invented with a scale between 0 and 25 and this score was used to evaluate the different RTCs. Although the mean score increased slightly over time, the conclusion was that the reporting quality remains suboptimal. It is of note that CONSORT scores derived for the different periods only came up to 46%, 48% and 53%, respectively.

It is indeed important to keep these shortcomings in mind when RCTs are used as basis for guidelines and treatment recommendations. There are RCTs of both high and low quality and only the fact that a publication is classified as RCT does not guarantee high quality.

References
1. Schulz KF, Altman DG, Moher D; CONSORT Group.
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomized trials.
Open Med. 2010;4(1):e60-8. Epub 2010 Mar 24. No abstract available.

2. Boutron I, Altman DG, Moher D, Schulz KF, Ravaud P; CONSORT NPT Group.
CONSORT Statement for Randomized Trials of Nonpharmacologic Treatments: A 2017 Update and a CONSORT Extension for Nonpharmacologic Trial Abstracts.
Ann Intern Med. 2017 Jul 4;167(1):40-47. doi: 10.7326/M17-0046. Epub 2017 Jun 20.

Treatment recommendations in urology are currently based on conclusions drawn from randomized controlled trials (RCTs) This is so for treatment guidelines formulated by both EAU and AUA. The bottom-line is that when one or more RCTs have been published on for instance different methods for stone removal, the level of evidence (LE) is set to 1b and when a meta-analysis has been published LE comes up to 1a. The clinical decisions accordingly are highly dependent on the quality of the RTC. To assure a good quality standard of data reporting, a specific checklist was invented: CONSORT (Consolidate standards of reporting data from trials) [1]. The latest update on nonpharmacological trials was published in 2017 [2]. The latter document is indeed extensive and comprises many items with the aim of increasing transparency of the report. The authors of the report above compared RCTs on studies with focus on methods for stone removal (mainly PCNL, URS and SWL) published during three different periods: 2002-2006, 2007-2011 and 2012-2017. A CONSORT score based on items from the checklist was invented with a scale between 0 and 25 and this score was used to evaluate the different RTCs. Although the mean score increased slightly over time, the conclusion was that the reporting quality remains suboptimal. It is of note that CONSORT scores derived for the different periods only came up to 46%, 48% and 53%, respectively. It is indeed important to keep these shortcomings in mind when RCTs are used as basis for guidelines and treatment recommendations. There are RCTs of both high and low quality and only the fact that a publication is classified as RCT does not guarantee high quality. References 1. Schulz KF, Altman DG, Moher D; CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomized trials. Open Med. 2010;4(1):e60-8. Epub 2010 Mar 24. No abstract available. 2. Boutron I, Altman DG, Moher D, Schulz KF, Ravaud P; CONSORT NPT Group. CONSORT Statement for Randomized Trials of Nonpharmacologic Treatments: A 2017 Update and a CONSORT Extension for Nonpharmacologic Trial Abstracts. Ann Intern Med. 2017 Jul 4;167(1):40-47. doi: 10.7326/M17-0046. Epub 2017 Jun 20.

Cancel Update Comment