Jonathan J Shuster*
Shuster. J Clin Transl Res 2023; 9(4):22-00019
Published online: July 12, 2023
Abstract
Background: A peer-reviewed Meta-Analysis methods article mathematically proved that mainstream random-effects methods, “weights inversely proportional to the estimated variance,” are flawed and can lead to faulty public health recommendations. Because the arguments causing this off label (unproven) use of mainstream practices were subtle, changing these practices will require much clearer explanations that can be grasped by clinical and translational scientists. There are five assumptions underlying the mainstream’s derivation of its statistical properties. This paper will demonstrate that if the first is true, it follows that the last two are false. Ratio estimation, borrowed from classical survey sampling, provides a rigorous alternative. Papers reporting results rarely fully disclose these assumptions. This is analogous to watching TV ads with the sound muted. You see high quality of life and do not hear about the complications. This article is a poster child for translational science, as it takes a theoretical discovery from the biostatistical world, translates it into language clinical scientists can understand, and thereby can change their research practice.
Aim: This article is aimed at future applications of Meta-Analysis of complete collections of randomized clinical trials. It leaves it to past authors as to whether to reanalyze their data. No blame for past use is assessed.
Methods: By treating the individual completed studies in the Meta-Analysis as a random sample from a conceptual universe of completed studies, we use ratio estimation to obtain estimates of relative risk (ratio of failure rates treatment: control) and mean differences, projecting our sample value to estimate the universe’s value.
Results: Two examples demonstrate that the mainstream methods likely adversely impacted major treatment options. A third example shows that the key mainstream presumption of independence between the study weights and study estimates cannot be supported.
Conclusion: There is no rationale for ever using the mainstream for Meta-Analysis of randomized clinical trials.
Relevance for patients: Future Meta-Analysis of clinical trials should never employ mainstream methods. Doing so could lead to potentially harmful public health policy recommendations. Clinical researchers need to play a primary role to assure good research practices in Meta-Analysis.
DOI: http://dx.doi.org/10.18053/jctres.09.202304.22-00019
Author affiliation
Department of Health Outcomes and Bioinformatics, College of Medicine, University of Florida, 2026 NW 34 Ter, Gainesville, FL 32605, United States of America
*Corresponding author
Jonathan J. Shuster
Department of Health Outcomes and Bioinformatics, College of Medicine, University of Florida, 2026 NW 34 Ter, Gainesville, FL 32605, United States of America
Tel: +1(352)682-0893
Email: shusterj@ufl.edu
Handling editor:
Michal Heger
Department of Pharmaceutics, Utrecht University, the Netherlands
Department of Chemistry, Utrecht University, Utrecht, the Netherlands
Department of Pathology, Erasmus Medical Center, the Netherlands
Department of Pharmaceutics, Jiaxing University Medical College, Zhejiang, China
Download
Full text PDF
