Abstract
Background
Selective reporting of results in published case-control studies has been widely suspected, but little comprehensive information on selective reporting is available with regard to case-control studies. We aimed to evaluate the concordance of findings between publications and the protocols of case-control studies and to assess the level of selective reporting of results in case-control studies.
Methods
The databases of Embase, Medline, Scopus, and Web of Science were searched to identify case-control study protocols published between January 1, 1990 and December 31, 2017. The numbers and characteristics of predefined exposures (or factors) were extracted from the protocols. The reported and unreported factors were both collected from the published studies and protocols. The frequency of selective reporting of results were estimated by identifying the discrepancies of factors between the protocols and the published studies. Study sample size and the extent of selective reporting of factors were measured by a Spearman correlation analysis.
Results
Fourteen protocols with 24 published studies and 159 factors were identified, of which eight protocols (57.1%) had discrepancies between the publications and protocols. The prevalence of incomplete reporting in published case-control studies was 42.9% (6/14), with participant characteristics, anthropometric and laboratory measurement variables more likely to be unreported. A total of 16,835 cases and 56,049 controls were recruited in the 14 protocols of case-control studies (sample size ranges from 428 to 52,596 per study). Sample size had no statistical significance with selective reporting of results (P > 0.05).
Conclusion
The study protocols should be publicly available prior to the completion of case-control studies so that the potential bias can be assessed by the readers. Our findings highlight the need for investigators, peer reviewers, and readers to exercise increased awareness and scrutiny due to the undesirable practice of selective reporting of results in medical sciences causing the loss of potentially important information, thus impacting quality of personalized attitude in healthcare in the context of the predictive, preventive, and personalized medicine.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs13167-019-00165-2/MediaObjects/13167_2019_165_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs13167-019-00165-2/MediaObjects/13167_2019_165_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs13167-019-00165-2/MediaObjects/13167_2019_165_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs13167-019-00165-2/MediaObjects/13167_2019_165_Fig4_HTML.png)
Similar content being viewed by others
Abbreviations
- CCS:
-
Case-control study
- CONSORT:
-
The Consolidated Standards of Reporting Trials
- FDA:
-
Food and Drug Administration
- ICMJE:
-
The International Committee of Medical Journal Editors
- NIH:
-
National Institutes of Health
- NOS :
-
Newcastle-Ottawa Scale
- PPPM:
-
Predictive, preventive, and personalized medicine
References
Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291(20):2457–65.
Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358(3):252–60.
Al-Marzouki S, Roberts I, Evans S, Marshall T. Selective reporting in clinical trials: analysis of trial protocols accepted by The Lancet. Lancet. 2008;372(9634):201.
Zhang S, Liang F, Li W. Comparison between publicly accessible publications, registries, and protocols of phase III trials indicated persistence of selective outcome reporting. J Clin Epidemiol. 2017;91:87–94.
Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med. 2009;361:1963–71.
You B, Gan HK, Pond G, Chen EX. Consistency in the analysis and reporting of primary end points in oncology randomized controlled trials from registration to publication: a systematic review. J Clin Oncol. 2012;30(2):210–6.
DeAngelis CD, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. JAMA. 2004;292(11):1363–4.
Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA. 1996;276(8):637–9.
Jones CW, Handler L, Crowell KE, Keil LG, Weaver MA, Platts-Mills TF. Non-publication of large randomized clinical trials: cross sectional analysis. BMJ. 2013;347:f6104.
Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The Clinical Trials.gov results database update and key issues. N Engl J Med. 2011;364:852–60.
Zarin DA, Tse T, Sheehan J. The proposed rule for U.S. clinical trial registration and results submission. N Engl J Med. 2015;372:174–80.
Weerasinghe M, Konradsen F, Eddleston M, Pearson M, Gunnell D, Hawton K, et al. Risk factors associated with purchasing pesticide from shops for self-poisoning: a protocol for a population-based case-control study. BMJ Open. 2015;5(5):e007822.
Estcourt LJ, Stanworth SJ, Collett D, Murphy MF. Intracranial haemorrhage in thrombocytopenic haematology patients--a nested case-control study: the InCiTe study protocol. BMJ Open. 2014;4(2):e004199.
Cramond F, Irvine C, Liao J, Howells D, Sena E, Currie G, et al. Protocol for a retrospective, controlled cohort study of the impact of a change in Nature journals’ editorial policy for life sciences research on the completeness of reporting study design and execution. Scientometrics. 2016;108:315–28.
Stosic M, Vukovic D, Babic D, Antonijevic G, Foley KL, Vujcic I, et al. Risk factors for multidrug-resistant tuberculosis among tuberculosis patients in Serbia: a case-control study. BMC Public Health. 2018;18(1):1114.
Soares AM, Szejnfeld VL, Enokihara MY, Michalany N, Castro CH. High serum 25-hydroxyvitamin D concentration in patients with a recent diagnosis of non-melanoma skin cancer: a case-control study. Eur J Dermatol. 2018;28(5):649–53.
Canova C, Anello P, Barbiellini Amidei C, Parolin V, Zanier L, Simonato L. Use of healthcare services at the end of life in decedents compared to their surviving counterparts: a case-control study among adults born before 1946 in Friuli Venezia Giulia. PLoS One. 2019;14(2):e0212086.
Golubnitschaja O, Costigliola V, EPMA. General report & recommendations in predictive, preventive and personalised medicine 2012: white paper of the European Association for Predictive, Preventive and Personalised Medicine. EPMA J. 2012;3(1):14.
Lu M, Zhan X. The crucial role of multiomic approach in cancer research and clinically relevant outcomes. EPMA J. 2018;9(1):77–102.
Hou H, Zhao Y, Yu W, Dong H, Xue X, Ding J, et al. Association of obstructive sleep apnea with hypertension: a systematic review and meta-analysis. J Glob Health. 2018;8(1):010405.
Leone M, Bottacchi E, Beghi E, Morgando E, Mutani R, Amedeo G, et al. A case-control study on alcohol and seizures: study design, protocol, and data collection. Ital J Neurol Sci. 1997;18(2):119–24.
Heinemann LA, Garbe E, Lewis M, van der Woude F, Graf H. Case-control study on analgesics and nephropathy (SAN): protocol. BMC Nephrol. 2005;6:9.
Boneberger A, Radon K, Baer J, Kausel L, Kabesch M, Haider D, et al. Asthma in changing environments--chances and challenges of international research collaborations between South America and Europe--study protocol and description of the data acquisition of a case-control-study. BMC Pulm Med. 2010;10:43.
Shariful Islam SM, Lechner A, Ferrari U, Froeschl G, Niessen LW, Seissler J, et al. Social and economic impact of diabetics in Bangladesh: protocol for a case-control study. BMC Public Health. 2013;13:1217.
Grigg MJ, William T, Drakeley CJ, Jelip J, von Seidlein L, Barber BE, et al. Factors that are associated with the risk of acquiring Plasmodium knowlesi malaria in Sabah, Malaysia: a case-control study protocol. BMJ Open. 2014;4(8):e006004.
Tovilla-Zárate CA, González-Castro TB, Juárez-Rojop I, Pool García S, Velázquez-Sánchez MP, Villar-Soto M. Study on genes of the serotonergic system and suicidal behavior: protocol for a case-control study in Mexican population. BMC Psychiatry. 2014;14:29.
Lewis DM, Assmann A, Heinemann L, Spitzer WO. Interim review of the transnational case-control study of oral contraceptives and health: approved protocol revisions through September 1995. Pharmacoepidemiol Drug Saf. 1996;5(1):43–51.
Wynn P, Stewart J, Kumar A, Clacy R, Coffey F, Cooper N, et al. Kee** children safe at home: protocol for a case-control study of modifiable risk factors for scalds. Inj Prev. 2014;20(5):e11.
Heinemann LA, Thomas DB, Möhner M. Multicentre international liver tumour study protocol of the case-control study on hepatocellular cancer. Pharmacoepidemiol Drug Saf. 1996;5(3):173–86.
Klussmann A, Gebhardt H, Liebers F, von Engelhardt LV, Dávid A, Bouillon B, et al. Individual and occupational risk factors for knee osteoarthritis - study protocol of a case control study. BMC Musculoskelet Disord. 2008;9:26.
Picot VS, Bénet T, Messaoudi M, Telles JN, Chou M, Eap T, et al. Multicenter case-control study protocol of pneumonia etiology in children: global approach to biological research, infectious diseases and epidemics in low-income countries (GABRIEL network). BMC Infect Dis. 2014;14:635.
Gomez-Marcos MA, Gonzalez-Sarmiento R, Recio-Rodríguez JI, Agudo-Conde C, Gamella-Pozuelo L, Perretta-Tejedor N, et al. Relationship between target organ damage and blood pressure, retinal vessel calibre, oxidative stress and polymorphisms in VAV-2 and VAV-3 genes in patients with hypertension: a case-control study protocol (LOD-Hipertension). BMJ Open. 2014;4(4):e005112.
Vinogradova Y, Coupland C, Hippisley-Cox J. Exposure to combined oral contraceptives and risk of venous thromboembolism: a protocol for nested case-control studies using the QResearch and the CPRD databases. BMJ Open. 2014;4(4):e004499.
Buckley CM, Ali F, Roberts G, Kearney PM, Perry IJ, Bradley CP. Timing of access to secondary healthcare services for diabetes management and lower extremity amputation in people with diabetes: a protocol of a case-control study. BMJ Open. 2013;3(10):e003871.
Summerskill W, Collingridge D, Frankish H. Protocols, probity, and publication. Lancet. 2009;373(9668):992.
Schulz KF, Altman DG, Moher D. Protocols, probity, and publication. Lancet. 2009;373(9674):1524.
Cheng T, Zhan X. Pattern recognition for predictive, preventive, and personalized medicine in cancer. EPMA J. 2017;8(1):51–60.
Chan AW, Krleza-Jerić K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ. 2004;171(7):735–40.
Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA. 2009;302(9):977–84.
Wayant C, Scheckel C, Hicks C, Nissen T, Leduc L, Som M, et al. Evidence of selective reporting bias in hematology journals: a systematic review. PLoS One. 2017;12(6):e0178379.
Calméjane L, Dechartres A, Tran VT, Ravaud P. Making protocols available with the article improved evaluation of selective outcome reporting. J Clin Epidemiol. 2018;104:95–102.
Swaen GMH, Urlings MJE, Zeegers MP. Outcome reporting bias in observational epidemiology studies on phthalates. Ann Epidemiol. 2016;26(8):597–9.
Golubnitschaja O, Baban B, Boniolo G, Wang W, Bubnov R, Kapalla M, et al. Medicine in the early twenty-first century: paradigm and anticipation - EPMA position paper 2016. EPMA J. 2016;7:23.
Funding
This work was supported by the Natural Science Foundation of Shandong Province, China (No. ZR2017MH100). The sponsor did not participate in the study design, data collection, data analysis, or manuscript preparation.
Author information
Authors and Affiliations
Contributions
HH and WW designed this study. GD, XZ, ZM, YZ, and ZG collected the data. HH, and DL performed the statistical analysis and interpretation of the data. JX provided statistical expertise. HH, WW, and JX wrote the manuscript. All authors have read and approved the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare that they have no competing interests.
Consent for publication
Not applicable.
Ethics approval and consent to participate
Not applicable.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
ESM 1
(DOCX 29 kb)
Rights and permissions
About this article
Cite this article
Hou, H., Ding, G., Zhao, X. et al. Comparisons between protocols and publications of case-control studies: analysis of potential causes of non-reproducibility and recommendations for enhancing the quality of personalization in healthcare. EPMA Journal 10, 101–108 (2019). https://doi.org/10.1007/s13167-019-00165-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13167-019-00165-2