Affordable Access

Access to the full text

Stakeholder views on publication bias in health services research

  • Williams, Iestyn1
  • Ayorinde, Abimbola A2
  • Mannion, Russell3
  • Skrybant, Magdalena4
  • Song, Fujian5
  • Lilford, Richard J6
  • Chen, Yen-Fu7
  • 1 Reader in Health Policy and Management and Director of Research, Health Services Management Centre, University of Birmingham, UK
  • 2 Research Fellow, Warwick Centre for Applied Health Research and Delivery, University of Warwick, UK
  • 3 Professor of Health Systems, Health Services Management Centre, University of Birmingham, UK
  • 4 Patient and Public Involvement and Engagement Lead, Institute of Applied Health Research, University of Birmingham, UK
  • 5 Professor in Research Synthesis, Department of Population Health and Primary Care, University of East Anglia, UK
  • 6 Professor of Public Health, Warwick Centre for Applied Health Research and Delivery, University of Warwick, UK
  • 7 Associate Professor, Warwick Centre for Applied Health Research and Delivery, University of Warwick, UK
Published Article
Journal of Health Services Research & Policy
SAGE Publications
Publication Date
Feb 03, 2020
DOI: 10.1177/1355819620902185
PMID: 32013573
PMCID: PMC7307418
PubMed Central


Objectives While the presence of publication bias in clinical research is well documented, little is known about its role in the reporting of health services research. This paper explores stakeholder perceptions and experiences with regard to the role of publication and related biases in quantitative research relating to the quality, accessibility and organization of health services. Methods We present findings from semi-structured interviews with those responsible for the funding, publishing and/or conduct of quantitative health services research, primarily in the UK. Additional data collection includes interviews with health care decision makers as ‘end users’ of health services research, and a focus group with patient and service user representatives. The final sample comprised 24 interviews and eight focus group participants. Results Many study participants felt unable to say with any degree of certainty whether publication bias represents a significant problem in quantitative health services research. Participants drew broad contrasts between externally funded and peer reviewed research on the one hand, and end user funded quality improvement projects on the other, with the latter perceived as more vulnerable to selective publication and author over-claiming. Multiple study objectives, and a general acceptance of ‘mess and noise’ in the data and its interpretation was seen to reduce the importance attached to replicable estimates of effect sizes in health services research. The relative absence of external scrutiny, either from manufacturers of interventions or health system decision makers, added to this general sense of ‘low stakes’ of health services research. As a result, while many participants advocated study pre-registration and using protocols to pre-identify outcomes, others saw this as an unwarranted imposition. Conclusions This study finds that incentives towards publication and related bias are likely to be present, but not to the same degree as in clinical research. In health services research, these were seen as being offset by other forms of ‘novelty’ bias in the reporting and publishing of research findings.

Report this publication


Seen <100 times