1 Department of Supply Chain and Business Technology Management, John Molson School of Business, Concordia University, 1455 De Maisonneuve Blvd, W, MB 12.359, Montreal, QC, H3G 1M8, Canada, 1 514-848-2424.

Background: The quality of health information on social media is a major concern, especially during the early stages of public health crises. While the quality of the results of the popular search engines related to particular diseases has been analyzed in the literature, the quality of health-related information on social media, such as X (formerly Twitter), during the early stages of a public health crisis has not been addressed.

Objective: This study aims to evaluate the quality of health-related information on social media during the early stages of a public health crisis.

Methods: A cross-sectional analysis was conducted on health-related tweets in the early stages of the most recent public health crisis (the COVID-19 pandemic). The study analyzed the top 100 websites that were most frequently retweeted in the early stages of the crisis, categorizing them by content type, website affiliation, and exclusivity. Quality and reliability were assessed using the DISCERN and JAMA (Journal of the American Medical Association) benchmarks.

Results: Our analyses showed that 95% (95/100) of the websites met only 2 of the 4 JAMA quality criteria. DISCERN scores revealed that 81% (81/100) of the websites were evaluated as low scores, and only 11% (11/100) of the websites were evaluated as high scores. The analysis revealed significant disparities in the quality and reliability of health information across different website affiliations, content types, and exclusivity.

Conclusions: This study highlights a significant issue with the quality, reliability, and transparency of online health-related information during a public health challenge. The extensive shortcomings observed across frequently shared websites on Twitter highlight the critical need for continuous evaluation and improvement of online health content during the early stages of future health crises. Without consistent oversight and improvement, we risk repeating the same shortcomings in future, potentially more challenging situations.