Can AI-Powered Chatbots Provide Reliable Health Information?

By Rasit Dinc

Artificial intelligence (AI) is rapidly transforming various sectors, and healthcare is no exception. AI-powered chatbots, in particular, are becoming increasingly popular as a source of health information. They offer the promise of instant, accessible, and personalized medical guidance. But can we truly rely on these digital assistants for accurate and safe health advice? This article delves into the current state of AI chatbots in healthcare, exploring both their potential and their pitfalls.

The Promise of AI in Healthcare

AI chatbots are designed to simulate human conversation and can be programmed to perform a wide range of tasks in a healthcare setting. As outlined in a comprehensive guide on the adoption of AI in healthcare, these chatbots can automate administrative duties like appointment scheduling, freeing up valuable time for medical staff [3]. They can also assist in patient triage by gathering symptom information and directing patients to the appropriate level of care. Furthermore, AI chatbots can serve as a valuable tool for patient education, providing information on treatments, medications, and healthy lifestyle choices [3]. The potential for these tools to improve efficiency and patient engagement is undeniable.

The Perils of Inaccuracy

Despite their promising capabilities, the reliability of AI chatbots for providing health information is a significant concern. A 2024 study published in the Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine evaluated the accuracy of ChatGPT-3.5 and ChatGPT-4 in disseminating resuscitation guidelines. The study found that while a high percentage of the chatbots' statements were in line with the guidelines, they failed to address a substantial number of key messages, and some of the information provided was superficial or incorrect [1]. The researchers concluded that healthcare professionals should not solely rely on these chatbots for up-to-date medical evidence [1].

Another study focused on the reliability of AI-powered chatbots for patient drug information painted an even more concerning picture. The research, highlighted in Inside Precision Medicine, revealed that a chatbot embedded in a search engine provided a considerable number of incorrect or potentially harmful answers to questions about medications [2]. The study's authors warned that if patients were to follow the advice given by the chatbot, it could lead to moderate or even severe harm [2]. This underscores the critical need for caution when using AI for medical advice, especially concerning medication.

Navigating the Digital Frontier: A Call for Caution

The evidence suggests that while AI chatbots can be a useful starting point for health-related queries, they are not yet a substitute for professional medical advice. Both patients and healthcare professionals need to approach these tools with a critical mindset. It is crucial to verify the information provided by chatbots with reliable sources, such as official health websites, and to consult with a qualified healthcare provider for diagnosis and treatment.

For healthcare organizations looking to implement AI chatbots, it is essential to choose reputable platforms and to have robust quality control measures in place. As the technology continues to evolve, it is hoped that the accuracy and reliability of these tools will improve. However, for now, they should be seen as a supplementary tool rather than a primary source of medical information.

Conclusion

AI-powered chatbots hold immense potential to revolutionize healthcare by improving efficiency and patient access to information. However, the current generation of these tools is not without its flaws. The risk of inaccurate, incomplete, or even harmful information is real and should not be underestimated. As we continue to navigate this new digital frontier, a balanced approach is necessary. We must embrace the potential of AI while remaining vigilant about its limitations. Ultimately, the expertise and empathy of human healthcare professionals remain irreplaceable.

References

[1] Beck, S., Kuhner, M., Haar, M., Daubmann, A., Semmann, M., & Kluge, S. (2024). Evaluating the accuracy and reliability of AI chatbots in disseminating the content of current resuscitation guidelines: a comparative analysis between the ERC 2021 guidelines and both ChatGPTs 3.5 and 4. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine, 32(1), 1-9.

[2] Chakraverty, A. (2024, October 11). AI-Powered Chatbots Unreliable for Patient Drug Information. Inside Precision Medicine.

[3] Ahramovich, A. (2025, August 28). AI Chatbots in Healthcare: An In-Depth Adoption Guide. Itransition.