Use of large language model-based chatbots in managing the rehabilitation concerns and education needs of outpatient stroke survivors and caregivers

 I've already concluded that AI is worthless without underlying research proving recovery. 

Here is my run through asking for 100% recovery:

ChatGPT on 100% recovery from stroke

The latest here:

Use of large language model-based chatbots in managing the rehabilitation concerns and education needs of outpatient stroke survivors and caregivers

Jin Rui E. Neo ^1* Joon Sin Ser ² San San Tay ¹

• ¹ Singapore Health Services Pte Ltd, Singapore, Singapore
• ² SingHealth Residency, Singapore, Singapore

Background 

 

The utility of large language model-based (LLM) artificial intelligence (AI) chatbots in many aspects of healthcare is becoming apparent though their ability to address patient concerns remains unknown. We sought to evaluate the performance of two well-known, freely-accessible chatbots, ChatGPT and Google Bard, in responding to common questions about stroke rehabilitation posed by patients and their caregivers. 

 

Methods 

We collected questions from outpatients and their caregivers through a survey, categorised them by theme, and created representative questions to be posed to both chatbots. We then evaluated the chatbots' responses based on accuracy, safety, relevance, and readability. Interrater agreement was also tracked. 

Results 

 Although both chatbots achieved similar overall scores, Google Bard performed slightly better in relevance and safety. Both provided readable responses with some general accuracy, but struggled with hallucinated responses, were often not specific, and lacked awareness of the possibility for emotional situations with the potential to turn dangerous. Additionally, interrater agreement was low, highlighting the variability in physician acceptance of their responses. 

Conclusions 

AI chatbots show potential in patient-facing support roles, but issues remain regarding safety, accuracy, and relevance. Future chatbots should address these problems to ensure that they can reliably and independently manage the concerns and questions of stroke patients and their caregivers.

Keywords: Stroke, Rehabilitation, Caregivers, artificial intelligence, Large Language Model, Chatbot, ChatGPT, Google Bard

Received: 03 Mar 2024; Accepted: 19 Apr 2024.

Copyright: © 2024 Neo, Ser and Tay. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Jin Rui E. Neo, Singapore Health Services Pte Ltd, Singapore, Singapore