Real-World Testing of an Artificial Intelligence Conversational Agent as an Early Intervention and Support Tool in the Mental Health Referral Care Pathway: A Mixed-Methods Study

Background: The incidence of mental health concerns is growing, and demand for support is exceeding service capacity.
Digital tools can provide additional support but risk causing harm if not delivered safely.
Aims: We aimed to establish real-world evidence of the impact of an artificial intelligence-based mental health
conversational agent (Wysa) on depression and anxiety in patients waiting for Talking Therapies treatment.
Methods: A mixed-methods randomized controlled trial was conducted with patients referred to Talking Therapies in
the Central and Northwest London NHS Foundation Trust. The primary outcome was change in depression severity
over 12weeks between groups; secondary outcomes included anxiety severity, quality of life, safety, engagement, and
app usage. Comparative analyses used linear regression; thematic analysis was conducted on qualitative data.
Results: 2,161 patients were screened, 625 were invited, 99 consented, and 76 were randomized (2:1). Thirty patients
were lost to follow-up. Descriptive analysis found that mean differences in depression were similar between arms, but
with large standard deviations (M=2.62, SD=5.07 and 6.56 for Wysa; M=2.59, SD=4.38 and 3.82 for control). Results
were similar for secondary outcomes. Wysa was potentially helpful, easy to use, and appreciated as an accessible source
of support, but limitations with the conversational agent negatively affected engagement.
Conclusions: Although sample size limited the analysis, participant feedback highlighted its potential to supplement
clinical services. Our study findings suggest that the change of depression score is similar in both arms thus indicating
that there is no evidence that Wysa treats depression in this study. However, limited sample size could have influenced
this. Key lessons to improve the quality of effectiveness studies of digital health technologies were identified.