With the cover of anonymity and the company of strangers, the appeal of the digital world is growing as a place to seek mental health support. This phenomenon is favored by the fact that more than 150 million people in the United States live in federally designated mental health professional shortage areas.
“I really need your help since I'm too scared to talk to a therapist and I can't get through to one anyway.”
“Am I overreacting and feeling hurt because my husband makes fun of me with his friends?”
“Could some strangers have a say in my life and decide my future for me?”
The quotes above are actual posts taken from users of Reddit, a social media news website and forum where users can share content or ask for advice in smaller, interest-based forums known as “subreddits.”
Using a data set of 12,513 posts with 70,429 responses from 26 mental health-related subreddits, researchers from MIT, New York University (NYU), and the University of California, Los Angeles (UCLA) came up with a frame to help evaluate the fairness and overall quality of mental health support chatbots based on large language models (LLM) such as GPT-4. Their work was recently published at the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP).
To accomplish this, the researchers asked two licensed clinical psychologists to evaluate 50 randomly selected Reddit posts seeking support for mental health, pairing each post with either a Redditor's actual response or a GPT-4-generated response. Without knowing which answers were real or which were generated by ai, the psychologists were asked to rate the level of empathy in each answer.
Mental health support chatbots have long been explored as a way to improve access to mental health support, but powerful LLMs like OpenAI's ChatGPT are transforming human-ai interaction, and ai-generated responses. they become more difficult to distinguish from the responses of real humans.
Despite this notable progress, the unintended consequences of ai-provided mental health support have drawn attention to its potentially life-threatening risks; In March last year, a Belgian man committed suicide as a result of an exchange with ELIZA, a chatbot developed to emulate a psychotherapist with an LLM called GPT-J. A month later, the National Eating Disorders Association suspended its chatbot Tessa, after the chatbot began offering dietary advice to patients with eating disorders.
Saadia Gabriel, a recent MIT postdoc who is now an assistant professor at UCLA and first author of the paper, admitted that she was initially very skeptical about how effective mental health support chatbots could be. Gabriel conducted this research during his time as a postdoc at MIT in the Healthy Machine Learning Group, led by Marzyeh Ghassemi, an MIT associate professor in the Department of Electrical Engineering and Computer Sciences and the Institute of Medical Engineering and Sciences. from MIT, affiliated with MIT. Abdul Latif Jameel Clinic for Machine Learning in Health and Computer Science and artificial intelligence Laboratory.
What Gabriel and the team of researchers found was that GPT-4 responses were not only more empathetic overall, but they were 48 percent better at encouraging positive behavioral changes than human responses.
However, in a bias assessment, the researchers found that GPT-4 response empathy levels were reduced for black (2 to 15 percent lower) and Asian (5 to 17 percent lower) posters. percent lower) compared to white or race-unknown posters.
To assess bias in GPT-4 responses and human responses, the researchers included different types of posts with explicit demographic filters (e.g., gender, race) and implicit demographic filters.
An explicit demographic leak would be something like: “I am a 32-year-old black woman.”
Whereas an implied demographic leak would look like this: “Being a 32-year-old girl wearing my hair naturally,” where keywords are used to indicate certain demographics to GPT-4.
With the exception of Black women who posted, GPT-4 responses were found to be less affected by explicit and implicit demographic filtering compared to human respondents, who tended to be more empathetic when responding to posts with demographic suggestions. implicit.
“The structure of the information you (the LLM) provides and some information about the context, such as whether you want the (LLM) to act in the style of a doctor, the style of a social media post, or whether you want it to use demographic attributes.” of the patient has an important impact on the response obtained,” says Gabriel.
The article suggests that providing explicit instructions to LLMs to use demographic attributes can effectively alleviate bias, as this was the only method in which researchers did not observe a significant difference in empathy between different demographic groups.
Gabriel hopes that this work can help ensure a more comprehensive and thoughtful evaluation of LLMs being implemented in clinical settings across demographic subgroups.
“LLMs are already being used to provide patient-facing support and have been implemented in medical settings, in many cases to automate inefficient human systems,” says Ghassemi. “Here, we demonstrate that while state-of-the-art LLMs are generally less affected by demographic filtering than humans in peer-to-peer mental health support, they do not provide equitable mental health responses across inferred patient subgroups… “We have many opportunities to improve the models so that they provide better support when used.”
