The ubiquity of smartphones and the rapid advance and widespread public adoption of “AI” tools like ChatGPT (especially among the young) have raised hopes among some researchers that such technologies can be harnessed in new ways to improve psychiatric assessment and treatment.

A newly published study by a team led by a two-time BBRF grantee reports results of a pilot study testing one way in which mobile phones and the large language model (LLM)-based ChatGPT tool can together aid in the assessment of behavioral changes that are important for therapeutic improvement in a behavioral therapy for adolescents with depression. Innovations of this kind could open doors to therapy adjustments made in real time based on indicators gleaned directly from patients.

The research, appearing in the journal NPP—Digital Psychiatry and Neuroscience, is the product of a project directed by Christian A. Webb, Ph.D. a 2018 and 2015 BBRF Young Investigator, and led by the paper’s co-first authors Dr. Hadar Fisher and Nigel M. Jaffe. All are affiliated with McLean Hospital and Harvard Medical School. The team included two other past BBRF grantees.

As the researchers note, while smartphones can passively collect rich behavioral data—with users’ consent—by tracking patterns of movement and activity, and while LLMs like ChatGPT can analyze text and language with impressive accuracy, it has remained unclear whether these technologies can provide insight into the emotional or motivational aspects of an individual user’s behavior.

This question helped to drive the team’s pilot study, which examined the two technologies in the context of a talk therapy called behavioral activation (BA), which aims to reduce the classic depression symptom of anhedonia (the loss of interest in pursuing rewards or pleasure) by targeting patterns of avoidance and withdrawal and increasing engagement with rewarding activities.

“A core assumption of BA is that increased ‘activation’ in daily life leads to symptom change, but few studies have rigorously measured whether this is the case,” the researchers noted. Most existing studies rely on self-report questionnaires, which can be burdensome and subject to memory bias.

“There is a pressing need for objective, low-burden methods to track activation in daily life,” they wrote. “Tools that assess activation between [therapy] sessions could help therapists monitor treatment progress and make timely adjustments, an objective consistent with calls for data-informed psychotherapy.”

Their pilot study made use of smartphone-based passive sensing of patients’ mobility patterns and LLM-derived ratings of daily text entries they provided during the 3-month therapy period. These data were examined in relation to daily assessments of positive and negative emotions, as well as weekly assessments of anhedonia, depressive symptoms, and a traditional self-report of activation.

The team recruited 38 adolescents, ages 13-18, from the Boston area to receive BA treatment targeting their anhedonia. Each was offered 12 weekly hour-long individual BA therapy sessions. Before each session, participants rated their anhedonia, depression, and “activation” (in essence, their engagement in daily activities and avoidance patterns), using self-report measures. A subset of 13 participants also contributed passive smartphone data—collected continuously via built-in accelerometer and GPS sensors.

Every other week during treatment all participants completed a “burst” of real-time assessments (“ecological momentary assessments”) over 5 consecutive days via 2-3 surveys delivered each day via a smartphone app. Each time they were prompted by the app, participants were asked to rate their positive and negative affect (6 items, to be rated on a 5-point scale, generating data about being “happy” “interested” and excited”; and “sad,” “nervous,” and angry”). Next, they were asked to provide responses via unrestricted texting, “about what they were doing the night before, with whom they were interacting, and the most enjoyable and stressful events since the previous prompt.”

The text responses were analyzed using OpenAI’s GPT-4o model, as well as by a human “rater” for one-fourth of responses, to evaluate consistency between human and AI ratings. The team found “substantial agreement” between the two.

As for the “silent” real-time monitoring made possible by the smartphone, this data was used to measure activity levels using the intensity of physical activity during the day; the percentage of time a participant spent at home; the distance and “mobility area” traveled each day relative to the home; and places visited each day.

While studies using passive smartphone data can be designed to be entirely unobtrusive, this project intentionally combined passive and interactive elements to test whether the combination of LLM-based text analysis and passive mobility data could provide real-time, clinically relevant insights. There was a cost: compliance of participants with the real-time surveys and texts declined over the course of the 12 weeks of treatment, from 62% in the first 2 weeks to 52% in the final 2 weeks.

This issue notwithstanding, the study produced interesting results. The most important, perhaps, was that “activation” among the depressed and anhedonic adolescents—as deduced from passive sensors in their smartphones—correlated positively with “activation ratings” generated by GPT analysis of their texts and patients’ activation self-report score over the course of therapy.

On the GPS side, greater activation corresponded to more places visited and more time spent away from home, though not distance traveled. In addition, increases in GPT-rated activation were associated with higher daily positive and lower daily negative affect. The team also found that smartphone passive sensing features could be used to forecast weekly improvements in anhedonia and depressive symptoms.

These associations—between the daily and weekly “activity” of patients and their level of symptoms—were found to work at the level of changes within individuals during the study, but not when analyzing differences between individuals. Put another way, changes in “activation” measured by the phone and LLM were found valid and potentially clinically useful for individuals.

The results also led the researchers to suggest that “LLMs such as GPT can extract psychologically meaningful information from unstructured text.” Past work had shown that LLMs can be used to analyze language from in-person psychotherapy sessions to inform clinical decision-making. “Importantly, our study shows that LLM-based assessments can also provide clinically relevant insights based on language generated outside the therapy room, offering scalable and unobtrusive ways to monitor therapeutic processes in patients’ daily lives.”

Full assessment of their results led the team to suggest that LLM-based linguistic analysis may be better suited to capturing short-term emotional changes, while passive sensing (e.g., location data via GPS) may tap into behavioral processes that unfold over longer periods.

“This implies that daily text assessments could help clinicians monitor” the response to behavioral activation therapy “in real time, while mobility patterns could indicate whether treatment is gradually translating into symptom improvement.” They concluded that the tools they tested “hold promise for advancing data-informed psychotherapy by tracking therapeutic processes in real time, reducing reliance on self-report and enabling personalized, adaptive interventions.”

The team also included: Diego A. Pizzagalli, Ph.D., 2017 BBRF Distinguished Investigator, 2008 Independent Investigator; and Erika E. Forbes, Ph.D., 2014 BBRF Independent Investigator, 2006 Young Investigator.