Poster CC4 - Exploration of large language models for thematic analysis of a communication ecosystem in a clinical trial

Introduction: Appearance counseling plays a crucial role in the consent process for breast reconstruction surgery. We developed BreastDecisions, a recommender system for appearance counseling that suggests photographs of previous patients that are tailored to a specific patient to help the patient form realistic expectations of their own reconstruction. This study aims to explore the utility of Large Language Models (LLMs) in thematic analysis of the communication ecosystem in a feasibility clinical trial of the BreastDecisions system.

Materials and

Methods: We recruited patients planning breast reconstruction and randomly assigned them to either the intervention or control group for a feasibility clinical trial of the BreastDecisions system. Patients in the intervention group utilized the BreastDecisions system on a tablet under the guidance of a researcher prior to their consultation. In both arms of the study, the patient underwent consultation with a plastic surgeon about breast reconstruction. Follow-up conversations in the form of phone calls and myChart messages were also assessed for both groups.

Results, Conclusions, and Discussions: We recruited a total of 35 patients, with 17 randomized to the intervention group and 18 randomized to the control group, for a feasibility clinical trial of the BreastDecisions system. Figure 1 illustrates the communication ecosystem of this clinical trial setting. There were three different types of actors in the communication ecosystem: patient (orange), researcher (purple), and provider team (blue). The provider team included a variety of different healthcare professionals, such as nurses, physician assistants, and surgeons. The communication ecosystem consisted of both (a) communications that were directly recorded: transcribed audio recordings (6-12 minutes per patient) and myChart messages, and (b) notes about communications: provider team’s notes in electronic health record, researcher’s field notes (n = 50), and provider team’s notes of telephone calls (n = 183). The communication ecosystem included face-to-face verbal communication explicitly paired with visual communication (patient using BreastDecisions system with researcher), face-to-face verbal communication (patient and provider consultation), audio-only communication (telephone calls), and written communication (myChart messages). We used LLM with prompts for inductive thematic analysis, including generating initial codes, searching for themes, defining themes, etc. to explore the potential of LLMs in thematic analysis of the diverse data streams in this communication ecosystem.

Acknowledgements (Optional): This study was supported by NIH grant R01 CA203984.