AI Science Tracker
EN

The University of Florida

1 discovery from 2025

Discoveries

Using Large Language Models to Detect and Understand Drug Discontinuation Events in Web-Based Forums: Development and Validation Study

Source check: Source checked. Not independent scientific confirmation.

Finding: Zero-shot BART, GPT-4 Turbo, GPT-4o and DeBERTa classified drug-discontinuation events and root causes in MedHelp forum comments. In a dataset of 1,000 comments (107 labeled events), BART achieved F1 0.86 for event detection, with 2.8% false positives and 6.5% false negatives; GPT-4o had 12.9% Hamming loss for root-cause classification. The result is a text-mining evaluation on this annotated forum sample, not evidence of medication adherence or clinical outcomes.

Technology: BART, GPT-4o and DeBERTa zero-shot classifiers

Family GPT-4: profile · models page

Publication: Peer-reviewed paper · Test context: retrospective or benchmark. Catalog categories, not an evidence grade.

AI contribution
BART, GPT-4 Turbo, GPT-4o and DeBERTa classify forum-comment sentences for drug-discontinuation events and root causes without task-specific fine-tuning.
Test and result
Evaluated zero-shot classifiers on MedHelp forum comments: among 1,000 comments, 107 were drug-discontinuation events; BART reached F1 0.86, 2.8% false-positive rate and 6.5% false-negative rate, while GPT-4o root-cause classification had 12.9% Hamming loss.
Limitation / still unknown
Performance is established only on the annotated MedHelp forum sample; it does not establish population prevalence, medication adherence, or clinical outcomes.
Evidence sources and annotation context

Evidence context (descriptions follow the cited sources; test type is a catalog classification):

Evidence sources: https://europepmc.org/article/MED/39883487 · https://europepmc.org/articles/PMC11826943 · https://doi.org/10.2196/54601

Source checked · editorial details

The source page was checked against this entry. Independent confirmation is not assessed by this label.

Source type: paper. Paper: not linked in this catalog; this does not mean no paper exists.

Review date: 22 Nov 2026. Review scope: Primary result-bearing material for the stated method, result and evaluation, with bibliographic version information..

No follow-up recorded; this is not a claim that no follow-up exists. Follow-up review scope and date are not recorded. No source-supported relationships recorded.

Editorial assessment: Impact 3/5 · Validation 2/5 · Novelty 3/5. Impact and novelty are editorial judgments; validation follows the annotated test type where one exists. Not an evidence grade. Scoring criteria.

Open the interactive profile for the timeline, or browse all organizations and model families.