Using Large Language Models to Detect and Understand Drug Discontinuation Events in Web-Based Forums: Development and Validation Study
Source check: Source checked. Not independent scientific confirmation.
Finding: Zero-shot BART, GPT-4 Turbo, GPT-4o and DeBERTa classified drug-discontinuation events and root causes in MedHelp forum comments. In a dataset of 1,000 comments (107 labeled events), BART achieved F1 0.86 for event detection, with 2.8% false positives and 6.5% false negatives; GPT-4o had 12.9% Hamming loss for root-cause classification. The result is a text-mining evaluation on this annotated forum sample, not evidence of medication adherence or clinical outcomes.
Technology: BART, GPT-4o and DeBERTa zero-shot classifiers
Family GPT-4: profile · models page
Publication: Peer-reviewed paper · Test context: retrospective or benchmark. Catalog categories, not an evidence grade.
- AI contribution
- BART, GPT-4 Turbo, GPT-4o and DeBERTa classify forum-comment sentences for drug-discontinuation events and root causes without task-specific fine-tuning.
- Test and result
- Evaluated zero-shot classifiers on MedHelp forum comments: among 1,000 comments, 107 were drug-discontinuation events; BART reached F1 0.86, 2.8% false-positive rate and 6.5% false-negative rate, while GPT-4o root-cause classification had 12.9% Hamming loss.
- Limitation / still unknown
- Performance is established only on the annotated MedHelp forum sample; it does not establish population prevalence, medication adherence, or clinical outcomes.
Evidence sources and annotation context
Evidence context (descriptions follow the cited sources; test type is a catalog classification):
Evidence sources: https://europepmc.org/article/MED/39883487 · https://europepmc.org/articles/PMC11826943 · https://doi.org/10.2196/54601
Source checked · editorial details
The source page was checked against this entry. Independent confirmation is not assessed by this label.
Source type: paper. Paper: not linked in this catalog; this does not mean no paper exists.
Review date: 22 Nov 2026. Review scope: Primary result-bearing material for the stated method, result and evaluation, with bibliographic version information..
No follow-up recorded; this is not a claim that no follow-up exists. Follow-up review scope and date are not recorded. No source-supported relationships recorded.
Editorial assessment: Impact 3/5 · Validation 2/5 · Novelty 3/5. Impact and novelty are editorial judgments; validation follows the annotated test type where one exists. Not an evidence grade. Scoring criteria.