Natural Language Processing of Seizure Outcomes from Clinic Notes at Scale: Feasibility and Further Validation
Abstract number :
2.146
Submission category :
4. Clinical Epilepsy / 4D. Prognosis
Year :
2022
Submission ID :
2204316
Source :
www.aesnet.org
Presentation date :
12/4/2022 12:00:00 PM
Published date :
Nov 22, 2022, 05:24 AM
Authors :
Kevin Xie, MS – University of Pennsylvania; Brian Litt, MD – Departments of Neurology and Bioengineering – University of Pennsylvania; Samuel Terman, MD, MS – Department of Neurology – University of Michigan; Chloe Hill, MD, MS – Department of Neurology – University of Michigan; Dan Roth, PhD – Department of Computer and Information Science – University of Pennsylvania; Colin Ellis, MD – Department of Neurology – University of Pennsylvania
Rationale: Mining the electronic health record for clinical outcomes has the potential to enable large scale retrospective research, such as comparative effectiveness trials. However, clinical outcomes are often contained in free text notes that are not easily mined. We recently developed and validated novel natural language processing (NLP) algorithms to automatically extract key epilepsy outcome measures from manually annotated epilepsy clinic notes [1]. Here, we sought to (1) determine the feasibility of extracting these seizure outcome measures at scale, (2) characterize the relationship between outcomes and ASM changes (relevant to retrospective trials), and (3) determine the generalizability of our methods to other departments and institutions.
Methods: Our NLP algorithms are extensions of pre-trained deep neural network transformer models from Google AI. We applied these algorithms to extract seizure freedom, seizure frequency, and date of most recent seizure from all outpatient visits at our epilepsy center from 2015-2018. We examined the dynamics of seizure outcomes over time using Markov models. We also extracted anti-seizure medication (ASM) prescription data for all patients and tested the relationship between ASM changes and seizure outcomes. Finally, as further validation, we tested the generalizability of our NLP models to other departments at our institution and at a second academic epilepsy center.
Results: We extracted seizure frequency and seizure freedom data from 13,587 clinic notes from 4,132 unique patients. Of these, 5,084 notes (37%) were classified as seizure-free since the last visit; 4,815 (35%) contained a quantifiable seizure frequency; and 6,632 (49%) contained the date of most recent seizure occurrence. Among patients with outcome data from at least 5 visits (6,002 notes from 810 patients, Figure 1), the probabilities of seizure freedom at the next visit ranged from 77% (in patients who were seizure-free at prior visits) to 14% (in patients not seizure-free at prior visits). The likelihood of seizure freedom was higher at visits following ASM dosage increases (X2(1)=6.4, p=0.04) and ASM dosage decreases (X2(1)=15.2, p=0.005) compared to no ASM dosage changes. Visits that followed an increase in number of ASMs were associated with greater likelihood of seizure freedom compared to other visits (X2(1)=11.1, p=0.004). Our models generalized well to notes from non-epilepsy providers and to epilepsy notes from a second institution.
Conclusions: Key epilepsy outcomes can be extracted accurately and at scale from clinical note text using NLP. This approach presents an unprecedented opportunity to perform retrospective clinical research and retrospective clinical trials at large scale by mining electronic health records.
_x000D_
Reference: _x000D_
1. Xie, Kevin, et al. Extracting seizure frequency from epilepsy clinic notes: a machine reading approach to natural language processing. JAMIA. 2022;29(5):873-881._x000D_
Funding: NINDS DP1 OD029758, NINDS K23NS121520, Mirowski Family Foundation, Rothberg Family, AAN, ONR Contract N00014-19-1-2620
Clinical Epilepsy