Abstracts

In many resource-constrained regions people with epilepsy face critical barriers to treatment due to limited access to trained neurologists. Primary care providers on the frontlines often report a lack of knowledge regarding the use of anti-seizure drugs. Artificial intelligence models trained on locally relevant data could serve as clinical decision aids. Here, we evaluate the performance of a large language model (LLM)-based pipeline to convert narrative medical history from clinical notes of neurologists from a resource-limited setting into structured data suitable for predicting treatment response to anti-seizure drugs.

334 children with epilepsy (age < 18 years), who were longitudinally followed at Uganda’s National Referral Pediatric Neurology clinic were included in the study. Narrative clinical notes documented seizure history, medication changes, and treatment responses. Using a prompt-engineered LLM, we extracted structured clinical features such as seizure type (e.g., focal vs generalized), chronicity, medications and counts, and seizure outcomes (categorized as decreased, increased or seizure freedom at 6-month follow-up). The LLM extracted structured features were used to train a CatBoost classifier to predict seizure outcome at a 6-month follow-up using stratified 5-fold cross validation.

The prediction model achieved strong generalization with a mean F1 score of 87% (variance 0.001) and precision of 88% across five folds (Figure 1). Performance remained consistent across metrics with minimal variance, demonstrating the pipeline’s robustness and low sensitivity to data partitioning. Feature importance analysis identified the following clinical factors as the most influential predictors of seizure outcomes: number of medications, age of seizure onset, and treatment change.