Authors :
Presenting Author: Daniel Freedman, DO – Dell Medical School
Kristina Julich, MD – Dell Medical School
Michael Silverii, BS – The University of Texas at Austin
Eunsol Choi, PhD – New York University
Rationale:
In rural areas, access to specialty care for children with epilepsy is limited and can contribute to delays in treatment. The initial diagnosis and management is often performed by community physicians. To address limitations in accurately identifying seizures in children in low resource areas, and to improve access to care, we are developing an AI-based “intelligent” questionnaire to diagnose seizures and differentiate them from seizure-mimics that can be used as a clinical decision support tool by community providers in rural areas.Methods:
We will evaluate a large-language model (LLM)’s ability to classify event descriptions into seizure likelihood. We will evaluate a suite of open-sourced LLMs in diverse inference conditions: (1) zero-shot prompting, (2) few-shot prompting, (3) few-shot prompting with explanations. Finally, we aim to develop a proactive model that will ask follow-up questions based on the event descriptions, collecting information that’s crucial for clinical judgements to determine the likelihood of the event being an epileptic seizure versus a nonepileptic seizure-mimic. The output will consist of a “seizure likelihood” rating to aid non-neurologists in distinguishing seizures from seizure mimics (Figure 1).Results:
To obtain pilot data, we reviewed randomly selected, de-identified hospital discharge summaries for seizure event descriptions. One hundred event descriptions were classified as “very likely seizure,” “likely seizure,” “neutral/uncertain,” “unlikely seizure,” and “very unlikely seizure,” extracting key words that helped the reviewers to classify the event. Of the randomly selected event descriptions, 42% were classified as “very likely”, 21% as “somewhat likely”, 18% as “unable to tell”, 8% as “somewhat unlikely”, and 11% as “very unlikely”.
Conclusions:
The pilot data will be used to generate a prediction model that will classify target event descriptions as seizure versus seizure mimic. The next step is to scale up using larger numbers, balancing the sample to mitigate bias, and assess the prediction accuracy of this model. Ultimately, the goal is to use this diagnostic tool as part of our existing epilepsy outreach project in rural areas of Texas to facilitate diagnosis of epilepsy and expedite subspecialty referrals if needed.
Funding: Internal funding from the University of Texas at Austin's IC2 Institute to develop AI-based tools that improve health outcomes and reduce health disparities.