EpiSemoLLM: A Knowledge Graph Guided RAG Enhanced Large Language Model for Epileptogenic Zone Localization Based on Seizure Semiology
Abstract number :
1.198
Submission category :
2. Translational Research / 2D. Models
Year :
2025
Submission ID :
809
Source :
www.aesnet.org
Presentation date :
12/6/2025 12:00:00 AM
Published date :
Authors :
Shihao Yang, MS – Stevens Institute of Technology
Neel Fotedar, MD – Epilepsy Center, Neurological Institute, University Hospitals Cleveland Medical Center, Cleveland, OH, USA
Xinglong Ju, PhD – Southern Utah University
Jun-En Ding, MS – Stevens Institute of Technology
Danilo Bernardo, MD – University of California at San Francisco
Yaxi Luo, MS – Stevens Institute of Technology
Xiaochen Xian, PhD – Georgia Tech
Yo-Tsen Liu, MD – Taipei Veterans General Hospital
Yen-Cheng Shih, MD – Taipei Veterans General Hospital
Hai Sun, MD, PhD – Rutgers University
Fang-Ming Hung, MD – Far Eastern Memorial Hospital
Vikram Rao, MD – Department of Neurology and Weill Institute for Neurosciences, University of California, San Francisco
Ioannis Karakis, MD, PhD – Emory University
Joshua Laing, BBiomedSci, MBBS, FRACP, PhD – Monash University
Felix Rosenow, MD – Goethe-University Frankfurt
Patrick Kwan, BMedSci, BM, BChir, FRACP, PhD, FAHMS – Monash Institute of Medical Engineering, Monash Univeristy
Shasha Wu, MD, PhD – University of Chicago
Presenting Author: Feng Liu, PhD – Stevens Institute of Technology
Rationale: Seizure semiology provides crucial information for localizing the epileptogenic zone (EZ), but interpretation requires specialized expertise. Recent advances in large language models (LLMs) offer potential to improve EZ localization accuracy by automatically interpreting seizure semiology descriptions. We introduce EpiSemoLLM, the first LLM fine-tuned specifically for mapping seizure semiology to corresponding EZs, built upon the Mistral-7B foundation model.
Methods: A total of 1372 cases, each containing seizure semiology descriptions paired with validated EZs via intracranial EEG recording and postoperative surgery outcome, were collected from 392 publications and 590 Electronic Health Record (EHR) from Taiwan Far Eastern Memorial Hospital (FEMH). These collected data cohort of seizure semiology descriptions and EZs, as the high-quality domain specific data, is used to fine-tune the foundational LLM to improve its ability to predict the most likely EZs. Based on the cleaned dataset, a knowledge graph was constructed and leveraged through Retrieval-Augmented Generation (RAG) technology to extract question-relevant information for enhancing LLM generation. This approach enables dynamic retrieval of contextually relevant knowledge entities and relationships to augment the factual accuracy and coherence of LLM outputs. To evaluate the performance of the fine-tuned EpiSemoLLM, 100 well-defined cases were tested by comparing the responses from EpiSemoLLM with those from a panel of 5 epileptologists. The responses were graded using the rectified Net Positive Inference Rate(rNPIR) and regional accuracy rate (RAR). Additionally, the performance of EpiSemoLLM was compared with its foundational model, Mistral-7B, and various versions of ChatGPT, Llama, and other representative biomedical LLMs.
Results: EpiSemoLLM achieved regional accuracy rates (RAR) of 60.71% (frontal), 83.33% (temporal), 63.16% (occipital), 45.83% (parietal), 33.33% (insular), and 28.57% (cingulate), with mean rectified Net Positive Inference Rate (rNPIR) of 0.535. Compared to epileptologists' RAR of 64.83% (frontal), 52.22% (temporal), 60.00% (occipital), 42.50% (parietal), 42.22% (insular), and 8.57% (cingulate), with mean rNPIR of 0.460. EpiSemoLLM demonstrated superior performance in temporal, parietal, occipital, and cingulate region localization. EpiSemoLLM significantly outperformed foundational models (Mistral-7B-instruct, ChatGPT-4.0, Llama variants, DeepSeek-R1) and state-of-the-art biomedical LLMs, particularly for insular and cingulate cortex localization, offering valuable support for presurgical EZ assessment.
Conclusions: EpiSemoLLM achieved comparable performance to epileptologists in localizing epileptogenic zones from seizure semiology, demonstrating its potential for presurgical assessment. The model excelled in temporal, parietal, occipital and cingulate localizations, while epileptologists performed better for frontal and insular regions. Superior performance over the foundational model confirms the value of domain-specific fine-tuning.
Funding: N/A
Translational Research