Abstracts

Rationale: Timely identification of nonconvulsive seizures and status epilepticus (SE) and continuous monitoring of these events remain a challenge. Point-of-care electroencephalography (POC EEG) systems coupled with artificial intelligence (AI) algorithms to monitor seizure activity offer a possible solution. However, the clinical utility of such algorithms depends on the algorithms’ performance in representative EEG samples acquired in real world scenarios. Here, we evaluate Clarity AI algorithm (Ceribell, Inc.) for the detection of suspected SE in a large dataset, when compared against expert consensus.

Methods: We analyzed Clarity AI’s performance on 1340 adult EEGs acquired with Ceribell EEG hardware. All EEGs were reviewed in parallel by two or more EEG experts, blinded to AI output and clinical information. Reviewers marked epochs of normal and pathological EEG activity, and from these a final EEG diagnosis was determined for each file per reviewer. Cases with seizure duration ≥5min or with cumulative 20% burden per hour were labeled as SE. Agreement ≥ 70% was used as consensus. Clarity AI diagnosed SE electrographically based on the new ACNS criteria, i.e., if i) the maximum seizure burden (SzB) was 90% or higher within a 5-minute window, ii) 10 minutes of continuous seizure activity was detected, or iii) seizure activity reached 20% burden within any hour time window.


Results: According to experts’ consensus 25 EEGs met criteria for SE, 51 for seizures, and 210 for highly epileptiform patterns (HEPs), while 188 had other pathological abnormalities, and 866 were diffusely slow or normal. A significant inter-rater variability was noted in the sensitivities of ten EEG experts in detecting SE ranged from 28.6% to 100% compared to the majority consensus while their specificities ranged from 83.3% to 99.5%. In a subset of 604 cases with the same 5 EEG reviewers, Gwet’s agreement coefficient was moderate (AC1 = 0.67). Clarity AI captured SE in 24 cases (sensitivity of 96.0%) and measured 10% SzB in the remaining SE case. Clarity correctly ruled out SE in 1246 cases, for a specificity of 94.8% and a negative predictive value (NPV) of 99.9%. Compared to human experts, Clarity’s sensitivity was close to the 75th percentile of reviewers, while its specificity was similar to the 50th percentile of reviewers. Finally, in the additional cases alerted for SE, 7 were seizures, 51 HEPs, 5 had other abnormalities, and 6 were normal or slowing. In four of these six cases, at least one reviewer noted seizures that met SE criteria, and all had abnormalities marked by at least one reviewer.

Conclusions: In this large validation dataset, we offer latest accuracy data for Clarity AI and provide context for evaluating its performance next to human EEG expert reviewers who were part of the majority consensus.  Clarity AI demonstrated high sensitivity and specificity to detect SE, at levels similar to expert reviewers.  The 99.9% negative predictive value of Clarity AI strongly supports its clinical utility in ruling out SE in neuro-emergencies, underscoring its value as a point-of-care tool to help optimize management and treatment decisions.

Funding: Funded by Ceribell, Inc.