Interrater Reliability among Epilepsy Centers: Multi-Center Study of Epilepsy Surgery.
Abstract number :
3.171
Submission category :
Year :
2001
Submission ID :
231
Source :
www.aesnet.org
Presentation date :
12/1/2001 12:00:00 AM
Published date :
Dec 1, 2001, 06:00 AM
Authors :
S.R. Haut, MD, Montefiore Medical Center/AECOM, Bronx, NY; A.T. Berg, PhD, Northern Illinois University, DeKalb, IL; S. Shinnar, MD, PhD, Comprehensive Epilepsy Management Center, Montefiore Medical Center/AECOM, Bronx, NY; C.W. Bazil, MD, PhD, The Neurol
RATIONALE: To measure the interrater reliability of presurgical testing and surgical decisions among epilepsy centers.
METHODS: In an ongoing, prospective multicenter study of resective epilepsy surgery, seven epilepsy centers independently reviewed 21 randomly selected surgical cases, 3 per center, for lateralization and localization by various preoperative studies, and for surgical decisions. No pre-training was administered. Overall consensus for each variable was determined, and interrater reliability was assessed using the Intraclass Correlation Coefficient(ICC), which is validated for use with multiple raters. ICCs were calculated in a two way random model based on absolute agreement.
RESULTS: ICC values [gt]=0.75 - excellent agreement; (0.60-0.74) - good agreement; (0.40-0.59) - fair agreement; [lt]=0.39 - poor agreement. Agreement of all or all but one center was considered consensus. Assessments from six centers were included, and one was excluded due to missing data. Agreement for scalp EEG lateralization was good (ICC = 0.738), and scalp EEG localization was excellent (ICC = 0.804). Agreements for MRI lateralization (0.952) and localization (0.913) were both excellent. Although the ICCs for neuropsychological lateralization (0.717) and localization (0.689) indicated good agreement, the ICC for WADA localization (0.945) was excellent. Agreement over whether to recommend intracranial monitoring after surface was only fair (ICC = 0.539). This in part reflected one center[ssquote]s tendency towards intracranial monitoring, as the ICC for the other five centers was 0.656. Agreement for EEG localization from intracranial monitoring was excellent (0.790). Overall agreement about the ultimate surgical decision was excellent (ICC = 0.831), with an 84.6% consensus for specific resective procedure.
CONCLUSIONS: Significantly high interrater reliability among six epilepsy centers was present for the interpretation of most components of presurgical testing. Although there was only fair agreement about the decision to perform intracranial monitoring or not, the ultimate decision about resective surgery, and the specific choice of resection demonstrated excellent agreement.
Support: NIH grant NS32375 (PI : Dr. Spencer); NIH grant K23 NS02192-01 (PI : Dr. Haut).