Abstracts

Rationale: Recent clinical trials have shown some efficacy in controlling seizures using deep brain stimulation (DBS) for patients with intractable epilepsies [1]. The large stimulus parameter space afforded by modern DBS therapy devices makes it difficult to determine the optimal therapy for patients with various forms of intractable epilepsy. DBS treatments can be improved by implementing adaptive closed-loop algorithms that systematically optimize stimulation parameters to maximize patient outcome and potentially minimizing side effects. In this research we utilize a temporal difference reinforcement learning algorithm, called SARSA(λ) [2] to optimize stimulation to suppress seizures in a in vivo rodent seizure model.Methods: The goal of the SARSA(λ) algorithm is to learn the optimal action, such as stimulation frequency, given the state, such as seizing or not seizing, to maximize reward. The reward in this case is inversely proportional to the high frequency activity which increases during epileptiform activity. The algorithm quickly learns the optimal action iteratively testing each action for every state. Animal surgery protocols followed an IACUC approved protocol. A single concentric bipolar stimulation electrode was placed into the ventral hippocampal commissure. Seizure like events (SLEs) were recorded in the hippocampus following focal injection of 4-aminopyridine. Stimulation was delivered with biphasic charged balanced pulses with 100µs pulse width and 50µA amplitude. The SARSA(λ) algorithm was implemented in a real-time system (http://RTXI.org) to determine the optimal stimulation frequency to mimimize SLEs over the course of the experiment.Results: We tested the algorithm to suppress SLEs in an in vivo rodent seizure model. The algorithm required three complete seizure like event cycles to identify parameters that affect seizures. With the SARSA(λ) algorithm active, SLEs were suppressed, although there were still residual electrophysiological correlates of epileptiform behavior. When the stimulation was turned off, SLEs returned to full form.Conclusions: The SARSA(λ) algorithm approach was shown to be effective at determining effective stimulation parameters in an in vivo model. This adaptive closed-loop algorithm does not require a model of the system to determine the optimal solution. This learning algorithm could be implemented to identify optimal parameters on a patient by patient basis, and is computationally efficient enough to implement on a low-power implantable device. References: Heck, C. N., King‐Stephens, D., Massey, A. D., Nair, D. R., Jobst, B. C., Barkley, G. L., ... and Morrell, M. J. (2014). Two‐year seizure reduction in adults with medically intractable partial onset epilepsy treated with responsive neurostimulation: Final results of the RNS System Pivotal trial. Epilepsia, vol. 55, no. 3, pp. 432-441. Sutton R.S., and Barto A.G. (1998). Reinforcement learning: An introduction. Vol. 1, no. 1. Cambridge: MIT press, 1998, (7.5).