Functional Electrical Stimulation (FES) can restore motion to a paralysed person's muscles. Yet, control stimulating many muscles to restore the practical function of entire limbs is an unsolved problem. Current neurostimulation engineering still relies on 20th Century control approaches and correspondingly shows only modest results that require daily tinkering to operate at all. Here, we present our state of the art Deep Reinforcement Learning (RL) developed for real time adaptive neurostimulation of paralysed legs for FES cycling. Core to our approach is the integration of a personalised neuromechanical component into our reinforcement learning framework that allows us to train the model efficiently without demanding extended training sessions with the patient and working out of the box. Our neuromechanical component includes merges musculoskeletal models of muscle and or tendon function and a multistate model of muscle fatigue, to render the neurostimulation responsive to a paraplegic's cyclist instantaneous muscle capacity. Our RL approach outperforms PID and Fuzzy Logic controllers in accuracy and performance. Crucially, our system learned to stimulate a cyclist's legs from ramping up speed at the start to maintaining a high cadence in steady state racing as the muscles fatigue. A part of our RL neurostimulation system has been successfully deployed at the Cybathlon 2020 bionic Olympics in the FES discipline with our paraplegic cyclist winning the Silver medal among 9 competing teams.
翻译:功能性电动刺激(FES)可以恢复瘫痪人的肌肉运动。然而,控制刺激许多肌肉以恢复整个肢体的实际功能是一个尚未解决的问题。当前的神经刺激工程仍然依赖20世纪的控制方法,因此只能显示需要每天修补才能操作的微小结果。在这里,我们展示了我们为实时适应性地刺激瘫痪的脚部而开发的深强化学习(RL)现状,以便实时地适应性地刺激瘫痪的脚部,而我们的方法的核心是将个性化神经机能部分纳入我们的强化学习框架,从而使我们能够在不要求病人参加延长培训课程和工作的情况下,有效地培训模型。我们的神经刺激工程部分包括肌肉和或调节功能的混合骨骼模型和肌肉疲劳的多州模型,以使神经刺激能对瘫痪的周期性瞬间运动肌肉能力作出反应。我们的方法的核心是将个性化神经机能构件组合成个性化的神经机能构件部分,让我们在精准和性性能方面对模型进行高效和性能分析,从而有效地培训模型模型,而无需要求与病人一起进行长期培训。我们神经机能性化的神经机能学部分,我们系统在不断的轨道运动运动运动运动运动的轨道上学会开始一个稳定地步步,以稳定地稳定地稳定地稳定地稳定地推进。