Quantum machine learning has the potential for a transformative impact across industry sectors and in particular in finance. In our work we look at the problem of hedging where deep reinforcement learning offers a powerful framework for real markets. We develop quantum reinforcement learning methods based on policy-search and distributional actor-critic algorithms that use quantum neural network architectures with orthogonal and compound layers for the policy and value functions. We prove that the quantum neural networks we use are trainable, and we perform extensive simulations that show that quantum models can reduce the number of trainable parameters while achieving comparable performance and that the distributional approach obtains better performance than other standard approaches, both classical and quantum. We successfully implement the proposed models on a trapped-ion quantum processor, utilizing circuits with up to $16$ qubits, and observe performance that agrees well with noiseless simulation. Our quantum techniques are general and can be applied to other reinforcement learning problems beyond hedging.
翻译:量子机器学习有潜力在金融等行业实现革命性影响。我们在工作中探讨了对冲问题,其中深度强化学习为实际市场提供了强大的框架。我们开发了基于政策搜索和分布式演员评论算法的量子强化学习方法,使用正交和混合层的量子神经网络体系结构,用于政策和价值函数。我们证明了我们使用的量子神经网络是可训练的,并进行了大量模拟,展示了量子模型可以减少可训练参数的数量,同时实现可比较的性能,并且分布式方法获得比其他标准方法(包括经典和量子)更好的性能。我们成功地在搭载最多$16$个量子比特电路的离子束缚量子处理器上实现了所提出的模型,并观察到与无噪声模拟相符的性能。我们的量子技术是通用的,可适用于除对冲之外的其他强化学习问题。