QSAN: A Near-term Achievable Quantum Self-Attention Network

A Quantum Self-Attention Network (QSAN) that can be achieved on near-term quantum devices is investigated. First, the theoretical basis of QSAN, a linearized and reversible Quantum Self-Attention Mechanism (QSAM) including Quantum Logic Similarity (QLS) and Quantum Bit Self-Attention Score Matrix (QBSASM), is explored to solve the storage problem of Self-Attention Mechanism (SAM) due to quadratic complexity. More importantly, QLS uses logical operations instead of inner product operations to enable QSAN to be fully deployed on quantum computers and meanwhile saves quantum bits by avoiding numerical operations, and QBSASM is a by-product generated with the evolution of QSAN, reflecting the output attention distribution in the form of a density matrix. Then, the framework and the quantum circuit of QSAN are designed with 9 execution steps and 5 special functional sub-modules, which can acquire QBSASM effectively in the intermediate process, as well as compressing the number of measurements. In addition, a quantum coordinate prototype is proposed to describe the mathematical connection between the control and output bits in order to realize programming and model optimization conveniently. Finally, a miniaturized experiment is implemented and it demonstrates that QSAN can be trained faster in the presence of quantum natural gradient descent method, as well as produce quantum characteristic attention distribution QBSASM. QSAN has great potential to be embedded in classical or quantum machine learning frameworks to lay the foundation for quantum enhanced Natural Language Processing (NLP).