Abstractors: Transformer Modules for Symbolic Message Passing and Relational Reasoning

A framework is proposed that casts relational learning in terms of transformers, implementing binding between sensory states and abstract states with relational cross attention mechanisms.