A Probabilistic Model for Discriminative and Neuro-Symbolic Semi-Supervised Learning

Recent progress has been made in semi-supervised learning (SSL) by combining methods that exploit various aspects of the data distribution, e.g. image augmentation and consistency regularisation, rely on properties of $p(x)$, whereas others, such as entropy minimisation and pseudo-labelling, pertain to the sample-specific label distributions $p(y|x)$. Focusing on the latter, we propose a probabilistic model for discriminative SSL that mirrors its classical generative counterpart, filling a gap in existing semi-supervised learning theory. Under this model, several well-known SSL methods can be interpreted as imposing relaxations of an appropriate prior over learned parameters of $p(y|x)$. The same model extends naturally to neuro-symbolic SSL, often treated as a separate field, in which binary label attributes are subject to logical rules. The model thus also theoretically justifies a family of neuro-symbolic SSL methods and unifies them with standard SSL, taking a step towards bridging the divide between statistical learning and logical reasoning.