We present TabPFN, a trained Transformer that can do supervised classification for small tabular datasets in less than a second, needs no hyperparameter tuning and is competitive with state-of-the-art classification methods. TabPFN is fully entailed in the weights of our network, which accepts training and test samples as a set-valued input and yields predictions for the entire test set in a single forward pass. TabPFN is a Prior-Data Fitted Network (PFN) and is trained offline once, to approximate Bayesian inference on synthetic datasets drawn from our prior. This prior incorporates ideas from causal reasoning: It entails a large space of structural causal models with a preference for simple structures. On 30 datasets from the OpenML-CC18 suite, we show that our method clearly outperforms boosted trees and performs on par with complex state-of-the-art AutoML systems with up to 70$\times$ speedup. This increases to a 3200$\times$ speedup when a GPU is available. We provide all our code, the trained TabPFN, an interactive browser demo and a Colab notebook at https://github.com/tabpfn-anonym/TabPFNAnonym.
翻译:我们介绍TabPFN这个训练有素的变压器,能够在不到一秒钟的时间里对小型表层数据集进行监督分类,不需要超分调,并且与最先进的分类方法具有竞争力。TabPFN在网络的权重中是完全的,该网络接受培训和测试样品,作为一套定值投入,并得出对整个测试的预测,在一个远道传球中,整个测试集成。TabPFN是一个数据适应网络(PFN),曾经接受过一次离线培训,以近似Bayesian对从我们以前提取的合成数据集的推断。这之前包含来自因果关系推理的观点:它包含一个巨大的结构性因果模型空间,倾向于简单的结构。在OpenML-CC18套件的30个数据集中,我们显示我们的方法明显超越了立式的树,并且与复杂状态的AutMLML系统同步,最高达70美元。当GPU可用时,它会增加到3200美元。我们提供了我们所有的代码,TabPFN/互动式BAmbN/MAmbon笔记本。