File formats for generic data structures, such as ROOT, Avro, and Parquet, pose a problem for deserialization: it must be fast, but its code depends on the type of the data structure, not known at compile-time. Just-in-time compilation can satisfy both constraints, but we propose a more portable solution: specialized virtual machines. AwkwardForth is a Forth-driven virtual machine for deserializing data into Awkward Arrays. As a language, it is not intended for humans to write, but it loosens the coupling between Uproot and Awkward Array. AwkwardForth programs for deserializing record-oriented formats (ROOT and Avro) are about as fast as C++ ROOT and 10-80$\times$ faster than fastavro. Columnar formats (simple TTrees, RNTuple, and Parquet) only require specialization to interpret metadata and are therefore faster with precompiled code.
翻译:通用数据结构的文件格式, 如 ROOT、 Avro 和 Parquet, 造成了一个解密问题: 它必须是快速的, 但其代码取决于数据结构的类型, 在编译时并不知道。 Just- in- community 可以满足这两种限制, 但我们建议了一个更便捷的解决方案: 专门的虚拟机器。 AwkwardForth 是将数据解密成 Awkward 矩阵的Forth 驱动的硬盘虚拟机器。 作为一种语言, 它不是供人类写入的, 但是它会解开 Uproot 和 Awkward Array 之间的连接。 AwkwardForth 将记录导向格式( ROOT 和 Avro) 解密的软件( ROOT 和 Avro) 与 C++ ROOT 和 10- 80\ times times away aways 一样快。 Collarar 格式( 简单的TTTrees, RNTuple, 和 Parquet) 只需要解释元码的速度就快。