The Userfault Object (UFO) framework explores avenues of cooperating with the operating system to use memory in non-traditional ways. We implement a framework that employs the Linux kernel's userfault mechanism to fill the contents of runtime objects on demand. When an object's memory is accessed the framework executes a user-defined function that generates a slice of the object. The back-end can generate data from thin air, calculate it from a formula, or retrieve it from persistent storage, the network, or other sources (with or without post-processing). UFOs follow the memory layout of standard runtime objects, so they can be introspected and written to safely. The framework manages the loading and unloading of object segments to ensure that memory is reclaimed as needed and data is never lost. This allows the UFO framework to implement larger-than-memory data structures that never materialize into memory in full. Implementing objects as UFOs also impacts performance, since overhead of populating memory is amortized by loading entire pages of data at a time. The host runtime can also rely on direct memory accesses into userfault object obviating the need for a special dispatch mechanism. We provide a proof-of-concept implementation of the UFO framework for the R language.
翻译:用户过失对象框架(UFO) 探索与操作系统合作的途径,以非传统的方式使用记忆。 我们实施一个框架, 使用 Linux 内核的用户断层机制来填充运行时间对象的内容。 当一个对象的内存被访问时, 框架会执行一个自定义的功能, 生成一个对象的切片。 后端可以从薄空气中生成数据, 从公式中计算数据, 或者从持续存储、 网络或其他来源( 不论是否后处理) 中获取数据。 UFO 遵循标准运行时间对象的内存布局, 以便它们能够被插入并写到安全处。 框架管理对象部分的上卸, 以确保根据需要重新恢复记忆, 数据永远不会丢失。 这样, UFO 框架可以实施更大的非模版数据结构, 永远无法在记忆中实现。 将对象作为UFO, 也影响性能, 因为充电存储存储存储的间接费用通过全页加载数据来进行摊算。 主机运行时间也可以依靠直接的内存权限访问系统, 将Ufalt 目标的测试框架。