Virtual Research Environments (VREs) provide user-centric support in the lifecycle of research activities, e.g., discovering and accessing research assets, or composing and executing application workflows. A typical VRE is often implemented as an integrated environment, which includes a catalog of research assets, a workflow management system, a data management framework, and tools for enabling collaboration among users. Notebook environments, such as Jupyter, allow researchers to rapidly prototype scientific code and share their experiments as online accessible notebooks. Jupyter can support several popular languages that are used by data scientists, such as Python, R, and Julia. However, such notebook environments do not have seamless support for running heavy computations on remote infrastructure or finding and accessing software code inside notebooks. This paper investigates the gap between a notebook environment and a VRE and proposes an embedded VRE solution for the Jupyter environment called Notebook-as-a-VRE (NaaVRE). The NaaVRE solution provides functional components via a component marketplace and allows users to create a customized VRE on top of the Jupyter environment. From the VRE, a user can search research assets (data, software, and algorithms), compose workflows, manage the lifecycle of an experiment, and share the results among users in the community. We demonstrate how such a solution can enhance a legacy workflow that uses Light Detection and Ranging (LiDAR) data from country-wide airborne laser scanning surveys for deriving geospatial data products of ecosystem structure at high resolution over broad spatial extents. This enables users to scale out the processing of multi-terabyte LiDAR point clouds for ecological applications to more data sources in a distributed cloud environment.
翻译:虚拟研究环境(VREs)在研究活动的生命周期中提供以用户为中心的支持,例如发现和访问研究资产,或制定和执行应用工作流程。典型的VRE通常作为一个综合环境实施,其中包括研究资产目录、工作流程管理系统、数据管理框架和用户之间促进合作的工具。Jupyter等笔记本环境允许研究人员快速原型科学代码,并以在线可访问的笔记本形式分享其实验。Jupyter可以支持数据科学家使用的多种流行语言,如Python、R和Julia。然而,这类笔记式环境对于远程基础设施的重计算或查找和访问笔记本内的软件代码没有完全的支持。本文调查笔记本环境与VRE之间的缺口,并提议为Jupyter环境提供嵌入的VRE解决方案,称为Notrebook-as-VRE(NaVRE)。NaVRE解决方案通过一个组件市场提供功能组件,允许用户在Jupyter环境的顶端上建立定制的VRE(RE-Rationerational-arder-D),从Vreal-arly dental dal-al dal dal-listalmas real livesess real list list list list list list list smas relist smas relist smal list smas a list smas a ligal sm sre a list sre sre sre smal smal smal smal smal smal smal smal smal smessmessmessmessmessmessmessmmm smessmessmessmaldowd smalds,我们可以在VD a smal smal smal smal sm sm sm sm sm sm sm smal smal smmmaldaldalmal smal smal sm sm sm sal sal sm sm sm sm sm sm sm sm sessm sal salsm smessmaldaldaldaldaldal samessmessmessmessm