粒子物理离线数据处理资源分配与作业管理双层调度算法研究

项目名称： 粒子物理离线数据处理资源分配与作业管理双层调度算法研究

项目编号： No.11475210

项目类型： 面上项目

立项/批准年度： 2015

项目学科： 数理科学和化学

项目作者： 石京燕

作者单位： 中国科学院高能物理研究所

项目金额： 92万元

中文摘要： 离线数据处理软件及其计算平台是粒子物理实验获得高质量物理成果的重要基础。本项目整合多个粒子物理实验的计算资源，研究基于cpu/核的资源弹性分配和作业动态调度的分层调度算法，用以提高粒子物理实验计算平台的资源利用率，减少作业运行时间，加快离线数据处理速度，帮助物理学家获得高质量的物理成果。针对粒子物理计算作业的特点，本项目改变传统的物理机集群调度算法，研究集成物理机集群与虚拟机集群计算资源，统一管理的双层调度算法。调度算法采用推、拉结合的方式，实现细粒度的资源分配，避免计算资源浪费；并进行动态作业调度，为粒子物理计算作业分配最适合的cpu/核，提高离线数据处理效率。

中文关键词： 离线数据处理；模拟计算；重建计算；数据分析；资源分配；作业调度算法；计算资源利用率

英文摘要： Offline data processing software and its computing platform are the important foundation for particle physics experiments to obtain the high quality physical results. The project integrates several computing resources of particle physics experiments, and studies the algorithm of flexible resource allocation and dynamic job scheduling algorithm based on cpu/core. The aim of the algorithm is to improve the resource utilization of particle physics experiment computing platform. The job running time could be reduced and the speed of offline data processing could be accelerated with the help of the algorithm. All of these could help physicist to obtain high quality physics result. Considering the characters of particle physics jobs, the traditional scheduling algorithm would be replaced by a new one. The new one is a double level scheduling algorithm to integrate computing resources of both physical cluster and virtual cluster. Push and Pull modes were adopted to realize the fine-grain resources allocation and dynamic job scheduling. It could eliminate the computing resources waste and fit jobs request by the most suitable cpu/cores.

英文关键词： offline data processing;simulation;reconstruction;data analysis;resource allocation;job scheduling algorithm;utilization of computing resource