The increasing complexity of modern high-performance computing (HPC) systems necessitates the introduction of automated and data-driven methodologies to support system administrators' effort toward increasing the system's availability. Anomaly detection is an integral part of improving the availability as it eases the system administrator's burden and reduces the time between an anomaly and its resolution. However, current state-of-the-art (SoA) approaches to anomaly detection are supervised and semi-supervised, so they require a human-labelled dataset with anomalies - this is often impractical to collect in production HPC systems. Unsupervised anomaly detection approaches based on clustering, aimed at alleviating the need for accurate anomaly data, have so far shown poor performance. In this work, we overcome these limitations by proposing RUAD, a novel Recurrent Unsupervised Anomaly Detection model. RUAD achieves better results than the current semi-supervised and unsupervised SoA approaches. This is achieved by considering temporal dependencies in the data and including long-short term memory cells in the model architecture. The proposed approach is assessed on a complete ten-month history of a Tier-0 system (Marconi100 from CINECA with 980 nodes). RUAD achieves an area under the curve (AUC) of 0.763 in semi-supervised training and an AUC of 0.767 in unsupervised training, which improves upon the SoA approach that achieves an AUC of 0.747 in semi-supervised training and an AUC of 0.734 in unsupervised training. It also vastly outperforms the current SoA unsupervised anomaly detection approach based on clustering, achieving the AUC of 0.548.
翻译:现代高性能计算(HPC)系统日益复杂,因此有必要采用自动化和数据驱动的方法,支持系统管理员增加系统可用性的努力。异常检测是改进可用性的一个组成部分,因为这样可以减轻系统管理员的负担,缩短异常现象与其分辨率之间的时间。然而,目前对异常现象检测的最先进(SoA)方法的监管和半监督是监督的,因此它们需要一个带有异常现象的人类标签数据套件――这在生产中收集往往不切实际。基于集群的不受监督异常现象检测方法,旨在减轻对准确异常数据的需求,迄今为止表现不佳。在这项工作中,我们通过推出新的不受监督的异常现象检测模型模型,克服了这些局限性。但是,目前对异常现象检测的最先进(SoA)方法比目前半监督和不受监督的SUA方法取得更好的结果。这是通过考虑数据的时间依赖性方法,在模型结构中包括长期短时间存储的存储细胞细胞细胞细胞。拟议的方法是在AA-A-A类的快速培训中,在AA-A类-A级系统10的完整历史中,在AS-AA-A-A-A级的快速培训中,在SU-A-A-A-A-A-A-A-A-A-A-A-A-A-A-A-A-A-A-A-S-S-S-S-S-S-S-S-S-S-S-A的10的10的10的10的10的10的10的完整培训中实现了一个10的10的10的完整的完整的完整培训中实现。