Operational networks are increasingly using machine learning models for a variety of tasks, including detecting anomalies, inferring application performance, and forecasting demand. Accurate models are important, yet accuracy can degrade over time due to concept drift, whereby either the characteristics of the data change over time (data drift) or the relationship between the features and the target predictor change over time (model drift). Drift is important to detect because changes in properties of the underlying data or relationships to the target prediction can require model retraining, which can be time-consuming and expensive. Concept drift occurs in operational networks for a variety of reasons, ranging from software upgrades to seasonality to changes in user behavior. Yet, despite the prevalence of drift in networks, its extent and effects on prediction accuracy have not been extensively studied. This paper presents an initial exploration into concept drift in a large cellular network in the United States for a major metropolitan area in the context of demand forecasting. We find that concept drift arises largely due to data drift, and it appears across different key performance indicators (KPIs), models, training set sizes, and time intervals. We identify the sources of concept drift for the particular problem of forecasting downlink volume. Weekly and seasonal patterns introduce both high and low-frequency model drift, while disasters and upgrades result in sudden drift due to exogenous shocks. Regions with high population density, lower traffic volumes, and higher speeds also tend to correlate with more concept drift. The features that contribute most significantly to concept drift are User Equipment (UE) downlink packets, UE uplink packets, and Real-time Transport Protocol (RTP) total received packets.
翻译:操作网络越来越多地使用机器学习模型来完成各种任务,包括发现异常现象、推断应用性能和预测需求。精确模型很重要,但准确性会随着时间推移而降低,因为概念的漂移使得数据随着时间的推移而变化的特点(数据漂移)或特征与目标预测或变化之间的关系(模型漂移),对于探测来说非常重要,因为基础数据或与目标预测的关系的变化可能要求模型再培训,这种再培训可能耗费时间和费用。概念在业务网络中发生下行,原因多种多样,从软件升级到季节性到用户行为的变化。然而,尽管网络流动很普遍,但是对预测准确性的影响和程度没有进行广泛研究。本文初步探索了美国大型蜂窝网络在需求预测范围内对一个主要大都市地区进行的概念漂移的问题。我们发现,概念的漂移在很大程度上是由于数据流,而且它出现在不同的关键业绩指标(IMUI)、模型、培训设定的大小和时间间隔。我们查明了概念向上流的来源,即从网络流动到季节性变化程度普遍,从高的流流流流动到不断流动到不断流动的频率和不断流动的频率,同时引入高水平和不断流流流流流动的频率也导致高的流流流流流流动。 流流流动的流流动的流流流流流动和高的流流流流流流流的流流流流流流流流流的流的流流的流和高。 流到高的流到流到高。 流到流到流到流到流到流到流到流到流到流到流到流到流到流到不断流到高流到流到流到高的流到流到高的流到流到流到流到流到高的流到流到流到流到流到流到流到流到高的流到流到流到高的流到流到高的频率。流到流到流到流到流到流到流到流到流到流到流到流到流到流流流流到流流流流流到流到流到流到流到流到流流流到流到流到流到流到流到高流到高流到流到流到流到流到流到流到流到流到流到流到