Evaluating the real-world performance of network protocols is challenging. Randomized control trials (RCT) are expensive and inaccessible to most, while simulators fail to capture complex behaviors in real networks. We present CausalSim, a trace-driven counterfactual simulator for network protocols that addresses this challenge. Counterfactual simulation aims to predict what would happen using different protocols under the same conditions as a given trace. This is complicated due to the bias introduced by the protocols used during trace collection. CausalSim uses traces from an initial RCT under a set of protocols to learn a causal network model, effectively removing the biases present in the data. Key to CausalSim is mapping the task of counterfactual simulation to that of tensor completion with extremely sparse observations. Through an adversarial neural network training technique that exploits distributional invariances that are present in training data coming from an RCT, CausalSim enables a novel tensor completion method despite the sparsity of observations. Our extensive evaluation of CausalSim on both real and synthetic datasets and two use cases, including more than nine months of real data from the Puffer video streaming system, shows that it provides accurate counterfactual predictions, reducing prediction error by 44% and 53% on average compared to expert-designed and supervised learning baselines.
翻译:评估网络协议的真实性表现是挑战性的。 随机控制试验(RCT)费用昂贵,多数人无法进入, 模拟器无法捕捉真实网络中的复杂行为。 我们展示了 CausalSim, 这是应对这一挑战的网络协议的追踪驱动反事实模拟器。 反事实模拟旨在预测在与给定跟踪相同的条件下使用不同协议将会发生什么。 由于在跟踪采集过程中使用协议的偏差, causalSim 使用一套协议下的初步RCT的痕迹来学习因果网络模型, 有效地消除数据中存在的偏差。 CausalSim 的 Key to CausalSim 正在绘制反事实模拟任务图, 以极分散的观测方式完成 。 通过对抗性神经网络培训技术, 利用来自RCT 中的数据的分布变异性来预测结果。 CausalSim 使得新式的索尔完成方法变得复杂, 尽管观察紧张。 我们对CausSimalSim 进行了广泛的评估, 在真实性和合成数据集和两个使用案例中, 提供了超过44个月的准确的预测, 通过平均预测, 和精确性的专家系统显示, 从44 的预测, 从实际的预测到正确性基线, 和精确的预测系统显示了53 和精确性基线, 从44 和精确的预测数据系统提供了比的预测, 从44 的预测, 和精确性的专家系统, 和精确的预测, 基线, 比较的预测数据系统提供了比。