CausalSim: Unbiased Trace-Driven Network Simulation

Evaluating the real-world performance of network protocols is challenging. Randomized control trials (RCT) are expensive and inaccessible to most, while simulators fail to capture complex behaviors in real networks. We present CausalSim, a trace-driven counterfactual simulator for network protocols that addresses this challenge. Counterfactual simulation aims to predict what would happen using different protocols under the same conditions as a given trace. This is complicated due to the bias introduced by the protocols used during trace collection. CausalSim uses traces from an initial RCT under a set of protocols to learn a causal network model, effectively removing the biases present in the data. Key to CausalSim is mapping the task of counterfactual simulation to that of tensor completion with extremely sparse observations. Through an adversarial neural network training technique that exploits distributional invariances that are present in training data coming from an RCT, CausalSim enables a novel tensor completion method despite the sparsity of observations. Our extensive evaluation of CausalSim on both real and synthetic datasets and two use cases, including more than nine months of real data from the Puffer video streaming system, shows that it provides accurate counterfactual predictions, reducing prediction error by 44% and 53% on average compared to expert-designed and supervised learning baselines.