Penalized and Decentralized Contextual Bandit Learning for WLAN Channel Allocation with Contention-Driven Feature Extraction

A multi-armed bandit (MAB)-based decentralized channel exploration framework both adapting unknown traffics of neighboring access points (APs) and ensuring convergence is proposed. As the throughput provided by a typical AP in wireless local area network (WLAN) is significantly affected by neighboring APs' channels due to carrier sense operations, the neighbor awareness, i.e., being aware of channels of neighboring APs, is valuable. The main scope of this paper is to incorporate this neighbor awareness into an MAB-based channel exploration as conventional MAB-based WLAN channel exploration schemes lacks this perspective. To this end, we propose contention-driven feature extraction (CDFE), which extracts the adjacency relation of a contention graph. This allows to formulate the traffic-adaptive channel exploration as contextual MAB (CMAB) problem with joint linear upper confidence bound (JLinUCB) exploration where the graph edge of the feature is leveraged as the weights of a linear throughput estimator. Moreover, we address the problem of non-convergence -- the channel exploration cycle -- which is an inherent difficulty in selfish decentralized learning. To prevent such a cycle, we propose a penalized JLinUCB (P-JLinUCB) based on the key idea of introducing a discount parameter to the reward for exploiting a different channel before and after the learning round.