Applying Opponent Modeling for Automatic bidding in Online Repeated Auctions

Online auction scenarios, such as bidding searches on advertising platforms, often require bidders to participate repeatedly in auctions for the same or similar items. We design an algorithm for adaptive automatic bidding in repeated auctions in which the seller and other bidders also update their strategies. We apply and improve the opponent modeling algorithm to allow bidders to learn optimal bidding strategies in this multiagent reinforcement learning environment. The algorithm uses almost no private information about the opponent or restrictions on the strategy space, so it can be extended to multiple scenarios. Our algorithm improves the utility compared to both static bidding strategies and dynamic learning strategies. We hope the application of opponent modeling in auctions will promote the research of automatic bidding strategies in online auctions and the design of non-incentive compatible auction mechanisms.