![]()
本文约2013字，建议阅读4分钟
本文介绍了9月28日15:00-20:30多位PhD带来的国际机器学习大会相关讲座，欢迎扫码报名观看直播。
9月28日 15:00~20:30
AI TIME特别邀请了多位PhD，带来ICML-6！
扫码关注AITIME哔哩哔哩官方账号
观看直播
链接：https://live.bilibili.com/21813994
贾彬彬：
从2017年9月开始在东南大学计算机科学与工程学院攻读博士学位，主要研究方向为多维分类(Multi-Dimensional Classification)，目前发表CCF A类期刊1篇、会议3篇，CCF B类期刊3篇，CCF C类会议1篇。
内容简介：
In multi-dimensional classification (MDC), there are multiple class variables in the output space with each of them corresponding to one heterogeneous class space. Due to the heterogeneity of class spaces, it is quite challenging to consider the dependencies among class variables when learning from MDC examples. In this paper, we propose a novel MDC approach named SLEM which learns the predictive model in an encoded label space instead of the original heterogeneous one. Specifically, SLEM works in an encoding-training-decoding framework. In the encoding phase, each class vector is mapped into a real-valued one via three cascaded operations including pairwise grouping, one-hot conversion and sparse linear encoding. In the training phase, a multi-output regression model is learned within the encoded label space. In the decoding phase, the predicted class vector is obtained by adapting orthogonal matching pursuit over outputs of the learned multi-output regression model. Experimental results clearly validate the superiority of SLEM against state-of-the-art MDC approaches.
吴齐天：
上海交大博士生，导师为严骏驰教授，研究方向为机器学习与数据挖掘。在ICML/NeurIPS/KDD发表多篇一作论文，获评2021年百度AI新星。
内容简介：
Recommendation models can effectively estimate underlying user interests and predict one's future behaviors by factorizing an observed user-item rating matrix into products of two sets of latent factors. However, the user-specific embedding factors can only be learned in a transductive way, making it difficult to handle new users on-the-fly. In this paper, we propose an inductive collaborative filtering framework that contains two representation models. The first model follows conventional matrix factorization which factorizes a group of key users' rating matrix to obtain meta latents. The second model resorts to attention-based structure learning that estimates hidden relations from query to key users and learns to leverage meta latents to inductively compute embeddings for query users via neural message passing. Our model enables inductive representation learning for users and meanwhile guarantees equivalent representation capacity as matrix factorization. Experiments demonstrate that our model achieves promising results for recommendation on few-shot users with limited training ratings and new unseen users which are commonly encountered in open-world recommender systems.
唐瀚霖：
从2017年9月就读于University of Rochester大学CS，于2021年7月获得CS博士学位。主要研究方向是大规模训练下的通信加速的优化器设计以及广义优化器研究。导师是刘霁老师。在ICML和NIPS上发表过相关论文。
内容简介：
Scalable training of large models (like BERT and GPT-3) requires careful optimization rooted in model design, architecture, and system capabilities. From a system standpoint, communication has become a major bottleneck, especially on commodity systems with standard TCP interconnects that offer limited network bandwidth. Communication compression is an important technique to reduce training time on such systems. One of the most effective methods is error-compensated compression, which offers robust convergence speed even under 1-bit compression. However, state-of-the-art error compensation techniques only work with basic optimizers like SGD and momentum SGD, which are linearly dependent on the gradients. They do not work with non-linear gradient-based optimizers like Adam, which offer state-of-the-art convergence efficiency and accuracy for models like BERT. In this paper, we propose 1-bit Adam that reduces the communication volume by up to 5×, offers much better scalability, and provides the same convergence speed as uncompressed Adam. Our key finding is that Adam's variance (non-linear term) becomes stable (after a warmup phase) and can be used as a fixed precondition for the rest of the training (compression phase). Experiments on up to 256 GPUs show that 1-bit Adam enables up to 3.3× higher throughput for BERT-Large pre-training and up to 2.9× higher throughput for SQuAD fine-tuning. In addition, we provide theoretical analysis for our proposed work.
牛帅程：
自2018年9月开始在华南理工大学软件学院攻读博士学位。导师为谭明奎教授，以及腾讯AI Lab的吴家祥和赵沛霖研究员。主要研究方向为神经网络结构搜索和迁移学习，并在相关领域会议和期刊发表论文多篇，包括ICML, CVPR, IJCAI, TIP, TKDE等。
内容简介：
In real-world applications, data often come in a growing manner, where the data volume and the number of classes may increase dynamically. This will bring a critical challenge for learning: given the increasing data volume or the number of classes, one has to instantaneously adjust the neural model capacity to obtain promising performance. Existing methods either ignore the growing nature of data or seek to independently search an optimal architecture for a given dataset, and thus are incapable of promptly adjusting the architectures for the changed data. To address this, we present a neural architecture adaptation method, namely Adaptation eXpert (AdaXpert), to efficiently adjust previous architectures on the growing data. Specifically, we introduce an architecture adjuster to generate a suitable architecture for each data snapshot, based on the previous architecture and the different extent between current and previous data distributions. Furthermore, we propose an adaptation condition to determine the necessity of adjustment, thereby avoiding unnecessary and time-consuming adjustments. Extensive experiments on two growth scenarios (increasing data volume and number of classes) demonstrate the effectiveness of the proposed method.
钟方威：
现为北京大学人工智能研究院博士后，获博士后创新人才计划资助。在此之前于2021年获得北京大学博士学位。他的研究兴趣是融合计算机视觉、机器人学习、多智能体、虚拟现实和认知科学等多个领域知识，实现高效自主的机器人。他已在人工智能领域顶级学术期刊和会议发表论文多篇，包括了IEEE TPAMI、ICML、ICLR、NeurIPS、CVPR、AAAI等。他多次受邀担任 NeurIPS、ICML、ICLR、CVPR、ICCV、AAAI等人工智能领域顶级国际会议程序委员/审稿人。
内容简介：
In active visual tracking, it is notoriously difficult when distracting objects appear, as distractors often mislead the tracker by occluding the target or bringing a confusing appearance. To address this issue, we propose a mixed cooperative-competitive multi-agent game, where a target and multiple distractors form a collaborative team to play against a tracker and make it fail to follow. Through learning in our game, diverse distracting behaviors of the distractors naturally emerge, thereby exposing the tracker's weakness, which helps enhance the distraction-robustness of the tracker.For effective learning, we then present a bunch of practical methods, including a reward function for distractors, a cross-modal teacher-student learning strategy, and a recurrent attention mechanism for the tracker. The experimental results show that our tracker performs desired distraction-robust active visual tracking and can be well generalized to unseen environments. We also show that the multi-agent game can be used to adversarially test the robustness of trackers.
孙胜扬：
加拿大多伦多大学的五年级在读博士生，主要研究方向是基于贝叶斯理论的不确定性估计，侧重于高斯过程，核方法，以及贝叶斯深度学习。曾在ICML，ICLR，NeurIPS，AISTATS等会议上发表文章。
内容简介：
We introduce a new scalable variational Gaussian process approximation which provides a high fidelity approximation while retaining general applicability. We propose the harmonic kernel decomposition (HKD), which uses Fourier series to decompose a kernel as a sum of orthogonal kernels. Our variational approximation exploits this orthogonality to enable a large number of inducing points at a low computational cost. We demonstrate that, on a range of regression and classification problems, our approach can exploit input space symmetries such as translations and reflections, and it significantly outperforms standard variational methods in scalability and accuracy. Notably, our approach achieves state-of-the-art results on CIFAR-10 among pure GP models.
微信扫码咨询专知VIP会员