ICML2019：Google和Facebook在推进哪些方向？

2019 年 6 月 13 日 专知

【导读】ICML2019正在加利福尼亚的长滩市如火如荼的进行着。作为一年一度的机器学习盛宴，来自世界各地的研究学者和顶尖公司都深度参与其中，引领着一年又一年的科技进步。那么，今年这些顶尖公司又在推进什么机器学习方向呢？小编简单分析了Facebook和Google在ICML2019上的投稿和参与的一些活动，一起来看看吧。

【Google at ICML 2019】

资料来自google官方博客

https://ai.googleblog.com/2019/06/google-at-icml-2019.html

组织方面：

在ICML2019的组织上，Google有：

ICML Board Member 董事会成员：4人
ICML Senior Area Chair（高级领域主席）：8人
ICML Area Chair（领域主席）：34人

投稿方面：

Google 共中稿 94篇：

Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
Learning to Groove with Inverse Sequence Transformations
Metric-Optimized Example Weights
HOList: An Environment for Machine Learning of Higher Order Logic Theorem Proving
Learning to Clear the Market
Shape Constraints for Set Functions
Self-Attention Generative Adversarial Networks
High-Fidelity Image Generation With Fewer Labels
Learning Optimal Linear Regularizers
DeepMDP: Learning Continuous Latent Space Models for Representation Learning
kernelPSI: a Post-Selection Inference Framework for Nonlinear Variable Selection
Learning from a Learner
Rate Distortion For Model Compression:From Theory To Practice
An Investigation into Neural Net Optimization via Hessian Eigenvalue Density
Graph Matching Networks for Learning the Similarity of Graph Structured Objects
Subspace Robust Wasserstein Distances
Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraints
The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Study
A Theory of Regularized Markov Decision Processes
Area Attention
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Static Automatic Batching In TensorFlow
The Evolved Transformer
Policy Certificates: Towards Accountable Reinforcement Learning
Self-similar Epochs: Value in Arrangement
The Value Function Polytope in Reinforcement Learning
Adversarial Examples Are a Natural Consequence of Test Error in Noise
SOLAR: Deep Structured Representations for Model-Based Reinforcement Learning
Garbage In, Reward Out: Bootstrapping Exploration in Multi-Armed Bandits
Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition
Direct Uncertainty Prediction for Medical Second Opinions
A Large-Scale Study on Regularization and Normalization in GANs
Learning a Compressed Sensing Measurement Matrix via Gradient Unrolling
NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks
Distributed Weighted Matching via Randomized Composable Coresets
Monge blunts Bayes: Hardness Results for Adversarial Training
Generalized Majorization-Minimization
NAS-Bench-101: Towards Reproducible Neural Architecture Search
Variational Russian Roulette for Deep Bayesian Nonparametrics
Surrogate Losses for Online Learning of Stepsizes in Stochastic Non-Convex Optimization
Improved Parallel Algorithms for Density-Based Network Clustering
The Advantages of Multiple Classes for Reducing Overfitting from Test Set Reuse
Submodular Streaming in All Its Glory: Tight Approximation, Minimum Memory and Low Adaptive Complexity
Hiring Under Uncertainty
A Tree-Based Method for Fast Repeated Sampling of Determinantal Point Processes
Statistics and Samples in Distributional Reinforcement Learning
Provably Efficient Maximum Entropy Exploration
Active Learning with Disagreement Graphs
MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing
Understanding the Impact of Entropy on Policy Optimization
Matrix-Free Preconditioning in Online Learning
State-Reification Networks: Improving Generalization by Modeling the Distribution of Hidden Representations
Online Convex Optimization in Adversarial Markov Decision Processes
Bounding User Contributions: A Bias-Variance Trade-off in Differential Privacy
Complementary-Label Learning for Arbitrary Losses and Models
Learning Latent Dynamics for Planning from Pixels
Unifying Orthogonal Monte Carlo Methods
Differentially Private Learning of Geometric Concepts
Online Learning with Sleeping Experts and Feedback Graphs
Adaptive Scale-Invariant Online Algorithms for Learning Linear Models
TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing
Online Control with Adversarial Disturbances
Adversarial Online Learning with Noise
Escaping Saddle Points with Adaptive Gradient Methods
Fairness Risk Measures
DBSCAN++: Towards Fast and Scalable Density Clustering
Learning Linear-Quadratic Regulators Efficiently with only √T Regret
Understanding and correcting pathologies in the training of learned optimizers
Parameter-Efficient Transfer Learning for NLP
Efficient Full-Matrix Adaptive Regularization
Efficient On-Device Models Using Neural Projections
Flexibly Fair Representation Learning by Disentanglement
Recursive Sketches for Modular Deep Learning
POLITEX: Regret Bounds for Policy Iteration Using Expert Prediction
Anytime Online-to-Batch, Optimism and Acceleration
Insertion Transformer: Flexible Sequence Generation via Insertion Operations
Robust Inference via Generative Classifiers for Handling Noisy Labels
A Better k-means++ Algorithm via Local Search
Analyzing and Improving Representations with the Soft Nearest Neighbor Loss
Learning to Generalize from Sparse and Underspecified Rewards
MeanSum: A Neural Model for Unsupervised Multi-Document Abstractive Summarization
CHiVE: Varying Prosody in Speech Synthesis with a Linguistically Driven Dynamic Hierarchical Conditional Variational Network
Similarity of Neural Network Representations Revisited
Online Algorithms for Rent-Or-Buy with Expert Advice
Breaking the Softmax Bottleneck via Learnable Monotonic Pointwise Non-linearities
Non-monotone Submodular Maximization with Nearly Optimal Adaptivity and Query Complexity
Agnostic Federated Learning
Categorical Feature Compression via Submodular Optimization
Cross-Domain 3D Equivariant Image Embeddings
Faster Algorithms for Binary Matrix Factorization
On Variational Bounds of Mutual Information
Guided Evolutionary Strategies: Augmenting Random Search with Surrogate Gradients
Semi-Cyclic Stochastic Gradient Descent
Stochastic Deep Networks

小编将这些文章的题目，剔除掉常用词，如Learning, Model, Deep等，然后可视化出来，结果如下：

Wordshop方面:

Google 参与组织了17个Workshop：

1st Workshop on Understanding and Improving Generalization in Deep Learning
Climate Change: How Can AI Help?
Generative Modeling and Model-Based Reasoning for Robotics and AI
Human In the Loop Learning (HILL)
ICML 2019 Time Series Workshop
Joint Workshop on On-Device Machine Learning & Compact Deep Neural Network Representations (ODML-CDNNR)
Negative Dependence: Theory and Applications in Machine Learning
Reinforcement Learning for Real Life
Uncertainty and Robustness in Deep Learning
Theoretical Physics for Deep Learning
Workshop on the Security and Privacy of Machine Learning
Exploration in Reinforcement Learning Workshop
ICML Workshop on Imitation, Intent, and Interaction (I3)
Identifying and Understanding Deep Learning Phenomena
Workshop on Multi-Task and Lifelong Reinforcement Learning
Workshop on Self-Supervised Learning
Invertible Neural Networks and Normalizing Flows

小编将这些workshop的题目，剔除掉常用词，如Learning, Model, Deep等，然后可视化出来，结果如下（注意，由于workshop的不多，又去掉了一些常用词和停用词，这个词云表达出来的信息可能有偏差，具体情况，可以自行浏览17个workshop的title）：

【Facebook at ICML 2019】

资料来自Facebook官方博客

https://ai.facebook.com/blog/facebook-research-at-icml-2019/

组织方面：

小编暂未找到Facebook在ICML2019会议中的参与组织的情况

投稿方面：

Facebook共中稿22篇

A Fully Differentiable Beam Search Decoder
AdaGrad Stepsizes: Sharp Convergence Over Non-convex Landscapes
Deep Counterfactual Regret Minimization
Discovering Context Effects from Raw Choice Data
ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero
First-Order Adversarial Vulnerability of Neural Networks and Input Dimension
Garbage In, Reward Out: Bootstrapping Exploration in Multi-Armed Bandits
GDPP: Learning Diverse Generations Using Determinental Point Processes
GEOMetrics: Exploiting Geometric Structure for Graph-Encoded Objects
Making Deep Q Learning Methods Robust to Time Discretization
Manifold Mixup: Learning Better Representations by Interpolating Hidden States
Mixture Models for Diverse Machine Translation: Tricks of the Trade
Multi-modal Content Localization in Videos Using Weak Supervision
Non-Monotonic Sequential Text Generation
Probabilistic Neural-Symbolic Models for Interpretable Visual Question Answering
Self-Supervised Exploration via Disagreement
Separating Value Functions Across Time-Scales
Stochastic Gradient Push for Distributed Deep Learning
TarMAC: Targeted Multi-Agent Communication
Trainable Decoding of Sets of Sequences for Neural Sequence Models
Unreproducible Research Is Reproducible
White-box vs. Black-box: Bayes Optimal Strategies for Membership Inference