对抗学习是一种机器学习技术,旨在通过提供欺骗性输入来欺骗模型。最常见的原因是导致机器学习模型出现故障。大多数机器学习技术旨在处理特定的问题集,其中从相同的统计分布(IID)生成训练和测试数据。当这些模型应用于现实世界时,对手可能会提供违反该统计假设的数据。可以安排此数据来利用特定漏洞并破坏结果。

VIP内容

论文题目:Learning to Weight Imperfect Demonstrations (ICML 2021)

作者:Yunke Wang, Chang Xu, Bo Du, Honglak Lee

论文概述:这篇论文主要解决的问题是如何在生成对抗模仿学习(GAIL)中为不完美专家演示加权。模仿学习期望智能体通过模仿专家的行为来进行学习,然而在许多现实世界的任务中专家也会犯错,由此产生的不完美专家演示将会严重误导智能体的学习。目前,已有的一些基于加权和偏好学习的解决不完美专家演示的方法往往依赖额外的先验信息,无法在更普遍和通用的模仿学习设置下使用。因此,本文提出了一种在生成对抗模仿学习的框架下为专家演示自动生成权重的方法,通过严格的数学证明,我们发现专家演示的权重可以在训练中由GAIL中的判别器和智能体策略估算得到。理论分析显示,当我们使用该估算的权重,智能体事实上在学习一个比原始给定的专家策略更优的策略。在Mujoco和Atari上的实验结果显示了算法的优越性。

http://proceedings.mlr.press/v139/wang21aa.html

成为VIP会员查看完整内容
0
8

最新内容

Self-supervised learning approach like contrastive learning is attached great attention in natural language processing. It uses pairs of training data augmentations to build a classification task for an encoder with well representation ability. However, the construction of learning pairs over contrastive learning is much harder in NLP tasks. Previous works generate word-level changes to form pairs, but small transforms may cause notable changes on the meaning of sentences as the discrete and sparse nature of natural language. In this paper, adversarial training is performed to generate challenging and harder learning adversarial examples over the embedding space of NLP as learning pairs. Using contrastive learning improves the generalization ability of adversarial training because contrastive loss can uniform the sample distribution. And at the same time, adversarial training also enhances the robustness of contrastive learning. Two novel frameworks, supervised contrastive adversarial learning (SCAL) and unsupervised SCAL (USCAL), are proposed, which yields learning pairs by utilizing the adversarial training for contrastive learning. The label-based loss of supervised tasks is exploited to generate adversarial examples while unsupervised tasks bring contrastive loss. To validate the effectiveness of the proposed framework, we employ it to Transformer-based models for natural language understanding, sentence semantic textual similarity and adversarial learning tasks. Experimental results on GLUE benchmark tasks show that our fine-tuned supervised method outperforms BERT$_{base}$ over 1.75\%. We also evaluate our unsupervised method on semantic textual similarity (STS) tasks, and our method gets 77.29\% with BERT$_{base}$. The robustness of our approach conducts state-of-the-art results under multiple adversarial datasets on NLI tasks.

0
0
下载
预览

最新论文

Self-supervised learning approach like contrastive learning is attached great attention in natural language processing. It uses pairs of training data augmentations to build a classification task for an encoder with well representation ability. However, the construction of learning pairs over contrastive learning is much harder in NLP tasks. Previous works generate word-level changes to form pairs, but small transforms may cause notable changes on the meaning of sentences as the discrete and sparse nature of natural language. In this paper, adversarial training is performed to generate challenging and harder learning adversarial examples over the embedding space of NLP as learning pairs. Using contrastive learning improves the generalization ability of adversarial training because contrastive loss can uniform the sample distribution. And at the same time, adversarial training also enhances the robustness of contrastive learning. Two novel frameworks, supervised contrastive adversarial learning (SCAL) and unsupervised SCAL (USCAL), are proposed, which yields learning pairs by utilizing the adversarial training for contrastive learning. The label-based loss of supervised tasks is exploited to generate adversarial examples while unsupervised tasks bring contrastive loss. To validate the effectiveness of the proposed framework, we employ it to Transformer-based models for natural language understanding, sentence semantic textual similarity and adversarial learning tasks. Experimental results on GLUE benchmark tasks show that our fine-tuned supervised method outperforms BERT$_{base}$ over 1.75\%. We also evaluate our unsupervised method on semantic textual similarity (STS) tasks, and our method gets 77.29\% with BERT$_{base}$. The robustness of our approach conducts state-of-the-art results under multiple adversarial datasets on NLI tasks.

0
0
下载
预览
参考链接
Top