题目

生成式对抗网络先验贝叶斯推断,Bayesian Inference with Generative Adversarial Network Priors

关键字

生成对抗网络,贝叶斯推断,深度学习,人工智能,计算物理学,图像处理

简介

当两者通过物理模型链接时,贝叶斯推断被广泛用于根据相关场的测量来推断并量化感兴趣场的不确定性。尽管有许多应用,贝叶斯推理在推断具有大维离散表示和/或具有难以用数学表示的先验分布的字段时仍面临挑战。在本手稿中,我们考虑使用对抗性生成网络(GAN)来应对这些挑战。 GAN是一种深层神经网络,具有学习给定字段的多个样本所隐含的分布的能力。一旦对这些样本进行了训练,GAN的生成器组件会将低维潜矢量的iid组件映射到目标场分布的近似值。在这项工作中,我们演示了如何将这种近似分布用作贝叶斯更新中的先验,以及它如何解决与表征复杂的先验分布和推断字段的大范围相关的挑战。我们通过将其应用于热噪声问题中的热传导问题中的推断和量化初始温度场中的不确定性的问题,论证了该方法的有效性,该问题由稍后的温度噪声测量得出。

作者

Dhruv Patel, Assad A Oberai

成为VIP会员查看完整内容
0
20

相关内容

生成对抗网络(GAN)是Ian Goodfellow及其同事在2014年设计的一类机器学习框架。两个神经网络在游戏中相互竞争(从博弈论的角度讲,通常但并非总是以零和博弈的形式)。 在给定训练集的情况下,该技术将学习生成具有与训练集相同的统计数据的新数据。 例如,受过照片训练的GAN可以生成新照片,这些新照片至少对人类观察者而言表面上看起来真实,具有许多现实特征。 尽管GAN最初是作为一种形式的无监督学习模型提出的,但它也已被证明可用于半监督学习,完全监督学习和强化学习。

知识荟萃

精品入门和进阶教程、论文和代码整理等

更多

查看相关VIP内容、论文、资讯等

自回归文本生成模型通常侧重于局部的流畅性,在长文本生成过程中可能导致语义不一致。此外,自动生成具有相似语义的单词是具有挑战性的,而且手工编写的语言规则很难应用。我们考虑了一个文本规划方案,并提出了一个基于模型的模仿学习方法来缓解上述问题。具体来说,我们提出了一种新的引导网络来关注更长的生成过程,它可以帮助下一个单词的预测,并为生成器的优化提供中间奖励。大量的实验表明,该方法具有较好的性能。

成为VIP会员查看完整内容
0
43

题目: A Survey on Distributed Machine Learning

简介: 在过去十年中,对人工智能的需求已显着增长,并且这种增长得益于机器学习技术的进步以及利用硬件加速的能力,但是,为了提高预测质量并在复杂的应用程序中提供可行的机器学习解决方案,需要大量的训练数据。尽管小型机器学习模型可以使用一定数量的数据进行训练,但用于训练较大模型(例如神经网络)的输入与参数数量成指数增长。由于处理训练数据的需求已经超过了计算机器的计算能力的增长,因此急需在多个机器之间分配机器学习工作量,并将集中式的精力分配到分配的系统上。这些分布式系统提出了新的挑战,最重要的是训练过程的科学并行化和相关模型的创建。本文通过概述传统的(集中的)机器学习方法,探讨了分布式机器学习的挑战和机遇,从而对当前的最新技术进行了广泛的概述,并对现有的技术进行研究。

成为VIP会员查看完整内容
0
80

题目:* Certified Adversarial Robustness with Additive Noise

摘要:

对抗性数据实例的存在引起了深度学习社区的高度重视;相对于原始数据,这些数据似乎受到了最小程度的干扰,但从深度学习算法得到的结果却非常不同。尽管已经考虑了开发防御模型的大量工作,但大多数此类模型都是启发式的,并且常常容易受到自适应攻击。人们对提供理论鲁棒性保证的防御方法进行了深入的研究,但是当存在大规模模型和数据时,大多数方法都无法获得非平凡的鲁棒性。为了解决这些限制,我们引入了一个可伸缩的框架,并为构造对抗性示例提供了输入操作规范的认证边界。我们建立了对抗扰动的鲁棒性与加性随机噪声之间的联系,并提出了一种能显著提高验证界的训练策略。我们对MNIST、CIFAR-10和ImageNet的评估表明,该方法可扩展到复杂的模型和大型数据集,同时对最先进的可证明防御方法具有竞争力的鲁棒性。

作者简介:

Changyou Chen是纽约州立大学布法罗分校计算机科学与工程系的助理教授,研究兴趣包括贝叶斯机器学习、深度学习和深度强化学习。目前感兴趣的是:大规模贝叶斯抽样和推理、深度生成模型,如VAE和GAN、用贝叶斯方法进行深度强化学习。

成为VIP会员查看完整内容
0
28

Sufficient supervised information is crucial for any machine learning models to boost performance. However, labeling data is expensive and sometimes difficult to obtain. Active learning is an approach to acquire annotations for data from a human oracle by selecting informative samples with a high probability to enhance performance. In recent emerging studies, a generative adversarial network (GAN) has been integrated with active learning to generate good candidates to be presented to the oracle. In this paper, we propose a novel model that is able to obtain labels for data in a cheaper manner without the need to query an oracle. In the model, a novel reward for each sample is devised to measure the degree of uncertainty, which is obtained from a classifier trained with existing labeled data. This reward is used to guide a conditional GAN to generate informative samples with a higher probability for a certain label. With extensive evaluations, we have confirmed the effectiveness of the model, showing that the generated samples are capable of improving the classification performance in popular image classification tasks.

0
4
下载
预览

Both generative adversarial network models and variational autoencoders have been widely used to approximate probability distributions of datasets. Although they both use parametrized distributions to approximate the underlying data distribution, whose exact inference is intractable, their behaviors are very different. In this report, we summarize our experiment results that compare these two categories of models in terms of fidelity and mode collapse. We provide a hypothesis to explain their different behaviors and propose a new model based on this hypothesis. We further tested our proposed model on MNIST dataset and CelebA dataset.

0
6
下载
预览

This paper studies the problem of generalized zero-shot learning which requires the model to train on image-label pairs from some seen classes and test on the task of classifying new images from both seen and unseen classes. Most previous models try to learn a fixed one-directional mapping between visual and semantic space, while some recently proposed generative methods try to generate image features for unseen classes so that the zero-shot learning problem becomes a traditional fully-supervised classification problem. In this paper, we propose a novel model that provides a unified framework for three different approaches: visual-> semantic mapping, semantic->visual mapping, and metric learning. Specifically, our proposed model consists of a feature generator that can generate various visual features given class embeddings as input, a regressor that maps each visual feature back to its corresponding class embedding, and a discriminator that learns to evaluate the closeness of an image feature and a class embedding. All three components are trained under the combination of cyclic consistency loss and dual adversarial loss. Experimental results show that our model not only preserves higher accuracy in classifying images from seen classes, but also performs better than existing state-of-the-art models in in classifying images from unseen classes.

0
6
下载
预览

Network embedding has become a hot research topic recently which can provide low-dimensional feature representations for many machine learning applications. Current work focuses on either (1) whether the embedding is designed as an unsupervised learning task by explicitly preserving the structural connectivity in the network, or (2) whether the embedding is a by-product during the supervised learning of a specific discriminative task in a deep neural network. In this paper, we focus on bridging the gap of the two lines of the research. We propose to adapt the Generative Adversarial model to perform network embedding, in which the generator is trying to generate vertex pairs, while the discriminator tries to distinguish the generated vertex pairs from real connections (edges) in the network. Wasserstein-1 distance is adopted to train the generator to gain better stability. We develop three variations of models, including GANE which applies cosine similarity, GANE-O1 which preserves the first-order proximity, and GANE-O2 which tries to preserves the second-order proximity of the network in the low-dimensional embedded vector space. We later prove that GANE-O2 has the same objective function as GANE-O1 when negative sampling is applied to simplify the training process in GANE-O2. Experiments with real-world network datasets demonstrate that our models constantly outperform state-of-the-art solutions with significant improvements on precision in link prediction, as well as on visualizations and accuracy in clustering tasks.

0
4
下载
预览

Recently introduced generative adversarial network (GAN) has been shown numerous promising results to generate realistic samples. The essential task of GAN is to control the features of samples generated from a random distribution. While the current GAN structures, such as conditional GAN, successfully generate samples with desired major features, they often fail to produce detailed features that bring specific differences among samples. To overcome this limitation, here we propose a controllable GAN (ControlGAN) structure. By separating a feature classifier from a discriminator, the generator of ControlGAN is designed to learn generating synthetic samples with the specific detailed features. Evaluated with multiple image datasets, ControlGAN shows a power to generate improved samples with well-controlled features. Furthermore, we demonstrate that ControlGAN can generate intermediate features and opposite features for interpolated and extrapolated input labels that are not used in the training process. It implies that ControlGAN can significantly contribute to the variety of generated samples.

0
5
下载
预览

We introduce an effective model to overcome the problem of mode collapse when training Generative Adversarial Networks (GAN). Firstly, we propose a new generator objective that finds it better to tackle mode collapse. And, we apply an independent Autoencoders (AE) to constrain the generator and consider its reconstructed samples as "real" samples to slow down the convergence of discriminator that enables to reduce the gradient vanishing problem and stabilize the model. Secondly, from mappings between latent and data spaces provided by AE, we further regularize AE by the relative distance between the latent and data samples to explicitly prevent the generator falling into mode collapse setting. This idea comes when we find a new way to visualize the mode collapse on MNIST dataset. To the best of our knowledge, our method is the first to propose and apply successfully the relative distance of latent and data samples for stabilizing GAN. Thirdly, our proposed model, namely Generative Adversarial Autoencoder Networks (GAAN), is stable and has suffered from neither gradient vanishing nor mode collapse issues, as empirically demonstrated on synthetic, MNIST, MNIST-1K, CelebA and CIFAR-10 datasets. Experimental results show that our method can approximate well multi-modal distribution and achieve better results than state-of-the-art methods on these benchmark datasets. Our model implementation is published here: https://github.com/tntrung/gaan

0
10
下载
预览
小贴士
相关VIP内容
生成式对抗网络GAN异常检测
专知会员服务
78+阅读 · 2019年10月13日
相关资讯
揭开GANs的神秘面纱
机器学习算法与Python学习
10+阅读 · 2019年2月27日
探幽深度生成模型的两种方法:VAE和GAN
AI前线
11+阅读 · 2018年3月10日
虚构的对抗,GAN with the wind
全球人工智能
4+阅读 · 2017年10月22日
Adversarial Variational Bayes: Unifying VAE and GAN 代码
CreateAMind
7+阅读 · 2017年10月4日
GAN | GAN介绍(2)
中国科学院网络数据重点实验室
40+阅读 · 2017年8月4日
GAN | GAN介绍(1)
中国科学院网络数据重点实验室
16+阅读 · 2017年7月26日
量化投资与建模基于贝叶斯系列
量化投资与机器学习
6+阅读 · 2017年7月12日
相关论文
Finding Needles in a Moving Haystack: Prioritizing Alerts with Adversarial Reinforcement Learning
Liang Tong,Aron Laszka,Chao Yan,Ning Zhang,Yevgeniy Vorobeychik
3+阅读 · 2019年6月20日
Quan Kong,Bin Tong,Martin Klinkigt,Yuki Watanabe,Naoto Akira,Tomokazu Murakami
4+阅读 · 2019年6月17日
Zhiming Zhou,Jiadong Liang,Yuxuan Song,Lantao Yu,Hongwei Wang,Weinan Zhang,Yong Yu,Zhihua Zhang
8+阅读 · 2019年2月15日
A Probe into Understanding GAN and VAE models
Jingzhao Zhang,Lu Mi,Macheng Shen
6+阅读 · 2018年12月13日
GAN Dissection: Visualizing and Understanding Generative Adversarial Networks
David Bau,Jun-Yan Zhu,Hendrik Strobelt,Bolei Zhou,Joshua B. Tenenbaum,William T. Freeman,Antonio Torralba
11+阅读 · 2018年12月8日
He Huang,Changhu Wang,Philip S. Yu,Chang-Dong Wang
6+阅读 · 2018年11月12日
Huiting Hong,Xin Li,Mingzhong Wang
4+阅读 · 2018年5月21日
Felix Kreuk,Assi Barak,Shir Aviv-Reuven,Moran Baruch,Benny Pinkas,Joseph Keshet
4+阅读 · 2018年5月13日
Minhyeok Lee,Junhee Seok
5+阅读 · 2018年5月1日
Ngoc-Trung Tran,Tuan-Anh Bui,Ngai-Man Cheung
10+阅读 · 2018年3月23日
Top