We propose a new kind of automatic architecture search algorithm. The algorithm alternates pruning connections and adding neurons, and it is not restricted to layered architectures only. Here architecture is an arbitrary oriented graph with some weights (along with some biases and an activation function), so there may be no layered structure in such a network. The algorithm minimizes the complexity of staying within a given error. We demonstrate our algorithm on the brightness prediction problem of the next point through the previous points on an image. Our second test problem is the approximation of the bivariate function defining the brightness of a black and white image. Our optimized networks significantly outperform the standard solution for neural network architectures in both cases.
翻译:我们建议了一种新的自动架构搜索算法。 算法将连接和添加神经元替换为连接和添加, 并且不仅限于分层结构。 这里的架构是一个任意方向的图形, 带有一定的重量( 加上一些偏差和激活功能), 这样网络中可能没有分层结构 。 算法将留在给定错误中的复杂性最小化 。 我们通过图像的前面的点来显示下一点的亮度预测问题的算法 。 我们的第二个测试问题就是确定黑白图像亮度的双轨函数的近似值。 我们优化的网络在两种情况下都大大超过神经网络结构的标准解决方案 。