In this note we prove a sharp lower bound on the necessary number of nestings of nested absolute-value functions of generalized hinging hyperplanes (GHH) to represent arbitrary CPWL functions. Previous upper bound states that $n+1$ nestings is sufficient for GHH to achieve universal representation power, but the corresponding lower bound was unknown. We prove that $n$ nestings is necessary for universal representation power, which provides an almost tight lower bound. We also show that one-hidden-layer neural networks don't have universal approximation power over the whole domain. The analysis is based on a key lemma showing that any finite sum of periodic functions is either non-integrable or the zero function, which might be of independent interest.
翻译:在本说明中,我们证明,对通用超高空飞机(GHH)的嵌套绝对价值功能的必要数量所构成的嵌套绝对值功能,对于代表任意的CPWL功能而言,我们有一个明显较低的约束。 上一个约束表示, $+1 的嵌套足以让GHH获得普遍代表权, 但相应的较低约束并不为人所知。 我们证明, 美元嵌套对于普遍代表权是必要的, 它提供了几乎更紧的下限。 我们还表明, 单层神经网络没有覆盖整个域的通用近似能力。 分析基于一个关键列姆马, 显示周期功能的任何有限总和要么不可加固, 要么是零功能, 可能具有独立的兴趣 。