Autoencoders as Tools for Program Synthesis

Recently there have been many advances in research on language modeling of source code. Applications range from code suggestion and completion to code summarization. However, complete program synthesis of industry-grade programming languages has not been researched extensively. In this work, we introduce a variational autoencoder model for program synthesis of industry-grade programming languages. Our model incorporates the internal hierarchical structure of source codes and operates on parse trees. By learning a latent representation of source code over trees, we capture more information and achieve a higher performance than standard autoregressive autoencoder models. Furthermore, due to the tree-structured nature of our model, the autoregressive operations are performed on paths of trees instead of linear sequences. Therefore, the size of the sequences that the autoregressive model processes, scales proportionally to the width and depth of the tree instead of the total size of the tree which mitigates the common problem of exploding and vanishing gradients.