In what will likely be a litany of generative-model-themed arXiv submissions celebrating April the 1st, we evaluate the capacity of state-of-the-art transformer models to create a paper detailing the detection of a Pulsar Wind Nebula with a non-existent Imaging Atmospheric Cherenkov Telescope (IACT) Array. We do this to evaluate the ability of such models to interpret astronomical observations and sources based on language information alone, and to assess potential means by which fraudulently generated scientific papers could be identified during peer review (given that reliable generative model watermarking has yet to be deployed for these tools). We conclude that our jobs as astronomers are safe for the time being. From this point on, prompts given to ChatGPT and Stable Diffusion are shown in orange, text generated by ChatGPT is shown in black, whereas analysis by the (human) authors is in blue.
翻译:在即将到来的愚人节,我们评估了现代 transformer 模型的能力,通过创作一篇论文来展示它们如何检测不存在的“Imaging Atmospheric Cherenkov Telescope (IACT) Array”和脉冲星风星云。 通过这种方式,我们评估这些模型仅基于语言信息能够解释天文观测和源的能力,并评估如何在同行评审期间识别欺诈性生成的科学论文(因为这些工具尚未部署可靠的生成模型水印)。 我们得出的结论是:作为天文学家,我们的工作目前是安全的。 ChatGPT 和 Stable Diffusion 给出的提示以橙色显示,由 ChatGPT 生成的文本以黑色显示,而由(人类)作者进行的分析则以蓝色显示。