Despite -- or perhaps because of -- their simplicity, n-grams, or contiguous sequences of tokens, have been used with great success in computational linguistics since their introduction in the late 20th century. Recast as k-mers, or contiguous sequences of monomers, they have also found applications in computational biology. When applied to the analysis of texts, n-grams usually take the form of sequences of words. But if we try to apply this model to the analysis of Sanskrit texts, we are faced with the arduous task of, firstly, resolving sandhi to split a phrase into words, and, secondly, splitting long compounds into their components. This paper presents a simpler method of tokenizing a Sanskrit text for n-grams, by using n-aksaras, or contiguous sequences of aksaras. This model reduces the need for sandhi resolution, making it much easier to use on raw text. It is also possible to use this model on Sanskrit-adjacent texts, e.g., a Tamil commentary on a Sanskrit text. As a test case, the commentaries on Amarakosa 1.0.1 have been modelled as n-aksaras, showing patterns of text reuse across ten centuries and nine languages. Some initial observations are made concerning Buddhist commentarial practices.
翻译:尽管 -- -- 或者可能是因为 -- -- 它们的简单性、正克或相毗的象征序列 -- -- 它们自20世纪末开始以来,在计算语言学中一直非常成功地使用它们的简单性、正克或相毗的象征序列 -- -- 也许是因为 -- -- 它们自20世纪末开始以来在使用它们的简单性、正克或相毗的象征物序列时,它们也发现在计算生物学中的应用。当用于分析文本时,正克通常采取文字序列的形式。但如果我们试图将这个模型应用于分析梵文文本时,我们面临着艰巨的任务,首先是解决沙希,将一个短语分为文字,其次是将长的化合物分为其组成部分。本文提出了一种简单的方法,通过使用正克沙拉或相毗连的线序列来象征正方格文本。这个模型减少了沙希决议的需要,使原始文本更容易使用。我们也可以在圣斯克里特-对文本上使用这个模型,例如,在圣斯克里特文本上用泰米尔语评注,然后将长的化合物分割成其组成部分。作为试验案例,Amarakak-asa 1 10世纪的文字模型展示了Amarasa 的理论。