TRepLiNa：基于层间CKA+REPINA对齐提升Aya-23 8B模型低资源机器翻译性能 (TRepLiNa: Layer-wise CKA+REPINA Alignment Improves Low-Resource Machine Translation in Aya-23 8B)

The 2025 Multimodal Models for Low-Resource Contexts and Social Impact (MMLoSo) Language Challenge addresses one of India's most pressing linguistic gaps: the lack of resources for its diverse low-resource languages (LRLs). In this study, we investigate whether enforcing cross-lingual similarity in specific internal layers of a decoder-only multilingual large language model (LLM) can improve translation quality from LRL to high-resource language (HRL). Specifically, we combine Centered Kernel Alignment (CKA), a similarity metric that encourages representations of different languages to align, with REPINA, a regularization method that constrains parameter updates to remain close to the pretrained model, into a joint method we call TRepLiNa. In this research project, we experiment with zero-shot, few-shot, and fine-tuning settings using Aya-23 8B with QLoRA across MMLoSo shared task language pairs (Mundari, Santali, Bhili) with Hindi/English pivots. Our results show that aligning mid-level layers using TRepLiNa (CKA+REPINA) is a low-cost, practical approach to improving LRL translation, especially in data-scarce settings.

翻译：2025年低资源场景与社会影响多模态模型（MMLoSo）语言挑战赛致力于解决印度最紧迫的语言资源缺口之一：多样化的低资源语言（LRL）缺乏可用资源。本研究探讨在仅解码器架构的多语言大语言模型（LLM）特定内部层中强制跨语言相似性，是否能提升从低资源语言到高资源语言（HRL）的翻译质量。具体而言，我们将中心核对齐（CKA）——一种促进不同语言表征对齐的相似性度量方法，与REPINA——一种约束参数更新使其接近预训练模型的正则化方法相结合，形成名为TRepLiNa的联合方法。在本研究项目中，我们使用Aya-23 8B模型与QLoRA技术，在MMLoSo共享任务的语言对（蒙达里语、桑塔利语、比利语）以印地语/英语为枢轴语的条件下，进行了零样本、少样本及微调设置的实验。结果表明，采用TRepLiNa（CKA+REPINA）方法对齐中间层级是一种低成本、实用的低资源语言翻译改进方案，在数据稀缺场景中尤为有效。