Khanh-Tung Tran, Barry O'Sullivan, Hoang D. Nguyen
Accepted to EACL 2026 (Main Conference)
Large Language Models (LLMs) have achieved impressive performance yet remain inconsistent across languages, often defaulting to high-resource outputs such as English. Existing multilingual alignment methods mitigate these issues through preference optimization but rely on external supervision, such as translation systems or English-biased signal. We propose Multilingual Self-Alignment (MSA), a preference optimization framework that leverages an LLM’s own latent representations as intrinsic supervision signals, rewarding lower-resource language outputs based on their alignment with high-resource (English) counterparts in the ``semantic hub''. We further introduce Language-Consistency MSA (LaCoMSA), which augments MSA with a final-layer language-consistency factor to prevent off-target generation. Integrated with Direct Preference Optimization, LaCoMSA improves a Llama 3 8B-based model multilingual win rates by up to 6.8% absolute (55.0% relatively) on X-AlpacaEval and achieves consistent gains across benchmarks and models. Our findings demonstrate that LaCoMSA can serve as an effective and scalable mechanism, opening a new venue toward multilingual self-alignment.
This repository contains the cleaned implementation for the LaCoMSA paper.
Alignment/ # DPO training scripts
Preprocess/ # Data & reward generation scripts
Data/
requirement.txt
- Environment:
python -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt- Prepare preference data:
cd Preprocess
bash preprocess.sh # more details available in the bash script- Train with. DPO:
cd Alignment
bash dpo.sh example.json@inproceedings{tran-etal-2026-lacomsa,
title = "{L}a{C}o{MSA}: Language-Consistency Multilingual Self-Alignment with Latent Representation Rewarding",
author = "Tran, Khanh-Tung and
O'Sullivan, Barry and
Nguyen, Hoang D.",
editor = "Demberg, Vera and
Inui, Kentaro and
Marquez, Llu{\'i}s",
booktitle = "Proceedings of the 19th Conference of the {E}uropean Chapter of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
month = mar,
year = "2026",
address = "Rabat, Morocco",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2026.eacl-long.224/",
doi = "10.18653/v1/2026.eacl-long.224",
pages = "4839--4853",
ISBN = "979-8-89176-380-7",
abstract = "Large Language Models (LLMs) have achieved impressive performance yet remain inconsistent across languages, often defaulting to high-resource outputs such as English. Existing multilingual alignment methods mitigate these issues through preference optimization but rely on external supervision, such as translation systems or English-biased signal. We propose Multilingual Self-Alignment (MSA), a targeted preference optimization framework that leverages an LLM{'}s own latent representations as intrinsic supervision signals, rewarding lower-resource language outputs based on their alignment with high-resource (English) counterparts in the ``semantic hub''. We further introduce Language-Consistency MSA (LaCoMSA), which augments MSA with a final-layer language-consistency factor to prevent off-target generation. Integrated with Direct Preference Optimization, LaCoMSA improves a Llama 3 8B-based model multilingual win rates by up to 6.8{\%} absolute (55.0{\%} relatively) on X-AlpacaEval and achieves consistent gains across benchmarks and models. Our findings demonstrate that LaCoMSA can serve as an effective and scalable mechanism, opening a new venue toward multilingual self-alignment."
}