Multilinguality of Language Models

• Chinese models’ access to Chinese data gives them strengths that models trained only on English data may not have, or more generally, each language comes with strengths greater than simply being able to speak the language (language_unique_strengths, 3)
• Bigger models, even open ones, can speak other languages well (different from above as speaking is different from knowing) (bigger_open_multilingual, 2)
• Open models, which also tend to be smaller, have less training on multilingual data than larger proprietary models (open_weak_multilingual, 1)
• In some regions, the ability for open models to speak their language is the dealbreaker (local_language_dealbreaker, 1)