Swedish AI Model Being Developed With the Help of a Spanish Supercomputer

AI Sweden and Fraunhofer IAIS are collaborating to develop language models for 45 European languages using the supercomputer Mare Nostrum 5. The project, which also aims to support minority languages like Sami, is expected to bring significant improvements for Swedish language models.

Share
Swedish AI Model Being Developed With the Help of a Spanish Supercomputer

AI Sweden and Fraunhofer IAIS now have access to the advanced supercomputer Mare Nostrum 5 in Barcelona to develop large language models for European languages, including Swedish.

A Collaboration for Stronger Language Models

AI Sweden and Germany's Fraunhofer IAIS have begun a collaboration to develop comprehensive language models for 45 European languages and dialects. Magnus Sahlgren of AI Sweden is optimistic: "The model will work really well for Swedish," says Sahlgren.

Benefits for Smaller Languages

Sahlgren, head of research in language understanding at AI Sweden, points out that smaller languages can benefit greatly from being trained together with larger languages. This phenomenon is known as transfer effects. "Smaller languages get better from having a mix of larger languages in the models," says Sahlgren.

Technical Details and Timeline

The project began in May, and the first language models are expected to be ready within a few months. The work requires 8.8 million compute hours on Mare Nostrum 5's cluster, which consists of 4,480 Nvidia H100 graphics chips. This capacity isn't available in Sweden, according to AI Sweden.

A Focus on Minority Languages

One of the goals of the project is also to support minority languages such as Sami. Despite challenges with data availability for Sami, there are plans to adapt the model afterward to include the language. "It really comes down to how much data is available for Sami. But even if we don't manage to include Sami in the actual training process now, the model can later be adapted to handle Sami afterward," Sahlgren explains.