Unique language model expertise strengthens Swedish and European independence

Large language models are the foundation for services and opportunities we have quickly come to take for granted, and for future public services and competitiveness. The work with large open models, such as within the European collaborative project OpenEuroLLM where Sweden and AI Sweden's language team play a key role, builds and retains advanced AI expertise in Sweden.

For AI Sweden, the collaboration with RISE and WASP WARA Media & Language on GPT-SW3, the first large language model for the Nordic languages, was an important milestone. The model was launched in the autumn of 2023.

GPT-SWE3 generated significant international coverage and provided AI Sweden's language team with valuable European relations.

Two years later, AI Sweden is involved in four major EU projects focusing on language technology.

"The most ambitious is OpenEuroLLM, with 20 leading European research institutions, companies, and supercomputer centers participating. This is the most comprehensive effort being made for Europe to catch up with international language model development," says Magnus Sahlgren.

The project has received a so-called STEP designation (Strategic Technologies for Europe Platform) from the European Commission, which means the project is considered to be of particular strategic importance for Europe. The goal is to build a family of high-performing, multilingual language models that can be used by both the private sector and the public sector.

In discussions with organizations in both the private and public sectors, AI Sweden's language team receives signals that there is a demand for open and transparent models developed within the EU.

Magnus Sahlgren continues:

"For many applications, transparency in the development of language models is important. It is not enough to be able to run it on your own servers and thereby avoid exposing sensitive data online. There are use cases where it is also important to know which training data was used and more details about how the training was conducted, partly so that we can trust them and feel secure that they are adapted to European values".

AI Sweden is a central partner in the OpenEuroLLM consortium and has a prominent role in several of the work packages. The NLU team at AI Sweden coordinates, among other things, the work of training the models' tokenizers, as well as the work on fine-tuning and post-training of the models.

"Our role ranges from expertise in data ethics and regulatory compliance to deep technical knowledge about model development," says Magnus Sahlgren.

Digital sovereignty high on the EU's agenda

The issue of digital resilience and digital sovereignty is increasingly high on the EU's agenda. This was evident, for example, in the phrasing of the AI strategy presented by European Commission President Ursula von der Leyen at the beginning of October.

"The strategy is clear on two things: That the use of AI must accelerate across Europe, in both the public and private sectors – and that the public sector, in particular, must strive to 'buy European' when making technology choices, preferably with a focus on open-source AI solutions," says Nina Ökvist.

These arguments start with geopolitical developments but quickly become concrete when it comes to the development and choice of technology. And it is an issue that is bigger than which commercial online services can or cannot be used to solve office tasks.

"As we increasingly seek to use language models as customized solutions for critical tasks, questions about who trained them and how they are made available will become increasingly central. If we are to use them to create value in healthcare, defense, and business, we need to take greater ownership of the technology. This includes both the development of the models, but also where in the world they are run," says Nina Ökvist, and continues:

"The project is significant for the sovereignty of Europe and Sweden. Open models reduce dependence, promote innovation, competitiveness, and diversity. Open source enables transparency and secure AI, allows for scrutiny, and builds trust. The project builds advanced AI expertise in Sweden, which is a strategic asset for the nation."

Unique and advanced expertise as the basis for value-creating applications

The unique knowledge being built up in the language team through these contexts not only creates the conditions for Sweden to be part of the major European projects. It is also incredibly valuable in concrete applications, not least in the work on a joint digital assistant for the Swedish public sector, "Svea."

"In the development of Svea, it has become apparent how incredibly complex it is to meet all the public sector's demands regarding linguistic ability, knowledge content, compliance with Swedish and European legislation, and so on. Thanks to the expert competence held by colleagues in the language team, we have been able to, among other things, create processes for producing training data now being used to develop customized models adapted to the information retrieval needs of municipalities, regions, and government agencies," says Jonatan Permert, project manager for Svea at AI Sweden.

Related content