NLP Seminar Series

The NLP Seminar Series at AI Sweden is a bi-weekly forum for people who work with or are interested in Natural Language Processing (NLP). Join us online or on-site at the AI Sweden Stockholm office on Folkungagatan 44. You can also take the opportunity to work from the AI Sweden Stockholm office for the entire day and/or join us for an NLP fika at 14.30. You are also welcome to join us at the AI Sweden Gothenburg office at some selected events.

Large Generative Models for a smaller language: Finnish GPT-3

Speaker: Sampo Pyysalo, research fellow at the University of Turku
Place: online & in-person in Stockholm and Gothenburg
Note: The Stockholm office has moved to Folkungagatan 44

Abstract: Large generative language models have demonstrated remarkable capabilities in a broad range of tasks within natural language processing and beyond. While increasing numbers of models are becoming openly available, the most capable models remain closed, and open models are limited in their coverage of smaller languages. This presentation introduces open GPT-3 -like models for Finnish, including monolingual models of sizes up to 13B parameters trained from scratch and a 176B-parameter multilingual model created by continuing the training of the largest BLOOM model. The talk will cover challenges and solutions in collecting sufficient data for training, data preprocessing and cleaning, training on the LUMI supercomputer, and the introduction of new evaluation datasets, and conclude with discussion of ongoing and future work building on the models.

Sign me up!

Recordings of previous NLP Seminars can be found here.

By practitioners, for practitioners

Each seminar features a 45-minute presentation by practitioners or researchers followed by a discussion. The NLP Seminar Series is organized by AI Sweden and RISE NLP Group.

Moderators:
Magnus Sahlgren, Head of Research, Natural Language Understanding at AI Sweden
Joakim Nivre, Head of Natural Language Processing at RISE.

Proposals for presentations can be sent to magnus.sahlgren@ai.se or francisca.hoyer@ai.se