| Shortname | Name and metadata | License | Location | Cite | Resource group and help | Apply | Publication year | Support level |
|---|---|---|---|---|---|---|---|---|
| Shortname | Name and metadata | License | Location | Cite | Resource group and help | Apply | Publication year | Support level |
FinEst BERT is a Bidirectional Encoder Representations from Transformers (BERT) multilingual model trained from scratch, covering three languages: Finnish, Estonian, and English. Used for various NLP classification tasks on the mentioned three languages, supporting both monolingual and multilingual/crosslingual (knowledge transfer) tasks. Whole-word masking used during data preparation and training; trained for 40 epochs with sequence length 128 and another 4 epochs with sequence length 512. FinEst BERT model published here is in pytorch format.
Corpora used:
Finnish – STT articles, CoNLL 2017 shared task, Ylilauta downloadable version
Estonian – Ekspress Meedia articles, CoNLL 2017 shared task
English – English wikipedia
Of this language corpus different versions are (or might be in the future) published in the Language Bank of Finland. The versions are available through the Language Bank Download Service and/or through the Korp concordance tool, or they are offered by another member organisation of FIN-CLARIN. The links to the different versions can be found from the list above.
Last updated: 07.11.2024
This resource group page has a Persistent Identifier: http://urn.fi/urn:nbn:fi:lb-2021091402
