The Newspaper and Periodical Corpus of the National Library of Finland, Kielipankki Version

Suomeksi

This corpus contains newspapers and magazines from Finland starting from 1770, compiled by the National Library of Finland.

• Listing of the newspapers and magazines included in the Finnish sub-corpus, version 1
• Listing of the newspapers and magazines included in the Finnish sub-corpus, version 2
• Listing of the newspapers and magazines included in the Swedish sub-corpus, version 1
• Listing of the newspapers and magazines included in the Swedish sub-corpus, version 2 (preliminary information)

NB: The Finnish acronym for the corpora The Newspaper and Periodical OCR Corpus of the National Library of Finland used to be ”Digilib”. Currently, however, the acronym ”klk” and the short names klk-fi-1874-dl and klk-fi-1920-dl are recommended instead.

Of this language corpus different versions/subcorpora are published in the Language Bank of Finland. The versions are available through the Language Bank Download Service and/or through the Korp concordance tool. The links to the different versions can be found from the list above.

Detailed information on the content of each version, user rights and licenses can be found from it’s specific metadata record in META-SHARE.

N-grams

Based on the KLK data, word-level collections of uni-, bi- and trigrams have been created and are available for download. These are their own data sets:

The N-grams of the Newspaper and Periodical Corpus of the National Library of Finland

Example queries from Korp

OCR quality

The corpora consist mainly of digitized versions of texts originally printed on paper. These physical papers have been scanned, and optical character recognition (OCR) was performed on the resulting images. The digitized material spans a long period and contains different kinds of texts, writing styles and fonts. Scanning some parts of the material is more complex than scanning other parts, and the physical condition of the original texts also varies. The OCR techniques used have also varied, and there is the possibility that some of the texts have gone through manual post-correction. This results in some parts of the corpora being of terrible quality while others are of good quality. We have collected a list of publications related to OCR quality and collection processing:

• Kettunen, K., & Pääkkönen, T. (2016, May). Measuring lexical quality of a historical finnish newspaper collection―Analysis of Garbled OCR data with basic language technology tools and means. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16) (pp. 956-961).

• Koistinen, M., Kettunen, K., & Kervinen, J. (2017). How to improve optical character recognition of historical Finnish newspapers using open source Tesseract OCR engine. Proc. of LTC, 279-283.

• Kettunen, K., Kervinen, J., & Koistinen, M. (2018, April). Creating and Using Ground Truth OCR Sample Data for Finnish Historical Newspapers and Journals. In DHN (pp. 162-169).

• Kettunen, K., Koistinen, M., & Ruokolainen, T. (2018, April). Research and Development Efforts on the Digitized Historical Newspaper and Journal Collection of The National Library of Finland. In DHN (pp. 474-479).

• Kettunen, K., & Koistinen, M. (2019, March). Open Source Tesseract in Re-OCR of Finnish Fraktur from 19th and Early 20th Century Newspapers and Journals-Collected Notes on Quality Improvement. In DHN (pp. 270-282).

• Drobac, S. (2020). OCR and post-correction of historical newspapers and journals. University of Helsinki.

• Hamdi, A., Jean-Caurant, A., Sidère, N., Coustaty, M., & Doucet, A. (2020). Assessing and minimizing the impact of OCR quality on named entity recognition. In Digital Libraries for Open Knowledge: 24th International Conference on Theory and Practice of Digital Libraries, TPDL 2020, Lyon, France, August 25–27, 2020, Proceedings 24 (pp. 87-101). Springer International Publishing.

• Kettunen, K., Koistinen, M., & Kervinen, J. (2020). Ground truth OCR sample data of Finnish historical newspapers and journals in data improvement validation of a re-OCRing process. LIBER Quarterly: The Journal of the Association of European Research Libraries, 30(1), 1-20.

• Jauhiainen, T., Piitulainen, J., Axelson, E., & Lindén, K. (2022, October). Language identification as part of the text corpus creation pipeline at the language bank of finland. In Digital Humanities in the Nordic and Baltic Countries Conference (pp. 251-259). CEUR.

• Kettunen, K., Keskustalo, H., Kumpulainen, S., Pääkkönen, T., & Rautiainen, J. (2022). OCR quality affects perceived usefulness of historical newspaper clippings–a user study. arXiv preprint arXiv:2203.03557.

• Rautiainen, J. (2022). Digitized Finnish Newspapers in Digital Humanities Research Projects: Challenges and Solutions from the Library Perspective.

This resource group page has a Persistent Identifier: http://urn.fi/urn:nbn:fi:lb-2021092404

Last updated: 19.6.2024