Project: FIN-CLARIAH
Grant agreement: Academy of Finland no. 345610
Start date: 01-01-2022
Duration: 24 months
WP 2.5: Report on Digital storage of The National Certificates of Language Proficiency test performances
Date of reporting: 2023-02
Report author: Ari Huhta (JYU)
Contributors: Ari Huhta, Sirkku Kronholm, Mika Halttunen (JYU)
Deliverable location:
Status: delayed due to external factors outside the control of the team (long and complex negotiations between Opetushallitus and JYU about the new 5-year contract on the NCLP examination); see below for details
The National Certificates of Language Proficiency (NCLP) is an official language examination in nine languages that targets adult learners. The exam is the responsibility of the Finnish National Agency for Education (Opetushallitus) which has delegated the design of the examination to the Centre for Applied Language Studies (CALS) at JYU. Since 1994 about 150 000 examinations have been taken, about 110 000 of which in the Finnish as a foreign or second language. With over 7 000 yearly exams in L2 Finnish, the NCLP has the potential for producing a significant amount of learner language for research purposes, particularly for written and spoken Finnish. However, we lack procedures by which examinee performances could be easily turned into a learner corpus and we lack a digital platform for storing these performances.
The NCLP examinations are still paper-based, which is a major issue for creating a digital learner corpus. The digitalisation of the exam has made some progress (e.g. databases for items and an online rating system for speaking performances have been created in recent years) but due to lack of resources the examinees reply to reading, listening and writing tasks on paper. The exception to this is the speaking test that the examinees complete by giving spoken responses in a language / computer lab; these responses are recorded digitally to be evaluated later by human raters.
The plans for creating systematic procedures and a digital platform for storing NCLP performances suffered a serious delay in early 2022 since the negotiations about a new contract concerning the NCLP between the Finnish National Agency and CALS / JYU took a whole year during which no further development (e.g. digitalisation) was funded by the Agency. The new agreement was finalised and signed in December 2022. It is only now that we can resume talks with the Agency about the digitalisation of the NCLP examination, which is a prerequisite for the creation of a new learner corpus.
Our survey of possible digital platforms for the NCLP has yielded one very potential candidate that would also make it possible to store examinee performances in the same system that is used to deliver and take the language tests. However, negotiations with the National Agency about the use of that (or any other) system have not yet started but are envisaged to commence in February 2023. If the National Agency agrees to use the above mentioned digital platform, the platform to store learner performances will, at least initially (and perhaps already in the latter part of 2023), be that digital system. However, it is likely that the performances need to be moved to a more permanent digital store at some later point.
Learner performances from the NCLP can be used in research on second/foreign language proficiency such as studies on the linguistic, textual, phonological etc features of learner language. Because learners’ performances are rigorously evaluated it is possible to conduct research that, for example, investigates the characteristics of learner language at different stages of development. However, NCLP speaking and writing task performances can also be used in other types of research. Some of the tasks ask examinees to tell about different aspects of their everyday life or express their opinions about various topics (the concreteness vs abstractness of the topics depends on the level of examination they take). Such spoken and written texts can shed light on learners’ (such as immigrants taking the Finnish or Swedish language examination as part of the naturalisation process) lives, experiences and opinions that can be of interest to sociologists and ethnologists, for example.
Thus, overall, any work on the envisaged deliverable 2.5.1 can only take place after the first 2-year stage of the Fin-Clariah project – unless the National Agency agrees to move forward with the digitalisation of the NCLP by adopting the digital test platform we have identified as the best option. However, we consider the deliverable 2.5.2 to be the main output of WP2.5 and even more important than 2.5.1 since it relates to and will be useful for several other Work Packages as well as for future work on making NCLP examinee performances available for the research community. A fair amount of learner language has already been collected in several past and present research projects at JYU (but also in the DigiTala project by HU, Aalto and JYU; see WP2.2). Therefore, we have enough data to evaluate the suitability of existing automated annotation systems for L2 Finnish. More data from the NCLP examinations would have been useful at this stage but not necessary or feasible since a systematic analysis of performances from e.g. new tasks from the NCLP would have taken more time and resources than were available for it.
