Integrating research infrastructures into teaching

The IRIS database, the ReLDI repository for data collection instruments, or the SLA Speech Tools repository are often used to search and download learning activities to help students improve their pronunciation in the classroom. The CMU-TalkBank repositories, which give access to well-documented CHILDES corpora are used to teach students how to analyse child language corpora. For example, the corpora have been widely used in undergraduate courses in language development to create handouts containing sample transcripts that students need to analyse and answer specific questions about language, or make more thorough analysis at the phonological, morphosyntax and discourse levels. Other teachers use the CHAT transcription program to teach students how to correct transcription errors or collect, record and transcribe child language data themselves. For more examples, please refer to the CHILDES Teaching Resources.

For parallel corpora, terminology databases or research tools for the study of translated texts, take a closer look at the CLARIN repositories, EuroParl, ELRA-ELDA catalogues, META-SHARE, EuroTermBank, and TAPoR.For example, these repositories can be used to find and download parallel corpora in .tmx format to use for a translation assignment in the Computer-Aided Translation classroom, extract domain-specific terminology to create a bilingual glossary or use the corpus to train a machine translation engine in AI/Machine Learning programmes. Furthermore, many corpora available in CLARIN are directly integrated into concordancers, such as NoSketch Engine, KonText, Corpuscle, and Korp, which can be freely used in the classroom for linguistic research.

To search for speech data and tools, the following repositories may be useful: the Bavarian Archive for Speech Signals (BAS), Speech Data & Technology platform, Open SLR,, and the Linguistic Data Consortium. The BAS repository shares speech resources of contemporary German and detailed information about what standards to use when compiling speech corpora and templates for informed consent forms for speakers participating in a research project. Moreover, the Speech Data and Tech platform offers a Transcription Portal for automatic transcription in English, German, Dutch and Italian.

The ELAR (Endangered Language Archives) repository provides multimedia collections of endangered languages from all over the world, which can be browsed free of charge. For more advanced use of the resources, registration is needed. The archive also provides documentation, guidelines and training on research data management, using ELAN to create, transcribe and translate files and Lameta to create consistent quality metadata.

In Austria, there are two knowledge centres:

1. CLARIN Knowledge Centre for Terminology Resources and Translation Corpora (TRTC) (hosted by the Centre for Translation Studies at the University of Vienna) provides material and training about the preparation and documentation of terminology resources (mono-, bi- and multilingual) and translation corpora, including helpdesk support. 
   • NB1: The terminology databases are publicly available and can be queried via a user-friendly terminology management system, Kaleidoscope. Users can send feedback on the terms and request to add new ones and/or translations. 
   • NB2: The corpora are available via the Austrian Language Resource Portal and can be used for translating Austrian public administration documents from Austrian German into English and for intercultural communication.
2. Phonogrammarchiv K-centre, hosted by the Austrian Academy of Sciences, provides audio and video archives from all research fields, including linguistics, focusing on Austrian scholars and institutions. The recorded materials are annotated to facilitate different modes of analysis and can be accessed via the Academy-CATalogPlus archive for scientific purposes only.

In Germany, SAW Leipzig, a Text + / CLARIN Centre, focuses on preserving language and lexical resources for underrepresented languages and offers various research-based applications to explore them. For example, the Vocabulary Portal gives access to over 30 million sentences of German-language newspaper corpora crawled from the Web. In addition, the centre offers a corpora portal that offers a search interface to more than 1000 corpus-based monolingual dictionaries in 293 languages. The corpora can be downloaded. The platform can be used to search for words with similar context, examples of use, and neighbour cooccurrences and visualise the relations in a word graph.

The CLARIN Knowledge Centre for Computer-Mediated Communication and Social Media corpora provides researchers and students with knowledge, support and training for developing and managing CMC corpora. More specifically, the centre provides FAIR guidelines on data management, such as using standards and formats, and advice on legal and ethical issues. Researchers, teachers and students can contact the centre via their helpdesk. More information about CMC corpora and how to work with this type of corpora is provided in 4.1.3.4. Computer-Mediated Corpora. ILC4CLARIN.IT offers services for browsing and querying corpora, file conversion, term extraction, lexical editing and text annotation, mainly for Italian languages.

Teachers of syntax and morphology can use language resources and training materials from INESS, the Norwegian Infrastructure for the Exploration of Syntax and Semantics. This centre is part of the CLARINO Bergen Centre and the CLARIN Knowledge Centre for Treebanking. It provides access to treebanks that are databases of syntactically and semantically annotated sentences. The platform is language-independent and can be used for building, accessing, searching and visualising treebanks. Data from INESS has also been used in a Master’s course on computational language models to show how empirical corpus data can strengthen or challenge hypotheses about grammar. Students quickly learn to use the system in projects for term papers and master’s theses. Victoria Troland’s master’s thesis, for instance, used INESS to extract syntactic markers from syntactically analysed Norwegian novels and subsequently used these markers as a basis for an author identification model. To teach linguists how to query treebanks, see the INESS Search Walkthrough and the Parseme tutorial: Studying MWE annotations in treebanks.

 The CLASSLA K-centre provides expertise and training on developing language resources and technologies for South Slavic Languages, including Slovenian, Slovene, Croatian, Bosnian, Serbian, Montenegrin, Macedonian, and Bulgarian. The platform gives access to research and tools for language processing, such as information extraction, language understanding, named entity recognition, processing of morphologically rich languages and speech recognition. The centre is managed by CLARIN.SI, the Institute of Croatian Language and Linguistics, and CLADA.BG. Moreover, it offers documentation on how to use the CLARIN.SI infrastructure in Slovene, Croatian and Serbian. A detailed description of the centre is available in Tour de CLARIN.

Teachers and researchers of bilingual language development and sign language are referred to the CLARIN K-centre for Atypical Communication Expertise (ACE), hosted by Radboud University in Nijmegen. Internationally, the centre is linked to the DELAD task force, an initiative that provides guidelines for data acquisition, processing and sharing of corpora and datasets that contain sensitive data (e.g. speech, audio and transcripts collected from people with language disorders). Moreover, the centre collaborates closely with both the Language Archive and the TalkBank repositories to host and give access to corpora of speech disorders securely and in a GDPR-compliant way. Educators can find some examples of well-documented speech corpora on the K-centre website, which can be used to teach students how to analyse disordered speech. Conversely, researchers can use the available corpora to refine the analysis methods and formulate and test hypotheses. For teaching and learning materials related to the impact of GDPR on language research and how to handle sensitive research data collected from human subjects, see this impact story: Navigating GDPR with Innovative Educational Materials. Students and researchers working with patient data learn how to perform a Data Protection Impact Assessment (DPIA) through role plays and use cases.

Several universities that are part of the CLARIN-CH consortium, develop language resources, datasets, and tools and provide expertise and training in language sciences. Teachers and students, searching for language resources, are referred to the collection of CLARIN-CH Resources and the national Linguistic Corpus Platform hosted by LiRI. Here, we highlight the computer-mediated communication (CMC) corpora collected in the What’s up, Switzerland? project. It contains 617 WhatsApp chats in all four national languages of Switzerland and their varieties, and they are freely available for linguistic research. To learn how to use and query the corpus, see the project website: https://whatsup.linguistik.uzh.ch/start. This project is also a good example for students and researchers who want to learn how to process, handle and annotate CMC corpora. 

One of CLARIN’s most essential language resources are corpora of various types, modalities and languages. Corpora are often used to answer research questions in both social sciences and humanities domains (McCarthy & O’Keefe, 2010), and in data-driven learning (DDL;  Johns, 1991) to encourage students to analyse corpus data independently using corpus query platforms, create hypotheses, identify linguistic patterns and formulate rules, and verify the validity of grammatical rules. DDL can benefit, for example, foreign language learners by providing them with access to corpora containing authentic texts and user-friendly tools to explore linguistic patterns and trends in language use and reach their own conclusions (Bernardini, 2002; Boulton, 2009 & 2017). This approach gives students autonomy over their learning process, helps them develop critical thinking skills and turns them into “researchers”. 

According to the literature (Boulton 2009; Gilquin & Granger, 2010), data-driven learning based on corpora has not yet been widely adopted in some language-related programmes because teachers are often unaware of the benefits of using corpora for pedagogical purposes. Second, teachers need a high level of corpus literacy (Mukherjee, 2006) to be able to develop corpus-based pedagogy (CBP), “the ability to integrate corpus linguistics technology into classroom language pedagogy to facilitate language teaching’’ (Ma et al., 2021, p 2).  Furthermore, teachers need to learn to switch to a “less central role’’ in the classroom than in traditional teaching and guide the students through the learning process (Gilquin and Granger, 2010). Finally, DDL may not suit all learner types because it requires certain technical expertise to work with the corpus technologies. It can also be time-consuming, as learners need to analyse concordance results and draw their own conclusions. 

While students appreciate the use of corpus concordancers in the translation classroom, it may take a long time to teach them how to collect enough evidence to be able to make generalisations. Students may also find it difficult to understand how to transform knowledge from another language to another, connect several language resources (e.g. WordNet vs Valency Lexicon), apply linguistic tests, and make decisions regarding translation equivalents. (Petya Osenova, Professor and Researcher in Syntax, Morphology and Corpus Linguistics, Faculty of Slavonic Languages, St. Kl. Ohridski University, Sofia, Bulgaria)

Teachers who never used corpora and corpus technologies in the classroom are recommended to first delve into the literature to understand whether incorporating corpus-based pedagogy (CBP) and data-driven learning in their curriculum would benefit their specific linguistic sub-discipline, teaching style and the students’ background, level and learning style. A good literature review of how corpora are used as a pedagogical tool in various areas of linguistics, see “Part III. Corpora, language pedagogy and language acquisition” in the Routledge Handbook of Corpus Linguistics (O'Keeffe & McCarthy, 2022). 

Other scenarios in which corpora are used in educational settings are teaching corpus linguistics as an academic subject, using corpora to inform syllabus design and development of educational resources (e.g. dictionaries and grammars) and involving students in the development of language resources (collect, design, and compile corpora) (Cheng & Lam, 2022). Finally, students can also be taught how to share and archive a corpus at the end of the project, solving all the issues related to handling personal and sensitive data. See Section 6 in the Moodle UPSKILLS learning content, Introduction to Language Data: Standards and Repositories.

References:

• Bernardini, S. (2002). Exploring new directions for discovery learning. Language and Computers, 42(1), 165–182.
• Boulton, A. (2009). Data-driven learning: Reasonable fears and rational reassurance. Indian Journal of Applied Linguistics, 35(1): 81-106.
• Boulton, A. (2017). Corpora in language teaching and learning. Language Teaching, 50(4), 483–506. DOI: 10.1017/S0261444817000167
• Cheng, W., & P.W.Y. Lam (2022). What can a corpus tell us about language teaching? In A. O'Keeffe, & M. McCarthy (Eds.), The Routledge Handbook of Corpus Linguistics (2nd edn), pp. 299-312. London: Routledge.
• Gilquin, G., & Granger, S. (2010). How can data-driven learning be used in language teaching? In A. O’Keefe & M. McCarthy (Eds.), The Routledge Handbook of Corpus Linguistics, pp. 359-370. London: Routledge.
• Johns, T. (1991). "Chapter 2: Should you be persuaded: Two examples of data-driven learning" (PDF). Classroom Concordancing. Birmingham: ELR.
• Ma, Q., S. Lin & J. Tang (2021). The development of corpus-based language pedagogy for TESOL teachers: A two-step training approach facilitated by online collaboration. Computer Assisted Language Learning, 1–30. DOI: 10.1080/09588221.2021.1895225
• McCarthy, M., & A O’Keefe, A. 2010. Historical perspective: What are corpora and how have they evolved? In A. O’Keefe & M. McCarthy (Eds.), The Routledge Handbook of Corpus Linguistics, pp. 3-13. London: Routledge.
• Mukherjee, J. (2006). Corpus linguistics and language pedagogy: The state of the art–and beyond. In S. Braun, K. Kohn, & J. Mukherjee (Eds.), Corpora and Language Pedagogy: New Resources, New Tools, New Methods, pp. 5-24. Frankfurt: Peter Lang.
• O’Keefe A & M. McCarthy (Eds.). (2022). The Routledge Handbook of Corpus Linguistics (2nd edn). London: Routledge.

Before integrating corpora or any other language resource or tools in the classroom, teachers should first gain some hands-on experience themselves. For example, learn basic methods in corpus linguistics, different types of corpora and corpus technologies, how to extract relevant data from corpora, count occurrences of phenomena, and do statistical analyses (Lin 2019). Some of the active learning content developed in UPSKILLS and available on Moodle can be a good starting point:

• First Steps into Scientific Research: Early-career teachers or trainers without knowledge of scientific research can use this learning block to learn about the main steps in the research process as applied to language, and how to formulate research questions and hypotheses.
• A Glimpse into Language Data Science: This learning block will help learners understand the core concepts related to language data science, such as data description, visualisation, testing of hypotheses and inference.
• Processing Texts and Corpora: This learning block shows teachers how to teach, design, compile, process and analyse corpora for linguistic research. Teachers will find corpus-based activities they can repurpose for their classroom. A final research-based student project has been included for an additional 1-2 ECTS, which also requires students to share and archive their corpus at the end of the project. 

Besides proficient use of corpora and corpus technology, teachers will also need pedagogical knowledge to design corpus-based activities and assessments that match their course's overall goals and students' learning needs (Ma et al., 2021). As practice shows, research is often prioritised over teaching, with lecturers having excellent research skills but poor pedagogical skills (van Dijk et al., 2020). Below, we recommend established workshops, summer schools and free online courses, which could help teachers build up both their technical and pedagogical skills:

• Lancaster University organises workshops, summer schools and online courses for continuing professional education, e.g. 
  • The Lancaster Summer Schools in Corpus Linguistics, usually include a summer school for language learning, teaching and testing, which is aimed at teachers, researchers and students who are interested in language data analysis and research using corpus methods. The programme combines lectures and practical hands-on sessions in computer labs, demonstrating how to explore the British National Corpus (BNC) with corpus-based methods and develop corpus-based materials for language teaching.
  • Corpus Linguistics: Method, Analysis, Interpretation is an eight-week course offered via Future Learn, starting each September. The course demonstrates the use of corpora in discourse analysis, sociolinguistics, and language learning and teaching. It suits teachers, researchers, and students who start working with corpora.
  • The ‘Corpus for Schools project’ offers ideas and activities for English Language teachers.
• The Corpus-Aided Platform for Language Teachers (CAP) is offered by the Department of Linguistics and Modern Language Studies at the Education University of Hong Kong. The platform offers a great collection of teaching activities and regularly organises training and workshops for teachers to help them design corpus-based activities for the classroom.
• The free online Moodle course by Agnieszka Lenko-Szymanska at the Institute of Applied Linguistics, University of Warsaw, is a great example of how to prepare corpus-based teaching materials and class activities for teaching languages, vocabulary, phraseology, grammar,  language for special purposes, discourse, and language skills.
• Although not free of charge, it is worth mentioning the Boot Camp provided by SketchEngine, both online and face-to-face: Boot Camp | Sketch Engine. The Boot Camp does not teach any theoretical background of corpus linguistics, text analysis or NLP but focuses on using the interface, understanding the functionalities and the search results. (For a gentle introduction to Corpus Query Language (CQL), Sketch Engine offers free tutorials and short videos for beginners and advanced users, e.g. CQL – basics | Sketch Engine. The same query techniques can be used in NoSketch Engine, the open-source variant implemented in the CLARIN.SI infrastructure to allow users to query the available corpora.)

Finally, we also recommend keeping an eye on the upcoming CLARIN workshops, which aim to educate and train various stakeholders on the use of corpora and other technologies. 

References:

• van Dijk, E.E., J.V. Tartwijk, M.F. Schaaf & M. Kluijtmans. (2020). What makes an expert university teacher? A systematic review and synthesis of frameworks for teacher expertise in higher education. Educational Research Review 31: 100365.
• Lin, M. H. (2019). Becoming a DDL teacher in English grammar classes: A pilot study. The Journal of Language Learning and Teaching, 9(1), 70–82.
• Ma, Q., Lin, S., & Tang, J. (2021). The development of corpus-based language pedagogy for TESOL teachers: A two-step training approach facilitated by online collaboration. Computer Assisted Language Learning, 1–30. DOI: 10.1080/09588221.2021.1895225

Sometimes, classes in Applied Linguistics, such as Corpus Linguistics and Computational Linguistics, and Translation Studies consist of students with mixed backgrounds (language and humanities vs. computer scientists) and coming from different countries. Such classes pose additional challenges in teaching and learning because the teacher needs to find ways to motivate language and humanities students to work with computational methods and tools. In contrast, the computer scientists need to learn about linguistics. Furthermore, access to multilingual language data repositories and resources is needed to design learning activities in the students’ preferred languages. For example, the Virtual Language Observatory and CLARIN Resource Families may be useful in multilingual language technology classes because they provide access to different types of corpora and datasets in multiple languages. Therefore, it is important to identify students’ backgrounds, languages, technical skills, research interests and their level of interest in technology to be able to tailor the language resources and research methods chosen for the classroom. If the learning activities cater to the needs of the students’ linguistic concerns or research interests, they will feel more motivated to engage with technology. Whenever possible, classes should be tailored to certain student groups instead of attempting to meet the needs of a varied group of students in each course (Baldridge & Erk, 2008).

Try to split the RBT course into feasible portions with clear learning outcomes and consider students’ own interests in making the connections between linguistics and the more technical parts (e.g. programming in Python, data handling). You could start with the linguistic questions, then use technical knowledge to address the questions and translate the answers back into the research domain. (Louis ten Bosch, Associate Professor of Deep Learning and Automatic Speech Recognition, Radboud University, the Netherlands)

References:

• Baldridge, J. & K. Erk. (2008). Teaching computational linguistics to a large, diverse student body: Courses, tools, and interdepartmental interaction. In Proceedings of the Third Workshop on Issues in Teaching Computational Linguistics, pp. 1–9, Columbus, Ohio. Association for Computational Linguistics.

After consolidating their knowledge about scientific research and corpus linguistics, teachers are invited to use this guide and the accompanying course on Moodle, Introduction to Language Data Standards and Repositories, to get acquainted with the CLARIN central services and identify those language resources and tools suitable to include in teaching. The exploration may be easier if it is based on a clear research question, study or project that teachers intend to formulate in the classroom. Consider the type of data the corpus should have, register, languages, size, availability, the time period that the corpus covers, etc. When testing and selecting corpus technology tools, it is recommended to test them properly to identify those functions that will help achieve a specific goal in the classroom. The final choices and implementation of language resources and tools in the classroom are often influenced by teachers’ own perceptions, attitudes, and confidence in applying corpus technology in the classroom (Leńko-Szymańska, A., 2017; Ma et al., 2022).

References:

• Leńko-Szymańska, A. (2017). Training teachers in data-driven learning: Tackling the challenge. Language Learning & Technology 21(3): 217–241.
• Ma, Q., R. Yuan, L.M.E. Cheung & J. Yang (2022). Teacher paths for developing corpus-based language pedagogy: A case study. Computer Assisted Language Learning 2: 1-32. DOI:  https://doi.org/10.1080/09588221.2022.2040537 

Once appropriate resources and tools have been identified for classroom use, the learning outcomes of the course should be adapted in order to target practical skills related to the use of infrastructure, create and curate learning materials and design corpus-based activities and assessments. To save time and effort, teachers are recommended to pick and choose units of individual learning activities from the UPSLILLS Moodle platform, build on them and share them with their students. Instructions on how to export and reuse learning content are included within each learning block. 

When creating learning content for technical-oriented tasks and using different tools, remember that tools change rapidly, so updating the content will require time and effort. Therefore, to save time, consider reusing tutorials provided by the infrastructure and tool providers, and adapting them to match the students’ learning objectives and levels. If learning materials need to be created  from scratch, the focus should be on making them as modular as possible so that other teachers can easily update and reuse them. Moreover, in multidisciplinary courses, learners come from different fields and backgrounds, and some may not yet have all the required skills. This can place an extra burden on the teacher. In online teaching, the issue might be partially solved by creating small, self-contained and well-described learning objects that can be used flexibly in many different courses and replaced or removed when the content becomes obsolete.

Effective planning is crucial when introducing corpora in the classroom. We have compiled a general framework of helpful teaching strategies from Sripicharn (2010), Tribble (2010), and Lessard-Clouston & Chang (2014) that can be referenced when starting to teach with corpora regardless of your linguistic sub-discipline.

1. First, assess students’ prior knowledge. 
2. Introduce corpora and data-driven learning.
3. Identify/create corpus-based tasks/activities for the classroom.
4. Identify and select relevant corpora and tools, providing access to them (preferably accessible online).
5. Show how to use corpus analysis tools, e.g. concordancers, and demonstrate types of different queries. A possible process to teach students how to use and read concordance lines, could involve the following steps:
   • Initiate a search for patterns in a set of concordance lines.
   • Interpret the concordance line results. 
   • Consolidate the results by looking for additional patterns and report them to others by explicitly writing down your observations.
   • Recycle your results by looking for further information, patterns and other contexts.
   • Check what other relevant sources (e.g. dictionaries) say about the patterns you identify.
6. Help students interpret search results and repeat the process with more data from the corpus until they can work independently.

After students have increased their corpus literacy skills, they can be introduced to data-driven learning by challenging them to work on a small-scale research project and answer a specific research question. Bennett (2010) proposes a general framework for using corpora in language teaching, similar to the one above but starting from a research question.

1. Have a clear research question.
2. Determine the register on which your students are focused.
3. Select a corpus appropriate for the register (e.g. spoken or written; general or specialised; contemporary, historical or diachronic; standard or non-standard). 
4. Use a concordance program for quantitative analysis.
5. Engage in qualitative analysis.
6. Create exercises for students.
7. Engage students in a whole-language activity.

According to lecturers’ testimonials in Simonovic et al. (2023), using technology in the classroom (even when starting from a simple task, such as organising data in an Excel file) can disrupt students’ learning experience when unexpected technical issues arise. Students will need guidance and support throughout the course. For example, more advanced students may be affected if the lecturer needs to invest more time in guiding those students who lack basic scientific knowledge and technical skills. A more interdisciplinary approach to teaching language and linguistics could help tackle such issues. For example, students could be recommended to follow computer science courses, while teachers could create joint projects and assignments to ease their workload.  However, if such an interdisciplinary approach is not possible in your programme, here are a few practical tips. Tool handouts, video tutorials, and a pre-defined folder structure for file organisation can help minimise technical disruptions during class. Collaborative online environments like Google Drive, Colab, GitHub, or Open Science Forum (OSF), containing all the files and instructions for the class, can help increase teaching efficiency and minimise disruptions. Additionally, creating a forum within the learning management system (such as Moodle or Blackboard) can assist in reducing the teacher's workload by allowing students to interact with their peers and ask questions about any technical difficulties they may face while completing homework or class assignments. This approach promotes independent problem-solving skills among students and only requires teacher intervention when necessary.

References:

• Bennett, G. (2010). Using Corpora in the Language Learning Classroom: Corpus Linguistics for Teachers. Michigan: University of Michigan Press ELT.
• Lessard-Clouston, Michael & Chang, Tracey. (2014). Corpora and English language teaching: Pedagogy and practical applications for data-driven learning. TESL Reporter 47: 1-20.
• O'Keeffe, A., & McCarthy, M.J. (Eds.). (2022). The Routledge Handbook of Corpus Linguistics (2nd ed.). London: Routledge.
• Simonović, M, van der Lek, I., Fišer, D. & Arsenijević, B. (2021). Guidelines for the students' projects and research reporting formats. UPSKILLS task report. DOI: 10.5281/zenodo.8297430 
• Sripicharn, P. (2010). How can we prepare learners for using language corpora? In A. O’Keefe & M. McCarthy (Eds.), The Routledge Handbook of Corpus Linguistics, pp. 371-384. London: Routledge.
• Tribble, C. (2010). What are concordancers and how are they used? In A. O’Keefe & M. McCarthy (Eds.), The Routledge Handbook of Corpus Linguistics, pp.167-183. London: Routledge.

Undertaking and monitoring research projects involving language resources can be challenging and require continuous guidance and feedback to help students improve their work. With many changes that may occur during the research process, it is easy to get lost and confused. However, keeping track of information at different stages can help clarify ideas, resulting in faster progress. In line with this, our consortium partners from the University of Zurich have developed an interactive research tracking tool that enables students to track their progress during projects. Students fill in the template using the student version of the tool. Teachers then use the teacher's version of the tool to provide brief feedback in the designated field. At the project's onset, it is up to the teachers and students to agree on the reporting frequency. Each submitted report should receive short feedback from the teacher. The preferred sharing mode (email, online drive, etc.) will also be decided. The research tracker can be downloaded from the landing page of this guide.

References:

• Simonović, M, van der Lek, I., Fišer, D. & Arsenijević, B. (2021). Guidelines for the students' projects and research reporting formats. UPSKILLS task report. DOI: 10.5281/zenodo.8297430 

Language resources and technologies can support teaching and research across various disciplines to teach students how to develop a data-driven mindset and draw insights and conclusions from large volumes of text.  While testing and using various language resources and tools corpus classroom use, collecting feedback from students on the usability of the resources, tools and infrastructure and their usefulness in educational settings is beneficial. For example, Lusicky and Wissik (2016) evaluated the usability of language resources like corpora and translation memories, disseminated through research data catalogues such as Virtual Language Observatory, Meta-Share, and ELRA, for translation studies scholars and students. According to the authors, repositories can help researchers make their language resources more FAIR and meaningful for translation studies by including specific metadata catering to the field. For instance, in the case of parallel corpora, it is essential to know the original language, the reliability of the source texts, whether the translation was obtained via post-editing machine-translation output, as well as the names of the translators and their native languages.

The following criteria could be used to evaluate your own experience with the tools and also collect insights from students.

• Accessibility (e.g. User interface design, clarity of instructions, ease of navigation)
• Efficiency (e.g. Search speed, data retrieval, overall tool performance)
• Functionality (e.g. Does the tool include the features you need to help you achieve your goal? e.g. linguistic processing, analysis, data manipulation and visualisation?
• Interoperability (e.g. Can you upload a dataset or a digital text collection you found in a repository into a tool of your choice for further exploration or more sophisticated analysis?)
• Quality of the documentation and other support, training materials provided with the resource and/or tool (e.g. Does the tool creator provide clear documentation, tutorials, and use cases on how the tool can be used for research or teaching?) 

All the feedback, suggestions for improvements or additional features could then be shared with the infrastructure and tools developers to improve the subsequent versions of their tools. See the testimonials below as an example:

The language resources available via the CLARIN infrastructure are very important for my teaching. It is vital that the same materials are persistently available and that they can be cited consistently. I try to teach first-year students to search for corpora and other resources that can be relevant to them via CLARIN platforms, and the more advanced students can benefit from tools, good practices and guidelines that help them to process and make available the resources they create. Praat and ELAN have been around for a long time, and teachers can be confident they will remain accessible in the coming years. University students need good examples of reproducibility, scientific references and citation practices for tools and data since these will be the building blocks they will work with in the future. (Mietta Lennes, Lecturer of Speech Technologies, University of Helsinki)

The students perceive corpora as something entirely new, they tend to get scared initially and need some time to familiarise themselves with different types of queries, and they tend to struggle with regex. But these challenges can be overcome. (Anonymous lecturer, UPSKILLS Questionnaire of Lecturers)

References:

• Lušicky, V., & T. Wissik, T. (2016). Evaluation of CLARIN services, user requirements, usability, VLO, and translation studies. In Selected Papers from the CLARIN Annual Conference 2016, Aix-en-Provence, France, pp. 63-75. Linköping University Electronic Press, Linköpings universitet.

If you teach language variation or pragmatics, you may be interested in using computer-mediated communication corpora (CMC) in the classroom because they include informal writing styles. The resource family of CMC corpora contains open CMC corpora in Slovenian, Czech, Dutch, Estonian, Finnish, French, German, Italian, and Lithuanian. Most corpora are tagged and available for exploration via integrated concordancers. 

If you are searching for multilingual CMC corpora, go to resource families and search for the following corpora: sms4science and What’s Up, Switzerland? . Both corpora have been compiled by Swiss researchers in the Swiss official languages and their varieties and are freely available for linguistic research through online corpus query platforms. The platforms allow you to perform simple and advanced queries, produce a frequency list and export the results. The corpora are also a great example of collecting, processing, cleaning, anonymising and annotating CMC corpora.

Another example of a well-documented CMC corpus is the Twitter corpus Janes-Tweet 1.0, a corpus of Slovenian tweets collected between 2013 and 2017.  This corpus is tokenised, and it can be queried both through NoSketch Engine and the KonText concordancers. Moreover, the corpus is published under a CC BY-NC 4.0 licence and can be downloaded from the CLARIN.SI repository. To learn how to use this corpus for research, see the PARTHENOS tutorial, Using social media corpora in CLARIN.

👉 PRO-TIP: If you or your students are interested in using CMC corpora for research purposes, we recommend the following learning resources:

• Go to PARTHENOS training module: Digital Humanities Research Questions and Methods (Bunout, E. et al., 2019) and read the following sections on CMC corpora:
  • Collections of Computer-Mediated Communication
  • Working with social media corpora
  • Boosting digital humanities research with CMC data
• Remember that on the website of the CLARIN K-Centre for CMC, you will find 
  • FAIR best practices for creating a CMC corpus, which actually can be adapted to any type of corpus, including guidance on the use of standards and formats.

Learner corpora are a valuable pedagogical tool for the teacher-researcher because they contain spoken and/or written texts produced by language learners. 

As per August 2023, the CLARIN resource family of L2 learner corpora contain 72 L2 corpora, out of which 11 are multilingual, while the rest are monolingual forms and various modalities (written, spoken, video). If you are teaching English as a Foreign Language (EFL), you may already use BAWE, BNC and COCA in your classroom. Here we would like to highlight a few corpora and tools that provide useful pedagogical materials for teachers.

The British Academic Written English Corpus (BAWE) is often used to teach linguistic analysis of different genres and academic writing registers, helping students understand academic conventions and develop writing skills. The corpus is hosted in the Oxford Text Archive Repository, and discoverable via the Virtual Language Observatory. Academic users can use it free of charge for academic purposes, and query it in Sketch Engine or Lextutor. To learn how to use it in SketchEngine, see the detailed Using SketchEngine with BAWE tutorial on the website, which consists of 7 lessons demonstrating how to make a simple concordance search, analyse collocations, learn how to use corpus query language and extract keywords. To make it easier for teachers and learners to use BAWE in the classroom, a collection of quicklinks to concordances in SketchEngine have been directly integrated as feedback directly in the students’ assignments to help them improve their academic writing errors. This is a great example of how corpora and data-driven learning (DDL) are used to improve learner autonomy.

The Das Digitale Wörterbuch der Deutschen Sprache (DWDS = Digital Dictionary of the German Language) corpora, developed at the CLARIN-D centre Berlin-Bradenburg Academy of Sciences and Humanities, are used at the University of Kansas as an open educational resource to teach German to English-speaking learners (Vyatkina, 2020a). The resource contains a guide to the main search functionalities of DWDS and a set of interactive corpus-based activities created in h5p. The learning activities have been designed based on guided induction, combining data-driven exploration with linguistic expertise to analyse the DWDS corpus. This pedagogical resource is a great example of how large open-source corpora in languages other than English can be integrated into the classroom (see Vyatkina, 2020b, for more details).

Another tool worth mentioning here, although it is not a CLARIN tool, is SkELL. This tool  is a free online version of SketchEngine specifically developed for English language learners and provides a very simple search interface to look up words and their meanings in a  one-billion-word web corpus. The tool includes a concordance tool, word sketches and a thesaurus functionality. The concordance results are presented in a sentence format. Teachers can use SkeLL to create exercises (gap-fill, matching and multiple choice) to teach students how to infer the meaning of words and phrases, analyse synonyms/polysemy, frequency, word association etc. For more examples of learning activities using SketchEngine and SkeLL, see Thomas (2017).

References:

• Thomas, J. (2017). Discovering English with Sketch Engine: A Corpus-Based Approach to Language Exploration (2nd edn, updated). Versatile.
• Vyatkina, N. (Ed.). (2020a). Incorporating corpora: Using corpora to teach German to English-speaking learners [Online instructional materials]. University of Kansas Open Language Resource Center. https://corpora.ku.edu
• Vyatkina, N. (2020b) Corpora as open educational resources for language teaching. Foreign Language Annals. 53: 359–370. 

Comparable corpora, defined as “a collection of texts composed independently in the respective languages and put together based on similarity of content, domain and communicative function” (Zanettin 1998:614), have extensively been used in educational settings. In translation teaching, for example, they are used as a pedagogical tool to help students enhance their understanding of the source language text and produce fluent translations

ParlaMint is one of the most well-documented multilingual comparable corpora in CLARIN. It comprises 33 parliamentary corpora that cover most of the EU languages. These corpora are an important multidisciplinary language resource for social sciences and humanities researchers. They can be accessed through online corpus query platforms and are available in many languages. This makes it easy to integrate parliamentary corpora into classroom teaching.

For example, search in Parliamentary Corpora Resource Family for ParlaMint.ana 3.0 (Erjavec et al., 2023). This collection contains a multilingual 26 comparable corpora of parliamentary debates, with each corpus being between 9 and 125 million words in size. What makes ParlaMint dataset easy to use both for research and teaching is that the corpora are tokenised and syntactically parsed using the Universal Dependencies (UD) framework,  and annotated with named entities, which enhances the understanding and analysis of the data. The corpora are fully open and available through the noSketch Engine concordancer, a user-friendly tool to introduce the students to concepts in corpus linguistics analysis. If you prefer to use the corpus in another concordancer, you can download it (with or without linguistic markup) from the CLARIN.SI repository. 

The corpora have been used in several editions of the Helsinki Digital Humanities Hackathon (Semparl - Cities in parliamint; Parliamentary Debates in COVID Times), various tutorials, university courses and student theses; see the examples below and more information on the ParlaMint project information page.  Here we would like to highlight two tutorials demonstrating the use of the parliament corpora in research, which teachers can integrate into any course or programme involving modern languages, digital humanities, social sciences and corpus linguistics.

• Voices of the Parliament: A Corpus Approach to Parliamentary Discourse Research. This tutorial uses the siParl 2.0 corpus to teach fundamental corpus linguistic methods to students and scholars of modern languages and learners from other fields, such as digital humanities and social sciences. After a brief introduction to corpora and corpus analysis methods and an introduction to the characteristics of parliamentary debates, the tutorial demonstrates how to use NoSketch Engine and KonText concordancers to analyse the corpora and explore topics female members of the parliament debate in the Slovenian Parliament and contrast their language use with that of their male counterparts. The tutorial takes about five hours to complete, and it has been included in the curriculum of Corpus Linguistics and the Use of Language Technologies in Lexicography course at the Postgraduate School of the Research Centre of the Slovenian Academy of Sciences and Arts. 
• What's on the agenda? Topic modelling parliamentary debates before and during the COVID-19 pandemic. This tutorial can be used to introduce students to basic text-mining concepts by applying topic modelling to a comparable corpus of parliamentary debates. The ParlaMint-GB corpora is used in the tutorial, which students can download from the CLARIN.SI repository. All the techniques demonstrated in the tutorial are language-independent, which means you can also apply them to parliamentary corpora in other languages. 

Both tutorials are reused under a CC-BY-NC-ND 4.0 licence, which means that you can adopt and integrate the tutorials in your classroom, giving appropriate credit, but you are not allowed to use the material commercially. You may not distribute the materials further if you modify and build upon them.

The ORVELIT corpus is a comparable monolingual corpus of original and translated Lithuanian consisting of four sub-corpora of original and translated fiction and popular science. Although the corpus is not available through a concordancer, it can be downloaded from the CLARIN-LT repository (both in a raw and morphologically annotated version). The corpus is being used in the curriculum of the MA course in “Contrastive Stylistics” at Vytautas Magnus University. Students get acquainted with building procedures and characteristics of the ORVELIT corpus, after which they are encouraged to think of possible research questions and registers that would be interesting to explore using the corpus. 

In addition, students learn to create a basic research data management plan. Other learning activities include downloading the morphologically annotated and raw versions of the corpus to investigate the features of original and translated Lithuanian independently with the help of a corpus analysis tool. Finally, students learn to generate and compare wordlists, keyword lists and concordances and discuss their findings on the similarities and differences between original and translated texts. The ORVELIT corpus has been used to design learning activities about the use of Lithuanian collocations in teaching, learning and translation in this online resource book:

→ Kovalevskaitė J., Rimkutė E., Vaičenonienė J. 2022: Lietuvių kalbos kolokacijos: vartojimas, mokymas(is) ir vertimas (Lithuanian Collocations: Usage, Teaching, Learning, and Translation). Kaunas: Vytauto Didžiojo universitetas. ISBN 978-609-467-524-9, https://doi.org/10.7220/9786094675249. (The activities in the book have been designed using many other Lithuanian language resources in the VLO, which are all listed in the tutorial entry on the CLARIN Learning Hub.)

References:

• Erjavec, Tomaž; et al., 2023, Linguistically annotated multilingual comparable corpora of parliamentary debates ParlaMint.ana 3.0, Slovenian language resource repository CLARIN.SI, ISSN 2820-4042, http://hdl.handle.net/11356/1488.
• Zanettin, F. (1998). Bilingual comparable corpora and the training of translators. Meta 43(4): 616–630. DOI: 10.7202/004638ar

Parallel corpora are defined as a “collection of texts in language A and of their correspondent translations into language B” (Baker, 1995). They can be bilingual or multilingual and contain published translations.  Due to the lack of parallel texts, parallel corpora are challenging to build and they are smaller than the monolingual or comparable corpora. For a discussion of issues and challenges in compiling and analysing parallel corpora, see Lefer M.-A. (2020).

Parallel corpora are often used in corpus-based contrastive linguistics and translation studies (Levishina, 2016). In translation studies, bilingual concordances can be used to analyse cross-linguistic correspondences between lexical items, syntactic patterns or grammatical structures and  discuss translation strategies for proper names and culture-specific elements (Lefer M.-A.,2020, p.263). In the practical translation classroom, parallel corpora are also used to teach students how to import it into their computer-aided translation tool and use it as a translation memory (e.g. using corpora from OPUS) or extract bilingual terms using such tools as Synchrotem to create bilingual glossaries. Furthermore, parallel corpora are used in the machine translation/AI classroom to teach students how to  train statistical machine translation engines. Last but not least, Bluemel, B. (2014) showed that parallel corpora can have a pedagogical value in foreign language learning provided the corpus is specifically designed for use in the classroom. He developed a Chinese/English parallel corpus to help high-school students improve their reading comprehension and writing skills. 

The CLARIN resource family provides access to 87 parallel corpora, out of which 5 contain language data in more than 50 languages. Here, we highlight those that can be easily integrated into the classroom.

Well-known parallel corpora used in corpus-based translation/contrastive studies and NLP are: EuroParl, DGT-Translation Memory, European Central Bank parallel corpus, Opus, Helsinki, ParaCrawl. You can view the most important metadata information in the CLARIN Resource Family overview.

InterCorp, a parallel corpus of more than 40 languages, is often used for comparative research in foreign language teaching, translation studies, theoretical studies, and NLP (Čermák 2019). The texts come from various sources, such as fiction, EU legal texts, film subtitles, and the Bible. The corpus can be queried via KonText. The Wiki page of the Czech National Corpus provides a nice tutorial consisting of 11 lessons about using KonText and performing queries in a parallel corpus (InterCorp), spoken corpus, diachronic corpus and syntactically annotated corpus. Each lesson contains clear examples of queries and an exercise at the end.

Compara is a parallel corpus of English and Portuguese, and it can be queried in a free and user-friendly parallel concordancer. The corpus is used in translation studies, not only for descriptive and empirical research but also by lecturers to prepare exercises and discuss translation problems with students.

Unfortunately, few concordancers are available for parallel corpora. In the CLARIN resource family of corpus query tools, we found AntPConc (free desktop-based tool), ParaConc (commercial tool), and SketchEngine (commercial).

References:

• Baker, M. (1995) Corpus linguistics and translation studies: Implications and application. In M. Baker, G. Francis & E. Tognini-Bonelli (Eds) Text and Technology: In Honour of John Sinclair. Amsterdam/Philadelphia: John Benjamins.
• Bluemel, B. (2014). Learning in parallel: Using parallel corpora to enhance written language acquisition at the beginning level. Dimension 1: 31–48.
• Čermák, P. (2019). InterCorp. A parallel corpus of 40 languages. In I. Doval & M. T. Sánchez Nieto (Eds.), Parallel Corpora: Creation and Applications, pp. 93-102. Amsterdam/Philadelphia: John Benjamins.
• Lefer M.-A. (2020). Parallel corpora. In M. Paquot & S. Th. Gries (eds). A Practical Handbook of Corpus Linguistics, pp. 257-282. Cham: Springer.
• Levshina, Natalia. (2016). Why we need a token-based typology: A case study of analytic and lexical causatives in fifteen European languages. Folia Linguistica 50 (2): 507-542.

In June 2023, CLASSLA-web, a new collection of large web corpora (each containing about 2 billion words), was launched. This collection is specifically for Slovenian, Croatian, and Serbian languages, and can be queried using the NoSketch Engine concordancer of CLARIN.SI. To learn how to analyse word usage in contexts, collocations, dictionary examples, and more, visit the tutorial on the CLARIN.SI website. The corpora consist of professional and informal texts, such as forums and blog posts, harvested from the web. This provides linguists and corpus linguists with valuable insights into non-standard language use. Additionally, the monolingual datasets have been linguistically annotated to make them more usable.

The previous Bosnian/Croatian/Montenegrin/Serbian Web corpora (srWaC, hrWaC, bsWaC, meWaC) were used in the MA course, From computational linguistics via clinical linguistics to forensic linguistics, at the University of Graz. The corpora can be downloaded from the CLARIN.SI repository and queried in NoSketch Engine and KonText.

Swiss-Al: A Multilingual Swiss Web Corpus for Applied Linguistics (Krasselt et al., 2020) contains about 8 million texts in German, French and Italian from selected resources on the web (i.e. news and specialist publications, governmental opinions, and parliamentary records, web sites of political parties, companies, and universities, statements from industry associations and NGOs). Unlike previous web corpora (e.g. WaCky; Baroni et al. 2009), Swiss-Al promotes data-based and data-driven research on societal and political discourses in Switzerland, not for NLP purposes. It can be queried via Swiss-AL Workbench and via CQPweb.

👉 PRO-TIP: If you are interested in more multilingual corpora in other languages, you can find a curated collection on the Wiki of the Association for Computational Linguistics.

References:

• Krasselt, J., P. Dreesen, M. Fluor, C. Mahlow, K. Rothenhäusler & M. Runte, M. (2020). Swiss-AL: A Multilingual Swiss Web Corpus for Applied Linguistics. In Proceedings of The 12th Language Resources and Evaluation Conference, 4138-4144.

Spoken corpora are compiled as audio and text transcriptions for linguistic purposes, including hypothesis testing and language teaching and for developing grammars and dictionaries.  In research, these tools serve various purposes, such as phonetics, conversation analysis, grammar, pragmatics, dialectology, language acquisition, and the creation of acoustic models for speech technology. Additionally, they are used to compile corpora of endangered languages in language documentation.

See the Language Archive repository for examples of speech corpora from worldwide languages.

According to Gut (2020), compiling and annotating spoken corpora is expensive, time-consuming, and poses numerous ethical challenges for researchers during data collection and dissemination. Therefore, there are scarce and much smaller in size than written corpora. Therefore, Gut calls upon corpus-based linguistic researchers to investigate how existing spoken corpora can be reused in research and teaching. She emphasises that proper reuse is hindered due to lack of documentation, lack of standardisation in annotations and data format, and lack of access and search tools (p.249).

As of August 2023, there are about 133 spoken corpora in the CLARIN resource families, mostly monolingual in 15 languages. Some  corpora are available for query directly in KonText or Korp. Here we would like to highlight a few spoken corpora used in educational settings.

TalkBank is a CLARIN K-Centre offering the world's largest open-access integrated spoken-language data repository. It provides language corpora and audio resources to support researchers in various fields of study, including Linguistics. Academic users can  transcribe sound files using the CHAT standard and analyse them in CLAN.  A user-friendly guide is available for teachers and students who want to learn how to use CLAN in combination with Praat to analyse speech data. The Slavic data collection from CHILDES corpora is used in the MA, PhD Language Acquisition in Slavic: Obtaining, Representing and Analysing Empirical data in Linguistics course at the University of Graz to help students enhance their research skills in language acquisition, as well as problem-solving and data-analysis skills. 

The web-based LABLASS platform and Bulgarian LabLing Corpus are included in the curriculum of the linguistic disciplines at Konstantin Preslavsky University of Shumen, Bulgaria. The corpus has been published on the CHILDES platform to facilitate cross-linguistic research, including other Slavic languages. Students work in groups on specific projects involving recording and transcribing children’s speech in the CHILDES universal format and data collection tasks in the LABLASS system.

The Database for Spoken German (DGD) is used in the German Linguistic department at the University of Mannheim  to teach BA and MA-level students how to analyse the usage of interaction signs, such as interjections and compare their frequency to written language data, such as postings on Wikipedia talk pages that are available via the Corpus Search, Management and Analysis System (COSMAS II). The latter is often used for cross-lingual studies. Students also explore collocations and create word development curves and word profiles with the help of corpus tools.

Teachers teaching Dutch as a second language should look at the OpenSoNaR corpus, which contains written and spoken corpora of Dutch from the Netherlands and Flanders (500 million words). The corpus is used for linguistic and human language technology research, and the development of NLP applications. It is easily accessible through a user-friendly interface that offers basic and advanced functionalities for corpus queries, including regular expressions.

OpenSoNaR Search Interface

Tutorials in Dutch are available in open access on Surf.nl, and video recordings on Vimeo. You can access the corpus using your university credentials; otherwise, apply for a CLARIN ERIC account.

References:

• Gut, U. (2020). Spoken Corpora. In M. Paquot & S.T. Gries (Eds) A Practical Handbook of Corpus Linguistics. Cham: Springer.

If you are new to incorporating natural language processing tools in your classroom and uncertain about which tools to choose for text or file processing and analysis, exploring the Language Resource Switchboard could prove beneficial. The service can process digital text collections found via the Virtual Language Observatory or other research data repositories (e.g. TextGrid) with matching NLP tools available through the infrastructure. Below, you will find the list of tools that connect to the Switchboard. Note that some tools require you to authenticate with your institutional credentials. 

NLP tools accessible via the Language Resource Switchboard

Before using Switchboard, you may need to introduce students to basic NLP methods for text analysis. On Moodle, you will find a basic tutorial showing the main types of automatic annotation methods useful for linguists, e.g. text segmentation, tokenisation, part-of-speech tagging, syntax/parsing, named entity recognition, and sentiment analysis.

The quick guide below shows how you can process a plain text file from the VLO with Switchboard directly from the search results interface. In Switchboard, depending on the level of the students and the task you want to demonstrate, you can select UDPipe for automatic NLP tasks or choose Voyant Tools to introduce students to basic text analysis in a visually appealing way. If you need to teach more advanced corpus query functionalities, you can also export the file from the repository where it is located and upload it to AntConc or SketchEngine. Finally, you can cite the language resource with the handle assigned by the repository.

Text processing and Analysis with Switchboard

Below, we highlight some tools accessible directly from Switchboard that may be suitable for classroom use.  

Corpus Analysis and Visualisation

We have already presented a few available corpora for browsing and linguistic exploration in tools such as NoSketch Engine, Korp, and KonText. We would now also like to mention Voyant Tools, which are directly accessible via the Language Resource Switchboard. Voyant Tools is a user-friendly web-based reading and analysis environment for digital texts offered by CLARIN-DK and is suitable for teachers and students with no technical background. The tools can also be downloaded and run on local computers. Teachers and students can use this online environment to analyse automatic text with functionalities such as word frequency lists, frequency distribution plots and KWIC displays. Students can use the tools to analyse an online collection of journals, blogs, and websites and include the link to their analysis directly in their research reports (if these are published online) so that the readers can view the results. 

👉 PRO-TIP: See the Resource Families of Corpus Query Tools if you are looking for more corpus-query tools and platforms. The overview includes desktop and online applications, covered languages, and links to user guides.

Automatic Annotation of Texts

UDPipe is a language-independent software that produces custom annotation pipelines for tokenisation, tagging, lemmatisation and dependency parsing of CoNLL-U files. The tool is based on the Universal Dependencies (UD) framework, which produced about 200 treebanks (consistently annotated by a human) for over 100 languages (Zeman et al, 2023). The framework is very popular because it is intuitive and does not adhere to any formal theory. LINDAT CLARIAH-CZ infrastructure provides UDPipe as a free web service for testing services. UD and the CoNLL-U format are  also supported by other annotation tools, such as the BRAT rapid annotation tool, WebAnno, and SketchEngine. An international team of volunteers maintains the framework, and everyone can join them to start building their own corpus through the TEITOK platform. 

UDPipe and the TEITOK UD 2.11. corpus are used in the NLP courses at the Institute of Formal and Applied Linguistics at Charles University for teaching MA students morphological analysis. Students acquire NLP skills by participating in real-life research projects related to the development and training of machine translation engines, corpora, lexicons, speech and dialogue systems, and text processing and semantics.

WebLicht (“Web-based Linguistic Chaining Tool”; hosted by the CLARIN centre at the University of Tübingen) is a user-friendly web service that can be used to teach and demonstrate automatic annotation of texts in English, German, Dutch, French, and Italian. It is included in the practical lab sessions on Corpus Linguistics at the University of Saarland, Germany, to teach corpus annotation. The service is integrated with the CLARIN infrastructure and accessible from Switchboard, Federated Content Search and the Virtual Collection Registry. Several NLP tools (e.g. sentence splitting, tokenisation, lemmatisation, POS tagging, morphological analysis, named entity recognition, dependency parsing, constituency parsing) are integrated to help researchers create and visualise custom processing chains easily. See below for an example of dependency parsing in WebLicht. The workflow is explained in detail in Hinrichs, M. (2022). Users can access the service through their academic institutions. 

Dependency parsing in WebLicht

👉 PRO-TIP: If you are searching for ready-made training materials to introduce your students to UD, check the following learning resources:

• The course Dependency Grammars and Treebanks, offered by Charles University, includes  a syllabus, lecture slides and information about the practical lab sessions.

• This MOOC course is shared via the CLARIN learning hub: Applied Language Technology by Tupmo Hiipala from the University of Helsinki. Part III covers a gentle introduction to NLP concepts for Linguists and Universal dependencies.

Manual Annotation of Texts

WebAnno is a user-friendly open-source web-based annotation tool for manual linguistic annotation tasks (e.g. morphological, syntactical, and semantic annotations) that can be introduced in teaching at the BA and MA level to introduce the students to annotation. The teacher can conduct multiple annotation projects in parallel and assign students different roles, e.g. annotator, curator, or project manager. Students can annotate in groups following the full annotation workflow, as in the figure below. Machine learning capabilities are integrated to make the work of the curator/reviewer easier. For example, WebAnno learns from pre-annotated data and makes suggestions. The annotator can accept or reject the suggestions with a single click. The suggestions help improve the training data. The project manager can assign workload to annotators and monitor their projects. The tool is also offered as a service to members of research institutions, e.g. CLARIN-D and Fin-CLARIN.

Workflow of a WebAnno project (Castilho, 2014)

INCEpTION is an upgraded version of WebAnno that is supported by UKP Lab at TU Darmstadt. This text-annotation platform offers a user-friendly interface for various collaborative semantic annotation tasks on written text, mostly for linguistic and machine-learning purposes. The platform is available both in a desktop version and online, and it includes a recommender system to assist in creating annotations quickly and easily. Additionally, it enables corpus creation by searching external document repositories and importing documents. Due to its user-friendly interface, user guide and tutorials, it can be used in the classroom to create semantic annotation projects with the students. INCEpTION is available as a service to CLARIN-EL members.

Examples of annotations in INCEpTION

Annotation of Speech

WebMaus is a web service developed by the Bavarian Archive for Speech Signals, and it can be used to automatically align transcriptions and speech signals. The results can be used in Praat. The service is part of the suite of BAS web tools for speech processing and provides word and phoneme alignment for more than 25 languages. Watch the short video tutorial that follows:

https://www.youtube.com/watch?v=7lI-gOShtFA

After getting acquainted with some of the tools in the Switchboard, access the service and try it. You can do so by following the steps in the quick guide that follows. Remember that the resources in plain text format from the Virtual Language Observatory can be submitted to Switchboard directly from the VLO interface.

Finding NLP Tools in Language Resource Switchboard

Note also that:

1. Language Resource Switchboard automatically recognises the format and language of input files.
2. It suggests a list of tools for processing the data, categorised by task.
3. The suggested tools include taggers, lemmatisers, named entity recognisers, chunking tools, etc. Select a tool from the list. If you are required to authenticate, try to use your university credentials.
4. Clicking on a tool provides additional information and opens it in a new browser tab. Your file is processed automatically. 

Finally, Switchboard can also be accessed from other research data repositories, e.g. CLARIN VLO, CLARIN Virtual Collection Registry, ARCHE, DARIAH-DE Repository, PARTHENOS VRE, and TextGrid.

If you are interested in teaching students how to compile and analyse large text corpora using programming languages but you do not know where to start, check these UPSKILLS courses on Moodle: 

• The Essence of Machine Learning for Linguists in Tech 
• Start Programming with Python in 10 Steps

Jupyter Notebooks provide a popular environment for programming, especially in educational contexts, to teach coding and concepts such as topic modelling. To understand how they can be used both for research and teaching, we recommend Quinn et al. (2019).

The tutorial demonstrates how to write a Jupyter notebook for data analysis as part of a research project and then adapt it for classroom use. 

The CLARIN centres have also implemented notebooks in their infrastructure to support researchers in data analysis as part of their research projects. You can find a collection of notebooks here. For example, Portulan CLARIN has implemented various NLP tools via a workbench (e.g. quantitative tools for syntax) and includes examples and documentation to help researchers and students design an experiment with Jupyter Notebooks (A great collection of open-source course materials on various NLP topics, including the use of Jupyter notebooks in educational settings, is available on GitHub). 

CLARIN Center of Estonian Language Resources has developed the EstNLTK python package for processing Estonian, as well as a series of tutorials in Jupyter Notebooks on different NLP components, such as text segmentation, morphological processing, syntactic analysis, word embeddings, etc. For educational materials in Estonian, please check the NLP course taught at the University of Tartu, available on GitHub. EstNLTK can be installed in Google Colab, which implements Jupyter Notebooks within the Google suite. This allows you to create a document containing executable code, which is stored on your GDrive and can be shared with peers for editing and commenting. It may be a more user-friendly option than GitHub for classroom use. If you want to learn more about EstNLTK, see this post in Tour de CLARIN.

Other helpful teaching and learning resources are:

• Introduction to Programming for NLP with Python, a web-based course taught by Koenraad de Smedt at the University of Bergen. This course focuses on text processing and data analysis related to linguistics, language studies, digital humanities and cognitive science. Students learn basic programming skills using Jupyter Notebooks that combine Python code examples with explanatory text. The course is suitable for teaching at the BA level.
• The Natural Language Toolkit (NLTK) and tutorial, Natural Language Processing with Python (Bird et al., 2009), are often used as teaching material in programming courses both at the undergraduate and graduate level. After students have acquired knowledge of basic text processing and corpora, you could use the tutorial to teach students how to access and analyse large text archives on the web  (e.g. Project Guttenberg), various types of corpora, and lexical resources. The tutorial contains a list of exercises of various levels of difficulty that can be integrated into the classroom. 
• Another platform often used in NLP training and education is the GATE NLP toolkit (Maynard et al., 2021). You find training materials on the CLARIN Learning Hub, additional information, and useful resources.
• If you teach programming to students and researchers with a strong background in the humanities, the Humanities Data Analysis: Case Studies with Python (Karsdorp et al., 2021) guide may be useful to you. It contains a series of case studies of data-intensive humanities research using Python programming language in Jupyter Notebooks to gather, clean, represent, and transform textual and tabular data. Exercises and resources are included, which turns the guide into a great educational resource.

👉 PRO-TIP: After acquiring basic programming skills, learn how to use Jupyter notebooks and CLARIN NLP tools to process large collections of texts from Virtual Language Observatory with the help of this tutorial: 

• Jupyter Notebooks for Europeana Newspaper Text Resource Processing with CLARIN NLP Tools. Version 1 Retrieved Aug 8, 2023, from the SSH Open Marketplace, https://marketplace.sshopencloud.eu/training-material/duVII1.

References:

• Bird, S., Klein, E., & Loper, E. (2009). Natural Language Processing with Python. O'Reilly. Retrieved July 28, 2023, from https://www.nltk.org/book/
• Maynard, D. K. Bontcheva, I. Roberts, X. Song, M. A. Greenwood, M. Bakir, J. Petrak & Y. Jiang (2021). GATE Training Course, https://gate.ac.uk/wiki/TrainingCourseFeb2021/.
• Quinn D., T. Gniady & D. Kloster. (2019). Introduction to Jupyter Notebooks. Programming Historian 8. DOI: 10.46430/phen0087

All the language technologies introduced in this section can support teaching and research across various disciplines to teach students how to develop a data-driven mindset and draw insights and conclusions from large volumes of text.  While evaluating various NLP tools for classroom use, it is beneficial to collect feedback from students on the usability of the tools used and the infrastructure in general to inform both the tool developers and other teachers who may be interested in using the same tools in the classroom.

The following criteria could be used to evaluate your experience with the tools and collect insights from students.

• Accessibility (e.g. User interface design, clarity of instructions, ease of navigation)
• Efficiency (e.g. Search speed, data retrieval, overall tool performance)
• Functionality (e.g. Does the tool include the features you need to help you achieve your goal? e.g. linguistic processing, analysis, data manipulation and visualisation?
• Interoperability (e.g. Can you upload a dataset or a digital text collection you found in a repository into a tool of your choice for further exploration or more sophisticated analysis?)
• Quality of the documentation and other support, training materials provided with the resource and/or tool (e.g. Does the tool creator provide clear documentation, tutorials, and use cases on how the tool can be used for research or teaching?) 

All the feedback, suggestions for improvements or additional features could then be shared with the infrastructure and tools developers to improve the subsequent versions of their tools.