Guidelines and Best Practices

This part introduces the topic area and the rationale for these guidelines and best practices, as well as the general structure of research-based courses described in the UPSKILLS best practices. In Section A1, we present the general concepts of research-teaching nexus and research-based teaching, discuss the challenges in implementing research-based teaching and outline how the present guidelines and best practices can be used to overcome these challenges. Section A2 discusses research-based teaching in the broader context of the UPSKILLS project, especially focusing on how research-based teaching can be beneficial to the employability of graduates of language-related programmes while also enabling them to make informed career choices. Section A3 presents topics covered in a typical UPSKILLS RBT course grouped into three topic blocks (Research design, Research infrastructures & techniques and Subject-specific aspects), the rationale behind this grouping, as well as the ideal time distribution across the topic blocks. In Section A4, we provide specific advice regarding the identification of the domain within the lecturer’s area of expertise that can be optimally covered in an RBT course, with special attention for the transferable skills that need to be acquired by the students. Finally, Section A5  addresses what students learn in an RBT course, devoting special attention to general research-related learning outcomes. The learning outcomes are organised in a structure that matches the structure of RBT topics presented in Section A3. 

Teaching and research constitute the two core activities in academia. However, these two activities are typically planned, performed and evaluated separately. It, therefore, comes as no surprise that the connection between them, usually discussed in the literature under the rubric of the research-teaching nexus, receives little attention in the reality of most academic institutions (for a notable exception, and a rich collection of experiences and ideas, see Bastiaens, van Tilburg & van Merriënboer 2017). 

The near absence of attention for the research-teaching nexus at the institutional level has ramifications for the ways academics see these two activities. Existing research on the research-teaching nexus indicates that academics conceptualise research and teaching separately. In a study of the nexus in humanities, Visser-Wijnveen (2009: 141) found that “academics’ conceptions of the research‐teaching nexus are related to their conceptions of teaching and not to their conceptions of research and knowledge”. Moreover, she found that “the conceptions of research and knowledge are more closely related to each other than to the conception of teaching”. The explanation offered by the author is that the conceptions of research and knowledge come from a different source than those related to teaching. For most academics, conceptions of research and knowledge are formed in the course of their research schooling (presumably during their postgraduate studies), whereas their conceptions related to teaching originate from their earlier experiences as students, as confirmed in studies on beginning teachers (e.g. Feiman‐Nemser & Remillard, 1996). 

Addressing the research‐teaching nexus explicitly is desirable at all levels, from individual to team, institutional and national. While higher levels are also crucial, our main goal in these guidelines is to assist individual lecturers in developing both well-elaborated views and putting these views into practice in the research‐teaching nexus. We trust that the most logical and rewarding first step is the inclusion of one’s own ongoing research into ongoing teaching in the form of research-based teaching (henceforth, RBT). While the other direction, the advancement of research based on teaching, should not be neglected (e.g. Giraud & Saulpic 2019), the focus of the UPSKILLS project is on the enhancement of students’ skills in ways that will enhance their employability. For this reason, the present document will concentrate on research-based teaching alone. 

Figure 1. Healey’s (2005, Jenkins et. al 2007) adapted model of the research‐teaching nexus (from Visser‐Wijnveen et al. 2010)

In the model proposed in Healey (2005) and refined in Jenkins et al. (2007), represented in Figure 1 below, research-based teaching is one of the four possible implementations of the research-teaching nexus. Research-based teaching focuses on the process of research rather than on the results of research and involves students as active participants rather than as audience. As such, research-based teaching introduces students to the role of researchers to a higher extent than the other three implementations shown in Figure 1. 

The rationale for focusing on research-based teaching in addressing the research-teaching nexus is threefold. First, exposing students to the actual research practices helps them acquire aspects of disciplinary knowledge (including the disciplinary conceptualisations of knowledge and research discussed above) that otherwise remain implicit and unaddressed. As previous research shows, “research awareness is stimulated by a carefully organised close look at their teacher’s research” (Visser-Wijnveen 2009: 141-142). Second, we trust that the everyday research practice of academics involves transferable skills which do not get taught to students during regular courses. Finally, for the lecturers, research-based teaching has the potential of obtaining valid research output from their students but also of discovering teachable aspects of their ongoing research activities that students will find useful or even necessary for their future (academic and non-academic) careers. 

According to Dekker  &  Wolf  (2016), the four manifestations of the research-teaching nexus in Figure 1 are ordered on a scale of the importance of the role of research as in Figure 2.

Figure 2. Dekker & Wolf (2016)’s scale of the research-teaching nexus

A further classification of the research‐teaching nexus, which does not perfectly overlap with the one shown above, is that into five profiles describing what the lecturer does in a course. These are I) teach research results, II) make research known, III) show what it means to be a researcher, IV) help to conduct research, and V) provide research experience (Visser-Wijnveen 2009: 131). Our implementation of research-based teaching involves elements of all profiles but its overlap with profile V) is the greatest, while that with profile I) is the most limited, with all the other profiles falling somewhere in between these two extremes. 

In courses with a strong research-based component of the kind that will be discussed in these best-practice guidelines, students are involved in activities of ongoing research projects, using the same research tools and infrastructures that the lecturer uses in their own research. Students can then acquire and actively apply problem-solving  and analytical skills to actual data, using approaches implemented in actual research. In some of the scenarios, when the lecturer is involved in this type of research activities, students can also be exposed to industry-based research, which they are likely to encounter in their post-university professional careers. In this setting, industry-based research refers to research that companies conduct as part of their business, rather than with the aim of advancing knowledge, as academics typically do, and comprises a number of tasks that graduates are bound to encounter in their day-to-day workflow when they gain employment in the industry (for more details, see our dedicated report on Integrating Industry-based Research into Teaching).

The concept of research-based teaching is often met with ambivalence by lecturers. Most first experiences with research-based teaching seem to be marked by considerable challenges and frustrations. An important part of these frustrations stems from underestimating how much additional work (but also what kind of additional work) is required in order to set up and run a research-based course. In many cases, research-based teaching is initially perceived as the intersection of two well-known domains (one’s own teaching and one’s own research) and therefore requires minimal additional effort. However, the actual implementation can involve setbacks that are not usually encountered in non-RBT courses.

The challenging aspects of these experiences are arguably traceable to the systematic lack of a well-elaborated view of the research‐teaching nexus, despite clear conceptualisations of research and of teaching. As already previewed above, one of the main goals of these best-practice guidelines is to facilitate the establishment of such a well-elaborated view.

The ambivalent attitude of lecturers towards research-based teaching does not mean that there is no recognition that this form of teaching is useful and necessary. As transpired from the survey we held within the preliminary needs analysis of our project, a bit over 50% of lecturers indicated that they integrate research into teaching, while around 75% indicated that they would be happy to follow dedicated training focusing on this. It is therefore safe to conclude that there is much room for expanding the domain of research-based teaching in the individual repertoires of the lecturers. Also, Visser-Wijnveen (2009: 141-142) found that students enrolled in courses with a strong research component report more learning outcomes on academic disposition and research awareness than their teachers had aimed for in their course designs.

Congruent signals are coming from the industry. The lack of training in research skills, data acquisition and data handling skills were cited by our informants from the industry as one of the major obstacles to integrating linguists into collaborative workflows (Miličević Petrović et al. 2021). 

Against the background sketched above, these best-practice guidelines have the goal of giving lecturers a headstart in developing their elaborated view of the research-teaching nexus and developing or improving their own research-based teaching. We complement the existing literature in at least three ways. First, our best-practice guidelines have a disciplinary scope, focusing only on courses in language-related disciplines. Second, they also take into account the employability of graduates of language-related programmes (see next section). Finally, they have a broad testing ground: they are based on experiences from sixteen courses taught by six consortium members.

These best-practice guidelines are especially targeting individual lecturers who are willing to integrate their ongoing research into their teaching. Clearly, they can also be used by research teams, departments etc. to create more variegated RBT courses. While not explicitly addressed, we believe that our best-practice guidelines will also be useful to administrators and policy makers at the university and beyond in articulating their agenda when it comes to research-teaching nexus.

We created these best-practice guidelines in an attempt to establish a general course-design framework that can be easily adapted to most university settings in Europe. However, we are aware that some of the contexts are less welcoming to research-based teaching than others. It is our hope that administrations and policy makers who get in touch with these best-practice guidelines will help take the necessary steps in adopting research-based teaching on a larger scale. 

References:

• Bastiaens E., van Tilburg J., van Merriënboer J. (eds.) (2016). Research-Based Learning: Case Studies from Maastricht University. Professional Learning and Development in Schools and Higher Education, vol 15. Cham: Springer, Cham. https://doi.org/10.1007/978-3-319-50993-8
• Dekker, H. & Wolf, W.S. (2016). Re-inventing research-based teaching and learning. Paper presented at the European Forum for Enhanced Collaboration in Teaching of the European University association, Brussels.
• Feiman‐Nemser, S., & Remillard, J. (1996). Perspectives on learning to teach. In F.B. Murray (ed.), The Teacher Educator’s Handbook (pp. 63‐91). San Francisco, CA: Jossey‐Bass.
• Giraud, F. and Saulpic, O. (2019). Research-based teaching or teaching-based research: Analysis of a teaching content elaboration process. Qualitative Research in Accounting & Management 16(4), 563-588. https://doi.org/10.1108/QRAM-10-2017-0097
• Healey, M. (2005). Linking research and teaching exploring disciplinary spaces and the role of inquiry-based learning. In R. Barnett (ed.) Reshaping the University: New Relationships between Research, Scholarship and Teaching (pp. 67-78). Maidenhead: Open University Press.
• Jenkins, A., Healey, M., & Zetter, R. (2007). Linking teaching and research in departments and disciplines. Report. York, UK: The Higher Education Academy.
• Miličević Petrović, Maja, Bernardini, Silvia, Ferraresi, Adriano, Aragrande, Gaia, & Barrón-Cedeño, Alberto. (2021). Language data and project specialist: A new modular profile for graduates in language-related disciplines. UPSKILLS report. https://doi.org/10.5281/zenodo.5030929
• Visser-Wijnveen, G. J. (2009). The research-teaching nexus in the humanities: Variations among academics. Phd thesis. Leiden University.
• Visser‐Wijnveen, G. J., Van Driel, J. H., Van der Rijst, R. M., Verloop, N., & Visser, A. (2010). The ideal research‐teaching nexus in the eyes of academics: Building profiles. Higher Education Research & Development 29(2), 195-210.  https://doi.org/10.1080/07294360903532016

The central goal of the UPSKILLS project is to identify skills gaps and mismatches in the curricula of language-related programmes and tackle these skills gaps and mismatches through the development of a new curriculum component and supporting materials to be embedded in existing linguistics and language-related programmes. These measures are intended to enhance students’ employability by providing them with the skills needed to compete for a wider range of positions in the labour market.

We trust that explicit attention to the research‐teaching nexus in general and research-based teaching in particular are beneficial not only to the employability of graduates of language-related programmes but also to their ability to make informed career choices. 

One aspect that is often forgotten in the discussions of the employability of graduates is that universities are employers as well and that there is no guarantee that a university will be inclined to hire its own graduates on a research project. At academic institutions where the gap between teaching and research is large and graduates have limited research skills, project leaders may have a tendency to hire graduates from other institutions or even other disciplines. Multiple consortium members can provide anecdotal evidence of this type, where especially graduates of programmes like psychology or computer science seem to have a headstart over graduates of language-related programmes, not because the position requires disciplinary knowledge related to psychology or computer science but because they are more versed in general research skills like data collection, data processing or statistics. 

A further negative consequence of a lack of involvement of students in actual ongoing research is that graduates are likely to develop a misguided notion of what research is. In study programs where lecturing and broad yet shallow handbooks predominate in teaching methods, students mistake introductions and overviews for the immediate results of research. This can for instance be observed when students are asked to come up with topics for research for a course paper or a BA or MA thesis, and instead of a research question, they come up with a domain that corresponds to a chapter in a textbook. “I want to investigate secondary predicates’”, or “My plan is to conduct research on the genitive case”. This indicates that students tend to think of research as a purely positivist enterprise in which encyclopedic knowledge is accumulated. This knowledge is then viewed as presentable in a new version of the textbook. Misconceptions of this type may lead the students to avoid a career path in research or research-related pathways or, paradoxically, to strive for one on the false grounds. Experience with real-life academic research is an important antidote to this, enabling students to make informed career choices. 

What is true of university-based career paths can be for a good part extended to other career paths. This is because most of the skills identified by the industry as lacking in graduates of language-related programmes in our needs analysis (Miličević Petrović et al. 2021) are essential to research activities performed at their universities and performed by their lecturers. As argued above, these skills are present, but not entirely transferable because there is a lack of attention for the research‐teaching nexus. 

While we trust that involvement in actual ongoing research is beneficial as such, we did develop a specific approach largely influenced by our disciplinary focus and the realities of our academic institutions. As already previewed, we are focussing on research-based teaching as the approach which enables the most direct involvement in research practices. A key aspect of our implementation of research-based teaching is the use of a research report as a dominant format of the final deliverable by the student. As will be elaborated below, the research report should have as many characteristics as possible of an ‘actual’ research report (in the sense of being presentable/publishable at an actual academic venue) and it should also be evaluated as such. 

As mentioned above, these guidelines are mainly based on our experience with RBT courses. The main instrument of quality assurance was piloting and obtaining feedback from both instructors and students. In the course of the project, sixteen courses were run. We have thus used and will continue to use the feedback from both students and lecturers to reshape the guidelines in order to ensure the broadest possible applicability.

In this section, we present the most general layout of UPSKILLS RBT courses. Many teachers are aware that the involvement of students in specific research projects may carry the risk of being too specific so that the students get a narrow perspective on research, failing to get acquainted with its full width and depth regarding topics, methods and techniques. This is a problem that riddles most involvement of students in internships and student assistantships: students perform tasks that are useful to the project, but without really getting the perspective of the entire research design in which they are participating. In order to avoid this pitfall, our research-based teaching guidelines envision a general introduction to research design as part of all RBT courses. Once the general background is in place, specific research topics can become the testing ground on which analytical and problem-solving skills can grow.  

In Table 1, we present the general overview of the topics applicable to any UPSKILLS RBT course. 

Table 1.  Overview of topics in an UPSKILLS course

The overview in Table 1 contains three main topic blocks. The order of the topic blocks reflects the order of appearance of the general topic in a majority of UPSKILLS courses (first we discuss the research design, then the infrastructures and techniques and only then do we spend substantial time on the disciplinary issues specific to the course). Also, the relative order of the topics within the topic block is the one shown in Table 1 for a majority of the UPSKILLS courses. However, in most RBT courses, topic blocks will overlap in time, i.e. some specific topics from the first block will be introduced after some of the topics from the second and the third block. (A more detailed overview of topics can be found here). In a detailed course plan, the colour-coding of the different topics can help the lecturer to keep track of the topic blocks and the time dedicated to each of them. 

The time dedicated to each of the topic blocks will naturally vary from course to course, but in Figure 3 below we show the ideal distribution of time over the three topic blocks. In the description of the topics below, we also point out the topics that in our experience often turn out to require more time than initially planned, so we recommend planning some extra time for them.

We now turn to each of the topic blocks. The topic block Research design contains topics that are common and essential to any research. In a RBT course outline, it should account for a substantial amount of course duration (ideally 35%, no less than 20% of time), since this block contains the most transferable skills inherent to conducting research. As discussed above, in our experience some of these skills may be so self-evident that lecturers may become aware of them only during the course. This is why, especially when the course is taught for the first time, we recommend keeping some room for developing these ‘covert’ skills in the course outline.

As stressed above, it is essential for each RBT course to contain a segment dedicated to General research design. An issue commonly encountered in teaching on this topic is that students have very different levels of previous knowledge and different previous research experience. An additional problem is that it often feels face-threatening to many students to inquire about the very basic aspects, as this is experienced as admitting lacking a clear idea of how research works. One resource which we found useful in shaping the classes dedicated to this topic is the course “Introduction to Research in Linguistics: Theory, Logic, Method”, which was created by some of the consortium members in a previous project. This online course (or parts of it) can be used as preparatory reading followed by a discussion session in which all the basic concepts are covered in order to arrive at a common ground which can be further elaborated in class. 

Once an understanding of the general research design is established, we can move on to Adapting general research design to the specific topic of interest, where some initial specific research designs are discussed as applying to the subject matter of the course. Our experience speaks in favour of planning ample time for this topic, as it is one of the topics where lecturers often discover that ways of thinking they take for granted need to be explicated at some length.

While it is useful to consider research designs as if there were no limitations on what can actually be done, once this step is completed, it is important to move on to Adapting the research design to the available research infrastructures. This is a further topic which in our experience usually requires more time than initially expected and should be planned accordingly.

Finally, once an idea of a research design is in place, we can turn to Research reporting. For this aim, we used the general research-report template designed by two UPSKILLS consortium members and presented here. As foreshadowed above, completing this template will lead to the final deliverable of the course, on which also a substantial part of the evaluation will be based. It is therefore instrumental for students to create their own copy of this template and start filling it in as soon as it is introduced to them. As discussed at some length in Section B2 , in adapting the research report template for a particular course, instructors inevitably face a degree of tension between creating a report that closely mirrors the conventions of the discipline (thus resembling a real-world research paper), and one that is as clear and transparent as possible by documenting each step in separate sections. In this case as well, it is helpful to encourage students to ask questions and seek clarification regarding both the disciplinary conventions and the documentation requirements.

The second topic block,  Infrastructures & techniques, focuses on the specific tools the students will use in order to implement their research design. This also includes identifying suitable datasets in repositories and using data responsibly. The way the topics in this block are structured is highly dependent on the specific infrastructures and techniques used by the lecturer in their research. However, even in cases where some of the tools are already provided, (e.g. all literature is provided beforehand, so Infrastructures & techniques for obtaining literature could be skipped) we recommend spending some time on each of these topics. For instance, if no literature search is necessary for the specific domain, a minor literature-search assignment can be added on a closely related domain. This theme block also has an important potential when it comes to transferable skills. Working knowledge of tools used in actual research is not only an important asset, but also an important springboard for learning how to use other tools.  

The final topic block, Subject-specific aspects, is used as an umbrella block for all topics which are strictly disciplinary. In all RBT courses a General question can be identified, whose answering typically requires addressing further Particular questions, tasks and skills. This is the domain in which the lecturer is an expert. This block is essential, but it should not cover more than 60% of the entire course outline (ideally 45%). With this in mind, we can now turn to the ways in which an optimal course subject can be selected.

As previewed above, a common feature of all RBT courses is that they are centered around a subject that the lecturer is currently working on or has extensive research experience with. To be sure, this does not exclude course scenarios where the students select the narrower topic for their projects. However, these topics should fall within the area of expertise of the lecturer. 

The reason for insisting on the research expertise of the lecturer lies in the fact that, as emphasised in previous chapters, the actual research practices of the lecturer are an important and underexploited resource. While designing and teaching an RBT course, the lecturer identifies research and transferable skills that can be taught to students.

Whereas on the side of the lecturer we recommend sticking to subjects within the area of expertise, on the side of the students no extensive background should be a prerequisite. Here again, the expertise of the lecturer is what guarantees that the necessary background will be provided (e.g., in the form of the best available overview of the state-of-the-art, short presentation, etc.).

In the selection of the course subject, we recommend considering which steps in answering the research question can be performed by the students without extensive assistance by the lecturer and making sure that such steps constitute a majority of the research trajectory. In many cases, there are steps that need to be performed by the lecturer or with extensive help by the lecturer (e.g., the final formulation of the research question, coming up with a data set or performing a complex statistical analysis). In such cases, these steps can be turned into classroom activities, where the students can appreciate what these steps entail and learn how the knowledge lacking at the moment can be acquired in order to perform this step independently.

A further beneficial aspect to be considered when choosing a suitable subject for an RBT course is the possibility of triangulation. If multiple methodological approaches can be applied in addressing the same research question, this is an important asset to be used at different steps of the research trajectory. Groups of students can work using different methods and exchange their experiences and results.

Finally, many research questions can be addressed by applying the same methodology to different samples. In this case, groups of students can work with “their” samples throughout the course and their results can be combined in a single conclusion. While we recommend using the type of data that the lecturer has experience with, the data should not be previously published or analysed (at least using the given methodology). The real-life-like aspect of an RBT course is exactly about the outcome being unknown before the course.

In this section, we address what students learn in an RBT course with special attention to general research-related outcomes. The topics listed in Table 1 map onto learning outcomes which we provide in Table 2.  

Table 2. Learning outcomes per topic block in an UPSKILLS course

A more detailed list of specific learning outcomes is available here. As will be discussed in the following section, we suggest using this more detailed list when creating an RBT course. Unsurprisingly, subject-specific learning outcomes (the yellow zone) are defined only in a very general way in this overview because they depend on the specific subject of the course.   

The learning outcomes in the first two topic blocks reflect competences necessary in research on which the course was based, ranging from very general research skills to familiarity with specific techniques and infrastructures. Each of these skills has application beyond the immediate academic context in which it was initially presented. 

Well-defined learning outcomes naturally connect to evaluation tools. As already mentioned above, we suggest using the research report as the main deliverable by the student, on which an important part of the final grade is based. In our experience, the complete report allows for evaluating the students’ work on all of the course outcomes. However, it does occur that students omit some of the information from the research report necessary for the evaluation. It is therefore commendable to plan at least one round of feedback before the final submission of the research report, so that specific aspects can be further elaborated. We discuss the assessment and evaluation in an UPSKILLS course in further detail in Section B3. 

With the necessary groundwork in place, in this part, we lay down specific recommendations for the creation and implementation of an RBT course. Section B1 presents a step-by-step roadmap from the selection of an RBT course subject to designing a preliminary course plan. Section B2  tackles the workflows in an RBT course, the optimal use of the research report in framing and organising the students’ work and the recommendations for writing (research) instructions for students. In Section B3, we address the optimal division between the portions of students’ work on which they only receive feedback and those that contribute to their final grade, as well as the importance of communicating this distinction to the students in order to foster creativity and encourage innovative thinking. Finally, Section B4 focuses on the issues of data reusability, giving the final feedback to students and debriefing them, as well as on the ways of receiving feedback from the students about the RBT course. 

We open these guidelines with a quick roadmap to making an RBT course outline. In our experience, the best way to use this roadmap is to open a separate document and start taking notes at each step, moving on to the next step only after completing the previous one. Of course, previous steps can be revised based on insights at later steps.

Based on our experiences with RBT courses we also formulated some DOs (👍) and/or DON’Ts (👎) for each step of the trajectory.  

RBT courses lend themselves excellently for an outcome-based and goal-oriented organisation. The final deliverable by the students, the research report, is experienced by the students as a tangible result of their efforts, but also as an important milestone in their academic development. As a consequence, students find activities much more meaningful if they view them as a step toward completing their own project report. We therefore encourage introducing the research-report outline at the outset of the course and referencing the specific parts of the outline whenever possible. 

Since the research report provides the general framework within which all specific activities are situated, we recommend devoting special attention to the adaptation of our research report template to the specific course. As mentioned in Section B1, the version prepared by us has dedicated chapters for all the major steps in the research process. We found this useful when working with students who have little experience with research reporting, as it helps them focus on the content of the report without having to address different steps in the same chapter. When teaching a more advanced course, there is more room for making the research report similar to an actual research paper by lumping together some of the steps in the report, following the mores in the relevant field. 

While the research report is a useful final goal, more ‘local’ goals and outcomes also have a role to play in the organisation of the work and motivation of the students. In most RBT courses, both teaching and the actual research activities are divided between homework and in-class activities. In both cases, the purpose of the specific activities and their function in the ‘big picture’ of the course goals may be much less obvious to the students than the lecturer might expect. We therefore recommend starting and closing each activity with an explicit reference to the learning outcomes. For example, if students are supposed to learn to extract a certain type of data from a corpus, and the lecturer provides a brief presentation on how to do that, this outcome should be explicitly mentioned before and after the presentation. At the next step, when the students are extracting data for their own projects, e.g., as part of their homework, this outcome should be referred to before and after this activity. While this may sound overly formal and potentially tedious, it takes less than a minute and our experience is that this helps students to appreciate the type and goal of the activity they are engaging in, as well as to make the most of it. In the absence of explicit tags, some students tend to miss the information on how general the skill is that they are acquiring or disregard the role of this skill in the more general research design. 

In most RBT courses, a good part of the steps in the research process is taken by the students as part of their homework. In this case, we recommend formulating the instructions on what needs to be done and how it needs to be documented in the research report. If possible, these instructions should be presented to the students in class and some time (10 to 15 minutes) should be allowed for a Q&A in which any ambiguity is discussed and cleared out of the way. It is actually useful to revise the instructions during the Q&A as necessary, so the same instructions can be reused for another group.

Ideally, the work submitted by students consists of parts of the research report (together with possible accompanying output, such as annotations, lists, descriptive statistics etc.). These are short enough to make it possible to give a sizable amount of feedback in class. In our experience, many comments apply to work submitted by multiple students. Here  as well, we recommend keeping track of common misinterpretations of the instructions (e.g., terms which have a specific meaning in the context of the research report). When revising the instructions, the lecturer can consider clearing the confusing formulations out of the way by making the instructions or the rubrics in the research-report outline more explicit.

An important advantage of the students’ work being centered around a single final deliverable, the research report, is that it also allows both the lecturer and the students to keep track of the progress made during the course. For the student, the research report can also serve as a useful indication of the number of completed steps and of the steps that still lie ahead. 

Clearly, all research-based courses require the lecturer to provide some necessary background, both on general research-related issues and on subject-specific issues. In our experience, students tend to find it useful if such less active parts of learning are clearly delineated and if the need for them is explicated. One option we recommend is providing the necessary background through a short unit that includes the following parts:

• preparatory reading,
• short presentation by the teacher,
• Q&A
• short (but obligatory) quiz.

While many of our courses have been taught to small groups and, in extreme cases, to single students, we have had some positive experiences with group work in RBT courses. In many courses, a sizable number of steps in the research process can be performed in a group. In this case, the groups are asked to submit a common report in which they also specify the specific contribution of the group members. These reports can then easily be included in the individual research reports. 

Quite independently from the amount of work done in groups, we recommend securing a common work space for the entire class both physically and virtually. Furthermore, a cloud or physical repository, where students can share their results, give each other feedback, etc. is a useful addition to any RBT course. 

The main bulk of individual supervision in an RBT course takes the form of feedback on intermediate versions of the research report. At least in some academic cultures, it is unusual for students to get a lot of feedback on the piece of work that eventually gets evaluated as the final deliverable of the course, so it is useful to be explicit on how the final grade is calculated and what contributes to it. This is also a way to ensure that the students feel free to try out approaches they are not perfectly confident about in intermediate versions of their research reports.

As previewed above, the main deliverable which also plays a central role in the assessment and evaluation is the final research report. Throughout the course, this research report grows with regular feedback provided both by the lecturer and by peers. 

In most RBT courses, there are other essential activities that can be evaluated, e.g., quizzes, group reports, intermediate reports etc. As mentioned above, evaluating intermediate versions of (parts of) the research report carries the inherent risk that the students may not be as creative as in the scenario where they just get feedback on intermediate versions. We therefore recommend including intermediate versions as necessary milestones towards the final research report without evaluating them separately (or evaluating them with a pass/fail mark in systems where this is common).

In Table 3, we give an overview of the general evaluation strategy. 

Table 3. General evaluation strategy for RBT courses

The first part of the mark reflects the lecturer’s general impression of the student’s participation in the course. Especially if there is a natural midterm evaluation point, we recommend giving intermediate feedback on this aspect to the students.

For output based on the research report, we suggest using the following evaluation form, which also includes questions to be answered in giving the specific mark.

Take into consideration each of the criteria below that apply to work produced in your course. 

👍 Make sure that at least two of the criteria predominantly involve transferable skills (e.g., the students choose and apply appropriate statistical tests, build a database, interview informants) and mark those criteria. Give those criteria a higher weight in the evaluation.

👍 Make sure to keep track of what the students learned in the course (so you don’t expect them to know things you never taught them).

Give the final mark.

Table 4. Evaluation for outputs based on the final report in an RBT course

One of the important results of many (ideally all) RBT courses are reusable research results. A way to make sure that these results get properly reused (and the student’s work gets credited) is having the students store their data in a repository as the final course activity. The lecturers who would like to know more about teaching the students how to deposit their dataset in a repository will profit from our dedicated guide on Integrating Research Infrastructures into Teaching and the accompanying UPSKILLS learning content on Moodle, Introduction to Language Data: Standards and Repositories.

Offering feedback on the final research report (in addition to the final mark) means additional work, but students tend to appreciate it. One tool that can be used for this is the evaluation form in Table 3 with answers supported by examples from the report. One advantage of using this tool over writing comments in the report is that positive aspects are elaborated on as well.

We also recommend debriefing the students after the course ends and (if applicable) informing them about the wider academic context of the course, e.g., the UPSKILLS project, further RBT courses etc. 

Once the students have received final feedback, it is useful to see how they perceived the course. We created a survey for course evaluation by students which can be used for these purposes. The survey can be further adapted and combined with assessment forms used at the institution where the course was taught.