Formative e-assessment improves performance in dental education

DOI: 10.53180/dzz-int.2022.0019

PDF

, ,

Keywords: dentistry e-seminars evaluation formative e-assessment gender

Abstract

Aim of the study: The purpose of this study was to investigate whether the integration of web-based learning in the form of e-seminars in dental education leads to improved student performance.

Methods: In the second semester of the clinical component of dental school, conventional seminars requiring attendance were replaced by e-seminars as part of the dental conservation course. In the control cohort, the theoretical teaching content was taught in the form of lectures as part of the conventional seminar; in the test cohort, this was done via guided self-study and e-seminars. The use of the e-seminars was evaluated with the help of a self-­designed questionnaire. To investigate the learning effect, the results of the final exam of two consecutive years were compared.

Results: Student acceptance of the e-seminars was positive. Students who attended the e-seminars achieved significantly better results on the final exam than those who attended conventional seminars.

Conclusion: E-seminars are an innovative and timely addition to traditional classroom instruction. They can help improve performance while making time management more flexible. They can lead to the relief of teachers and learners and simultaneously to the increase of the education quality.


Department of Conservative Dentistry, Periodontology and Preventive Dentistry, Hannover Medical School, Carl-Neuberg-Str. 1, 30625 Hannover, Germany: Veronika Rogos, Alexander Rahman, Silke Jacker-Guhr

Citation: Rogos V, Rahman A, Jacker-Guhr S: Formative e-assessment improves performance in dental education. Dtsch Zahnärztl Z Int 2022; 4: 158–163

Peer reviewed article: submitted: 03.12.2021, revised version accepted: 21.04.2022

DOI.org/10.53180/dzz-int.2022.0019

Introduction

The traditional study of dentistry essentially consists of elements of face-to-face teaching: lectures, seminars, exercises, practical courses and tutorials [4]. These are characterized by the local and timely constraints as well as by the physical presence of the learners and the teaching staff [3]. In this context, the transfer of knowledge is mainly controlled by the teachers. Dental students at Han­nover Medical School (MHH), Germany, participate in more than 40 hours of lectures, seminars and practicals per week during the clinical section. In doing so, students report a constant lack of time for pre- and post-processing of the content. During lectures in particular, the amount of content, limited interaction between teachers and learners, lack of feedback from fellow students, and the resulting pas­sivity of learners are major drawbacks [9, 15, 17]. In order to promote learning, new teaching methods therefore rely on greater interactivity between instructors and learners as well as learners among themselves, replacing passive participation in courses [8]. In particular, supplementing face-to-face teaching with electronically supported learning opportunities enables flexibility in terms of time and space in teaching [1, 24] and can effectively support face-to-face teaching.

With this in mind, electronic seminars (e-seminars) have been integrated into the clinical section of dental studies since 2010 as a form of formative e-assessment. The e-seminars are realized with the Learning Management System (LMS) ILIAS (Integrated Learning, Information and Work Cooperation System, ILIAS open source e-Learning e.V., Cologne, Germany), which is the central learning platform at the MHH. Being able to regularly check one’s own learning progress should enable students to self-reflect on their own level of knowledge, thus helping them to identify and effectively address learning deficits and consequently improve their knowledge. This offering of e-seminars is still in the development phase and has not yet been tested for effectiveness.

The use of e-seminars in the first semester of the clinical section of dental school was therefore tested by evaluating students. The aim of the study was first to determine the acceptance of this alternative teaching method. Furthermore, one objective was to analyze the effectiveness of the e-seminars. The null hypothesis states that this teaching format does not show a performance-enhancing effect in the form of a better result in the final exam. Additionally, gender differences regarding study load were investigated.

Methods

Samples

The study included two cohorts. They were students in the Dental Conservation Science I course from two consecutive years. The cohort that participated in the e-seminars included 54 students out of a total of 55. The study was conducted during the second clinical semester (7th semester) of dental school. The sample consisted of 37 female (68.5%) and 17 male (31.5%) students. Data from the second clinical semester of the previous cohort were used for the comparison sample of the traditional teaching format cohort. This sample consisted of 76 data sets.

The study project was approved by the Ethics Committee (Study No.:1780–2013) of Hannover Medical School.

Procedure

In the second clinical semester (7th semester) of the dental program, a mandatory seminar course with a pronounced lecture character was converted into an e-seminar. The introductory lecture on the e-seminar procedure was held in person for all students. In the following eight weeks, the students worked independently or in groups on patient cases from the fields of endodontology, periodontology and restorative dentistry. In addition to the clinical content, students also worked on topics such as the Medical Devices Act, hygiene in the workplace, dental instrument preparation and the handling of hazardous substances. The students had one hour per week to work on the content, which was also specified in the timetable. The scope and content thus corresponded to the courses of the previous years.

Each week, the students received a scientific article on one of the topics for independent work and as preparation for the respective seminar lesson. This was made available to the students via the learning platform of the ILIAS learning management system. In each e-seminar, the students completed a multiple-choice (MC) test online with questions on the article provided. This test consisted of five single-choice questions (single selection from five offered answers) and was activated in the learning management system ILIAS, limited in time to this hour. The test could be started at any time within the seminar hour and could be worked on for a maximum of 15 minutes. While working on the test, the students were allowed to discuss the questions. The time allocation was up to the students and was intended to be used to deal with the contents, the questions and problems of the topic.

 

Subject area/question

Evaluation options

N (absolute)

N (relative in %)

M ± SD

1. Gender

Female

Male

 

37

17

68.5

31.5

 

2. Study workload

How would you categorize your stress level as a dental student?

high level

moderate level

low level

49

4

1

90.7

7.4

1.9

 

 

How many hours per week do you attend formal courses, seminars or lectures?

0–10h

10–20h

20–30h

30–40h

>40h

7

0

7

27

13

13

0

13

50

24

 

3. E-seminars

Knowledge control in the form of e-seminars is innovative.

1–4

  

3.28 ±

0.76

 

I find the learning effect of e-tests positive and useful.

1–4

  

3.06 ±

0.86

 

I was able to check my learning progress better in this course.

1–4

  

2.93 ±

0.82

4. What materials did you use to answer the questions?

Professional discussion with fellow students

1–4

  

3.46 ± 0.79

 

Additional professional literature

1–4

  

2.76 ±

0.87

 

ILIAS (literature, pdf files)

1–4

  

3.28 ±

0.81

5. Was it helpful to work with fellow students to solve the questions?

I do not find it useful to discuss the questions with fellow students.

1–4

  

2.89 ±

1.3

 

I solved the questions myself.

1–4

  

2.93 ±

0.75

 

I have learned by discussing the questions.

1–4

  

3.37 ±

0.73

6. Alternative form of learning compared to a required course

For me, the alternative learning option with the test questions led to more time freedom.

1–4

  

3.63 ±

0.59

 

The targeted learning based on the test questions was more effective than a mere lecture.

1–4

  

2.74 ±

0.76

 

It would have been better if I could have answered the questions at any time.

1–4

  

3.09 ±

0.88

Abbreviations: N = sample size, M = mean value, SD = Standard deviation

Table 1 Evaluation questions

Participation in the tests was mandatory for the students and was required for admission to the electronic written final exam of the course. The tests were conducted in the e-learning room of the Center for Oral and Maxillofacial Surgery of the MHH for students who were present in the clinic at the time. Any remaining time of the seminar hour was at the free disposal of the participants. They could use this for organi­zational and administrative activities such as dealing with the patient history, documentation, contacting patients or laboratory activities. It was also possible to perform the test on the road or at home, as there was no compulsory attendance for this seminar hour. Only the time slot was predetermined due to the class schedule. Within the semester, a total of 8 e-seminars with the corresponding short tests of this type took place. At the end of the course, the e-seminars were evaluated with the help of a self-developed, paper-based questionnaire. Statistical validation was not performed. The questionnaire consisted of 29 questions on 10 aspects of investigation (Table 1). The investigated aspects included questions about the general stress situation in the study, the learning effect through the use of the e-seminars, the preparation methods for the e-seminar, the use of communication possibilities in the context of the e-seminar, the acceptance of the alternative learning form, the independence of the task solution and the use of the provided free time. These were essentially closed questions with rating scales (frequency, intensity and evaluation scales) and questions on personal attitudes (Table 1). The evaluation was based on a 4-point Likert scale with the response categories 4 completely agree, 3 rather agree, 2 rather disagree, 1 not at all agree. Furthermore, students were able to give an overall evaluation of the e-seminars using the school grading system and free text comments. In order to assess the effectiveness of the e-seminars, the data of the final exams of the respective years were compared. A maximum of 20 points could be achieved in the written exam. To pass the written exam, at least 60% of the questions had to be answered correctly.

Statistical analysis

Statistical analysis was performed using SPSS Statistics software (SPSS 26.0 IBM, Armonk, NY, USA). The Kolmogorov-Smirnov test showed that there was no normal distribution of the data. To compare the results between groups, the Mann-Whitney U test was applied. The significance level was set at p < 0.05. Cohen’s d was calculated to represent the effect size. In a further step, an analysis on gender differences was performed. For this purpose, cross-tabulations were made for the relationship between gender and the responses in the questionnaire and possible dependencies were tested using Pearson’s chi-square test.

Results

Analysis of the evaluation of the e-seminars

The workload in the semester of dental school under study was considered high by 49 individuals (90.7%), medium by 4 individuals (7.4%), and low by one individual (1.9%). The majority of the time invested was 30 to 40 hours per week. 76.1% of the students stated that they had gained more personal time through the e-seminars. Regarding the form of processing, the distribution between group work and individual work was about equal (Table 2).

The evaluation for the e-seminars was assessed in the majority (56%) with the school grade of 10–12 points.

Analysis of the exam results

The exam scores of the cohort of students who participated in the e-seminars had a mean (M) of 17.24 points (86.18% ± 1.05). The comparison group cohort scored a mean of 15.25 points (76.25% ± 1.76). This means that the test cohort achieved significantly better exam results than the comparison cohort (p < 0.001) (Table 2). The effect size in this case is high ([d] = 0.963) and therefore the effect is large. Thus, the null hypothesis can be rejected.

N

M

SD

95% CI

Range

Point values

Cohort e-seminars

Cohort comparison

55

76

17.24

15.25

1.05

1.76

[16,95; 17,52]

[14,85; 15,65]

[15; 20]

[11; 19]

Point values

Cohort e-seminars

Cohort comparison

55

76

86.18

76.25

5.27

8.8

[84.76; 87.61]

[74.24; 78.26]

[75; 100]

[55; 95]

Abbreviations: N = sample size, M = mean value, SD = Standard deviation

Table 2 Examination results of both years

Gender-specific differences

To test for a gender difference in stress perception, a chi-square test was performed. This showed no significant differences (χ2 = 4.05; p = 0.132). In terms of hours invested in the course, there was a trend for gender to matter (χ2 = 6.81; p = 0.078). 25% of male respondents were less likely than their female peers (75%, of female respondents) to participate in 40 or more hours of instruction per week, according to the study.

Discussion

In the present study, the integration of a web-based learning tool in the form of e-seminars into the clinical study section of dental training was investigated. In addition to evaluating the acceptance of the e-seminars, summative performance assessments were also conducted and gender differences were taken into account.

The form of assessment used here is one of the formative assessments. Formative assessments serve to support learners in the learning process, to confront them with their current level of competence, and thus to motivate them to eliminate any learning deficits that may exist. According to Miller [1990], MC questions are suitable for this purpose, as they not only ask for specific technical knowledge, but also test higher levels of competence [18, 22]. The combination of self-study and the previously provided content and the subsequent query of the learning status by the web-based tests, which could be worked on alone or collaboratively by the students, had a positive effect on the results of the final exam. Thus, the formative assessment not only resulted in an improvement of the previous teaching structure in this semester, but also had a positive effect on the summative assessment in the form of the final exam.

The e-seminars were positively received by the students overall and showed a positive influence on learning success. This is not only illustrated by the direct effectiveness of the seminars in terms of end-of-semester performance, but also by the broader benefits. For instance, the opportunity to independently review learning progress in order to identify one’s own gaps in knowledge, and the indirectly positively influenced variables such as increased time efficiency, personal responsibility, and self-discipline point to the consistently positive effects. A superiority of e-seminars compared to a pure face-to-face teaching with lecture character can thus be proven with clear results, with which this study joins the large number of research results on the favorable effects of e-learning [2, 9, 20].

Nearly all students surveyed reported high levels of stress – a finding that has emerged repeatedly from other research [3, 23]. Studies show that female students are more stressed than male students when new courses are introduced [14, 21]. The present study could not confirm this correlation. In terms of content, the primary trigger for this was cited as the lack of time in the daily study routine as a result of the very tightly scheduled and long day during lecture periods and the resulting impairment of performance, which can come to a head, especially during the phase of entering the clinic [8]. Exactly which factors were responsible for the high score in terms of stress perception in the present study cannot be answered here, as the students were already in their 7th se­mester, and requires further detailed studies. In the survey conducted, the creation of free time, which the students themselves could decide how to use, was rated positively. It can therefore not be ruled out that the lack of time in the very tightly structured dental studies is also a significant factor for stress in this respect, as the free spaces created have led to a relief for many. Further advantages of the e-seminars, which additionally counteract this stress factor, are the regulated range of knowledge and the possibility of targeted and time-saving learning [2].

Nevertheless, the evaluation found that face-to-face teaching is rated as extremely important by students, as it provides them with structured knowledge transfer as well as personal contact with lecturers. A combination of e-seminars and face-to-face teaching is a promising option for dental education. But the sole sta­tistical comparison of the exam results of the two groups of the traditional face-to-face method or the online equivalent, as conducted here, does not yet support this recommendation with the necessary evidence. However, indications arise from the subjective statements of the students that the fixed face-to-face appointments to discuss the learning material had a positive benefit. Evaluation studies previously conducted showed that the combination of both forms of teaching can lead to valuable synergies [1, 8]. For this purpose, the exact composition of different teaching methods is investigated under the didactic concept of “blended learning”.

Today, gender differences are also an indispensable aspect of research in social and cultural sciences. The discipline of dentistry is now predominantly chosen by women [25]. In order to draw correctly differentiated conclusions, differentiation of outcomes by gender has become indispensable. In the study by Arrenberg and Kowalski, female students were found to be more active in using e-learning services than their male peers [1]. The present study confirms this outcomes. The female students rated the e-seminars as more attrac­tive and also performed better in the final exam than their male fellow students. They worked more independently and also spent more time attending classes.

Currently, new learning concepts are to be developed by integrating new projects into dental studies. On one hand they improve teaching, and on the other hand they also relieve the teaching staff and thus create optimized conditions for effective studies [6, 7].

A central aspect that is significant from an organizational point of view is the increase in the efficiency of e-assessment use through an effort-saving application of infrastructures for the acceptance of e-assessments [16]. Recent solutions advocate mobile devices that can be used in an organized manner. In addition, the development effort is very high and the presentation of learning content is often determined by technical rather than didactic factors. In view of the current state of science, further research is needed on didactic, methodological, organizational, and technical aspects of e-assessment to promote the development of appropriate systems [11–13, 19].

Nevertheless, e-assessments are a good supplement to conventional teaching methods, and a development towards digital teaching formats can clearly be seen. It could be observed that more formative e-assessment forms, as opposed to diagnostic examinations, are used in universities and colleges than in the past [5]. At universities, the proportion of formative assessments increased, e.g., “interim tests during self-learning,” “quizzes for motivation or review,” “follow-up and e-exercises,” and “audience response in the lecture hall” [5]. Legal issues also need to be considered [5]. A legal opinion on “Legal aspects of e-assessments at universities” can be used as a guide for this [10].

Conclusions

Formative e-assessments in the context of e-seminars can be well integrated into dental education as an innovative teaching method. The present study shows that e-seminars have a positive effect on the performance achieved by students in dental education and thus hold great potential for the sustainability of teaching. Acceptance on the part of students is the prerequisite for teaching and learning success.

Further scientific studies on the topic of e-assessment can help to further increase the acceptance and use of e-assessments. In addition, the new licensing regulations for dentists offer many opportunities to integrate online-based teaching right from the start. The importance of this is particularly evident in the COVID-19 pandemic, in which face-to-face teaching had to be largely replaced by online teaching formats.

Conflict of interest

The authors declare that there is no conflict of interest as defined by the guidelines of the International Committee of Medical Journal Editors.

References

  1. Arrenberg J, Kowalski S: Lernen Frauen und Männer unterschiedlich? Eine Studie über das Lernverhalten von Studierenden. Fakultät für Wissenschaften der Fachhochschule Köln, Arbeitsbericht 2007; http: //www.gesundheitsfoerdernde-hochschulen.de/Inhalte/05_Materialien_Links/Literatur/10775–2007.pdf. (access: 18.11.2018).
  2. Azizi SM, Farajollahi MF, Seraji F, Sarmadi MR: Synthesis research on the effectiveness of e-learning in medical sciences education and its design and implementation requirements. Iran J Med Edu 2017; 17: 270–287.
  3. Basudan S, Binanzan N, Alhassan A: Depression, anxiety and stress in dental students. Int J Med Edu 2017; 8: 179–185.
  4. Beck CH: Geschichte der Universität in Europa. C.H. Beck Verlag, München 2010.
  5. Berkemeier M, Bilo A, Fischer Y et al.: E-Assessment in der Hochschulpraxis: Empfehlungen zur Verankerung von E-Assessments in NRW. Zentrale Serviceeinrichtungen, Zentrum für Informations- und Mediendienste (ZIM) 2017.
  6. Crome M, Adam K, Flohr M, Rahman A, Staufenbiel I. Application ofthe inverted classroom model in the teaching module “new classificationof periodontal and peri-implant diseases and conditions” during theCOVID-19 pandemic. GMS J Med Educ. 2021;38:Doc89.
  7. Crome M, Rahman A, Iversen RM, Lührs AK: Synchronous vs. asynchronous education: Questionnaire-based survey in dental medicine during the COVID-19 pandemic. Dtsch Zahnärztl Z Int 2021; 3: 207–215.
  8. Dick M, Riesen Olten C: Klinikeinstiege in der Zahnmedizinischen Ausbildung: spezifische Belastungen und Lösungsansätze. Schlussbericht für die Stiftung Suzanne und Hans Biäsch. Institut für Kooperationsforschung und -entwicklung IfK, 2011. https: //www.psychologie.uzh.ch/dam/jcr: 00000000-0be1-​a873-ffff-ffffd065cd21/Bericht_​Dick_​2010_03.pdf (access: 19.06.2018).
  9. Ehlers JP, Möbs D, v.d. Esche J, Blume K, Bollwein H, Halle M: Use of formative e-assessment in lectures. GMS Z Med Ausbild 2010; 27: Doc59.
  10. Forgó N, Graupe S, Pfeiffenbring J: Rechtliche Aspekte von E-Assessments an Hochschulen. Gutachten im Auftrag des Verbundprojektes E-Assessment NRW. 2016 https: //doi.org/10.17185/duepublico/42871.
  11. Irlbeck M: Digitalisierung und Energie 4.0 – Wie schaffen wir die digitale Energiewende?. In: Doleski O (Hrsg.): Herausforderung Utility 4.0. Springer Vieweg, Wiesbaden 2017, 135–148.
  12. Jeremias XV, Sellmer B: Chancen und Herausforderungen von E-Assessments im Anwendungskontext. In: Barton T, Müller C, Seel C (Hrsg.): Hochschulen in Zeiten der Digitalisierung. Springer Verlag, Wiesbaden, 2019, 123–137.
  13. Jeremias XV, Rabe C, Sellmer B: Drei Aspekte der Qualitätssicherung bei E-Assessments. DeLFI 2016 – Die 14. E-Learning Fachtagung Informatik, 2016.
  14. Li J, Shang L, Wang T, Siegrist J: Measuring effort-reward imbalance in school settings: a novel approach and its association with selfrated health. Journal of Epidemiology 2010; 20 (2): 111–118.
  15. Lochner L, Wieser H, Waldboth S, Mischo-Kelling M: Combining traditional anatomy lectures with e-learning activities: how do students perceive their learning experience? Int J Med Edu 2016; 7: 69–77.
  16. Malzahn B, Groß T, Claßen I: Eine mobile Infrastruktur zur Durchführung von E-Prüfungen. In: Barton T, Müller C, Seel C (Hrsg.): Hochschulen in Zeiten der Digitalisierung. Springer Verlag, Wiesbaden, 2019, 107–122.
  17. Meguid EA, Collins M: Students’ perceptions of lecturing approaches: traditional versus interactive teaching. Adv Med Edu Pract 2016; 8: 229–239.
  18. Miller GE: The assessment of clinical skills/competent/performance. Acad Med 1990; 65: 63–67.
  19. Moskaliuk J, Thillosen A, Hesse, FW, Cress U: Erfolgsfaktoren für den Einsatz digitaler Medien in der Hochschullehre. Digitalisierung der Hochschulen: Forschung, Lehre, Administration, 2017; 22–40.
  20. Regmi K, Jones L: A systematic review of the factors – enablers and barriers – affecting e-learning in health sciences education. BMC Med Edu 2020; 20: 1–18.
  21. Schmitter M, Liedl M, Beck J, Rammelberg P: Chronic stress in medical and dental education. Medical teacher 2008; 30(1): 97–99.
  22. Shumway JM, Harden RM: AMEE Guide No. 25: the assessment of learning outcomes for the competent and reflective physician. Medical Teacher 2003; 25(6): 569–584.
  23. Singh P, Aulak DS, Mangat SS, Aulak MS: Systematic review: factors contributing to burnout in dentistry.Occup Med 2016; 66: 27–31.
  24. Tolks D: eLearning in der medizinischen Aus-, Weiter- und Fortbildung. In: Fischer F, Krämer A (Hrsg.): eHealth in Deutschland. Springer Verlag, Berlin, Heidelberg, 2016, 223–239.
  25. Wilcke AJ, Buchmann M, Reißmann DR, Pohontsch N, Aarabi G, Heydecke G: “Feminisierung” der Zahnmedizin: Herausforderungen an einen Wandel des Berufsbildes. Dtsch Zahnärztl Z 2016; 71: 217–224.

Corresponding Author:

Dr. Silke Jacker-Guhr

Department of Conservative Dentistry, Periodontology and Preventive Dentistry,

Hannover Medical School, Carl-Neuberg-Str. 1, 30625 Hannover, Germany

Jacker-Guhr.Silke@mh-hannover.de

Photo: MHH


related files

PDF

(State: 07.09.2022)

Latest Issue 5/2022

In Focus

  • The Journal Impact Factor 2021
  • Knowledge and skills of paramedics in handling patients with traumatic dental injuries
  • Formative e-assessment improves performance in dental education