If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Practical training on the patient is crucial in medical students’ last year education - the so-called practical year (PJ) in Germany. Due to difficulties in combining student training with the everyday tasks on ward, it is often criticised as not sufficient for a good preparation for later practical work. The Medical Faculty of the University of Jena therefore designed a project called “PJplus”. The project includes mentoring and workplace-based assessment by means of Mini-Clinical Evaluation Exercise (Mini-CEX) in combination with training workshops for supervisors. Three years after the first clinical departments started voluntary participation, the project was evaluated by comparing the experience and the self-assessed learning progress of students from departments participating in this project (PJplus group) with those non-participating (control group).
Methods
An online questionnaire was sent to all medical students registered at the University of Jena for PJ between March 2016 and April 2017. The students were invited to participate at the end of each section of their PJ within that period. The answers of the PJplus group were compared to the answers of the control group using descriptive and multivariable analysis.
Results
201 students participated in the survey and filled out 257 questionnaires. PJplus was recommended by 80% of the students participating in the project. The PJplus group (n = 92) was significantly more satisfied with their PJ and felt significantly better prepared for work than the control group (n = 165). The project's elements mentoring and feedback could lead to a better improvement in practical medical skills. However, only 17% of the students managed to conduct the required amount of three Mini-CEX during their PJ rotation and 52% of the students seemed to have lost contact to their mentor or did not have one at all. These difficulties arose due to unfamiliarity with the project, shortage of time and staff on ward or due to lack of motivation among supervisors.
Conclusion
Adding mentoring and feedback to the PJ helps to better prepare students for their practical work after finishing studies. With the project presented, it is feasible to integrate these elements in a structured way. Nevertheless, a good control of the elements’ implementation and consistent training of the supervising physicians is needed to ensure long-term success.
Zusammenfassung
Hintergrund
Die praktische Ausbildung am Patienten ist von entscheidender Bedeutung im Praktischen Jahr (PJ) des Medizinstudiums. Aufgrund von Schwierigkeiten die Ausbildung der Studierenden mit den alltäglichen Stationsaufgaben zu verbinden, wird sie oft kritisiert und als nicht ausreichend für eine adäquate ärztliche Berufsvorbereitung empfunden. An der Friedrich-Schiller Universität Jena wurde daher zur Förderung der praktischen Ausbildung der PJ-Studierenden das Projekt “PJplus” ins Leben gerufen. Es kombiniert Mentoring und Arbeitsplatz-basiertes Assessment mittels Mini-Clinical Evaluation Exercise (Mini-CEX) mit Trainingsworkshops für supervidierende Ärzte. Drei Jahre nachdem die ersten Kliniken mit der Teilnahme begonnen hatten, erfolgte eine Evaluation, indem Erfahrungen und selbst eingeschätzter Lernfortschritt Studierender von an dem Projekt teilnehmenden Kliniken (PJplus-Gruppe) mit denen nicht teilnehmender Kliniken (Kontrollgruppe) verglichen wurden.
Methoden
Zwischen März 2016 und April 2017 wurden alle an der Universität Jena für ihr PJ eingeschriebenen Studierenden zu einer Online-Umfrage eingeladen. Die Einladung erfolgte jeweils am Ende eines jeden PJ-Tertials innerhalb dieses Zeitraumes. Die Antworten der PJplus-Gruppe wurden mit den Antworten der Kontrollgruppe mittels deskriptiver und multivariabler Analyse verglichen.
Ergebnisse
201 Studierende nahmen an der Umfrage teil und füllten 257 Fragebögen aus. PJplus wurde von 80% der an dem Projekt teilnehmenden Studenten empfohlen. Die PJplus-Gruppe (n = 92) war signifikant zufriedener mit ihrem PJ und fühlte sich besser auf die praktische Arbeit vorbereitet als die Kontrollgruppe (n = 165). Die Projektelemente Mentoring und Feedback konnten eine stärkere Verbesserung der praktischen medizinischen Fähigkeiten bewirken. Allerdings erreichten nur 17% der Studierenden die geforderte Anzahl von drei Mini-CEX pro Tertial und 52% schienen während des Tertials den Kontakt zu ihrem Mentor verloren zu haben oder wurden nicht von einem Mentor betreut. Diese Schwierigkeiten ergaben sich durch fehlende Vertrautheit mit dem Projekt, durch Zeit- und Personalmangel oder durch fehlende Motivation der betreuenden Ärzte.
Schlussfolgerung
Mentoring und Feedback helfen die Studierenden besser auf ihre praktische Arbeit nach dem Studium vorzubereiten. Mit dem vorgestellten Projekt können diese Elemente strukturiert in die PJ-Ausbildung integriert werden. Für den langfristigen Erfolg ist eine gute Kontrolle der Umsetzung der Projektelemente und ein konsequentes Training der betreuenden Ärzte erforderlich.
In most countries, the last year of medical education is devoted to practical training on the patient to finally provide the students with the skills and knowledge they need to independently work as a doctor after graduation. In Germany, this period is called practical year (PJ for German ‘Praktisches Jahr’), divided into three parts à 16 weeks (‘PJ-tertial’). During the PJ the students should learn how to apply their acquired theoretical knowledge on the individual patient's case [
Approbationsordnung für Ärzte (ÄApprO); Approbationsordnung für Ärzte vom 27. Juni 2002 (BGBl. I S. 2405), die zuletzt durch Artikel 5 des Gesetzes vom 17. Juli 2017 (BGBl. I S. 2581) geändert worden ist. Bundesministerium für Gesundheit, Bonn. 2002.
] and train appropriate practical methods on the patient. They have to work practically, but they are not allowed to do any work on the patient alone to ensure patient safety. The students are supposed to perform all medical activities under continuous supervision, until the supervising doctor knows that delegation is safe [
Approbationsordnung für Ärzte (ÄApprO); Approbationsordnung für Ärzte vom 27. Juni 2002 (BGBl. I S. 2405), die zuletzt durch Artikel 5 des Gesetzes vom 17. Juli 2017 (BGBl. I S. 2581) geändert worden ist. Bundesministerium für Gesundheit, Bonn. 2002.
Medizinstudenten im PJ.2013; (https://www.spiegel.de/lebenundlernen/uni/medizinstudenten-im-pj-das-haette-auch-schiefgehen-koennen-a-914791.html. Accessed 5 May 2019)
Medizinstudenten im PJ.2013; (https://www.spiegel.de/lebenundlernen/uni/medizinstudenten-im-pj-das-haette-auch-schiefgehen-koennen-a-914791.html. Accessed 5 May 2019)
Patient safety during final-year clerkships: A qualitative study of possible error sources and of the potential of Entrustable Professional Activities.
]. To better include supervision and feedback in undergraduate medical education, the concept of workplace-based assessment (WPBA), originally introduced for postgraduate training [
]. Training in correct self-assessment is urgently needed in medicine, as studies show how poorly students and young doctors manage their self-evaluation [
]. For WPBA, different structured assessment-forms were created. One is Mini-Clinical Evaluation Exercise (Mini-CEX), in which the student is supervised for 15 to 20 minutes in a real clinical situation and receives a structured formative feedback directly afterwards [
In 2012 the project “PJplus” was initialized at Jena University Hospital initially funded by the Federal Ministry of Education and Research in Germany. It combines WPBA for improvement of practical training with individual mentoring of PJ-students to support their educational success [
]. PJplus was primarily promoted to the head physicians of the Jena University Hospital in 2013, promotion to the academic teaching hospitals followed in 2017. The project is still ongoing and participation is voluntary. Until May 2017 16 of the 27 clinical departments from Jena University Hospital (where PJ is possible) and different departments from 8 of the 23 teaching hospitals were affiliated to the project. First feedback by students and mentors was mainly positive [
]. Regarding the amendment of the German Licensing Regulations for Doctors planned by the German Ministry of Health by the year 2025, the project will become more obligated for all institutions offering PJ [
]. The amendment aims a better interlinking of theoretical and practical knowledge with orientation towards clinical competences and the introduction of obligatory formative examinations. To outline the impact of PJplus, we conducted an online survey three years after its practical initiation.
Methods
The PJplus project
PJplus contains one-to-one mentoring and workplace-based assessment (WPBA) as main components. These elements are accompanied by a brochure for PJ-students with specific organizational information about the individual department (“PJ-Wegweiser”), trainings in medical didactics for supervisors and continuous program evaluation. The Mini-CEX was chosen as assessment tool of WPBA. The projects’ objective is to include the PJ-students actively into daily hospital routine and to supervise them while doing exactly what they will be expected to do after graduation. Trainings in medical didactics are offered regularly for supervising physicians [
For the implementation of PJplus we determined the following goals: (a) department had to create the PJ-Wegweiser, (b) at least one mentor per clinic had to be trained at one of the training workshops for PJplus, (c) every PJ-student had to have an individual mentor, (d) the mentor was supposed to have at least three meetings with his mentee during the PJ-tertial, (e) the PJ-student had to have at least three Mini-CEX per tertial.
For assistance and documentation the tertial-meetings between mentor and mentee as well as the Mini-CEX are anonymously recorded on sheets (Appendix A, Additional Files 1 and 2). The Mini-CEX form was based on the rating form introduced by Norcini et al., 2003 [
], the mentoring form was our own work. Both forms include helpful information on the procedure. The students and mentors are asked to send a copy of the forms to the PJplus project, so the number of Mini-CEX and tertial-meetings carried out can be recorded. The participation in the project is voluntary.
Evaluation
During June 2016 and May 2017 a web-based anonymised questionnaire survey was conducted via the tool LimeSurvey (www.limesurvey.org, LimeSurvey GmbH, Hamburg; hosted at Jena University Hospital server) to compare students participating in PJplus (PJplus group) with those non-participating (control group). Students were classified as participating in PJplus if the department they attended was officially listed as being part of the project at the time of the survey. Invitations were sent via e-mail two weeks before the end of every PJ-tertial. One reminder was sent two weeks later. Not invited were students attending their PJ in General Medicine, at a different German University or abroad, also students who interrupted their PJ or students who did not respond two times to the invitation during the tertial before. Participation was voluntary and without reward incentives. The students received an individual link to the survey, which could only be used once. This way their participation was confirmed, but their answers were anonymised, so connections between the students and their answers could not be made. Since the invitations were sent out continuously at the end of every tertial, some students were asked to participate up to three times. The student survey was accompanied by a survey among their supervisors (results available from the authors). The Ethics Committee of the Jena University Hospital confirmed innocuousness of the study.
The questionnaire
The questionnaire in German language contained 22 to 29 questions with nominal-scaled items to assess on a 6-point Likert-scale and singular questions with free-text items (Appendix A, Additional File 3, translated by the authors). Demographical data was not raised within the questionnaire, but drawn from the profile of the responding students after they had been marked to have participated. Depending on the project-participation, the students received slightly different questions to specifically address the projects components. The questions related to the students’ general satisfaction with teaching and supervision, their feeling of preparedness for future work, the possibility to care for own patients and how well they thought they could improve in 13 practical clinical competencies. Students after their third PJ-tertial were specifically asked if they now felt able to care for a ward on their own.
For general questions, the 6-point Likert-scale ranged from 1 = very much to 6 = not at all, corresponding with the German rating system for school grades. The neutral position was excluded to avoid loss of information by central tendency bias [
].The clinical competencies were assessed at the beginning and the end of the current PJ-tertial. The levels of competence were set relating to the levels of competence used in the National Competence Based Catalogues of Learning Objectives for Undergraduate Medical Education (NKLM) in Germany, which are named as follows [
Level 3a = competent in demonstrating and performing under supervision
•
Level 3b = competent independent performance appropriate to the situation and in knowledge of all possible consequences
Between NKLM level 3a and 3b a level “insecure independent performance” was added. The students also could choose “no knowledge” or skip a competency if not being relevant for their discipline. For the chosen clinical competencies, the last two levels were considered as target of PJ. We performed a pilot study with three medical students and four colleagues from the medical and psychological department for review of comprehensibility and completeness.
Power analysis
A power analysis performed in advance showed that 64 cases per group were needed to detect a significant mean effect of d = 0,5 for a difference between two groups with a probability of 80% (PASS 2014, NCSS Software LCC, Kaysville, Utah, USA).
Data analysis
Data was analysed using IBM SPSS Statistics 25 (Armonk, NY, USA). Figures were created with Microsoft Excel 2015 for Windows (Redmond, WA, USA). For quantitative analysis, relative frequency, median and interquartile range were calculated. Since only end-points of Likert-scales were anchored, we assumed that the scales were metrically interpreted by our respondents and added mean values with standard deviation. Incomplete questionnaires were included if the student had answered more than the general questions about time and place of his PJ. To compare two groups, either Chi-quadrat-test for nominal-scaled items or Mann-Whitney-U-test for items using Likert-scales was chosen. To compare variables with more than two groups Kruskal-Wallis-test was used for first analysis and Mann-Whitney-U-test to reveal the position of significance. Competencies were analysed with Wilcoxon-test. For the comparison of competencies between groups, the individual differences of each student between the beginning and the end of PJ-tertial were calculated. Results thus obtained were compared using Mann-Whitney-U-test or Kruskal-Wallis-test. A multivariable analysis was added using univariate variance analysis (ANOVA) to sort out the influence of previous practical experience (PJ-tertial), different teaching hospitals or speciality and for assessing differences between the frequency students obtained special teaching methods (mentoring, Mini-CEX, feedback, care for own patients). P values < 0.05 were considered statistically significant.
As a relevant part of students did not correctly know if they had participated in PJplus or not (n = 27), the answers were divided into groups using the given names of the hospital or hospital department, that were known to have participated in the project or not. If students chose they did not participate, but in fact they did, they missed the project specific questions (n = 20). After the complete export from LimeSurvey, raw and evaluated data is kept anonymously on the authors’ server.
Results
Respondent profile
In six rounds, 497 invitations were sent to 372 medical students. 106 of them were invited two or three times due to performing more than one PJ-tertial in Thuringia. 201 students took part in the survey and filled out 257 questionnaires. 78.6% of the students participated only one time, 19,4% two times and 2% three times. Response rate - as ratio between all usable responses and sent invitations - was 51.7%. The responders were comparable to the eligible group of all students concerning sex, PJ hospital, speciality and project participation (Appendix A, Additional File 4). More students responded after their first PJ-tertial than after the second and third (p = 0.018). The characteristics of students’ cases in PJplus and the control group are seen in Table 1.
Table 1Characteristics of students’ cases and their participation in PJplus.
Overall most students were satisfied with their PJ and felt well prepared for work (Appendix A, Additional File 8). 49.2% of the students had individual mentoring and 73.1% received feedback. 56.8% recorded learning goals with their supervisors or mentors. 62.6% of the students were able to care for own patients. 28.8% of the students felt exploited during their tertial. Among others, mentioned for this feeling were taking blood, IV cannulation, admission of patients without feedback and changing bandages without help or instruction. At the beginning of the PJ-tertial, 39% of the students felt motivated to later work in the current speciality, at the end this increased up to 54.9%. As reasons for the change of motivation, 48.2% named the speciality of the current department and 40.9% named the experienced supervision. Concerning the competencies, the students were mostly able to improve (Table 1, Appendix A, Additional File 5). For six competencies, 80% of the students at the end of their third PJ-tertial (n = 72) achieved the targeted level of competence (taking patient history, physical examination, IV cannulation, patient file management, write medical reports, patient conversation management). Less than the half (48.2%) of these students felt competent to care for a ward in their future speciality on their own.
Impact of PJplus
Unifactorial data analysis showed a general higher satisfaction with the PJ for the PJplus group than for the control group (Figure 1). Within the PJplus group it was more likely that the students set learning goals (77.2% vs. 45.5%, p < 0.001), had an introductory meeting (79.7% vs. 44.6%, p < 0.001) and had individual mentoring (67.4% vs. 38.9%, p < 0.001). Students of the PJplus group felt less exploited for simple duties (17.8% vs. 35%, p = 0.008). 80% of the PJplus group recommend the project. In free comments the benefits of PJplus were mostly seen in the individual feedback and supervision, more structure and orientation, a higher commitment for teaching and the supervisors did know the learning goals for PJ.
Figure 1Results of students’ survey for PJplus group and control group.
There was no significant difference between the two groups in the amount of received feedback, in the experienced quality of feedback, the number of patients students took care of on their own and the feeling of being able to run a ward round independently. There was also no significant difference in the acquisition of clinical competencies between PJplus and the control group (Table 2). In multivariable analysis these findings were not confounded by the number of tertials the students had already done or the current speciality (Table 1, Appendix A, Additional File 6).
Table 2Mean change in competencies for PJplus and control group.
relative and absolute frequency of students at the end of PJ who reached the last two levels of competence
taking patient history
84
0.6 ± 0.92
100% (25)
154
0.6 ± 0.93
97.9% (47)
0.972
physical examination
83
0.6 ± 0.88
100% (25)
152
0.6 ± 0.91
97.9% (47)
0.537
IV cannulation
82
0.3 ± 0.54
100% (24)
154
0.5 ± 0.81
100% (48)
0.166
patient file management
81
1.1 ± 1.31
83.3% (20)
152
1.2 ± 1.37
83.4% (40)
0.367
establish a diagnostic plan
81
1.0 ± 1.18
72.0% (18)
154
1.1 ± 1.18
66.7% (32)
0.825
prescribe medication
81
1.1 ± 1.02
66.6% (16)
153
1.3 ± 1.18
61.7% (29)
0.096
evaluate radiological findings
80
1.1 ± 1.07
59.8% (14)
155
0.9 ± 1.08
53.9% (26)
0.109
interpret lab results
82
0.9 ± 1.06
79.2% (19)
154
1.1 ± 1.15
68.8% (33)
0.220
write medical reports
80
1.2 ± 1.45
79.2% (19)
153
1.4 ± 1.69
83.4% (40)
0.182
register patient for technical procedures
82
1.6 ± 1.74
87.5% (21)
153
1.4 ± 1.75
70.2% (33)
0.707
patient conversation management
81
0.7 ± 0.97
87.5% (21)
153
0.7 ± 1.15
79.2% (38)
0.786
wound care
79
0.8 ± 1.00
52.1% (11)
151
1.0 ± 1.49
64.3% (30)
0.609
perform a necropsy
71
0.5 ± 0.84
4.5% (1)
144
0.4 ± 0.96
16.3% (7)
0.853
Mean ± SD of the individual differences between begin and end of PJ-term for all students. An individual difference of 1 means the student improved by one level of competence. p-values calculated for mean change. p-values < 0.05 in bold.
* relative and absolute frequency of students at the end of PJ who reached the last two levels of competence
Analysis revealed that in 59% (n = 151) of cases PJplus was not fully implemented or was performed insufficiently (Table 3). A relevant number of students did not have a mentor or did not receive feedback. Only a small group of students had received the determined amount of three Mini-CEX per tertial or three individual tertial-meetings with their mentor. 55.4% did not receive any Mini-CEX at all. In free text comments, students mostly mentioned lack of staff and time as reason for the failure to implement PJplus (Supplement 7, translated by the authors).
The data was re-analysed after dividing the cases into three new groups depending on the amount of received mentoring or feedback (in form of Mini-CEX or otherwise). We performed a multivariable analysis to directly sort out findings confounded by the PJ-tertial or the current speciality. The first group of students got feedback and mentoring (n = 106), the second group feedback or mentoring (n = 98) and the control group neither feedback nor mentoring (n = 53). This analysis showed significant differences in almost all items of the survey (Figure 2). The achievement of clinical competencies was higher for students with mentoring and/or feedback than in the control group for the following: taking patient's history (p = 0.028), physical examination (p = 0.038), interpret lab results (p = 0.010) and write medical reports (p = 0.045) (Table 2, Appendix A, Additional File 5). Analysing the amount of feedback in four groups (both mentoring and feedback, only feedback, only mentoring and control) was similar to the analysis in three groups.
Figure 2Results of students’ survey depending on the amount of mentoring and feedback.
Mean ± SD of Likert-scaled items, range from 1 = very much approval to 6 = not at all. Control group did neither have mentoring nor feedback. p< 0.001 for all items.
Among the students who have had a mentor (n = 127) analysis revealed further, that as soon as they have had three tertial-meetings, the students were more satisfied with their PJ (for no meetings vs. three: p = 0.043, no meetings vs. more than three: p = 0.004). They also felt better prepared for work (for no meetings vs. three: p = 0.008).
Among the students who received feedback (n = 185) we could not prove a better satisfaction or performance of students who have had Mini-CEX (n = 43). Nevertheless, students with at least three or more Mini-CEX (n = 16) compared to students with less than three Mini-CEX (n = 27) felt their feedback was more helpful (p = 0.023).
Impact of caring for one's own patients
For a third consideration students’ answers were divided into two groups depending on whether or not they had been able to care for patients on their own. Results of this multivariable analysis showed that students who could care for own patients were consequently more satisfied with their PJ (Figure 3). They also felt less exploited for simple duties (22.5% vs. 40%, p = 0.036) and reported a significant increase in motivation to later work in the current speciality (p = 0.039). A significant change in the levels of competence showed up for the following (Table 3, Appendix A, Additional File 5): prescribe medication (p = 0.031), interpret lab results (p = 0,027), write medical reports (p = 0.006), register patient for technical procedures (p = 0.039) and perform a necropsy (p = 0,020).
Figure 3Results of students who could care for own patients compared with those who could not.
In further multivariable analysis the factors amount of teaching (i.e. mentoring and feedback) and the possibility to care for one's own patients were correlated with each other. Although students receiving mentoring and feedback were significantly more often able to care for their own patients (p < 0.001), the two factors turned out to be mainly independent main effects (Table 2, Appendix A, Additional File 6).
Students at the end of their studies did not feel competent to manage a ward
For all three considerations we could not find any significant differences for the question whether or not students at the end of PJ (n = 72) did now feel competent to care for a ward in their future speciality on their own. In all cases, more than 50% of the students stated they did not feel ready for this. Most mentioned as reasons were missing feedback, lack of training or care of own patients (Appendix A, Additional File 7).Students who were able to care for 1-3 patients during the past PJ-tertial felt the most insecure (75%; p = 0.064). Least insecure felt students who could care for 4 or more patients (37%); students who could not care for any patients were in the middle (57.6%).
Discussion
Impact of PJplus and its elements
This survey showed that students who were cared for in departments participating in PJplus were more satisfied with their PJ and felt better prepared for practical work after finishing studies. It showed a high acceptance of the project among the students and their supervisors. The advantages of the project were seen in more distinct feedback, better support in practical activities and in better basic teaching conditions (PJ-students were expected and welcomed, goals were discussed, problems addressed). However, the survey revealed a highly inadequate implementation of the project's key elements: 15% of the students officially participating in PJplus received neither mentoring nor feedback, another third received only one of both. So overall, the PJplus group had not been able to improve more in their practical skills than the control group. On the other hand, re-evaluation with shifted focus on the actual implementation of mentoring and feedback showed that students who received mentoring or feedback were indeed more able to improve in some competencies than students without mentoring and feedback. They also felt more competent in conducting a ward round. We conclude, if mentoring and feedback are carried out as intended, PJplus can have a great and measurable impact on the success of teaching during PJ. Additionally, this study showed that the possibility to care for one's own patients should be strongly promoted independently, as students with the possibility to care for own patients felt significantly better prepared for practical work than the control group..
Although PJplus includes feedback training, the study did not show that students with PJplus felt their feedback was more helpful than in the control group. We do not think that feedback training is therefore of no use. We rather suspect that the students lacked the comparison between trained and untrained supervisors and that they basically appreciated any form of feedback compared to no feedback.
Reflections about the poor implementation of PJplus-elements
It is most important to discuss why the implementation of PJplus did not proceed as intended. Some Mini-CEX and tertial-meetings may have been just missed in this survey, because it was conducted two weeks before the end of the PJ-tertial. Still the results were far below our requirements. Since Mentoring and WPBA were so far unknown to our faculty, the benefits might not yet have been commensurate to the expenses [
]. So we assume, that the project might still need more time to be fully acknowledged. Concerning the implementation of the elements during daily hospital routine, many students reported missing time as the major problem. This is a known problem during the implementation process of WPBA [
]. It can lead to role conflicts between caring for patients and teaching and lower the acceptance of new methods. As suggested by Lörwald et al., fostering the learning culture within the faculty by introducing protected time for teaching and feedback may help [
]. Also, financial resources must be made available to ensure time for training. If enough staff is available, a low personal motivation might be another hindering factor for the implementation of WPBA [
]. Conversations with the participants of our workshops had shown that some were engaged by their head of department to participate. Therefore some trainers may have been motivated only extrinsically. More information and training is needed to highlight the benefits of PJplus and WPBA and to motivate for teaching. To promote the teaching skills of active mentors the University of Saarland successfully initiated peer review for PJ-teaching [
]. For further support, follow-up workshops may be another idea. We also discussed a better quality management with specific consequences if the claimed quality standard could not be maintained within the department participating in PJplus. This concerns, for example, if the number of incoming Mini-CEX and mentoring sheets was counted too low. Also checkpoints, whether the department is still employing enough trained supervisors and has not lost them due to rotation and staff changes, could help. If necessary, it should be possible to withdraw the “PJplus-label”. This might be needed to keep quality at a high level and to maintain students’ approval of the project. Additionally, student feedback must be integrated more strongly. A supervisor-related online evaluation, for example, can provide more distinct feedback [
Current state of practical medical education in general
Independently of PJplus or any teaching method we like to call attention to the fact that more than half of our students at the end of PJ did not feel competent to manage a ward on their own (52,8%). Furthermore, we found that only 38% of the students felt competent in preparing medications; which maybe mainly a nursing task, but also a doctor should be able to prepare infusions and syringes at least in emergency cases. Lastly, but most concerning, our study showed that more than a third of the participating students did not care for any patient during their PJ-tertial at all. In this context, we have to be aware that training conditions do not improve after graduation[
Die große HB-Assistenzarztumfrage 2018/19: Zwischen Arbeitszeit, Fehlermanagement und Digitalisierung: Wie gut können Sie gute Ärzte werden?, Berlin. 2019. https://www.hartmannbund.de/berufspolitik/informationen/umfragen/. Accessed 13 Jun 2019.
]. Most recent, the Hartmannbund, a professional association of physicians in Germany, revealed that many junior doctors (1-4 years after graduation) feel overwhelmed with work overload and confronted too early with unprepared situations; 20% even daily [
Die große HB-Assistenzarztumfrage 2018/19: Zwischen Arbeitszeit, Fehlermanagement und Digitalisierung: Wie gut können Sie gute Ärzte werden?, Berlin. 2019. https://www.hartmannbund.de/berufspolitik/informationen/umfragen/. Accessed 13 Jun 2019.
Die große HB-Assistenzarztumfrage 2018/19: Zwischen Arbeitszeit, Fehlermanagement und Digitalisierung: Wie gut können Sie gute Ärzte werden?, Berlin. 2019. https://www.hartmannbund.de/berufspolitik/informationen/umfragen/. Accessed 13 Jun 2019.
]. In US, Raymond et al. reported that a significant number of new residents performed critical activities (e.g. tell a patient bad news, perform thoracentesis) without supervision [
]. These findings again raise the question about the PJ's objective. We may ask, how medical students without any supervised work on the patient can be adequately prepared for their profession. There still is a need for more supervised hands-on experience in medical education [
]: It is the tendency of less competent people to overestimate their own abilities because they do not know what challenges them. This may lead to the bias that good performing students may assess their skills lower than poor performing students. Also, students who have made the experience of being in charge on their own may evaluate themselves more critically. One hint to this problem may be the result that students who could care for a few patients, felt the most insecure to manage a ward. As many PJ-students overestimate their abilities compared to the assessment by seniors [
]. In order to gain a more reliable impression of all students, in further studies objective assessments should be added to evaluate the impact of PJplus (e.g. Objective structured clinical examination = OSCE).
When designed, this survey was intended for internal evaluation only. For thus we did not assess reliability or validity of the questionnaire. Knowing that there is no standard in Germany for PJ evaluation [
Approbationsordnung für Ärzte (ÄApprO); Approbationsordnung für Ärzte vom 27. Juni 2002 (BGBl. I S. 2405), die zuletzt durch Artikel 5 des Gesetzes vom 17. Juli 2017 (BGBl. I S. 2581) geändert worden ist. Bundesministerium für Gesundheit, Bonn. 2002.
], we developed our own questionnaire. We also did not assess all aspects of PJ, but focused on PJplus specifically, because this survey was on top of the regular PJ-evaluation in Jena. These aspects lower the comparability with surveys from other universities. In 2018 the University of Mannheim published a PJ questionnaire with recommendations for use in all over Germany [
Erfassung der Zufriedenheit mit dem Einsatz im Praktischen Jahr (Ma-FEZ-PJ) zur Qualitätssicherung der Lehre im Praktischen Jahr – Psychometrie eines Selbsteinschätzungsfragebogens.
]. It may be helpful to refer to this in later surveys. Since PJplus was not yet established in the teaching hospitals while the survey was conducted, the sizes of groups to compare sometimes differed considerably. Mostly, the required number of cases for the demanded power was obtained. However, in some cases it was expressively undercut. This concerns, for example, the question about the influence of Mini-CEX on the success of PJ. Especially our sample group of students with regular, meaning three or more Mini-CEX, was very small (n = 16). We do not think that Mini-CEX does not have an effect in general, but our study suggests that even informal feedback is better than no feedback. Mini-CEX can help feedback to be more structured and more reliable [
]. Our survey contained some questions in which the students were asked to assess their abilities retrospectively. This may have caused a relevant recall bias, especially when linked to the Dunning-Kruger-Effect mentioned above. We aimed to minimize this effect by conducting the survey while the students were still experiencing the PJ. A two step survey could have reduced the willingness to participate noticeably. Further, we have to consider the personal motivation of the students not as a consequence but as a prerequisite. Our results showed that students who were able to care for their own patients started significantly more motivated into the PJ-tertial. They might have been able to care for more patients just because they demanded it. Similarly, we see a selection bias in the voluntary nature of PJplus participation. An already high level of motivation for teaching certainly increased the willingness to participate. Arguably, some departments participating in PJplus were already receiving good evaluations from students before the introduction of the project. However, our regular final PJ-evaluations showed a trend that some hospitals have improved after the introduction of PJplus [
]. Lastly, as with other studies, participation was voluntary and those who responded to the survey may be those with particularly strong views or particularly good or bad experiences.
The strength of our study lies in the direct comparison of students participating in the project with students non-participating. To our knowledge, this is the first PJ-project with WPBA which is presented and evaluated this way in Germany. The survey was conducted over the period of one year to include several cohorts of PJ-students.
Conclusion
Our study showed that improvement of practical medical education is achievable. Students need supervision and structured feedback e.g. WPBA and they need to care for own patients. The presented project is a feasible proposal for adding these elements to the PJ in a structured way. However, the path to full implementation is difficult and needs time to be acknowledged and resources for training and motivation of staff. It also needs financial resources because in our current clinical routine it is nearly impossible to allocate time for teaching without more staff. Nevertheless, a better quality of teaching is urgently needed. It has to be pursued and - most important - passed on to the attention of trainers on ward. It is our duty - for the patients and our young colleagues - to offer the best practical education. Todays students may be our doctors tomorrow.
Funding
PJplus was part of the project “ProQualität Lehre” at the Friedrich Schiller University of Jena, which was funded from 2012 - 2016 by the German Ministry of Education and Research (BMBF), grant number 01PL12071. Funders had no role in study design, data collection and analysis or presentation.
Ethics approval and consent to participate
The Ethics Committee of the Jena University Hospital confirmed innocuousness of the study with consented, anonymized and voluntary participation (reference number 479-05/16).
Acknowledgement
The authors would like to thank all participating students and supervising doctors for their feedback. They like to thank Dr. Thomas Lehmann form the statistical institute of Jena University Hospital for assistance and data review. The also like to thank Dominique Ouart and Isabell Woest as former project managers of PJplus and Marjan Govaerts and Bas Verhoeven from the Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences at Maastricht University who have substantially contributed to the development of the project.
Conflict of interest
The authors have no conflict of interests to declare.
Approbationsordnung für Ärzte (ÄApprO); Approbationsordnung für Ärzte vom 27. Juni 2002 (BGBl. I S. 2405), die zuletzt durch Artikel 5 des Gesetzes vom 17. Juli 2017 (BGBl. I S. 2581) geändert worden ist. Bundesministerium für Gesundheit, Bonn. 2002.
Medizinstudenten im PJ.2013; (https://www.spiegel.de/lebenundlernen/uni/medizinstudenten-im-pj-das-haette-auch-schiefgehen-koennen-a-914791.html. Accessed 5 May 2019)
Patient safety during final-year clerkships: A qualitative study of possible error sources and of the potential of Entrustable Professional Activities.
Die große HB-Assistenzarztumfrage 2018/19: Zwischen Arbeitszeit, Fehlermanagement und Digitalisierung: Wie gut können Sie gute Ärzte werden?, Berlin. 2019. https://www.hartmannbund.de/berufspolitik/informationen/umfragen/. Accessed 13 Jun 2019.
Erfassung der Zufriedenheit mit dem Einsatz im Praktischen Jahr (Ma-FEZ-PJ) zur Qualitätssicherung der Lehre im Praktischen Jahr – Psychometrie eines Selbsteinschätzungsfragebogens.
00109-4&id=gr1.jpg){kind=link}
00109-4&id=gr2.jpg){kind=link}
00109-4&id=gr3.jpg){kind=link}