Confidence in essential procedural skills of Thai medical graduates

Corresponding Author:

Chitkasaem Suwanrath
Department of Obstetrics and Gynecology, Faculty of Medicine; Prince of Songkla University; Hat Yai, Songkhla 90110; Thailand
Tel: +66 74 429 617
Fax: +66 74 429617
E-mail: schitkas@yahoo.co.uk; schitkas@medicine.psu.ac.th

Abstract

Background: This study is aimed to survey the confidence levels as well as clinical experience of Thai medical graduates in performing essential procedures.

Methods and findings: A questionnaire was developed to survey the confidence levels of the new graduates during the academic year 2012 in performing the 17 selected essential procedural skills, using a 1-5 rating scale with 4-5 deemed ‘satisfactory’ along with their learning experience, categorized as 0, <5, 5-10 and >10 cases. Ninety-nine medical graduates completed the questionnaire (62.7%). The mean confidence levels of the essential procedures varied from; 3.35 to 4.57, with the highest in wound suturing whilst the lowest was in newborn resuscitation. Confidence was significantly related to clinical experience. More than 90% of the graduates with experience of more than 10 cases indicated satisfactory confidence in all procedures with the exception of newborn resuscitation. With clinical experience of 5-10 cases, more than 70% of graduates were confident in performing all procedures except endotracheal intubation.

Conclusions: Confidence in performing essential procedural skills varied among procedures, depending on clinical experience and the type of procedure. Because of this, we strongly recommend the establishment of minimum requirements, for each procedure, be implemented. Our recommendation would be that at least 10 cases for life-saving skills and 5-10 cases for other essential procedures be required.

Keywords

Confidence; Procedural skill; Medical graduates; Clinical experience; Essential procedure

Introduction

In some developing countries, where medical personnel are inadequate, the graduates must have enough confidence along with competence to perform essential procedural skills independently. In Thailand the Thai Medical Council has decreed the 46 basic procedural skills which medical graduates should be competent in [1]. Medical schools have to provide clinical experience for medical students who are expected to have competence in all 46 procedural skills by graduation. The Thai medical education program is a 6-year curriculum, consisting of; premedical year (first year), preclinical years (second and third years) and clinical years (fourth to sixth years).The clinical years are devoted to rotations designed to provide comprehensive clinical exposure. They are expected to attain mastery of history taking, physical examinations along with procedural skills.

Performing tasks on live patients caused anxiety for new clinical students [2]. To prepare medical students, so as they are able to perform the basic procedures competently, we have initially approached this by; using skill-lab and simulation, followed by performing required procedures under supervision. The preceptors evaluated their performance until they achieved competence, and then gave them permission to perform these procedures independently. All required procedures were taught in the fourth and fifth years. The sixth year was the period of externship, in which we provided an opportunity for medical students to perform essential procedural skills in a live setting, under supervision, so that they could acquire clinical competence and confidence in a “really life” situation, with hands on practice before graduation. After graduation, they had to complete a 1-year internship before going into practice in a community or pursuing specialist training. In the Faculty of Medicine, Prince of Songkla University, the externs (6^th year medical students) were divided equally into two groups, with one working at Songklanagarind Hospital (The hospital of the Faculty of Medicine of Prince of Songkla University) while the others worked at 6 affiliated hospitals. This was for the first 24-week period, following which the two groups switched places. In both periods, the externs were divided into small groups for successive rotations within internal medicine, surgery, pediatrics, obstetrics and gynecology, orthopedics and emergency departments. Working within these affiliated hospitals provided more opportunities for medical students to attain essential procedural skills. From the guidelines, obtained for the Thai Medical Council, data from logbooks and feedback information, from the medical students, it was revealed and plainly apparent that the students performed some of the required procedures infrequently, thus having a low confidence level when performing these procedures. Knowing the real situation is needed for planning the education program, so that the graduates will attain the required skills competently. Most studies concerning this topic were from developed countries, which revealed that medical schools could not prepare all students to perform basic procedures independently [3-6]. In addition, a gap in desired versus actual level of competence existed for all procedures [6].

Self-confidence is of importance for a medical student’s professional development and lack of such confidence may foster stress. The relationship between procedural competence and confidence is complex. Confidence may reflect competence, but the correlation is quite poor in some studies [7-9]. There might be no relationship between self-reported levels of confidence and formally assessed performance [8]. However, one study showed that confidence was associated with competency only after training [10].

Self-assessed competency is correlated with experience, therefore to ensure an adequate number of procedure attempts may very well be a possible solution for preparing medical students to work confidently [4]. The confidence levels of the medical graduates in performing essential procedural skills, and the evaluation of their learning experience have not been previously assessed within our institution. Therefore, we conducted a cross-sectional study to survey the confidence levels as well as the experience of the medical graduates completing externship training in performing essential procedures.

This information will be beneficial, in particular, for setting the minimum requirements of each essential procedure in the curriculum. Moreover, specific deficiencies in procedural skills will be assessed.

Methods

A study survey concerning confidence in performing essential manual skills in the new medical graduates was conducted at the Faculty of Medicine, Prince of Songkla University, Thailand, after approval of the Ethics Committee. Our approach for training procedural skills in the externs was via use of the mentoring system. Before being externs, they had to attend an intensive cardiopulmonary resuscitation workshop. Selected staff doctors of each department were charged with monitoring the progression of the learning experience of the externs. They focused on essential procedural skills and then provided feedback using a logbook. During this process, the students had an opportunity to acquire necessary clinical skills and became familiar with patient responsibility under close supervision. A questionnaire was developed to survey the confidence levels of the new medical graduates completing externship training in the academic year 2012 in performing the 17 selected essential procedural skills using a 1-5 rating scale (1=least confidence and 5=most confidence), with 4-5 deemed satisfactory together with learning experiences. The 17 procedures were purposefully selected from the 46 basic procedural skills identified by the Thai Medical Council, which were considered to be sophisticated, essential as well as being those required in daily practice, especially in an emergency situation.

These consisted of; normal labor, episiotomy repair, amniotomy, pelvic examination with Pap smear, manual vacuum aspiration (MVA), wound suturing, excision of benign tumor, skin biopsy, incision and drainage, debridement, gastric lavage, lumbar puncture, external splinting, digital nerve block, newborn resuscitation, cardiopulmonary resuscitation (CPR) and endotracheal intubation.

Learning experience, defined as procedures which they have performed as the first operator (personally operated by more than 50%), was categorized into 4 groups based on the number of cases performed; 0, <5, 5-10 and >10. The questionnaire responses were anonymous. The data were analyzed using descriptive statistics and chi-square test. A p value of <0.05 was considered significant.

Results

Ninety-nine of the graduates completed the questionnaire (response rate 62.7%). Among the 17 essential procedures, the mean confidence levels varied from; 3.35 to 4.57, with the highest confidence in wound suturing and the lowest being in newborn resuscitation (Figure 1). Mean confidence levels were satisfactory (>4.0) in 11/17 (65%) of the procedures. The percentage ratings of the confidence levels for each procedure are shown in Figure 2. More than 50% of these medical graduates rated satisfactory confidence (level 4 and 5) in all procedures, with the exception of newborn resuscitation. Evaluation of the clinical experience found that the procedures for which more than half of the graduates had performed more than 10 cases were; pelvic examination and Pap smear, wound suturing, incision and drainage, debridement, external splinting, digital nerve block, CPR and endotracheal intubation (Figure 3). Procedures in which the graduates had low experience were MVA, skin biopsy and lumbar puncture, of which more than half had performed less than 5 times.

Figure 1: Mean confidence levels for the 17 essential skills surveyed.
NL, normal labor; PV, pelvic examination; MVA, manual vacuum aspiration; I&D, incision and drainage; LP, lumbar puncture; CPR, cardiopulmonary resuscitation; ET, endotracheal intubation

Figure 2: Essential manual skills and percentage ratings of confidence levels.
NL, normal labor; PV, pelvic examination; MVA, manual vacuum aspiration; I&D, incision and drainage; LP, lumbar puncture; CPR, cardiopulmonary resuscitation; ET, endotracheal intubation

Figure 3: Clinical experiences of essential manual skills.
NL, normal labor; PV, pelvic examination; MVA, manual vacuum aspiration; I&D, incision and drainage; LP, lumbar puncture; CPR, cardiopulmonary resuscitation; ET, endotrachealintubation
Clinical experience classified into 4 groups based on number of cases; 0, <5, 5-10 and >10.

Table 1 shows an analysis of confidence based on experience in performing essential procedures, which shows that with our graduates, confidence was significantly related to clinical experience. More than 90% of the graduates with clinical experience of more than 10 cases were confident in all procedures, again with the exception of newborn resuscitation. With clinical experience of 5-10 cases, more than 70% of the graduates were confident in performing all procedures, except endotracheal intubation. The procedures which had quite similar rates of confidence between clinical experience of 5-10 cases and >10 cases were MVA, wound suturing, excision of benign tumor, skin biopsy, and incision and drainage. All procedures except endotracheal intubation had significant differences in confidence rates between clinical experience of <5 cases and 5-10 cases. A marked difference in confidence levels in performing endotracheal intubation was noted when clinical experience exceeded 10 cases.

Table 1: Clinical experience and confidence.

Discussion

In this survey, we evaluated the self-assessed confidence levels of new medical graduates in performing procedural skills, and their experience in one institution. Our data provide information of medical student procedural skill attainment along with the deficient areas, which require further evaluation. We selected the seventeen procedural skills from the 46 basic procedures identified by the Thai Medical Council, which were considered to be sophisticated and essential in daily practice. The results of this study reflect the quality of curriculum management and the mentoring system. Mean confidence levels among procedures were highly variable, and only two-thirds of the selected procedures achieved satisfactory levels which were; amniotomy, pelvic examination and Pap smear, wound suturing, excision of benign tumor, incision and drainage, debridement, gastric lavage, external splinting, digital nerve block, cardiopulmonary resuscitation and endotracheal intubation. These skills were not so difficult and/or available resources were adequate. For the remaining skills, wherein confidence levels were not satisfactory achieved included; normal labor, episiotomy repair, MVA, skin biopsy, lumbar puncture and newborn resuscitation. These results might be related to more advanced skills and/or inadequate resources. In regards to patient safety concerns, more advanced procedures were performed by postgraduate trainees. In our hospital, we had postgraduate training in all fields of service. Factors affecting confidence might be the difficulty of the procedure and the number of cases which the students had performed. Significant correlations between clinical experience and confidence level were found among all procedures. More than 90% of the graduates with clinical experience of >10 cases achieved satisfactory confidence in performing almost all procedures. With experience of 5-10 cases, more than 70% of medical graduates attained satisfactory confidence, with the exception of endotracheal intubation. Compared to previous studies from developed countries, the data showed that our medical graduates had more clinical experience in procedural skills [4,6]. Only few procedures (manual vacuum aspiration, skin biopsy and lumbar puncture) were reported as having zero experience within a small proportion in our study, while several procedures in the previous studies reported zero experience in high percentages [4,6]. Medical personnel within most hospitals are adequate in most developed countries, so the medical students have less opportunity to perform on live patients. The strategy we employed was to provide more opportunity of clinical experience for the medical students by making use of the affiliated hospitals in the region, where numerous learning resources were available. The externs had spent a 6-month period in the affiliated hospitals. In addition, most of the affiliated hospitals did not have much in the way of formally postgraduate training, so the externs worked under close supervision of medical staff doctors with a ratio of approximately 1:1. In conjunction with more clinical exposure, we also used a mentoring system in all departments. Each extern was assigned to be monitored for procedural skill development. For each department rotation, the staff mentors had evaluated the required procedural skills and gave feedback to the externs at least twice (midway and final week of every department rotation). We believed that this process was of great importance. There was a study, which suggested that experience in performing skills increased confidence, but not competence, giving more feedback and pointing out erroneous corrections on performance during training may improve the relationship between confidence and competence [11].

Our data found that confidence in newborn resuscitation was the lowest, even though we have provided workshops for all medical students to prepare for this skill before the extern year. Newborn resuscitation is a complicated procedure. Successful resuscitation depends upon many factors. In an emergency situation, these cases were managed by pediatric residents or staff doctors, so the externs had less opportunity to perform as the first operator. To increase the opportunities for learning this essential skill, making use of simulations should be offered with repeated learning until competence is achieved. Earlier studies have also shown this to be a well-accepted teaching method [12] along with formative assessment. For life-saving procedures including endotracheal intubation and CPR, satisfactory confidence levels were achieved in both procedures. Although these were advanced skills, our medical students displayed high confidence, and most of them had performed these skills many times. No “zero experience” was reported. When comparing to the study of Dehmer [6], a high proportion (62%) of medical students had never performed intubation [6]. The confidence in these skills was significantly related to clinical experience.

More than 10 cases should be required so that >90% of graduates can become confident in performing these lifesaving skills. Since these are very essential skills, systematic teaching and chances of practicing are provided. All medical students are required to attend an intensive CPR workshop before their extern year. In addition, learning resources are also available in all hospitals. Low clinical experience was observed in three procedures including MVA, skin biopsy and lumbar puncture. For MVA, it was not made available in some affiliated hospitals, hence the low clinical experience obtained. In Songklanagarind Hospital, most cases came in as an emergency condition; therefore these were for the most part performed by residents in obstetrics and gynecology. There were limited resources for skin biopsy and lumbar puncture within most of the affiliated hospitals. The benefit of our study was that we surveyed self-assessed confidence of the graduates in performing essential procedures on graduation day, so the students could still remember their experience, implying that data of clinical experiences were quite accurate. This information can be used to set minimum requirements for each procedure within the education program. The responsible departments will be directly informed of these findings to help them establish effective means of teaching as well as assessment to enhance competency. This should additionally establish a higher rate of confidence within the student body in performing procedural skills.

We believe that it is useful, not only in our curriculum, but also in those of other countries where medical personnel are scarce and where physicians should be able to perform essential procedures independently with satisfactory confidence. Limitations of this study should be addressed such as; when such a self-assessment method is employed, the students may systematically overestimate- underestimate their work especially the outliers. In addition, the data reflect the experience and confidence of new medical graduates only within one academic year at one institution. Therefore, these findings could not possibly be generalizable to all medical graduates across the country. Finally, self-assessed confidence is not a guarantee of competency.

Conclusion

Confidence in performing essential procedural skills varied among procedures, and was dependent on clinical experience along with the difficulty of the procedure, with the highest being in wound suturing and the lowest in newborn resuscitation. Although, we have systematically provided learning opportunities for clinical experience within our education program, this survey reflected some deficiency areas that need to be improved. Confidence levels were significantly associated with clinical experience. We recommend that for lifesaving skills, a minimum requirement should be set at least at 10 cases, and for the other essential procedures, 5-10 cases should be required. Additionally, we propose seven strategies, which we believe would be effective in enhancing clinical skills along with confidence in the training program namely;

• Providing adequate real learning resources

• Making use of simulations in limited-resources procedures

• Establishing minimum requirements for each procedure, which must be met before graduating

• Creating innovative practical and effective means of teaching and assessment

• Close monitoring for clinical experience and confidence

• Early recognition of skill deficiencies in the trainees, so timely remedial action can be taken

• Formative assessment and narrative feedback

Conflict of interest

The authors declare no conflict of interest.

References

1. http://www.tmc.or.th/file_08062012.pdf
2. Moss F, McManus IC.The anxieties of new clinical students. Med.Educ.26(1):17-20 (1992).
3. Sanders CW, Edwards JC, Burdenski TK.A survey of basic technical skills of medical students. Acad.Med.79(9):873-875(2004).
4. Promes SB, Chudgar SM, Grochowski CO, et al.Gaps in procedural experience and competency in medical school graduates. Academic.Emergency.Medicine.16(2 Suppl): S58-62(2009).
5. Wu EH , Elnicki DM , Alper EJ , et al.Procedural and interpretive skills of medical students: experiences and attitudes of third-year students. Acad.Med.81(10SUPPL):S48-51(2006).
6. Dehmer JJ, Amos KD, Farrell TM,et al.Competence and confidence with basic procedural skills: the experience and opinions of fourth-year medical students at a single institution. Acad.Med.88(7):682-687(2013).
7. Fitzgerald JT, White CB, Gruppen LD, et al.A longitudinal study of self-assessment accuracy. MedEdu.37(7):645-649(2003).
8. Barnsley L, Lyon PM, Ralston SJ, et al. Clinical skills in junior medical officers: a comparison of self-reported confidence and observed competence. Med.Edu.38(4): 358-367(2004).
9. Morgan PJ, Cleave-Hogg D. Comparison between medical students’ experience, confidence and competence. Med.Edu.36(6): 534-539(2002).
10. Clanton J, Gardner A, Cheung M, et al.The relationship between confidence and competence in the development of surgical skills.J.Surg.Edu.71(3):405-412(2014).
11. Marteau TM, Wynne G, Kaye W,et al. Resuscitation: experience without feedback increases confidence but not skill. BMJ.300(6728): 849-850(1990).
12. Cooke JM , Larsen J , Hamstra SJ , et al.Simulation enhances resident confidence in critical care and procedural skills. Fam.Med.40(3):165-167(2008).