The use of interprofessional peer examiners in an objective structured clinical examination: can dental students act as examiners?

KEY POINTS IN BRIEF This article: * Illustrates interprofessional education. * Uses both peer group teaching and assessment. * Increases awareness by future medical practitioners of the role

of the dental surgeon. ABSTRACT OBJECTIVE To assess whether final year dental students could act as reliable examiners within an Objective Structured Clinical Examination (OSCE) by

comparison with results obtained by an experienced member of staff. DESIGN A station testing examination of the mouth was included in the second year medical undergraduate summative OSCE

examination. SETTING Concurrently run in three different examination venues on the Ninewells Hospital campus. SUBJECTS 147 medical students and 3 pairs (A, B, C) of examiners. Each examining

pairing consisted of one member of staff and one dental student (blind to each other's marking). METHOD A checklist of 13 tasks to be performed was provided to the examiners. One mark

awarded for a completed task, no mark for no attempt at the task, and half a mark for attempt at task. RESULTS Paired results were available for 125 medical students. Using Mann-Witney

analysis, the non-parametric 95% confidence intervals for the difference in scores between the 3 paired teams were group A (–0.5, 0), group B (–0.5, 0.5), group C (–0.5, 0). In only 4

students (out of 125) did the difference between the individual pair differ by 2 or more marks. CONCLUSION On the basis of this pilot study final year dental students may be used as

examiners in OSCEs where basic technical skills are to be evaluated. This development from peer group teaching provides further evidence supportive of interprofessional education. You have

full access to this article via your institution. Download PDF MAIN There can be few dental surgeons who are not involved in some aspect of educating members of the dental team (eg dental

nurses or vocational/general professional trainees). As the role of the dental practitioner evolves, it is important to share educational innovations within the journals they read. Such a

strategy has been adopted by the _British Medical Journal_1 and is supported by the creation of an Education Section within the _British Dental Journal_. This recognises that education and

service are complementary to one another. To that end examinations should seek to be reliable and valid in assessing performance.2 The method of assessment should also encourage lifelong

learning strategies through a deep approach to learning.3 In the clinical environment the Objective Structured Clinical Examination (OSCE)4 is increasingly being used at an undergraduate

level to assess competence. Peer teaching has also been shown to be effective in promoting learning in the undergraduate medical curriculum.5 In a problem-based course in Brasilia scores of

meaningfulness of course experience and group work usefulness were higher in the peer tutored group than in the teacher tutored group. There was no loss of cognitive achievement in the peer

tutored group.6 The natural progression from the peer teaching and learning process is a move to peer assessment. In the context of an OSCE the assessment can be dissected to a certain

extent to measure the procedural rather than the contextual aspects of a particular skill. When students are novices at the commencement of their clinical programme it is essential to

examine discrete aspects of a task. For example, the OSCE can be used to determine a student's performance to demonstrate a procedural skill in the correct sequence. However, in the

assessment of performance, it is often difficult to develop an OSCE station that only assesses one aspect of the task. One has to obtain reliability in assessing a task without losing its

validity within the context that it is set in. The need to employ an 'expert' examiner may not be necessary. However, to achieve reliability, training in the use of a checklist is

required. Arguably less training is required for those with prior knowledge of the subject, since an expert can evaluate the quality of the behaviour and its appropriateness in the given

context.7 Hence we used dental students in the assessment of oral cavity examination by medical students. The World Federation for Medical Education Summit in 1993 highlighted the need for

feasible educational strategies to equip doctors with the clinical skills to promote health and prevent illness, and to practice in collaboration with other healthcare professionals.8

Collaboration in healthcare practice requires a knowledge of the scope and unique functions of each contributor to patient care. Collaborative education can prepare students to work

collaboratively in later professional life.9When the barriers of professional jargon and ignorance of training programmes are removed there are opportunities for better patient care through

shared insight.10 Medical curricula have traditionally had little emphasis on oral health issues,11 yet in the Health Education Board for Scotland's strategic plan dental and oral

health was identified as one of the eight priorities for health education.12 Diseases affecting the teeth and mucosa are generally higher in Scotland than in the rest of the UK.13,14 Often

patients attend their doctor rather than their dentist if they perceive there is something wrong with their mouth.15,16 This is often caused by ignorance of the role that GDPs play in the

diagnosis and management of oral disease. There is obviously a need to identify areas for collaboration between doctors and dentists in order to offer optimal patient care. In this paper we

report on the performance of final year dental students as assessors of second year medical students using an OSCE. METHOD Following the publication of _Tomorrows Doctors,_17 the GMC's

recommendations on undergraduate medical education in 1993, the University of Dundee developed a spiral integrated three-phase curriculum.18 The first two phases are based on the 11 body

systems, with the student moving from normal structure function and behaviour in Phase 1 to abnormal structure, function and behaviour in Phase 2. There are 12 curriculum outcomes which

determine the attributes of a Dundee medical graduate. The students are assessed during each phase of the curriculum on their progress in the 12 outcomes. The gastroenterology systems

teaching in Phase 2 is a 5-week programme developed by an interprofessional group of healthcare professionals supported by a basic scientist, microbiologist and pathologist. Oral health is

the focal topic of the first week's teaching. Students are given lecture opportunities in relation to caries prevention and an overview of systemic disease affecting the oral cavity,

small group activities which relate basic sciences to the topic, and each student participates in a 2-hour clinical skills session. At the end of the week students attend the integrated

teaching programme which focuses on the integration between healthcare professions, the integration between clinical presentation and pathophysiology, and the integration between primary and

secondary care. The 2-hour clinical skills session uses small group teaching methods and consists of four components: * Examination of the oral cavity * Identification of common oral

lesions using a video programme * Taking a history of an oral complaint from a patient * Examination of patients with oral disease. The expected outcome from this session is that students

would be able to develop their competence in the technical skill of oral examination in the correct sequence and using the appropriate equipment. Final year dental students were chosen at

random from a class of 50 to participate in the teaching programme. The students were undertaking a core teaching attachment in oral surgery. The clinical programme for medical students is a

compulsory programme so all students in second year attended. The students work in groups of eight, randomly allocated at the beginning of Phase 2. Eight dental students participated in the

programme, two attending each session. The oral surgeon in charge of the programme met with the students prior to the start of the G.I. course. Using the clinical skills tutor manual

guidelines given to other medical tutors participating in the programme the students were taught what to cover. The dental students were thus appraised of the course outcomes and their

responsibilities in relation to the teaching and learning of oral examination to the second year medical students. The dental students had to demonstrate examination of the oral cavity to

each group of eight to ten medical students. In the demonstration of this, one dental student acted as a 'patient', while the other was the examining 'doctor'. This

re-enacted the role play that they had been exposed to prior to entering clinics 2.5 years previously. However, it reflected what the dental students were doing practically every day. The

'tools' employed were those readily available in any general medical practice setting (ie disposable gloves, gauze, wooden tongue spatula, light). They then had to oversee the

medical students' practise and refine their techniques as required. The medical students were encouraged to take up self revision skills sessions and students were also given the

opportunity to refine their skills in oral examination in the integrated teaching programme in the following week. At the end of the 24-week core teaching programme all 144 medical students

underwent a 24-station OSCE4 covering all areas taught in the 'integrated programme'. Errors of rating were minimised by a meeting in which the examiners for all three venues

(staff and dental student) were briefed as to their role, the use of the checklist of actions to be performed and the marking system (Fig. 1). The examination was run over the course of a

morning, in three venues within the hospital with two runs of each examination, to ensure all candidates were assessed. The medical students changed stations every 4 to 8 minutes. At one

station medical students were given a written scenario in relation to examination of the oral cavity. The dental student 'examiners' were taken from those that had helped teach the

medical students some 3 months earlier. The checklist (Fig. 1) gave marks for communication skills as well as the sequence of the oral examination. Two examiners were placed at each site.

One a final year dental student and the other a member of the dental faculty staff (with at least 5 years postgraduate experience). They were given an identical set of examination papers

containing the checklists and medical student matriculation numbers. They were informed that one of their checklists would be used for the final student mark but were not informed which (in

the event, only the marks recorded by the member of staff were used). The two examiners were requested not to confer throughout the examination process and the site supervisor in each venue

was able to observe the examiners, as the station was near the central organizational point. There were no gaps between medical student assessments and therefore no time for discussions

between student and staff examiner. RESULTS The OSCE station relating to examination of the oral cavity had a checklist of 13 points, each with a value of 1 (if correct) or 0 (if incorrect).

The examiners were allowed to score 0.5 if the student attempted the examination procedure but did not carry it out properly. For each medical student, the total score for the OSCE (maximum

13 points, minimum 0 points) marked by the member of staff was compared with that obtained by the dental student. A total of 147 medical students sat the exam, although the results were not

made available to us for 22 of the medical students. Thus the results for 125 medical students were analysed. Since there were three pairings of staff plus dental student (ie one OSCE held

simultaneously in three venues), comparisons were made initially for each pairing separately. When all three dental students were combined and compared with all three members of staff, mean

values of 0.92 and 0.93 respectively, were obtained. Thus a very close agreement was identified, between student and staff members, in terms of the overall result for the OSCE questions on

oral examination. Using Mann-Whitney tests, the hypothesis that the medium score of marks awarded by the dental student was the same as the median score of marks awarded by the member of

staff was not rejected in each of the three pairings A, B, C. Non-parametric 95% confidence intervals for the difference in scores in the three pairings were: A: (–0.5, 0); B: (–0.5, 0.5);

C: (–0.5, 0). The overall agreement can be seen in Table 1, which lists the frequency in which each pairing gave the same score for the OSCE, one mark difference, two marks difference and so

on. When we allowed for a difference of one mark (eg a total score of 12 by dental student, 13 by member of staff), the percentage agreements for pairings A, B and C were 95%, 100% and 80%

respectively. When all were combined, an overall level of agreement (allowing for a maximum difference of one mark) was 93%. A high level of consistency was thus achieved. Indeed, a

difference of more than one mark only occurred for nine medical students examined (out of the total available, _n_ = 125). Even then the maximum difference recorded was three marks. There

were no cases in which the total mark allocated by the dental student compared with that from the member of staff, differed by more than three marks. DISCUSSION The level of correlation

between the dental student examiners and the senior examiners was high. However, certain anomalies were apparent. Overall, pairing A had the closest agreement on the majority of items on the

checklist. However, on certain items pairing C were better correlated yet overall had less agreement. These findings need to be discussed in the context of current knowledge of the role of

the examiner in the assessment process. Classical test theory and Generalisability theory assumes that behaviour does not change over time. However, in the assessment process the performance

of the examinees as well as the examiners changes.19 Examiners with experience often judge performance in the light of their own experience.7 There may be concern that the final year dental

students, while able to assess individual components of the task, may have been unable to identify concerns in the overall quality of performance. In maximising the reliability of the

examination, random errors were controlled by minimising the fluctuations in the external environment and by the discussion of the checklist by the examiners prior to the exam, in other

words having a shared view of the behaviour paradigm being assessed. However, concern has previously been expressed that expert examiners are often constrained by checklists. Norman has

indicated that breaking down the task into discrete parts using a checklist does not necessarily make the assessment more reliable than a global rating.19 This may have influenced the

results in this pilot. Refinements to the checklist and the use of a global rating scale may help to increase the reliability of this station. Freidman and Menin have also suggested that

expert examiners are able to differentiate the effects of systematic error from those of random error and their effect on performance outcome.7 This would imply that there should be less

correlation between the dental students and their expert co-examiners. However, we would suggest that from our results in the assessment of a procedural skill, that may be used in a wide

variety of clinical contexts, the need for expert examiners is not necessary. To some extent our conclusions are only valid for the three randomly selected dental students. It could have

arisen by chance that the three most reliable dental students had been chosen (although as previously described, those students had not been specifically selected to teach the medical

students). However, given the constraints of the examination and space available, it was not possible to have more than two examiners for this station. There may be concern at the use of

students to mark fellow undergraduates. However, the dental students were all about to qualify and therefore were fully aware of their professional role. Furthermore, the results indicate no

significant difference between the staff and student examiner. There was also a member of staff present as an examiner. Even if the dental student was the sole examiner, inappropriate

communication would be unlikely given that: a) A site supervisor regularly patrolled the station, and b) The simulated patients were trained lay people. The station could also be videoed.

One of the future considerations for this station is to video the assessment process and ask an expert and a novice medical marker to mark the student performance and then to compare this

with the examiners present at the station. We do not know what measure of agreement may have been obtained if comparing scores from two members of staff. This will be undertaken in a future

study. Rater bias is considered not a problem if all students are scored by the same rater and a relative standard issued.20 This was not possible in our study, as the examination took place

in three different venues at the same time. 'Hawks' and 'doves' always exist in any assessment process.20 The briefing meeting and the discussion of the checklist prior

to the examination should have minimised this well recognised cause of inter rater reliability. On some occasions, however, the students appeared to be more hawkish. This may have been due

to a systematic error affecting realiability caused by dental student bias. They had been involved in the teaching programme and therefore had previous exposure to the medical students and

perhaps had higher expectations of performance. One aspect to emerge from the study is the identification of tasks within the OSCE that should be considered for deletion or modification. In

this area there has not been close agreement between dental student and staff member, perhaps suggesting that there is difficulty in interpreting what the medical student does successfully,

eg as in response to inspection of facial asymmetry or inspection of the lips. The process of this study has enabled us to reflect and focus on what the required outcome is in relation to

this station and the programme on gastroenterology and oral health that is offered in the undergraduate medical curriculum.21 In relation to this it is also important to question the

appropriate representation of the station in relation to its generalisability to real practice. To that end, feedback was obtained for both the medical and dental students (paper in

preparation). CONCLUSION This pilot study illustrates that peer groups that have been used to teach other health professionals can be used successfully as examiners in an OSCE assessment.

Although it may be said to be only true for these three randomly selected dental students, it does nevertheless, provide further evidence supportive of interprofessional education. An area

which in the future may lead to better collaboration between different health professionals, to the benefit of their patients.22 They also gain insight into evaluating their own level of

understanding.5 This practical peer collaboration within the joint Faculty of Medicine, Dentistry & Nursing in the assessment process can only lead to a better understanding of

professional roles in the delivery of healthcare. REFERENCES * Education Group for Guidelines on Education Guidelines for evaluating papers on educational interventions. _Br Med J_ 1999;

318: 1265–1267. * Davenport E S, Davis J E C, Cushing A M, Holsgrove G J . An innovation in the assessment of future dentists. _Br Dent J_ 1998; 184: 192–195. Article  Google Scholar  *

Newble D I, Entwhistle N H . Learning styles and approaches: Implications for medical education. _Med Educ_ 1986; 20: 162–175. Article  Google Scholar  * Harden R M, Gleeson F A . Assessment

of clinical competence using an objective structured clinical examination (OSCE). _Med Educ_ 1979: 13: 41–54. PubMed  Google Scholar  * Heylings D J, Stefani L A J . Peer assessment

feedback marking in a large medical anatomy class. _Med Educ_ 1997; 31: 281–286. Article  Google Scholar  * Sobral D T . Productive small groups in medical studies: training for cooperative

learning. _Med Teacher_ 1998; 20: 118–121. Article  Google Scholar  * Friedman M, Mennin S P . Rethinking Critical Issues in Performance Assessment. _Acad Med_ 1991; 66: 390–395. Article 

Google Scholar  * Hamilton J . Training for skills. _Med Educ_ 1995; 29: 83–87. Article  Google Scholar  * Multidisciplinary education. Editorial. _Med Educ_ 1995; 29: 329–331. * Horder J .

Interprofessional co-operation: the vision and the challenge. Conference Reports. _J Int Prof Care_ 1992; 6: 73–75. Google Scholar  * Bowden J, Scully C . Dentistry and total oral health.

_Br Med J_ 1989; 298: 186. Article  Google Scholar  * _Strategic Plan 1992–1997_. Health Education Board for Scotland, 1993. Edinburgh, published by HEBS. * Macfarlane G J, Boyle P, Scully C

. Rising mortality from cancer of the tongue in young Scottish males. _Lancet_ 1987 VOL 2: 912. Article  Google Scholar  * O'Brien M . Office Population Census & Surveys.

_Children's Dental Health in the UK_. London: HMSO, 1994. Google Scholar  * Scully C, Malamos D, Levers B G H, Porter S R, Prime S S . Sources and patterns of referrals of oral cancer;

role of general practitioners. _Br Med J_ 1986; 293: 599–601. Article  Google Scholar  * Guggenheimer J, Verbin R S, Johnson J T, Horkowitz C A, Myers E N . Factors delaying the diagnosis of

oral and oropharyngeal carcinomas. _Cancer_ 1989; 64: 932–935. Article  Google Scholar  * General Medical Council. _Tomorrow's Doctors. Recommendations on Undergraduate Medical

Education_. London: General Medical Council, 1993. * Harden R M, Davis M H, Crosby J R . The new Dundee medical curriculum : a whole that is greater than the sum of the parts. _Med Educ_

1997; 31: 264–271. Article  Google Scholar  * Norman G R . Reliability and Construct Validity of some Cognitive Measures of Clinical Teaching. _Teach Learn Med_ 1989; 1: 194–199. Article 

Google Scholar  * Newble D, Dawson B, Dauphinee D, Page G, Macdonald M, Mulholland H, Swanson D, Thomson A, van der Vleuten C . Guidelines for Assessing Clinical Competence. _Teaching &

Learning Med_ 1994; 6: 213–220. Article  Google Scholar  * Ogden G R, Ker J . Re: Kissun et al. General medical practitioners' knowledge of the specialty of oral and maxillofacial

surgery (letter). _Br J Oral & Maxillofac Surg_ 1998; 36: 479–480. Article  Google Scholar  * Horder J . Interprofessional Education. _Med Educ_ 1992(b); 26: 427–528. Article  Google

Scholar  Download references AUTHOR INFORMATION AUTHORS AND AFFILIATIONS * Section of Oral & Maxillofacial Surgery, University of Dundee, G R Ogden * Lecturer, Department of Mathematics,

Clinical Skills Unit, University of Dundee, M Green * Lecturer, Ninewells Hospital & Medical School, University of Dundee, J S Ker Authors * G R Ogden View author publications You can

also search for this author inPubMed Google Scholar * M Green View author publications You can also search for this author inPubMed Google Scholar * J S Ker View author publications You can

also search for this author inPubMed Google Scholar ADDITIONAL INFORMATION Refereed Paper RIGHTS AND PERMISSIONS Reprints and permissions ABOUT THIS ARTICLE CITE THIS ARTICLE Ogden, G.,

Green, M. & Ker, J. The use of interprofessional peer examiners in an objective structured clinical examination: Can dental students act as examiners?. _Br Dent J_ 189, 160–164 (2000).

https://doi.org/10.1038/sj.bdj.4800711 Download citation * Received: 05 August 1999 * Accepted: 05 January 2000 * Published: 12 August 2000 * Issue Date: 12 August 2000 * DOI:

https://doi.org/10.1038/sj.bdj.4800711 SHARE THIS ARTICLE Anyone you share the following link with will be able to read this content: Get shareable link Sorry, a shareable link is not

currently available for this article. Copy to clipboard Provided by the Springer Nature SharedIt content-sharing initiative