Introduction
Endotracheal intubation is a core skill of anaesthetists, intensive care physicians and pre-hospital practitioners [1]. Trainees usually learn how to perform the procedure on a manikin before they are permitted to practise on patients. However, there is no validated method to assess proficiency The MATTI (Manikin Assessment Tool for endoTracheal Intubation) task was developed, using a task analysis approach, by two consultant anaesthetists to assess endotracheal intubation proficiency [2]. The MATTI task has fourteen components (Table). Each component may be awarded either 2 (if completed correctly on the first attempt), 1 (if partially completed, completed out of sequence or completed after a prompt) or 0 marks (if not completed). Total score (out of 28) and time taken to complete the MATTI task are the recorded parameters. This study aimed to assess the construct validity, the test-retest reliability and the inter-rater reliability of the MATTI task [3].
Methods
Ethical approval was obtained from the Imperial College Research Ethics Committee (20IC6445). Thirty-six participants (12 Novices with no anaesthetic experience; 12 Intermediates with 2 years anaesthetic experience) were recruited to test the construct validity and the test-retest reliability (intubation group). A separate 20 participants (5 with anaesthetic experience; 5 with teaching experience; 5 with both; 5 with neither) were recruited to assess the inter-rater reliability (rater group). Both groups watched an instructional video before participating and the intubation group were allowed to practise the task. The intubation group were asked to perform the MATTI task three times, with an interval of 1 hour between attempts. The rater group were asked to score five pre-recorded videos of the MATTI task which contained between 0 and 10 deliberate mistakes. The “gold standard” score for each video was decided by two investigators (NC and CJM).
Results
Intermediates and Experts performed the MATTI task faster than Novices (p < 0.05; Figure). The median (IQR [range]) total score was 27 (26-28 [23-28]) out of 28, but the total score did not differ with experience level (p = 0.54). Neither time (p = 0.64) nor score (p = 0.69) differed between attempts. The median (IQR [range]) difference between raters’ scores and the “gold standard” was 1 (0-2 [0-11]) per video and Cronbach’s alpha for the rater group’s scores was 0.99.
Conclusion
The results demonstrate that the MATTI task is a valid and reliable tool to examine proficiency at endotracheal intubation of a manikin. Although the total score in the intubation group did not differ with anaesthetic experience level, its inclusion in the assessment is necessary in order to demonstrate correct completion of the MATTI components. The rater group results demonstrate that neither anaesthetic nor teaching experience is required in order to mark candidates’ proficiency. Further work should assess the validity and reliability of MATTI to assess endotracheal intubation in patients.