Development of a Standardized Assessment of Simulation-based Extracorporeal Membrane Oxygenation Educational Courses
Abstract
Background
In 2020, the Extracorporeal Life Support Organization education task force identified seven extracorporeal membrane oxygenation (ECMO) educational domains that would benefit from international collaborative efforts. These included research efforts to delineate the impact and outcomes of ECMO courses.
Objective
Development of a standardized online assessment tool to evaluate the effectiveness of didactic and simulation-based ECMO courses on participants’ confidence, knowledge, and simulation-based skills; participant satisfaction; and course educational benefits.
Methods
We performed a prospective multicenter observational study of five different U.S. academic institution–based adult ECMO courses that met Extracorporeal Life Support Organization endorsement requirements for course structure, educational content, and objectives. Standardized online forms were developed and administered before and after courses, assessing demographics, self-assessment regarding ECMO management, and knowledge examination (15 simple-recall multiple-choice questions). Psychomotor skill assessment was performed during the course (time to complete prespecified critical actions during simulation scenarios). Self-assessment evaluated cognitive, behavioral, and technical aspects of ECMO; course satisfaction; and educational benefits.
Results
Out of 211 participants, 107 completed both pre- and postcourse self-assessment forms (97 completed both pre- and postcourse knowledge forms). Fifty-three percent of respondents were physician intensivists, with most (51%) practicing at academic hospitals and with less than 1 year of ECMO experience (50%). After the course, participants reported significant increases in confidence across all domains (cognitive, technical, and behavioral, P < 0.0001, 95% confidence interval [CI], 1.2–1.5; P < 0.0001, 95% CI, 2.2–2.6; and P = 0.002, 95% CI, 1.7–2.1, respectively) with an increase in knowledge scores (P < 0.001; 95% CI, 1.4–2.5). These findings were most significant in participants with less ECMO experience. There were also significant reductions in times to critical actions in three of the four scored simulation scenarios. The results demonstrated participants’ satisfaction with most course aspects, with more than 95% expressing that courses met their educational goals.
Conclusion
We developed and tested a structured ECMO course assessment tool, demonstrating participants’ self-reported benefit as well as improvement in psychomotor skill acquisition, course satisfaction, and educational benefits. Course evaluation is feasible and potentially provides important information to improve ECMO courses. Future steps could include national implementation, addition of questions targeting clinical decision making to further assess knowledge gain, and multilanguage translation for implementation in international courses.