An online immunohematology educational resource for post-graduate hematology trainees: LearnSerology.ca

As a pre-requisite of receiving Royal College certification in hematology, Canadian post-graduate medical trainees must demonstrate general competency in transfusion medicine []. However, these competencies are defined in broad terms (i.e., an expectation that trainees be able to “describe the red blood cell and platelet antigen systems and the principles of transfusion medicine”), and a recent evaluation of final year hematology trainees, using a standardized knowledge assessment tool (aka, exam), revealed significant gaps in understanding of core concepts in immunohematology []. This exam was developed by a panel of experts, composed of Canadian hematology program directors and transfusion medicine subspecialists, using a modified Delphi approach with further enhancements made via Rasch analysis. As published previously, the average correct answer rate on this exam, as completed by 28 final year hematology residents from 12 different Canadian universities, was 65.1 % ( ± 15.6) a lower-than-expected level of transfusion knowledge. In addition, significant variation in performance was observed between trainees [].

Teaching immunohematology to hematology trainees can be challenging. In the absence of standardized teaching objectives, teaching is often focused on the active cases encountered during a trainees’ 1–2 month transfusion medicine rotation, which may disadvantage the teaching of important but more rarely encountered phenomena, such as weak or partial D antigen expression. Similarly, teaching is more likely to focus on compatibility testing platforms (tube, gel microcolumn or solid-phase) which are in local use. Finally, teaching trainees how to properly perform an antibody investigation can be time consuming as it often requires explanation of the theory and the rationale behind each step or elimination performed.

Internet-based learning has been advocated as an educational strategy that can improve teaching efficiency, particularly when incorporating feedback and interactivity, as it facilitates self-paced and asynchronous learning [, ]. It was therefore hypothesized that the development of a structured, interactive and self-directed immunohematology teaching platform (LearnSerology.ca) would be a useful means of improving trainee education. To evaluate the effectiveness of learnserology.ca, a case-control study was conducted, comparing performance on a standardized exam by residents exposed and not exposed to this resource.

A total of 57 trainees from 13 hematology programs completed the pre-test; one of these programs exposed its trainees (N = 10) to the LearnSerology.ca resource; 45 trainees from 14 universities (6 exposed to the LearnSerology.ca resource) completed the post-test. Overall test-performance, including breakdown by whether residents were in their 1st or 2nd year of hematology training, is shown in Table 1. Post-test results are further subdivided by the inclusion of the 2 additional questions relating to an antibody investigation panel.

Residents selected for exposure to the LearnSerology.ca resource had similar baseline performance on the pre-test (averaging 57.5 % correct answers) as those who did not have this exposure (55.5 % correct). Performance on the post-test, by comparison, revealed superior scores by residents who had been exposed to the LearnSerology.ca resource (73.8 vs 62.8 % correct). The difference in exam performance was particularly striking on the two questions relating to the antibody panel, with an average score of 91.7 % correct amongst those exposed to the LearnSerology.ca resources vs. 53.8 % amongst those who were not exposed. The statistical significance of these differences, as shown in Table 2, was limited by the small sample size but showed a strong trend, with an odds ratio of 0.600 (CI 0.355–1.014) that residents without LearnSerology.ca exposure would perform as well on the post-exam as those who had. When the analysis was limited to performance on the panel-related questions, the difference did achieve statistical significance (OR 0.106, CI 0.012–0.934).

When results were analyzed in terms of change in test score between the 12-multiple choice question pre- and post-exam (Table 3), it was observed that while a significant improvement in test performance could be shown for all residents (OR 1.451, CI 1.141–1.845), the change in score was not statistically significant for participants from ten out of thirteen universities (including the one institution with LearnSerology.ca exposure) when each training program was analyzed separately. When subgroup analysis was performed by level of training, as shown in Table 4, the level of improvement seen amongst second year trainees was much larger amongst those with LearnSerology.ca exposure, and did achieve statistical significance (13.9 % vs 3.9 % improved average test score, p < 0.0001). Notably, however, the cohort of 2nd year trainees in the exposed group who wrote the post-test consisted of a single individual.

An internet-based learning program called LearnSerology.ca has been developed which comprises a series of six self-learning modules that present different topics related to immunohematology knowledge as well as a virtual simulator with a library of interactive antibody cases from various levels of difficulty. Evaluation of the effectiveness of LearnSerology.ca in terms of acquired knowledge and skills showed that this new e-learning platform is at least as effective as more traditional teaching methods and superior in terms of acquired complex skills of antibody investigation as demonstrated by the statistically significant highest score at the two questions related to an antibody case obtained by the exposed trainees. The latter is likely partly due to the interactive virtual simulator with automated real-time feedback following digital antibody eliminations that allows asynchronous repetitive exposure to cases of variable difficulty levels. As has been contended previously by opinion leaders in the field of medical education, the opportunity to engage in deliberate and repeated practice of a new skill is a key requirement for development of expertise []. However, the superior performance that was also seen on the multiple choice questions, although smaller, suggests that the LearnSerology.ca curriculum was also effective in improving knowledge as well as skills when compared to traditional teaching methods. Some of this may be attributed to the longitudinal structure of the curriculum, which allowed the gradual teaching of concepts over a six-month period, rather than relying on a two-month concentrated course structure. While the benefits of longitudinal medical education have been described in most detail in the context of direct patient care [], there is growing enthusiasm for its application to the teaching of more theoretical concepts such as quality improvement [] and the risks vs benefits of blood transfusion []. The efficacy of the LearnSerology.ca curriculum may also have been a function of its provision as a web-accessible eModule, a format which allows learners to go at their own pace and to review the learning content as often as needed [, , , , , ]. This is an especially important advantage in teaching immunohematology, which is typically provided during a blood bank rotation and infrequently repeated elsewhere in the curriculum. E-learning has also the potential to be more efficient in terms of teacher workload and time, and permits learning standardization across a wide population of learners from different institutions, geographic locations and even different professions (e.g. blood bank technologists). Somewhat unexpectedly, despite being developed specifically for use by hematology residents at the University of Toronto, site statistics have revealed that the LearnSeology.ca website has been discovered and accessed by users from over 125 different countries since it was first launched.

An important limitation of the study is the inability to assess improvement in knowledge by each individual trainee. This is largely due to the desire to maintain trainee anonymity, a step which we felt was an important to disincentivize proscribed activities in the non-invigilated pre-test, such as use of reference materials or collaboration between trainees. However, it was also apparent that not all exposed trainees were able to complete both the pre- and post-tests. As a result, test results for each institution were by necessity combined and averaged before analysis. Nonetheless, it was possible to perform a subgroup analysis by level of training, which interestingly appeared to demonstrate a stronger benefit of the LearnSerology.ca curriculum in 2nd vs 1st year trainees. This result should be interpreted with caution due to the very small number of trainees included in the analysis, however, which was another important limitation of this study. Indeed, it was likely statistical underpowering that accounted for the lack of a statistically significant difference in scores between the pre- and post-test in the LearnSerology.ca cohort. In addition, while the pre- and post-test were administered a year apart, with trainees not provided their individual results or a copy of the exam to review, it is possible that the improved performance noted when results for all trainees were pooled was in part due to recollection of the questions, as well as the improved knowledge in transfusion medicine that comes with the traditional instruction in transfusion medicine that all trainees received in the interim. This secular effect could have also served to obscure the pedagogical benefit of the LearnSerology.ca curriculum. Despite these limitations, however, it is notable that trainees exposed to LearnSerology.ca still appeared on the post-test to outperform those trainees without such an exposure, particularly in regards to the interpretation of an antibody panel.

In summary, we have developed the internet-based learning platform LearnSerology.ca which appears to provide a meaningful supplement to traditional immunohematology teaching, particularly in terms of acquisition of complex skills of antibody cases investigation, with the benefit most pronounced amongst senior trainees. Future studies on a larger number of exposed trainees as well as surveys among users could look at other potential benefits of internet-based learning. Outcome measures of interest would be not simply improved knowledge of immunohematology, but confidence in providing medical oversight of a blood transfusion laboratory.