Click to see each type of abstract or scroll to see them all!

Design and Development of an Automated Prescription Exam Parser and Marker

Keshavjee, Karim; Khatami, Alireza; Holbrook, Anne; Levinson, Anthony

INTRODUCTION
Teaching medical students the art and science of safe and cost-effective prescribing is challenging yet essential for the health care system. However, the assessment of prescription writing and providing high-quality feedback to students is time consuming, labour intensive and requires expertise. There is a need for an automated solution that can assist educators to provide high-quality feedback to trainees at different stages of training in a timely manner.
The authors designed, developed and alpha-tested a prescription parser-marker to validate the concept of using computer readable very short answers for marking prescriptions.
METHODS
Requirements for a Prescription Parser-Marker were gathered through literature review and expert panel consensus. The first draft design was a Parser-Marker software that could handle a number of different question sets using a Marking File that had standard features but had customized content for each new question set.  The first draft software was built using RapidMiner, a data science software that has extensive text management capabilities.  As proof of concept, the free text responses of 10 students who wrote prescriptions for 5 clinical case scenarios were processed through the software.  Errors in parsing were categorized and mapped onto a cause-and-effect fishbone diagram. The software was redesigned to solve key problems identified in the cause-and-effect analysis.
RESULTS
The major cause of poor performance of the software arose from the high variability in responses by the students.  Misspellings and non-responses were a minor cause of errors.  Variability in responses arose from 6 key areas: 1) student knowledge about prescribing standards and preparedness for writing a prescribing quiz, 2) ambiguity or imprecision in questions posed in the quiz, 3) the exam writing software and exam writing environment, 4) lack of standard nomenclature for drug prescribing (e.g., multiple ways of writing frequency, dosing and route), 5) inability to represent a wide range of synonyms for drug names, frequencies of administration, dosing and route in a standard Marking File and 6) how much or how little the parsing software is ‘hard-coded’ for addressing variability in the responses.
When sources of variability are reduced, the software generates scores quickly and accurately.  
CONCLUSION
We demonstrated that automation of prescription marking with response-specific feedback to learners, is possible and feasible.

Exploring the development of standardized patient educational experiences to support equity, diversity, and inclusivity mandates of healthcare training programs

Sibbald, Matthew; Last, Nicole; Keuhl, Amy; Azim, Arden; Sheth, Urmi; Khalid, Faran; Bhanja, Farhan; Geekie-Sousa, Aaron; Uzelli Yilmaz; Monteiro, Sandra

INTRODUCTION

Standardized patient (SP) programs are increasingly being asked by health professions training programs to create learning and assessment opportunities integrating equity, diversity and inclusivity (EDI). However, little is known about the optimal approach and many SP programs struggle to meet these growing needs. While theory on instructional design, sociologic translation and simulation design could be helpful, no cohesive framework exists to guide SP programs. 

METHODS

We interviewed 14 key informants in 2021 involved in creating or managing SP based education in response to health professions training EDI mandates. Verbatim transcripts were analysed qualitatively in an iterative coding process, anchored by direct content analysis methodology. Repeated cycles of data collection and analyses continued until themes could be constructed, aligned with existing theories, grounded in empirical data, and with sufficient relevance and robustness to inform educators and curricular leads.

RESULTS

Three themes emerged. First, SP programs have various ways to create safety for SPs who are paid to be vulnerable. Second, fidelity issues not only reflected who plays the role, but the integration of traditionally underrepresented or marginalized groups in multiple aspects of SP programs, from co-creating content to debriefing.  Finally, structures that lead to health inequities similarly challenge participation in education. Building relationships with communities, and empowering members who are also health providers and/or educators facilitates community mobilization for SP based education.

CONCLUSIONS

SP programs must be mindful of the oppressive nature of educational structures, and be allies and advocates to traditionally underrepresented groups, so they may collaborate and share governance of the educational development process from its outset. SP programs play important roles in (1) advocating with curricular leadership, (2) building relationships with community and empowering mobilizers, (3) facilitating co-creation and co-delivery of educational content and (4) building inclusivity and safety into simulation processes.

How workplace-based assessments guide learning in postgraduate education: a scoping review

Martin, Leslie; Blissett, Sarah; Johnston, Bronte; Tsang, Michael; Gauthier, Stephen; Ahmed, Zeeshan; Sibbald, Matthew

INTRODUCTION

Competency-based medical education (CBME) led to the widespread adoption of workplace-based assessment (WBA), with the promise of achieving assessment for learning. Despite this, studies have illustrated tensions between the summative and formative role of WBA, which threaten trainee learning. Models of workplace-based learning (WBL) provide insight into achieving assessment for learning, however, these models either predated or systematically excluded WBA. This scoping review synthesizes the body of primary literature addressing the role of WBA to guide learning in postgraduate medical education (PGME). By analyzing findings through the lens of WBL theory, we identify gaps that can be addressed through future studies. 
METHODS

The search was applied to OVID Medline, Web of Science, ERIC and CINAHL databases. Titles and abstracts were screened by two independent reviewers, followed by a full text review to determine inclusion. Two team members independently extracted and analyzed quantitative and qualitative data from each paper using a descriptive-analytic technique rooted in Billet’s four premises of learning through healthcare work. Themes were synthesized and discussed amongst the team until consensus.  
RESULTS

Thirty-three papers were included, all focused on the perception of learning through WBA. The majority applied qualitative methodology (70%), and only eleven studies (33%) made explicit reference to theory that informed the study. Aligning with Billet’s first premise, results reinforce that learning always occurs in the workplace, however WBA helped guide learning goals and enhanced feedback frequency and specificity. Billet’s remaining premises provided an important lens to understand how tensions that existed in WBL have been exacerbated with frequent WBA. For example, as individuals engage in work, including WBA, they are slowly transforming the workplace. Culture and context frame individual experiences and the perceived authenticity of WBA within the WBL environment. Finally, individuals will have different goals, and learn different things from the same experience.  
CONCLUSION

Analysing the literature that focuses on the intersection between WBA and WBL through Billet's WBL theory has reframed the previously described tensions. We propose that future studies attend to learning theory, with clear alignment to the researchers’ philosophical position, to advance our understanding of assessment-for-learning in the workplace.

Student Perceptions of Hybrid Delivery of Interprofessional Anatomy. The Best of Both Worlds?

Mezil, Yasmeen; Oliveira, Ana; Akhtar-Danesh, Noori; Palombella, Andrew; Rockarts, Jasmine; Wojkowski, Sarah; Wainman, Bruce

INTRODUCTION

Interprofessional education (IPE) prepares healthcare students for collaboration in their future careers. The Education Program in Anatomy at McMaster offers an interprofessional course where students discuss case studies, deliver presentations on their scope of practice, and perform cadaveric dissection. The onset of the pandemic shifted the IPE course to a virtual environment, replacing the in-person events with online discussions and dissection videos. With the transition back to in-person events, the Education Program in Anatomy adopted a hybrid approach for IPE, offering the first 4-weeks in a virtual environment (Microsoft Teams) and the second 4-weeks in-person (in the anatomy lab). We aim to explore the impact of the hybrid delivery of the IPE course on the perceptions of healthcare students towards interprofessional and anatomy education.

METHOD

28 students (14 females; 24.8±6.3 years old) from different programs (4 Physician Assistant; 2 Midwifery; 3 Speech-Language Pathology; 4 Physiotherapy; 3 Occupational therapy; 4 Nursing; 8 Medicine) participated. Students were asked to complete two surveys pre- and post-course: the Readiness for Interprofessional Learning Scale (RIPLS) and Interdisciplinary Education Perception Scale (IEPS). A higher score in these questionnaires indicate a higher readiness for interprofessional learning. Students were also asked to complete a Q-methodology survey by sorting 40 statements about the virtual and in-person environments of the IPE course.

RESULTS AND CONCLUSIONS

In the pre-course RIPLS, participants scored a median of 84/95 [interquartile range 78-87]. Sub-scale mean scores were as follows: Teamwork & Collaboration 41/45 [37.7-42]; Negative Professional Identity 13/15 [12-14]; Positive Professional Identity 18.5/20 [16-20]; and Roles & Responsibilities 12/15 [10.7-12]. IEPS pre-course total scores were: 64.2/72 [60.5-66.5] points. Sub-scale mean scores were as follows: Competency & Autonomy 26.8/30 [25-28.5], Perceived Need for Cooperation 11.2/12 [10-12], and Perception of Actual Cooperation 26.3/30 [24-30]. Post-course scores, as well as qualitative information from Q-methodology pertaining to virtual and in-person course satisfaction, will be collected at course termination and included in the presentation. The feedback collected will be critical for the future development of this course using a hybrid approach and will highlight aspects of the course that contribute to students’ interprofessional and anatomical expertise. 

A Simulated Call to The Wards to Improve Preparedness in The Transition to Discipline for Incoming Internal Medicine Residents

Sharma, Rishi; McDonald, Rachel; Spatafora, Laura; Khalid, Zahira

INTRODUCTION
The transition from undergraduate to postgraduate medical education poses significant stress for incoming residents, particularly during on-call shifts due to reduced supervision and heightened responsibility. This period has been associated with poorer patient outcomes. There is a paucity of data investigating the role of non-procedural simulation training during this period. This project aimed to explore the role of simulation-based curricula in improving confidence and easing anxiety during on-call shifts in the transition to internal medicine residency.

METHODS
A simulation curriculum was designed based on a needs assessment completed by staff and current/incoming residents. 35 incoming residents were assigned to intervention or wait-list control groups by matched pairs design. The intervention group completed the simulation prior to residency and the wait-list control group one month following. Participants completed pre- and post-simulation surveys, a structured debrief following the simulation, and a focus group one-month post simulation. A mixed-method approach to data analysis with quantitative analysis, descriptive statistics and thematic analysis was utilized.

RESULTS
Approaching unstable patients was the skill associated with the most stress and feelings of lack of preparedness yet was identified as one of the most important competencies by faculty. These scenarios were found to be the most useful components of the simulation curriculum with residents in both groups reporting an increase in confidence and decrease in stress levels when approaching unstable patients. There was a greater impact across all skills when the simulation was completed prior to the start of clinical training. 

CONCLUSION
The simulation curriculum was well-received by residents and had a greater impact when completed earlier in the transition to residency period. As residents begin to complete overnight call shifts, practical experiences also improve confidence and stress levels. However, early-in-training residents appreciate additional practice in a simulation setting and would likely benefit from the development of longitudinal simulation curriculum development. Ultimately, our simulation bootcamp improved confidence and reduced anxiety for several key competencies required to successfully and independently perform patient care on overnight call shifts. This curriculum may be easily implemented across other internal medicine programs. 

Developing a Canadian Clinical Pharmacology e-Curriculum with Online Assessment for Medical Students

Holbrook, Anne; Levinson, Anthony J.; Keshavjee, Karima; Perri, Dan; Levine, Mitchell; Rudkowski, Kill; McLeod, Heather; Ford, Angela; Maxwell, Simon

INTRODUCTION
Clinical pharmacology knowledge and prescribing skills are essential for all medical students to acquire, regardless of career path. Our group has previously shown that a) fewer than 48% of final-year Ontario medical students pass the Canadian Prescribing Safety Assessment, b) all Canadian medical schools struggle to ensure prescribing competence, and c) e-curricula resources and online assessment could be invaluable, since there are very few Clinical Pharmacology & Toxicology (CPT) faculty and the pandemic has accentuated the need for quality online medical education.   
We were funded in a competition by eCampus Ontario to develop a package of Canadian clinical pharmacology, therapeutics and prescribing competence e-curriculum resources with concomitant online assessment. 
METHODS
We planned a multi-component pilot including a) evaluating a selection of e-curriculum resources in CPT for e-learning quality and relevance then optimizing for the Canadian undergraduate medical training environment, b) developing and testing methods of parsing written prescriptions to allow entirely computerized marking of prescription writing and other very short answer questions, and c) development of processes and evaluation metrics for an online Observed Structured Clinical Exam (OSCE) in CPT using SIMULATIONiQ. 
RESULTS
Building on a structured review of all accessible CPT e-curricula, we rated the quality and relevance of Australian National Prescribing e-curriculum modules and created Canadian ‘translation’ preambles. An online Prescription Parser Marker, which was developed in RapidMiner using RegEx, was able to successfully identify and mark the 7 components of a free-hand prescription. A rapid scoping review of OSCEs found a lack of evidence of unique contribution to CANMEDS role-based education compared to written exams, or high-quality comparisons of virtual versus in-person OSCEs. A pilot virtual CPT OSCE was created and may be feasible for both synchronous and asynchronous assessment.
DISCUSSION
A Clinical Pharmacology & Toxicology e-curriculum with innovation in assessment beyond usual multiple choice written exams to include computerized assessment of prescription writing and virtual OSCEs is likely to be useful to meet new medical licensing objectives and is generalizable. Our next step is to expand development into a full online Canadian CPT course with assessment guided by a national needs assessment survey.

Investigating Canadian Radiology residents understanding of financial literacy: a nation-wide survey.

Demoobad Sharifabadi, Anahita; Bellini, Jonathan; Alabousi, Abdullah; Monteiro, Sandra; Al-Arnawoot, Basma

INTRODUCTION

Resident physicians throughout Canada face large debts to manage the burden of financing their costly education, often using loans to afford the medical school they attend. The literature suggests debt and money management both play a large role in both the day-to-day life and in deciding the future of physicians. The objective of this observational study is to assess the financial literacy of Canadian Radiology residents through their financial knowledge and an understanding of their financial situation. 

METHODS

We conducted an in-depth national survey assessing demographics, current financial status, a financial literacy quiz and plan for developing a curriculum moving forward. All survey questions were reviewed for accuracy and clarity by a physician financial expert, as well as a research and education expert before distribution. After a pilot run to ensure robustness, the survey was distributed nationally to Canadian Radiology residents via program administrators and directors on Google Forms. Descriptive analyses on preliminary data and the association between level of training and financial quiz scores were obtained.

RESULTS

There were 92 valid responses from 15 universities. Respondents’ levels of training were spread across training levels (PGY1=17, PGY2=19, PGY3=24, PGY4=12, PGY5=20). 53% of residents indicated they had above 150K debt level (n=49), yet only 10% of residents (n=9) stated they had formal financial residency curriculum. The average score of residents on the financial quiz was 11/15 and survey quiz scores were not associated with training level (p=0.29). 89% of residents rated 3 or 4 (out of 4) for their interest in a financial literacy curriculum (n=82). 

CONCLUSION

Our findings demonstrate unique insight into the financial literacy and demographics of Canadian Radiology residents; residents face high burdens of debt and majority do not have adequate financial literacy curriculum in their residency programs. Interestingly, residents scored a high average on our financial quiz. Despite this, nearly all residents were interested in a formal financial literacy curriculum to further help their finances. These results indicate that the current resident physicians value a formal financial literacy curriculum as a part of their residency program despite existing financial knowledge. 

Does your group matter? How group function impacts educational outcomes in problem-based learning: a scoping review

Li, Athena; Bilgic, Elif; Keuhl, Amy; Sibbald, Matthew

INTRODUCTION

Problem-based learning (PBL) is a common instructional method in undergraduate health professions training. Group interactions with and within PBL curricula are well known to influence learning outcomes, yet very few studies have synthesized the existing evidence. This scoping review thus summarized the literature examining the influence of group function on PBL outcomes. The aspects of group function were (1) group composition (identities and diversity), (2) group processes (facilitation and dynamics), and (3) PBL processes (tutorial activities). Outcomes were examined at an experiential, academic, and behavioural level using the Kirkpatrick framework.

METHODS

A literature search was conducted using Medline, Cinhal, and AP PsychInfo from 1980-2021, with the help of a librarian. English-language empirical studies and reviews that related group function to learning outcome, as defined, in undergraduate health professions PBL curricula, were included. Relevant references from included articles were also added if the eligibility criteria was met. The methods, results, discussions, and limitations of the sample were summarized narratively.  

RESULTS

The final sample (n=48) varied greatly in context, design, and results. Most studies examined junior medical students (n=31), used a quantitative questionnaire design (n=29), and reported immediate cross-sectional outcomes (n=34). Group composition was the most studied aspect of group function (n=23), followed by group processes (n=18) and PBL processes (n=7). The relationship between group function and outcomes were not consistent across studies. PBL experiences were generally highly-rated, but favourable student experiences were not reliable indicators of better academic outcomes. Conversely, problematic group behaviours were not predictors of poor student experiences or grades. Common confounders of outcome measurements included exam pressure and self-study. 

CONCLUSIONS

The main findings of the review suggested that (1) group function does not reliably predict experiential, academic, and behavioural PBL outcomes, and (2) different Kirkpatrick levels of outcomes are not highly correlated to each other. More research is needed to understand the overarching complexity of group function in PBL tutorials, in order to better inform tutor and student training as well as curriculum design. Standardized tools for measuring PBL group function may be required for more conclusive findings. 

The Systematic Online Academic Resource (SOAR) Review as a novel methodology to curate Free and Open Access Medical Education Resources

Britto, Joanne; Alshenaiber, Abdullah; Cantor, Nathan; Chen, Andrew; Grzela, Patrick; Sadeghian, Shadi; Singh, Devika; Zhang, Cindy; Douketis, James; Chan, Teresa; Mithoowani, Siraj

INTRODUCTION

The Free Open Access Medical Education (FOAM) movement has resulted in an explosion of open access online educational resources. High-quality FOAM resources are difficult to aggregate; they are often decentralized and of variable quality. A Systematic Online Academic Resource (SOAR) review is a novel methodology to curate and filter FOAM resources within a domain. The resulting collection can be integrated into existing or novel curricula. SOAR reviews have been conducted in two other domains; renal/genitourinary, and endocrine, metabolic, and nutritional disorders. We report the use of SOAR methodology on the topic of periprocedural anticoagulation management. 

METHODS

We conducted a search of 3 search engines (FOAMSearch.net, blogsearchengine.org, listennotes.com) and the Top 100 educational websites on a FOAM impact index (Social Media Index). A librarian specializing in searching “grey literature” helped identify search terms, MeSH headers and colloquial synonyms. All freely accessible educational resources on periprocedural anticoagulation management were included. We used the revised METRIQ score to assess the quality of resources and classified them by topic area and by the Four-Component Instructional Design (4C/ID) model.

RESULTS

Of the 5453 screened resources, 63 blog posts, 22 podcasts and 66 point-of-care articles were included for data abstraction. More than 60% of the resources were published after 2016. The target audience was explicitly stated in 10% of resources; 87% were targeted to internists, 75% to surgeons and only 2.6% to family physicians. The majority of resources were “supportive information” (84.8%) or “just-in-time information (15.2%) according to the 4C/ID model. The average rMETRIQ score was 11.9 ± 3.74 of 21; the lowest score was 1, and highest score any resource received was 18. 19.8% of resources (30/151) were identified as high quality (rMETRIQ ≥16). 

CONCLUSION

We systematically identified, described, and curated FOAM resources on the topic of periprocedural anticoagulation management, yielding a list of high-quality resources that can be easily accessed by trainees and educators. Although only 20% of posts were of high quality, this is comparable to SOARs conducted in other domains. Future directions will be to develop generic data collection forms that can be adapted to SOAR reviews of other topics, and to develop methods to keep SOAR reviews current amidst new FOAM publications.

Perspectives on Physician Leadership: Exploring the Role of Character-Based Leadership in Medicine

Torti, Jacqueline; Inayat, Ali; Inayat, Hamza; Sultan, Nabil

INTRODUCTION

Physician leadership is multi-faceted, but leadership training in medicine is not. Leadership training for physicians is rarely grounded in conceptual leadership frameworks and suffer from a primary focus on cognitive leadership domains. Character-based leadership is a conceptual leadership framework that moves beyond cognitive competencies and articulates dimensions of character that promote effective leadership. Although character-based leadership is a well-recognized approach in business, we know very little about how character-based leadership translates to the distinctive space of effective physician leadership. The purpose of this study was to explore character-based leadership in the medical context.

METHODS

This qualitative descriptive study used semi-structured interviews to explore health care professionals’ perceptions of character in relation to effective leadership in medicine. All interviews were audio-recorded, transcribed, and analyzed by the research team. Consistent with descriptive qualitative inquiry, a qualitative latent content analysis was used. A constant comparative approach incorporating character-based leadership as a sensitizing concept was used to code and categorize transcriptions. Researchers met regularly to clarify coding structures and categorize codes until sufficiency was reached.

RESULTS

Twenty-six individuals (twelve doctors, five nurses, four hospital administrative staff, two social workers and two allied health professionals) participated. Character was deemed essential for effective physician leadership. Participants reflected on different dimensions they attributed to an effective physician leader, including collaboration, humility, and humanity. Participants shared examples working in interdisciplinary healthcare teams to illustrate these in practice. Moreover, effective leadership was not dependent on positional leadership roles. Rather members of the interprofessional healthcare team who demonstrate character stand out as leaders regardless of their position or career stage. 

CONCLUSION

Our findings highlight two conceptual shifts for leadership in medicine. The first is expanding traditional cognitive leadership competencies to include character attributes, and the second is a shift from positional to dispositional leadership. These findings provide important insights into character-based leadership in medicine and help lay the foundation for more robust leadership education and training. 

Leadership Development in Postgraduate Medical Education: A Systematic Review of the Literature

Torti, Jacqueline; Sultan, Nabil; Haddara, Wael; Inayat, Ali; Inayat, Hamza; Lingard, Lorelei

INTRODUCTION

Leadership is increasingly becoming a vital competency expected of physicians, and leadership development and training are particularly important for residents in postgraduate medical education (PGME). Medical bodies and institutions have responded to the need for physician leadership by adopting guidelines to encourage leadership development during medical training. While such guidelines reflect a growing awareness of the importance of leadership training in medical education, there is a lack of synthesis of leadership curricula and the extent to which any PGME leadership education is informed by theory. The purpose of this study was to conduct a systematic review of interventions aimed at leadership development in PGME to better understand leadership development in residency training.

METHODS

The authors conducted a systematic review of peer-reviewed, English-language articles from four databases published since 1980. Articles were included if they described interventions aimed at leadership development. They characterized the educational setting, curricular format, learner level, instructor type, pedagogical methods, and conceptual leadership framework, including intervention domain.

RESULTS

Twenty-one articles met inclusion criteria. The classroom was the most common educational setting (n=17). Most curricula (n=13) were not longitudinal (less than one year long) with all curricula ranging from three hours to five years. Leadership curricula were available to residents at various levels of their education. The most common instructor type was clinical faculty (n=13). The most used pedagogical method was small group/discussion, followed by didactic teaching (n=15 and n=14, respectively). Less than half of the studies (n=9) provided an explicit conceptual leadership framework as the basis of the intervention design and most leadership interventions (n=19) taught knowledge and skills related to the cognitive domain.

CONCLUSION

The results reveal that interventions for developing leadership during PGME employ varied curricular formats, lack longitudinal integration, lack grounding in conceptual frameworks, and focus primarily on cognitive leadership domains. Both the lack of theoretical grounding and the narrow focus on cognitive domains are particularly troubling. Medical educators should design future leadership interventions grounded in established conceptual frameworks and pursue a comprehensive approach to leadership training.

Exploring the Role of Organisational Factors in the Emergence of Burnout in Surgery at McMaster University

McNeill, Kestrel; Yang, Ilun; Sonnadara, Ranil

INTRODUCTION
Burnout is a psychological syndrome prevalent across surgical specialties associated with reduced quality of patient care, medical errors, depression, and suicidality. Recent work has emphasized the importance of addressing burnout at a departmental level due to the context-specific and institutional factors associated with this phenomenon. Given the negative sequelae associated with burnout, it is critical that continuing professional development initiatives are developed to address this issue in surgical settings.

METHODS
The goal of this study is to identify the drivers of burnout in the Department of Surgery at McMaster University and implement changes that will mitigate and minimize these drivers through continuing professional development programming. To accomplish these goals, a convergent mixed methods approach embedded within a case-study design will be employed which involves the simultaneous integration of quantitative and qualitative data to explore a context-specific phenomenon. 

ANTICIPATED RESULTS
The Areas of Worklife (AW) model will serve as the theoretical framework for analysis and interpretation in this study. This framework was developed to identify drivers of burnout in an organization that can be translated into stakeholder-informed strategies for change. Surgeons will be recruited to participate in a cross-sectional survey to evaluate the prevalence and drivers of burnout in the department through Maslach’s Burnout Inventory and the AW model’s scale which will be analyzed using multiple regression. Semi-structured interviews will be conducted exploring surgeons’ experiences and strategies to mitigate burnout which will be analyzed using reflexive thematic analysis. The qualitative and quantitative results will be merged to develop a strategic plan for addressing burnout in the department and metrics by which the plan will be evaluated. 

CONCLUSION
Given the prevalence of burnout in surgery, it is critical that system-level interventions be developed in order to promote sustained and meaningful change. Continuing professional development provides the opportunity to address this issue by identifying key drivers that exist at the institutional level. This study will also provide a framework that other institutions may use to investigate context-specific causes of burnout and will lead to the creation of strategies for addressing this issue in surgical settings through continuing professional development.

Clinician responses to cannabis use during pregnancy and lactation: a systematic review and integrative mixed-methods research synthesis

Panday, Janelle; Taneja, Shirpa; Popoola, Anuoluwa; Pack, Rachael; Greyson, Devon; McDonald, Sarah D.; Black, Morgan; Murray-Davis, Beth; Darling, Elizabeth; Vanstone, Meredith

INTRODUCTION

Perinatal cannabis use is increasing, and clinician counselling is an important aspect of reducing the potential harm of cannabis use during pregnancy and lactation. We conducted a systematic review and integrative mixed-methods synthesis to understand current counselling practices and to inform health professional education and practice. We addressed the question “how do perinatal clinicians respond to pregnant and lactating patients who use cannabis?”
METHODS

We searched 6 databases up until 2021-05-31. Eligible studies described the attitudes, perceptions, or beliefs of perinatal clinician about cannabis use during pregnancy or lactation. Eligible clinicians were those whose practice particularly focused on pregnant and postpartum patients. The search had no limitations on study design, geography, or year. We used a convergent integrative analysis method to extract relevant findings for inductive analysis.
RESULTS

Thirteen studies were included; describing perspectives of 1,366 clinicians in 4 countries. We found no unified approach to screening and counselling. Clinicians often cited insufficient evidence around the effects of perinatal cannabis use and lacked confidence in counselling about use. At times, this meant clinicians did not address cannabis use with patients. Most counselled for cessation and there was little recognition of the varied reasons that patients might use cannabis, and an over-reliance on counselling focused on the legal implications of use.
CONCLUSION

Current approaches to responding to cannabis use might result in inadequate counselling. Counselling may be improved through increased education and training which would facilitate conversations to mitigate the potential harm of perinatal cannabis use while recognizing the benefits patients perceive. Health professional education in this area should focus on up-to-date, comprehensive understandings of the evidence about perinatal cannabis use, why patients might use cannabis, and how to facilitate informed decision-making about use. 

Faculty facilitator perspectives on the One Room Schoolhouse: Insights on a novel curricular intervention

Lopes, Jillian; Tokalty, Natalie; Acai, Anita; Snelgrove, Natasha; McCutchen, Ben; Fallen, Robyn; Corey, JoAnn; Harms, Sheila; 

INTRODUCTION

Data widely suggest that academic half-day (AHD) learning lacks educational engagement, which can be amplified in distributed campus settings with remote learning circumstances. Addressing this need led to the development and evaluation of the One Room Schoolhouse (ORS). It was hypothesized that implementing novel pedagogical elements reflective of best practices, such as case-based learning and focusing on learner engagement would result in better test scores as well as improved learner satisfaction. While these positive changes have been observed among residents, there has yet to be an exploration of the curricular design from faculty facilitators. This study aimed to better understand faculty facilitators’ perspectives on the ORS curricular design, their perceptions of resident engagement, and experiences within their facilitator roles.  

METHODS

All faculty facilitators involved in the ORS from McMaster University’s Postgraduate Psychiatry Training Program were invited to participate in semi-structured qualitative interviews. Interviews were transcribed and analyzed using thematic analysis. 

RESULTS

Five of seven eligible faculty facilitators consented to participate. Participants reported that they felt the ORS curriculum fostered increased self-efficacy and meaningful, collaborative engagement with the community at large, as well as simulated a more authentic learning experience reflective of an independently practicing physician. Faculty facilitators also perceived residents to be more engaged within the ORS model, observing increased participation compared to didactic sessions. With respect to facilitators’ perceptions on their role, they noted the challenge of having to adjust to the role of a fellow learner among the residents, as opposed to being subject matter experts, and the importance of adopting a mindset of humility and transparency. 

CONCLUSION

The ORS offers an opportunity for educators to reimagine what medical education might consist of beyond the traditional confines of didactic learning during AHD. Creating opportunities to enhance personal agency, inspiring engagement, and incorporating interdisciplinary learning were identified as strengths of the ORS design. While the ORS has been shown to be a meaningful supplement for postgraduate medical learners, it also contributes to ongoing faculty professional development and the demonstration of a commitment to lifelong learning. 

Can interprofessional education occur virtually for undergraduate healthcare learners? A scoping review

Xi, Cheng En; Bilgic, Elif; Keuhl, Amy; Sibbald, Matthew

INTRODUCTION
With the COVID-19 pandemic restricting in-person education activities, virtual simulations have replaced in-person simulation education activities for health professions students. Converting interprofessional education (IPE) simulations to virtual formats has unique challenges. Effective IPE tends to rely on high-quality interactions between learners of different programs, so they can learn “from, with, and about each other”. It is unclear whether such objectives can be achieved in the virtual environment. A scoping review was conducted to identify the evidence around virtual simulation for IPE. 
METHODS
Comprehensive searches were conducted in OVID Medline with the help of a librarian. Duplicate results were removed, and two independent reviewers conducted title/abstract screening. The resulting articles underwent full-text screening by the primary author and data were extracted from eligible studies using a standardized form. The primary author compiled common themes that emerged in the studies based on the data extracted, such as the way virtual IPE was conducted and the outcomes of such education programs. 
RESULTS
A total of 417 articles underwent title/abstract screening and 45 articles went on to the full-text review. A total of 20 eligible studies were included in this review. The majority of eligible studies took place in the US, and the most common virtual simulation modalities were virtual worlds with interactive avatars (8), video conferencing (4), and interactive case modules (3). All simulations occurred in small groups, involving students from multiple undergraduate healthcare programs. Students found virtual simulations to be engaging and sufficiently met learning objectives. 
CONCLUSION
A variety of virtual platforms can facilitate IPE for undergraduate healthcare students in small groups. Virtual simulations can increase accessibility and standardization for students and educators but can involve significant cost and technical difficulties. More research is needed to compare virtual with in-person simulations focusing on higher-level Kirkpatrick interprofessional education outcome measures.

Virtual Interprofessional Education: Building a Novel Theoretical Model for Interprofessional Learning in the Virtual Space

Azim, Arden; Kocaqi, Etri; Wojkowski, Sarah; Uzelli-Yilmaz, Derya; Foohey, Sarah; Sibbald, Matthew

INTRODUCTION

Virtual interprofessional education (IPE) has emerged as a promising alternative to traditional in-person models of IPE. However, theoretical frameworks to support virtual interprofessional learning are not well elaborated. Two theoretical frameworks emerged as relevant to virtual IPE: (1) the Canadian Interprofessional Health Collaborative (CIHC) interprofessional learning framework and (2) Dornan’s Experience-Based Learning Model (ExBL) of workplace learning. In this study, we sought to explore virtual IPE using both frameworks to develop new theoretical understandings and identify assumptions, gaps and barriers.
METHODS

Seventeen semi-structured interviews were conducted with medical and nursing students (n = 14) and facilitators (n =3) who participated in one of two virtual IPE workshops (“Handover of Care” and “Virtual Code Blue”). Transcripts were analyzed qualitatively in an iterative process, informed by the CIHC framework and Dornan’s ExBL model and using directed content analysis methodology. Emerging themes were explored using novel mind-mapping transitional coding. Data collection and analysis was continued until themes emerged with adequate conceptual depth, relevance and plausibility.
RESULTS

Three themes emerged: (1) emphasis of the personal over the professional, (2) decreased sociologic fidelity, and (3) threatened interpersonal connections. Professional distinctions and hierarchies are blurred virtually. This contributed to an increased sense of psychological safety among most learners and lowered the threshold for participation. Separation from the sociologic complexity of the workplace facilitated role clarification and communication objectives. However, loss of immersion in the virtual setting may limit deeper engagement. Interprofessional objectives that rely on deeper sociological fidelity, such as conflict resolution, may also be threatened. Informal interactions between learners are also hindered, which may threaten organic development of interprofessional relationships.
CONCLUSIONS

Role clarification and communication objectives are preserved in virtual IPE. Educators should pay close attention to psychological safety and sociologic fidelity – both to leverage advantages and guard against threats to connection and transferability. Virtual IPE may be well-suited as a primer to in-person activities, or as scaffolding towards interprofessional practice.

Indirect patient care activities in Family Medicine residency: Optimizing the balance of education and service

O'Toole, Danielle; Sadik, Marina; Inglis, Garbielle; Weresch, Justin; Vanstone, Meredith

INTRODUCTION

In Family Medicine, only about 50% of care is done while directly interacting with a patient. Activities that do not require this face-to-face interaction are known as indirect patient care activities (IPCA), and include documentation, reviewing investigations and filling out forms. While an essential part of patient care, rising rates of IPCA work correspond with increases in physician burnout and job dissatisfaction.  It is not known how best to prepare family medicine residents for this aspect of their career. This study aimed to understand how to optimize IPCA work during residency to best prepare residents for future practice.

METHODS

Using Constructivist Grounded Theory, focus groups and individual interviews were conducted with 42 clinicians (19 family medicine residents, 16 family physicians in the first five years of practice, and 7 family physician educators). All participants were connected to a single residency program. Interview data were analyzed iteratively, using a staged approach to constant comparative analysis. 

RESULTS

While residents, early career physicians, and educators perceived the implications of IPCAs differently, they all reported IPCAs as a necessary weight that family physicians carry throughout their career. Some residents described IPCAs as a burden, creating inequities in workload and interfering with other learning and personal opportunities. In contrast, IPCA work in residency was conceptualized by educators as an opportunity to build strength by developing the skills required to carry the weight of IPCAs throughout the physician’s career. Specific recommendations for helping residents recognize this educational opportunity related to increasing clarity, equity, purpose, and consistency of this type of work.

CONCLUSION

All participants described IPCAs as a necessary, though often unfulfilling part of family medicine practice. Independent physicians emphasized the need to use residency to develop competency and strategies for managing IPCAs in a sustainable way for long-term practice. IPCAs are a key competency for family medicine residents, but require explicit pedagogical attention. If the educational opportunities are not made explicit, residents may miss the opportunity to develop strategies for practice management, professional boundaries, and administrative efficiencies.

Interprofessional education readiness among health science learners: 3-year cohort

Quach, Shirley; Oliveira, Ana; Akhtar-Danesh, Noori; Brewer-Deluce, Danielle; Bondy, Linda; Wojkowski, Sarah

INTRODUCTION

Interprofessional collaboration is an essential component of patient-centered care and interprofessional education (IPE) training is mandatory across Canadian healthcare programs. Positive IPE experiences may be influenced by students’ pre-existing attitudes and perceptions about learning with students from other programs. However, this has been explored in small cohorts, without considering yearly differences in students’ perceptions. This study’s focus was to evaluate IPE readiness in a subgroup of incoming first year students in the Faculty of Health Sciences (FHS), McMaster University from 2019 to 2021.

METHOD

First year students in the FHS (e.g., health professions and bachelor of health sciences) were invited and randomly allocated to complete the Readiness for Interprofessional Learning Scale (RIPLS) using Q-methodology or a 5-point Likert scale. This study reported the results from the Q-methodology. Students ordered the 19 statements of the RIPLS on a Q-sort table with a scale ranging from -3 (strongly disagree) to +3 (strongly agree). 
Participants also provided written feedback to justify statements ranked -3 or +3. Data was analyzed using STATA and by-person factor analysis was used to identify factors (i.e., groups of students similarly rated Q-sorts). The research team subjectively labelled factors by statements that were loaded onto each factor.

Participant characteristics (i.e., age, sex, program, and degree) for each factor were summarized and compared using Chi-square tests and ANOVA.

​RESULTS

A total of 785 students were invited to complete the RIPLS Q-sort and 253 (32%) responded. From those, 198 students (25%; mean age 21.2 [SD=3.8] years) loaded into three factors: Personal learning (n=85); Delivery of healthcare (n=65) and Collaborative learning (n=48). Participants loaded in factor 2 (mean age= 19.7 [SD=3.0] years) were younger than those in factor 1 and 3 (mean age= 21.7 [4 .0]years, p=0.006; mean age= 22.4 [3.8] years, p=0.001). There were statistically significant differences across factors for program (professional vs. general; p=0.01), degree (undergraduate vs. graduate; p=0.01), but no differences between factors for gender (p=0.59).

From 2019 to 2021, incoming FHS students were sorted into 3 factors, each with its own IPE value.

CONCLUSION

Differences in IPE value may be related to the students’ age, program, and degree. These results provide an impression of incoming students’ perception of IPE and considerations for IPE training development.

Comparing Efficacy of an Interprofessional Education Dissection Elective for Nursing Students versus Other Healthcare Students

Sonha, Sakshi; Todd, Alexandra; Palombella, Andrew; Rockarts, Jasmine; Wainman, Bruce C

INTRDUCTION

Interprofessional education (IPE) allows healthcare students to collaborate in hopes of preparing students for their future collaborative workplace environment. Two tools are often used to measure healthcare students’ perception and acceptance of IPE: Interdisciplinary Education Perception Scale (IEPS) and Readiness for Interprofessional Learning Scale (RIPLS). IPE courses emphasize the importance of interprofessional cooperation and improve mutual respect and understanding. The role of a nurse is broad and far-reaching, and spans from providing care for patients to support for the family members. As such, it is important to understand the effect of an IPE course on the attitudes and perceptions of nurses towards a collaborative healthcare approach.

METHODS

Nursing students participated in an interprofessional dissection elective course, which was offered for three hours once a week over two months. Clinical scenarios were discussed, and specific cadaveric dissections were carried out in mixed teams of nursing, medical, midwifery, occupational therapy, physician assistant, physiotherapy, and speech language pathology students. IPE perceptions were recorded, using IEPS and RIPLS, before and after taking the course. A 12-item IEPS was used, which is split into three categories: Perception of Actual Cooperation, Perceived Need for Cooperation, and Competency and Autonomy. A modified 18-item RIPLS was used, which is split into four categories: Teamwork and Collaboration, Negative Professional Identify, Positive Professional Identity, and Roles and Responsibilities. IEPS and RIPLS data for the past 11 years (2011-2021) was analyzed.

RESULTS

A paired samples t-test was conducted. On the IEPS, nursing students had a statistically significant increase in their total scores, and the Perception of Actual Cooperation and Competency and Autonomy subscores. On the RIPLS, nursing students had a statistically significant increase in Teamwork and Collaboration and Positive Professional Identity subscores, while Negative Professional Identity and Roles and Responsibilities subscores significantly decreased after taking an IPE course.

CONCLUSION

IPE courses have a significant positive effect on the attitudes and behaviours of nurses towards an allied health approach. Complete data analysis in April 2022 will provide a clearer picture of the efficacy of IPE courses for nursing students compared to other healthcare students.

How trainees use entrustable professional activities for learning: A cross-center comparison

Martin, Leslie; Johnston, Bronte; Blissett, Sarah; Gauthier, Stephen; Gilchrist, Tristen; Madan, Sunchit; Hatala, Rose; Zhang, Lindsey; Sibbald, Matthew; 

INTRODUCTION

Canadian residency training has shifted to a competency based medical education (CBME) model where entrustable professional activities (EPAs) have been implemented with the goal of achieving assessment-for-learning in the workplace. Previous work demonstrates several tensions associated with EPAs which threaten this goal, such as trainee orientation around performance versus learning. Informed by existing workplace learning theory and the role of learning cues, this study seeks to understand if, when and how EPAs facilitate workplace learning across different sociocultural environments. 
METHODS

Trainees from all postgraduate training programs in four institutions (McMaster University, the University of British Columbia (UBC), Western University, and Queens University) were invited to participate focus groups. Inclusion criteria required trainees to be in a CBME model for a minimum of two years. We anticipate reaching theoretical sufficiency after twelve focus groups given our narrow research question and tight link to theory, yet diverse participant sample. An iterative, deductive analysis is being conducted concurrently with data collection, sensitized by learning theory, including Watling’s learning cues and Schumacher’s model for developing the master learner.  
RESULTS

At the time of submission, five focus groups are complete. The research team has iteratively updated the focus group guide, and coding is underway. Sample initial codes include learning cues, inhibitors of learning, EPA implementation, feedback, EPA design, reflection, and professional identity formation. We anticipate having near-completed our data collection and analysis at the time of presentation.  
CONCLUSION

EPAs were designed as an outcomes-based assessment blueprint within CBME, and merit ongoing research to understand if or how they facilitate learning in the workplace. Examining this question through the lens of learning cues provides an opportunity to understand how EPAs fit with our existing understanding of learning through clinical work.

Provider and Patient Experiences of Health Inequity in Primary Care: Navigating the Entwinement of Social and Medical Needs

Molinaro, Monica; Vanstone, Meredith; Inglis, Gabrielle; Agarwal, Gina

INTRODUCTION

The COVID-19 pandemic has brought into sharp relief the relationship between health outcomes and social inequities experienced by Canadians, especially those with disabilities and/or who are Black, Indigenous, People of Colour, poor, migrant workers, and newcomers to Canada. Simultaneously, public and scholarly attention has focused on the moral distress experienced by health care providers. This presentation describes an ongoing program of research that engages with these conversations to investigate two central aims. First, primary care providers’ (PCPs’) stories of moral distress in caring for patients experiencing health needs related to social inequities will be analyzed, and this presentation shares preliminary findings from this data. Subsequently, patients’ narratives of experiencing social inequities and seeking primary care will be elicited.

METHOD

The intent of critical narrative inquiry is to generate rich narratives that illuminate the ambiguities that characterize complex experiences. Informed by moral distress, this study will be attuned to how broader contexts and discourses are inflected, reproduced, and challenged by participants’ narratives. Participants will engage in two narrative interviews, allowing them the freedom to narrate their experiences of moral distress and social inequity. Rigorous narrative analyses will occur concurrently and iteratively with data collection through holistic readings followed by analysis of distinct phrasing and themes.

RESULTS

At this point in time, twenty PCPs with varied experiences have been recruited and participated in one narrative interview. Their interviews have highlighted their experiences navigating the health care system with their patients, the tensions they experience in their identities as agents of the health care system who have beliefs that counter those of the health care system, the meanings they ascribe to providing care, and the incongruence perceived between their medical education and practices.

CONCLUSION

The stories of the PCPs suggest several implications for education, practice, and policy. Specifically, their stories draw attention to how elements of collaborative approaches to care benefit patients, how particular remuneration programs de-incentivize caring for patients experiencing inequity, how medical education can address inequity, and how the way our society, governments, and health care system can change how we see patients experiencing inequity.

Remote Education Strategies Training Oncology Residents for End-of-Life Discussions (The RESTORED study)

Levine, Oren; Jekic, Danica; Steinberg, Lean; Myers, Jeff; Incardona, Nadia; Seow, Hsien

INTRODUCTION

High quality communication in oncology care benefits patients, families, and clinicians. Within oncology residency programs, communication skills training (CST) is limited. The adoption of virtual care in the context of the COVID19 pandemic has further limited opportunities for evaluation of communication skills among learners. How best to teach communication skills in the context of virtual care is unclear. A formal CST curriculum could help to standardize teaching and evaluation, and remote learning will mitigate current challenges in the pandemic. We will adapt two recognized educational tools, electronic learning modules (ELMs) and standardized patients (SPs), to create a novel virtual CST program. We will explore the relative impact of each educational resource through a randomized trial. In this initial study, we define the content of the novel curriculum through consensus methods.

METHODS

We conducted a Modified Delphi Consensus process to define content for a CST curriculum focusing on difficult conversations in oncology and end of life care. An initial proposal was informed by literature review, training objectives from the Royal College, validated CST resources, and best practice in physician-patient communication. Experts were consulted for content validation. A stakeholder panel was convened including clinician educators across different Canadian institutions from oncology and palliative care specialties, and psychosocial oncology. Patient and family advisors and trainees were also recruited. Stakeholders rated item relevance on a 7-point scale through 2 iterative surveys and provided explanatory comments. Consensus was achieved for items scoring 6-7 among 80% of respondents. Non-consensus items were analyzed, revised according to thematic analysis of comments and returned to the panel for second review along with new items.

RESULTS

The initial proposal included 34 items across 3 domain for curriculum content: types of conversations, conversation structure, and general skills/behaviours. Expert review led to revised language, and 8 new items within a 4th domain (additional considerations). 22 stakeholders joined the panel for consensus development and iteratively rated item relevance in 2 rounds. Response rate was 100% for both rounds. In the first round, 33 items reached consensus. 9 items did not reach consensus and were modified according to comments, and 4 new items were generated. Thus, 13 items went on to the second round. Only 5 items did not reach consensus in the second round and were not included in the final CST curriculum. Reasons for non-consensus were different ideologies, different experiences, and relevance-ratings reflecting different contexts.

CONCLUSION

Through consensus methods, content is defined for a novel CST curriculum in oncology. This serves as the foundation for developing the ELMs and SP resources for our pilot randomized trial.

Transforming the Ontario Midwifery Education Program through social justice curriculum: a modified Delphi study

Ramlogan-Salanga, Claire; Campbell, Audrey; Grierson, Lawrence; Darling, Elizabeth K

The study is using a modified Delphi approach, which includes focus groups and interviews with subject matter experts, two rounds of surveys with key informants, and a final member checking session. Participants were recruited through both purposeful and snowball sampling. The Delphi methodology involves an iterative data analysis process, therefore upon completion of each focus group and round of surveys, analysis has been conducted and used to form the following rounds of the Delphi.  Qualitative data has undergone primary coding, secondary coding, and then theme formation.  Quantitative data has been gathered using REDCap and analyzed using descriptive statistics. Based on the social justice elements identified by participants, development intended learning outcomes will be compared to the national midwifery competencies to determine alignment. Recommendations to the Ontario Midwifery Education Program (OMEP) will be reviewed at a final member checking session. The future implication of this study is the revision of the Midwifery Education Programs curriculum and to provide an example for other Canadian MEPs. The broader implications include integration of social justice curriculum by other Canadian MEPs and other health professions education programs.

Learning to Discuss Goals of Care in Internal Medicine: A Replication Study

Lee, Claire; Martin, Leslie; Brandt Vegas, Daniel

INTRODUCTION

Establishing patients’ goals of care (GOC) is a critical component of patient-centred care. The task of discussing GOC with patients often falls to Internal Medicine (IM) physicians. Studies have shown poor performance of IM trainees and attending physicians in standardized patient scenarios involving GOC discussions. It is unclear how IM clinicians develop their GOC discussion skills, nor what would best support them in their training. An unpublished study from McGill University found that clinicians developed their conceptualization and approach to GOC conversations over many years, suggesting that GOC educational curricula should also be longitudinal. This study aims to expand on the previous findings
by looking at how IM trainees and attending physicians at McMaster University develop or refine their GOC discussion skills, and what factors support or impede this process.

METHODS

The authors performed a qualitative study using one-on-one interviews with core IM trainees and attending physicians, using a semi-structured interview guide to explore their experiences with developing GOC discussion skills. Interviews were audio recorded and transcribed, and then analyzed by the authors using applied thematic analysis.
PRELIMINARY RESULTS

Trainees experienced cognitive dissonance between how they understood GOC should be discussed and the circumstances in which they felt external pressure to discuss GOC (on call, in noisy settings, lacking privacy). Suboptimal conditions contributed to limiting these conversations to a simple confirmation of the patient’s code status. They described challenges improving their GOC discussion skills, citing a lack of formal training, as well as a paucity of opportunities for direct observation. Trainees felt unprepared to provide specific recommendations to patients regarding their treatment trajectory. On the other hand, attending physicians were more comfortable with being directive in providing patient-centred guidance. Both trainees and attending physicians alike felt that there would be value in observing and/or practicing discussing GOC in challenging scenarios.
PRLEIMINARY CONCLUSIONS

IM trainees experience challenges developing GOC discussion skills due to receiving minimal feedback or formal educational supports. Attending physicians seem to gain experience and confidence in this skill over years. A formal curriculum may help provide trainees increased practice and skills over a shorter timeframe.

Through the Eyes of the Beholder: What CanMEDS roles do clinician teachers see when interpreting EPAs?

LoGiudice, Andrew; Sibbald, Andrew; Monteiro, Sandra; Sherbino, Jonathan; Norman, Geoffrey R.; Chan, Teresa

INTRODUCTION

The CanMEDS framework guides postgraduate medical education in Canada by representing the broader needs of the public. Nonetheless, the CanMEDS roles are now less prominent following a recent shift to Entrustable Professional Activities (EPAs) as cornerstones of assessment. Therefore, to evaluate whether these new assessment standards still capture society’s needs, we explored how well clinician teachers share an understanding of the roles when interpreting specific EPAs. 

METHOD

We conducted a cross-sectional study of 3 specialties (adult cardiology, emergency medicine, anesthesia) to measure how consistently clinician teachers mapped roles onto EPAs. Clinician teachers received a set of EPAs from their specialty via an online survey, and for each EPA selected which roles they thought informed the EPA. We then used these responses to measure agreement with classifications found in formal EPA guides (% matching responses) and agreement among participants (inter-rater reliability via intraclass correlation; ICC). 

RESULTS

Thirty-eight clinician teachers participated with no attrition. Overall, across roles and specialties, classifications of EPA by role matched formal guides approximately 62% of the time, with Communicator (75%; SEM = 2.6) and Collaborator (69%; SEM = 2.3) showing the greatest agreement. Inter-rater reliability was poor across all included roles and specialties (Cardiology ICC = 0.28; Emergency Medicine ICC = 0.24; Anesthesia ICC = 0.17). 

DISCUSSION

Our findings suggest frontline clinician teachers in Canada often “see” different roles when interpreting EPAs, and thus that this new EPA-based assessment system does not easily map onto the pre-existing CanMEDS framework. 

Does Allowing Access to Electronic Differential Diagnosis Support Threaten the Validity of a Licensing Exam?

Sibbald, Matthew; Pugh, Debra; Sherbino, Jonathan; Morin, Maxim; Norman, Geoffrey R.; Monteiro, Sandra

INTRODUCTION
Newer electronic differential diagnosis supports (EDS) are efficient and effective at improving diagnostic skill.  While these supports are encouraged in practice, they are prohibited in medical licensing examinations. The purpose of this study is to determine how using an electronic differential diagnosis support (EDS) impacts examinees’ results when answering clinical diagnosis questions.

METHOD
The authors recruited 100 medical students from McMaster University (Hamilton, Ontario) to answer 40 clinical diagnosis questions in a simulated examination in 2021.  Of these, 50 were first year students and 50 were final year students. Participants from each year of study were randomized to one of two groups. During the survey, half of the students had access to Isabel (an EDS) and half did not. Differences were explored using ANOVA and reliability estimates were compared for each group.

RESULTS
Test scores were higher for final year versus first year students (53土 13% versus 29土10, F=124.9 p<0.001) and higher with the use of EDS (44 土 28% versus 36 土 26%, F=11.6 p<0.001). Students using the EDS took longer to complete the test (p<.001). Reliability (Cronbach’s alpha) increased with EDS use among final year students, but was reduced among first year students, although the effect was not significant. 

CONCLUSION
EDS use during diagnostic licensing style questions was associated with modest improvements in performance, increased reliability in senior students and increased testing time. Given that clinicians have access to EDS in routine clinical practice, allowing EDS use for diagnostic questions would maintain ecologic validity of testing while preserving important psychometric test characteristics.

The value of group interaction in diagnostic reasoning?

Sherbino, Jonathan; Sibbald, Matthew; Monteiro, Sandra; LoGiudice, Andrew; Lee, Mark; Keuhl, Amy; Norman, Geoff

INTRODUCTION
Involvement of multiple clinicians has been proposed to reduce diagnostic error using two broad strategies– interactive groups, where active discussion occurs, and nominal groups, where independent individual judgments are combined statistically. The two processes are not necessarily equivalent. 

Objective
The objective of the study was to compare the diagnostic accuracy of nominal and interactive groups of different sizes (3 and 6).
METHOD

Sample: 36 internal medicine and emergency medicine residents from McMaster University. 

Cases: 12 written general medicine cases, A to L, were administered using the design shown below.

Resident                  Group Size
                   1                   3                  6    
1-12        ABCD        EFGH        IJKL
13-24        EFGH        IJKL        ABCD
25-36        IJKL        ABCD        EFGH
    
Initially, participants individually reviewed 4 cases online and provided a differential diagnosis. Residents then formed online groups of 3 and reviewed 4 new cases, and formed a group consensus. Finally, they were combined into groups of 6 and the consensus process repeated with 4 new cases. Later, nominal groups were created by combining the individual records into groups of 3 and 6. 

Outcome: Diagnostic accuracy was scored as 0,1 or 2 for each case. Analysis was conducted on a) number of unique hypotheses, b) accuracy of primary diagnosis, c) accuracy of differential diagnosis. Nominal groups were scored by presence of correct diagnosis in one or more records.

RESULTS
As shown in the Table, nominal groups considered more hypotheses than interactive groups. (p<.0001). The accuracy of primary and differential diagnosis was similar for nominal and interactive groups and for group size 3 vs. 6.

                   No. Hypotheses           Primary Dx (/2)            Diff Dx (/2)
Sample size        3            6               3            6                3            6            

Nominal        7.23    12.5        1.10    1.08        1.46    1.46
Interactive        2.81    2.67        1.15    1.15        1.31    1.42                

CONCLUSIONS
The study showed that accuracy was no greater for groups of 6 than 3. There was no benefit of group discussion. 

Are two (or more) heads better than one?

Sibbald, Matthew; Sherbino, Jonathan; Monteiro, Sandra; LoGiudice, Andrew; Lee, Mark; Keuhl, Amy; Norman, Geoffrey

INTRODUCTION
Involvement of multiple clinicians in diagnosis has been proposed to reduce diagnostic error using two broad strategies– interactive groups, where active discussion occurs, and nominal groups, where individual judgments are retrospectively combined statistically. While in practice interactive discussion is likely the primary mode, research is dominated by studies of nominal groups based on mammogram or ECG. We found only one study with interactive groups and clinical cases, but this was limited to pairs of medical students. How (or if) more expert clinicians benefit from group interactions, and the relation to group size, has not been studied.
OBJECTIVE
The objective of the present study was to compare the diagnostic accuracy of individual clinicians to groups of size 1, 3 and 6.

METHOD

Sample: 36 internal medicine and emergency medicine residents from McMaster University were recruited as participants. 

Cases: 12 general medicine written case vignettes were used. Initially, participants individually reviewed 4 case vignettes and provided a diagnosis and differential.. Residents were then formed into Zoom groups of 3 (4 groups), given 4 new cases and asked to come to a consensus differential diagnosis. Finally, they were further combined into 2 groups of 6 and again given 4 new cases. 

Outcome: For each individual and group, diagnostic accuracy was scored as 0,1 or 2 for primary diagnosis and differential diagnosis. Analysis looked at accuracy by group size using ANOVA.

RESULTS
Average accuracy of primary diagnosis was 0.71 for individuals, 1.10 for n=3 and 1.08 for n=6.(F= 3.85. p=.05) For differential diagnosis, accuracy was 0.95, 1.46 and 1.46 respectively, (F = 8.25, p=.004). 

CONCLUSIONS
Working in groups substantially increased accuracy. However, increasing group size from 3 to 6 resulted in no advantage. Eliciting diagnoses from groups may reduce diagnostic error. 

An Innovative method of Learner and Faculty Interaction in Teaching Evidence-Based Medicine

Thrall, Sam; Chambers, Larry 

INTRODUCTION
Evidence-Based Medicine (EBM) enables clinicians to combine evidence with clinical expertise and patient values. In the Niagara Regional Campus (NRC), accessing research faculty to EBM skills can be difficult. 

McMaster’s MD Program includes asynchronous recordings of EBM lectures online and one three-hour interactive session. The NRC faculty organized optional, virtual, synchronous “EBM Primer Sessions” with learner/faculty interaction.

We will report the content, process, and feedback about a session on “What is involved in making an evidence-based decision about a treatment?” 

METHODS
Session earning objectives: apply CONSORT checklist to assess methodological quality of reporting of RCTs; describe role of RCTs in the hierarchy of evidence about treatment effectiveness; describe what makes evidence of treatment effectiveness less certain; and describe importance of PICO to determine internal and external validity of trials.

The following education design principles outlined by Green and Fankish (1994) were used to create the learning resources and organization of the session: individualization, relevance, feedback, reinforcement, and facilitation. 

The session comprised of pre-and post session questionnaires, presentation of slides, ‘chats’ throughout session, question pauses, and student/presenter interactions.

RESULTS
Five of seven students who attended the session completed the questionnaires. All reported following summaries of evidence about treatments using “UpToDate”. 

Students reported an improvement in their knowledge related to the learning objective: ‘apply CONSORT checklist to assess methodological quality of reporting of RCTs’ (p < 0.04).

Three students reported confidence the session achieved the four learning objectives and four reported session satisfaction. 

CONCLUSIONS
A recording of the virtual session will be made available to all 82 students to determine uptake and impact.

In four other EBM sessions, student attendance was less than 10 of 82 NRC students. Steps to increase session attendance included: confirming the session did not overlap with other student events in the MD Education Program, a revised advertising poster with pictures and bios of the presenters, list of the learning objectives and clinical vignette of a treatment decision-making dilemma. 

Future work will be to create learning formats meeting student learning needs and adding value to the demanding MD Program curriculum. 

Evidence of Effectiveness of a Spiral Curriculum

McAssey, Karen; Whyte, Rob; Profetto, Jason; Lee-Poy, Michael; Rudkowski, Jill; Grierson, Lawrence; Sibbald, Matthew; Norman, Geoffrey R

INTRODUCTION
The idea of a spiral curriculum, where individual domains are revisited has been a subject of curriculum discussions for about 5 decades (Harden, 1973). However, evidence of effectiveness is weak (Kulasegaram K, 2018).  Following a curriculum reform at McMaster in 2019, an opportunity arose to evaluate one component of the curriculum, an “Integration (Synthesis) Foundation” which occurred at the end of the first academic year and was designed to review concepts that had been previously covered in organ system modules in the context of complex, multi-system diseases and chronic illness.

To assess the effectiveness of this module, students in four classes, 2 before the introduction of the Integration Foundation and 2 after, were assessed using a Progress Test, a 180-item multiple choice test drawn from all of medicine and administered to all classes 3 times a year, in use since 1992. The study hypothesis was that the Integration Foundation would result in an increase in performance from the June to September administrations compared to the control classes without the module.

METHODS
Sample: Four undergraduate classes, graduating in 2020, 2021, 2022 and 2023, with class size from 202 to 204. 

Outcome: Performance on the Progress Test in June of Year 1 and September of Year 2.

Analysis: Repeated measures ANOVA on overall scores, with 3 variables – intervention/control, pre/post, and class (2020/2021 and 2022/2023).
RESULTS
Control classes improved from 36.0 to 39.2 and 36.1 to 38.3 (Class of 2020 and 2021). By contrast intervention groups (Class of 2022 and 2023) improved from 37.2 to 48.9 and 38.4 to 49.2. All SEs were between 0.54 and 0.56. Mean difference in change between intervention and control was 8.6, F = 112.7, p<.001. ES =  1.1.

CONCLUSIONS
The 12 week Integration Foundation led to a large and significant increase in knowledge replicated across two classes.

A Survey-Based Study of Physician Experiences in Clinical Fellowship Programs

Kottaras, Steven; Acai, Anita; Pike, Rebecca; MacNay, Ramsay; Wang, Ding; Al-Sawafi, Aza; Brotea, Alina; 

INTRODUCTION

Clinical fellowship programs offer physicians the opportunity to gain a deeper level of clinical competence via exposure to clinically based learning, research, and scholarly activities that are typically not covered to the required extent during residency training. Such learning experiences typically occur over the course of a one- or two-year term and are offered by many departments in the Faculty of Health Sciences. The purpose of the present study was to provide a comprehensive look into the unique experiences of clinical fellows at McMaster University and to isolate strengths and areas for improvement. METHODS

Survey questions were adapted from the Postgraduate Medical Education Office’s Resident Satisfaction Survey. The survey consisted of 66 open- and closed-ended questions related to demographics, prior education, the fellows’ quality of work-life, faculty, burnout, the impact of COVID-19, mistreatment, professional goals, CanMEDS roles, and other related topics. The survey was distributed to all clinical fellows at McMaster University in October 2021 and kept open until December 2021. Clinical fellows from all programs were encouraged to complete the survey. RESULTS

Overall, 58 clinical fellows within the Faculty of Health Sciences completed the full survey with 84 participants responding to at least one question. Strengths included having supportive faculty, excellent learning experiences, and the development of collaborative relationships with colleagues from diverse specialties. Key areas in need of improvement included providing additional funding, personal work/leisure spaces within the clinical setting, work schedules that allow for a better balance between clinical and non-clinical duties, and protected time for self-study and formal learning. Work was a major contributor to burnout for some clinical fellows, and the COVID-19 pandemic was reported to have hurt some aspects of training, such as assessment experiences.

CONCLUSION

Fellowship programs at McMaster University offer a unique learning environment and are recommended by most clinical fellows. Increased funding, dedicated space and better structuring of fellows’ time warrant further consideration. A follow-up survey could examine if, in fact, the fellowship program adequately prepared fellows for independent practice in their chosen area of clinical competence. Future studies could also examine the experiences of clinical fellowship programs elsewhere. 

From Empathy to Impact: Evaluating a Design Thinking Innovation in Online Health Professions Education using the Kirkpatrick Framework

Nasser, Muhammadhasan; Bayer, Ilana

INTRODUCTION

Design thinking enables educators to empathize with the genuine needs of learners and engage them in constructing and implementing innovations to enhance learning experiences. This two-step project evaluated a design thinking innovation in asynchronous online discussion (AOD) within the context of online, graduate health professions education using Kirkpatrick’s Four Levels (KFL). Specifically, KFL was chosen as it is a commonly used and well-known program evaluation model which conceptually aligned with the principles of testing in design thinking. 
METHODS

KFL guided a sequential mixed-methods evaluation. In step one, three online surveys were distributed via email to learners in a health professions education graduate course before, during, and after the innovation implementation. Likert scales were analyzed through descriptive statistics and open-ended responses analyzed through open and axial coding with deductive thematic analysis informed by Levels 1 to 3 of KFL along with sub-outcomes based on findings from earlier design thinking stages. In step two, survey data informed semi-structured interviews with learners to gain a deeper understanding of experiences, learning, and behaviours associated with the innovation (KFL Levels 1-3). Interviews were similarly coded and analyzed using deductive thematic analysis. 
RESULTS

Survey results demonstrated mixed reactions and experiences (Level 1) with the AOD innovation; factors impacting reactions were alignment with learning outcomes and personal relevance, engagement, and usability. In terms of respondent learning (Level 2), value of AOD to learning increased and was seen as valuable to learning insofar as it exposed learners to different perspectives relevant to their experiences, but value began to deteriorate with increasingly long posting behaviour. Learner behaviour (Level 3) involved timing, modality, length, and content of AOD posting, which generally remained unchanged for respondents. Preliminary interview results revealed the importance of motivation, alignment with learning outcomes, and pre-existing behaviour and peer connections. 
CONCLUSION

This project highlights the use of KFL to evaluate a design thinking education innovation in online health professions education. Overall, the evaluation revealed that the design thinking innovation in AOD had mixed impact on learner reactions, increased the learning value of AOD, and had no demonstrable impact on learner behaviour. 

Evaluation of a Blended Learning Electroconvulsive Therapy Curriculum for Psychiatry Residents to Treat Depression in Older Adults

Owais, Sawayra; Acai, Anita; Payne, Sarah; Brown, Michael; Levinson, Anthony J.; Saperson, Karen; 

INTRODUCTION

Electroconvulsive therapy (ECT) is a highly effective treatment for major depressive disorder, including in older adults. Despite its effectiveness and standardization of training guidelines, ECT remains heavily underused. Few Canadian psychiatry residents feel competent to administer ECT and up to 60% of Canadian healthcare centres that administer ECT do not have formal ECT training for their psychiatry residents. Blended learning, which combines online and face-to-face learning, may have the potential to optimize ECT training for psychiatry residents. The purpose of this study was to evaluate the online component of a blended curriculum for general psychiatry learners on the use of ECT to treat depression in older adults with or without a cognitive disorder.
METHODS

Second- and third-year psychiatry residents from McMaster University completed a blended learning curriculum during their core geriatric psychiatry rotation. The curriculum consisted of didactic seminars, hands-on clinical management, and two online clinical cases focused on the management of late-life depression with ECT. Learner satisfaction, clinical relevance, and instructional design were measured through a self-report questionnaire adapted from the Medical E-Learning Evaluation Survey (MEES). Knowledge acquisition following the online module completion was measured using a nine-question multiple-choice test. 
RESULTS

A total of 24 participants completed both online modules. Of these, 10 (42%) completed the adapted MEES, and 21 (88%) completed the knowledge test. All participants agreed or strongly agreed that the modules were relevant to their clinical work, evidence-based, able to be completed in a reasonable amount of time, and a valuable learning experience. All participants stated that they would recommend the modules to a colleague. The average score on the knowledge test, after removing one outlier, was 81%.
CONCLUSIONS

Psychiatry residents were very satisfied with the content and delivery of the online component of a blended curriculum for understanding the use of ECT for late-life depression. Future work should examine satisfaction with the remainder of the curricula as well as the impact on longer-term knowledge acquisition and patient care.

Everchanging Perceptions: A Q-method Analysis of the Transition from Online to In Person Learning in Undergraduate Anatomy

Metla, Sai Gayathri; Akhtar-Danesh, Noori; Saini, Jessica; Bayer, Ilana; Wainman, Bruce C.; Brewer-Deluce, Danielle

INTRODUCTION

As a result of the Covid-19 pandemic, anatomy education has had to adopt online modes of delivery. Previous research conducted on student views towards an online undergraduate anatomy and physiology course revealed areas of both strong preference (e.g., asynchronous lectures) and strong dislike (e.g., virtual specimens) within the online format. As restrictions are lessened and students transition back into in-person learning, there exists the unique opportunity to examine the views of a consistent cohort of students towards virtual and in-person modes of course delivery. 
Objective: The current study seeks to compare the views of a single cohort of students experiencing both the online and in-person anatomy and physiology course. To avoid the considerable drawbacks of course evaluations relying on Likert scales and traditional qualitative analysis, Q-methodology is being used to measure the different perspectives of students between the two teaching modalities.
METHOD

Q-methodology was used to assess opinions of students enrolled in the same undergraduate introductory anatomy and physiology course. A list of 41 opinion-based statements regarding anatomy education was compiled. Students sorted the statements in a quasi-normal grid based on their degree of agreement with the statements. The rankings underwent a by-person factor analysis which categorized students with shared perceptions into groups allowing educators to better understand and respond to the needs of students. 
RESULTS

Data was collected from 243 students in the primarily online fall semester and 189 students in the primarily in person winter semester. By-person factor analysis revealed three distinct subgroups in the cohort. Factor one (%n = 55.6 fall, 59.4 winter) was overall satisfied with the course materials and delivery. Factor two (%n = 29.5 fall, 10.9 winter) had a deep dislike of online learning. Factor three (%n = 14.8 fall, 29.7 winter) had a strong preference for online learning.
CONCLUSION

While many students were comfortable in both online and in-person learning environments that was not the case for all learners. In addition, more students preferred in person learning during an online semester, but an increase in appreciation for online learning was observed following the transition to in person learning during the winter semester. Modifying course elements based on Q-methodology results can bring measurable change in students’ experience.