The Challenge
I was hired as an Instructional Design Specialist to work on a project titled “Accelerating the Adoption of Artificial Intelligence (AI) in Healthcare”. The project was a collaboration between the Michener Institute, the University Health Network, and the Vector Institute. It was funded by the Government of Canada’s Future Skills Centre and focused on preparing the healthcare community for AI-enabled care by transforming their mindset, skillset, and toolset through research projects, symposia, roundtables, certificate programs, and a mentorship component.
​
As an Instructional Design Specialist, my main role was to lead the deliverables relating to certificate programs and a mentorship program. Secondary to this, I help with the engagement activities, including symposia and partnership round tables, and with the overall research component that aims to better understand what the healthcare community needs to accelerate AI.
The Solution
The documents the research team initially shared with me included their knowledge transfer protocol that outlined the research plan for the entire project, including evaluation activities. They were open to my input and help with the evaluation plan and tools. Similar to the needs assessment, they were happy to have my help with interviews and data analysis to gain a deeper understanding of their findings and to simultaneously prepare re-design plans and presentations.
​
During our initial meeting, we discussed how to involve our two patient partners in evaluation activities. We decided to incorporate their feedback at the different stages of program development such as reviewing objectives and program outlines, reviewing patient-related curriculum, co-creating patient-related curriculum, recording videos, being guest speakers, and auditing the program. We also decided to incorporate:
-
Surveys and interviews for program participants
-
Reflections, debriefing meetings, and interviews with the program faculty
-
Reflections and interviews with our patient auditors
-
Reflections and debriefing meetings with the team.
​
I was able to draft the interview scripts to ensure program priorities and key needs assessment questions were captured as well as research focuses. I also adjusted and aligned questions to the inclusion, diversity, equity, and accessibility program creation framework the team was developing. I then organized the questions for the Research Ethics Board application so that they aligned with the RE-AIM planning and evaluation framework for translating public health research into practice the research team centered the project evaluation around:
-
Reach – total number of participants and their representativeness
-
Effectiveness – impact of intervention on outcomes
-
Adoption – number, proportion, and representativeness of settings and intervention agents operating the program
-
Implementation – how well the intervention protocol is followed
-
Maintenance – the extent of intervention adoption and individual-level outcomes after at least six months
​
​
After the evaluation data was analyzed, I was able to outline program improvement priorities in presentations for project leadership buy-in and in design documents for program faculty and SMEs.
Refernces
Gaglio, B., Shoup, J. A., & Glasgow, R. E. (2013). The RE-AIM framework: A systematic review of use over time. American Journal of Public Health, 103(6), e38-e46. https://doi.org/10.2105/AJPH.2013.301299
​
Graham, I. D., Logan, J., Harrison, M. B., Straus, S. E., Tetroe, J., Caswell, W., & Robinson, N. (2006). Lost in knowledge translation: Time for a map? Journal of Continuing Education in the Health Professions, 26(1). https://doi.org/10.1002/chp.47
Moore Jr., D. E., Green, J. S., & Gallis, H. A. (2009). Achieving desired results and improved outcomes: Integrating planning and assessment throughout learning activities. Journal of Continuing Education in the Health Professions, 29(1), 1-15. https://doi.org/10.1002/chp.20001
​
Wiljer, D., Salhia, M., Dolatabadi, E., Dhalla, A., Gillan, C., Al-Mouaswas, D., Jackson, E., Waldorf, J., Mattson, J., Clare, M., Lalani, N., Charow, R., Balakumar, S., Younus, S., Jeyakumar, T., Peteanu, W., & Tavares, W. (2021). Accelerating the Appropriate Adoption of Artificial Intelligence in Health Care: Protocol for a Multistepped Approach. JMIR Research Protocols, 10(10). https://doi.org/10.2196/30940