SGEM Xtra: Rock, Robot Rock – AI for Clinical Research - The Skeptics Guide to Emergency Medicine

Subscribe: RSS

Date: February 11, 2025

Dr. Ross Prager

Guest Skeptic: Dr. Ross Prager is an Intensivist at the London Health Sciences Centre and an adjunct professor at Western University. His expertise in critical care medicine is complemented by his research interests in critical care ultrasound and evidence-based knowledge translation.

This is another SGEM Xtra. On today’s episode, we’re diving into a fascinating and evolving topic—how artificial intelligence (AI) shapes clinical research. AI has the potential to streamline many aspects of medical research, from study design to statistical analysis and even manuscript preparation. But as always, we need to approach these innovations skeptically.

There are a lot of promises being laid on the shoulders of AI, and increasingly, it can be difficult to separate the hype from reality. I certainly believe that AI will change what clinical research looks like in the next decade, but at its core, it will be the synergy between researchers and technology that drives innovation, not either in isolation.

The easiest way to reflect on how AI might be used in clinical research is to think about the research lifecycle. Layered on top of this are themes like collaboration/team efficiency, security and privacy, and other general administrative efficiencies (accounting + Meeting scheduling + email management).

Study inception and design
Protocol Generation
Ethics application
Study Facilitation and Recruitment
Data Extraction
Data Analysis
Manuscript writing
Manuscript submission
Knowledge Mobilization

Eleven Questions on Artifical Intelligence and Clinical Research

Listen to the SGEM Podcast to hear Dr. Prager’s responses to my eleven questions.

1. Designing a Study with AI – Formulating a PICO Question: Every good clinical study starts with a clear and well-defined research question. AI tools are now being used to help formulate the PICO (Population, Intervention, Control, Outcome). How can AI assist researchers in this first critical step?

2. Identifying Potential Study Participants from EMRs: One of the biggest challenges in research is identifying eligible patients. Traditionally, this has been done manually and has been a very time-intensive process (think medical students). How can AI help streamline this?

3. Determining the Most Important Patient-Oriented Outcome: Research should prioritize outcomes that matter to patients. How can AI help determine the most clinically meaningful and patient-centred outcomes for a study? In other words, can AI help us find the POO?

4. Estimating Effect Size and Sample Size Calculations: To conduct a well-powered study, researchers need to estimate the expected effect size and determine the required sample size. Can AI assist with these calculations?

5. AI for Statistical Analysis and Data Visualization: Once data is collected, the next step is analysis. How can AI assist with statistics and visualizing complex data?

6. AI-Assisted Manuscript Writing and Editing: Writing a research paper is a time-consuming process, especially for non-native English speakers. A friend of mine is a clinical researcher and editor for a major journal. They talk about knowing some brilliant researchers who cannot write/communicate well. Can AI help these people and improve the clarity and readability of their scientific manuscripts?

7. Verifying Citation Accuracy: We will be talking about the issue of inaccurate citations in the medical literature with Dr. Nick Peoples. His research reported that citations are not correct up to 25% of the time (reference). Concerns have been raised about AI hallucinated citations. We want to make things better, not worse, by using AI. How can AI be used to ensure accuracy and prevent misinformation in referencing?

8. AI in Systematic Reviews and Meta-Analyses: Another form of clinical research is performing systematic reviews and meta-analyses. These can be very labour-intensive. Hiring a research librarian can help. How could AI streamline the process while maintaining rigour?

9. Automating Ethics Approval and Journal Submission: Applying for IRB approval and formatting manuscripts for journal submission can be bureaucratic nightmares. See how I’m slipping two questions into one? How can AI help researchers automate these processes?

For manuscript formatting, Ross gave a self-plug a platform during the podcast that he has been working on for the past year. His team built a tool called Resub, that automatically formats manuscripts for any journal in minutes. This is a hugely time-consuming, painful, and tedious process. Worldwide, this wastes almost 24 million researcher hours per year on formatting alone. In the next couple of years, the delays from editorial rejections account for almost 43,000 years of delay in new scientific knowledge reaching the bedside.
His thought was to create Resub software that is like a reference manager but for the whole paper. This wasn’t possible before the advent of LLMs, because the technology needs to recognize the entire meta-structure of the manuscript, check against a database with all of the journal requirements, and then automatically format what it can or flag items to the user.
Resub can separate title pages, tables, and figures; change headings, format tables, etc., but in other circumstances, it just flags that the abstract/manuscript wordcount is too long or that certain sections, statements, or items are missing in the manuscript.
The idea is that Ross and his team want to take something that often takes days and reduce it to minutes. They have users from around the world and are building a whole bunch of cool features to support the entire submission process (think cover letters and figure editing). You can check it out at Resub.app!

10. Ethical Considerations in AI Research: With all these potential benefits, we must also consider the ethical challenges AI brings to research. It is like considering using a medication; we must think about the potential benefits, but we must also contemplate the potential harms. What are the biggest ethical concerns researchers should be aware of?

11. The Future of AI in Clinical Research: We’ve covered a lot, but I want to give you the floor. What’s one area where you think AI will most significantly impact clinical research in the next decade?

Final Thoughts by Dr. Prager: “Even taking a step back, I think as researchers, clinicians, and just humans in society, we must realize that AI is going to have a profound impact on our lives. You can either be at the table or on the menu. In a health system where healthcare and research costs are ballooning and there are unmet needs, we owe it to our patients and the public to provide the most efficient, cost-effective, and quality care possible. The same goes for research. I think AI in clinical medicine will allow individual researchers and small teams to accomplish the same feats as was previously accomplished only by massive research labs. This democratization of high-quality research will help advance the speed of innovation in medicine and change the shape of medicine for decades to come.”

The SGEM will return next week with a structured critical appraisal of a recent publication. We shall continue in our effort to cut the knowledge translation window down from over ten years to less than one year using the power of social media. Our ultimate goal is for patients to get the best care, based on the best evidence.