00;00;05;04 

MAPS 

Welcome to this episode of the Medical Affairs Professional Society podcast “Elevate”. The views expressed in this recording are those of the individuals, and do not necessarily reflect on the opinions of MAPS or the companies with which they are affiliated. This presentation is for informational purposes only and is not intended as legal or regulatory advice. And now for today’s “Elevate” episode. 

00;00;33;09  

Ruth Nicholson 

Welcome to episode four of the SciTech Critical Evaluation of Literature and AI series. Smarter summaries, better decisions AI across Medical Affairs. I’m Ruth Nicholson, I’m a Global Medical Excellence and Medical Affairs Leader, and I’m joined today by Adeola Davis and Sarah Snyder. And Adeola, would you like to introduce yourself to everyone. 

00;00;56;00 

Adeola Davis 

Yeah. Thank you for having me today, Ruth. Hi, I’m Adeola Davis, Senior Director of R&D, Learning and Development at a global pharmaceutical company. And Sarah. 

00;01;07;12 

Sarah Snyder 

Hi everyone! My name is Sarah Snyder and I am from MSL Mastery. 

00;01;12;08 

Ruth Nicholson 

Thank you both so much for joining me here today. So this podcast is part of a series of podcasts and resources produced by MAPS to ensure that Medical Affairs professionals have the skills on the critical evaluation of literature that they need in today’s world. So we’ve already discussed in previous podcasts how important it is to really separate the gravel from the gold, to not be blindsided by fancy titles, to really evaluate the data that we’re looking at. We’ve discussed how important it is to, be aware of preprints, grey literature, maybe false signals in a very busy world. And we’ve also spoken about using AI to help us do what we need to do best. So this episode really brings our series together. The question we’re going to discuss today is, how do we turn all this good evidence into good decisions across Medical Affairs? 

00;02;19;12 

Sarah Snyder 

I love this topic, Ruth. The question that comes in my brain is. How do we make sure the evidence is solid before we do all the work? We summarize it, we. Scale it, and we. Cascade it. What do you think Adeola? 

00;02;33;06 

Adeola Davis 

I think, you know, as we get into this a little bit more, as we think about how AI can scale insight, we have to recognize it can also scale mistakes. 

00;02;44;09 

Ruth Nicholson 

So great points, but let’s start at the very beginning. So imagine a high impact, highly anticipated outcomes paper has just been published. So what should be our very first step? 

00;02;58;13 

Sarah Snyder 

Good question. We’ve got to start with the basics, which is critical appraisal. Before AI touches that article. 

00;03;06;24 

Ruth Nicholson 

So, Adeola, how can we undertake that rapid critical appraisal in Medical Affairs? 

00;03;13;29 

Adeola Davis 

One of the things that we can apply is the MAPS clear framework and clear stands for credibility. The L is logic and design, E is evidence strength, A applicability and R is relevance. And when you put all of those together, you can see how if you filter some of your information you’re looking at through this framework, it can really allow for you to appraise the literature in a meaningful way that then you can translate to your function within Medical Affairs. 

00;03;46;09 

Sarah Snyder 

In addition to that we can do a sanity check. So we can use other familiar frameworks as well. CONSORT is a great one for trial quality. And Prisma. If it was a review. 

00;03;59;20 

Ruth Nicholson 

So the rule here is simple. Don’t summarize what you haven’t evaluated. So once that first check has been undertaken and the evidence has passed, I guess then the real challenge begins. And that’s cascading across the different functions of Medical Affairs. So each function is going to have a slightly different lens or slightly different requirements. So Sarah how could this be undertaken for field medical. 

00;04;29;16 

Sarah Snyder 

For field medical for Medical science liaisons. The questions are always going to be around. How will this apply to the individuals I speak with on a daily basis? And that question when you drill down to it, it. Which patients does this data actually apply to? 

00;04;50;24  

Adeola Davis 

I would add medical information needs precision. No interpretation. Just what the study did and didn’t show, as you can imagine. as they’re creating FAQs and SRLs. The precision is key. 

00;05;03;19 

Sarah Snyder 

To bridge on that for the publications teams. They’re going to look at novelty. So what’s this bringing that wasn’t there before. And then any positioning in the broader evidence landscape. 

00;05;17;20  

Adeola Davis 

And then when thinking about HEOR, they are focused on outcomes, comparators and generalizability. 

00;05;24;17 

Ruth Nicholson 

So each of these functions requires something slightly different in terms of AI producing decision ready summaries and outputs for them. So how can AI actually be used, for each of these functions? 

00;05;43;20 

Sarah Snyder 

After we do that critical appraisal that we’ve all been trained to do, we can use AI to generate the first pass summary. 

00;05;52;29  

Adeola Davis 

And I think to when we think about using AI, the most effective prompts were structured. And so as an example, as a structured prompt we can ask AI to summarize key findings, limitations and applicability for field medical teams in under 150 words. Or we can have a prompt that says extract study design and endpoints without interpretation for medical information. 

00;06;17;08 

Sarah Snyder 

Absolutely. So the prompts are the most important part to determine the output that you’re going to get. Going back to those different audiences, the medical information, the publications, the field medical, then that output can be done and validated by that department. They can check the source accuracy, the scope and the tone. 

00;06;41;19  

Ruth Nicholson 

Fantastic. So we’ve critically evaluated the data. We’ve understood the requirements of the different functions. And then we’ve tailored our AI prompts to get applicable outputs and summaries for each of these functions. Again, it sounds too good to be true. You know where can this go wrong? 

00;07;02;18  

Adeola Davis 

If context has been removed. Yeah. Some of the early AI summaries looked very positive until we layered in population restrictions and unmet needs. 

00;07;11;18 

Sarah Snyder 

AI can only do what it what it can do with what you put into it. So if you don’t have a good prompt, you don’t have the right data in there, you’re not using the right tool, then it’s going to be one of those garbage in, garbage out type things. So we’ve got to make sure we use approved tools, good prompting and validate the outputs very carefully. 

00;07;33;23 

Ruth Nicholson 

So the context isn’t just a nice to have it’s risk control. So thank you both so much for discussing this with me today. So let’s close the series with some final takeaways. Sarah, what are your final thoughts or top tips? 

00;07;50;07 

Sarah Snyder 

The biggest takeaway for all these episodes is going to be to put our critical appraisal hat on, and in this case, evaluate the evidence before you summarize it. I can’t replace that critical appraisal human piece. 

00;08;06;04 

Adeola Davis 

I would like to add to that is that as we use AI, tailor your prompts, your outputs by your function. Right. There’s going to be the same data, but you can use different questions. And as Sarah mentioned just a few moments ago, you if you put garbage in, you’re going to get garbage out. 

00;08;24;17 

Ruth Nicholson 

And you’re absolutely right, both of you, to expand on that context and judgments are so vital. We can’t go through this process without that human aspect. And to really help us turn information into insights. I think we can all agree that AI scales the information, but it’s the people in Medical Affairs that really safeguards the decisions we make based on that data. So thank you, everyone for joining us today. Please have a look at the Medical Affairs Knowledge Center on the MAPS website for further resources.