Linn Southall   00:00

Welcome to the Medical Affairs Professional Society’s three-part podcast series on how to prioritize, track and measure your medical communications plan. In the second podcast episode, we will discuss considerations for measurable engagements for Medical Affairs. I am Linn Southall. I will be moderator for this podcast. I am a Medical Communications Director at Novartis Gene Therapies based in Zurich, Switzerland, and I currently serve as a member of the Medical Communications Focus Area Working Group. The views expressed in this recording are those of the individuals and do not necessarily reflect the opinions of MAPS or the companies with which they are affiliated. This presentation is for informational purposes only and not intended as legal or regulatory advice. This podcast series is for Medical Affairs professionals asking how do I know that my Medical Affairs programs have the expected impact? In this episode, we discuss in particular how to approach engagement and complicated objectives across Medical Affairs. Thinking about measurement, what you need to consider when you do and what this might look like in practice. I would like to say welcome and a big thank you to today’s panelists for sharing their subject matter expertise for the MAPS members. So before we get started, I will let the speakers briefly introduce themselves and their background in the industry. Jonathan, do you want to start?

Jonathan Oldershaw   01:28

Hi, I’m Jonathan Oldershaw, Vice President of Insights at AXON Communications. My work within insights and data for the last eight years that’s been with a consultancy that specializes in healthcare and Medical Affairs. But I also work within a range of other sectors helping organizations understand their impact.

Christina Gordon   01:46

Christina? Hi, I’m Christina Gordon, Associate Director of Scientific Education and Innovation with the US Medical Affairs team at Novartis Pharmaceuticals. I’ve been in the industry for 14 years both consulting in house and outside of the organization. And I’ve worked across multiple therapeutic areas in both commercial and medical. Prior to pharma I worked in additional industries.

Soroosh Aidun   02:08

And hello, everyone, my name is Soroosh Aidun. I am the Global and US Medical Communications, Digital and Congress lead at Amgen. I am also adjunct faculty at UCSF School of Pharmacy, and I have served in various commercial and medical pharmaceutical roles specializing in business development, medical communications, and Medical Affairs.

Linn Southall   02:31

Thank you, and thank you all for being here today. As discussed in episode one, we’re sensing a shift within Medical Affairs to make the case for and and system better measurements. When taking the approach and marrying strategic needs into implementation, how can Medical Affairs teams focus on alignment of wants versus needs. We’ve today’s panel will address these questions and look at novel and cutting edge approaches to measurement in medical engagements. So as also discussed in the first episode, MAPS has been active in driving a discussion about measurement and published a white paper titled mastering the art of meaningful metrics in the past year. This paper gives a very comprehensive overview of the of the topic and we definitely suggest that you read this if you want more in depth knowledge. So starting from that, theoretically, we all understand the reason to measure and the importance there. However, when considering the tremendous availability of touch points that are emerged on a daily basis, and so many approaches with so many different channels, audiences messages, where do we even start with our measurements? What are your thoughts on this? Starting with the Soroosh.

Soroosh Aidun   03:51

Thank you, Linn. For me, it always starts with first defining what your core objectives are, and what success means, then you need to understand the platform you’re executing your strategy on. Together that should inform you what you should be measuring. Because not all objectives and platforms are the same. Your metrics should provide meaningful and actionable insights. And that should give you a clearer sense of whether you have met your objectives and the effectiveness of your strategy and how it can be improved in the future. It’s not about capturing all the measurement points. It’s about capturing the right ones.

Linn Southall   04:34

Christina, to build on what Soroosh just said, how do you know which ones are the right measurement points?

Christina Gordon   04:42

So for my approach, I look at the measurable activities strategically, not just does the math work along the tactics and cost time audience numbers, which are all really completely irrelevant and important. I really look at the more in depth approach of how I will support the audience. In some way we can jointly achieve, aligning the strategy to how well I can deliver given that particular environment. And knowing what those audience gaps are is really important to understand how to leverage the right channels with the different approaches for each activity, understanding the audience is just as critical as knowing if the activity is measurable.

Linn Southall   05:19

Lastly, Jonathan, with your experience across industries and companies, what are your thoughts on what solution and Christina just shared?

Jonathan Oldershaw   05:28

Yeah, I absolutely, absolutely agree with them entirely. I think the purpose of measurement is to show how well you’re delivering on your strategy. So for communications, and particularly Medical Affairs type activities, the strategy is genuinely about changing attitudes and behaviors, and your audience in some way through engaging with them. So in terms of measurement, you need to look at whether and how your audience has engaged with you and the content that you produce. And then the second part to your measurement is to assess how HCP attitudes or behaviors have been changed from this.

Soroosh Aidun   05:59

And just looking at our commercial colleagues, they often measure everything by looking at the ROI, the return on investment, which for medical, it’s not appropriate. But where we can, we need to measure everything by looking at the ROO, return on objective. The ROO helps tell you whether your educational campaign has been successful. So you need to understand your medical objectives. And if you’re trying to raise awareness, engagement, or have deep interactions.

Linn Southall   06:32

Right, so so good metrics start with well-defined objectives and expected outcomes. So what is the next step, then? What do we do with the data that we capture? If we look at the MAPS white paper? Already 4% of the respondents to the survey answered No or unsure if the metrics captured are used in their Medical Affairs Association. Would you think a useful first step would be to use the metrics people already have? Or should we go and start capturing other metrics?

Soroosh Aidun   07:05

I can weigh in here, we do not necessarily need to start generating more data and sophisticated algorithms, if the data we already have isn’t being utilized. The first step in these cases is to start looking at what measurement points already exist for relevant insights to further inform you on the next steps.

Jonathan Oldershaw   07:25

Yeah, I agree. I think from our experience working in any industry, we rarely see companies where the problem that they have is not having enough data. I don’t think it’s too much data. And I think there is a challenge in picking out the right data to use. But it’s also important to establish those opportunities to reflect on what the data show, and what can be done in response to that. So there’s not really that much use in knowing whether something’s not working. If you can’t really do anything with that, and figure out how you could improve the situation.

Christina Gordon   07:53

I agree. And I would also suggest that if we don’t have the time to invest into these exercises, to measure our results, and analyze what it means, did we do the right thing by engaging in the exercise to begin with, I find that the retrospective time requires a commitment on my end and the diligence to commit my time invested in the exercise, but because I value the insights from from that activity, and what I can learn from the data that will help me, you know, define my next plan and how I will implement that. And that gradual growth might be more valuable than that individual activity on its own.

Linn Southall   08:29

Thank you. Jonathan, as you mentioned earlier, medical education activities are about engagement and driving that behavior change. So thinking about that, what are helpful metrics in terms of understanding what engagement your audience have had with your deliverables or content? What do you look for? Or how do you approach this?

Jonathan Oldershaw   08:51

When we’re thinking about engagement, and how that drives behavior change, what we’re trying to do is build a picture of what the engagement looks like by an audience, make sure that it’s one that’s plausible. So often, you see very large numbers, probably from a technology platform vendor, and they say something like 20,000 people could have seen this content produced. And what you need to do that is drill down further and you see that maybe a small number actually click through and then even smaller number stayed until the end and finished watching whatever it was you produced. All of these really useful metrics to understand where the engagement is high, where it drops off, and identify what to do about it next time.

Soroosh Aidun   09:30

And to Jonathan’s point, we could actually use this podcast as an example. For this podcast, we will not measure and define success solely based on the number of listeners, there are going to be some people who may have not been paying attention. Others may have listened maybe for the first two minutes of the podcast, but then they heard my voice and decided to jump off. We should consider other key measurement points, like what percent of individuals dropped off after listening to 25% of podcasts 50% 75% or 100%. You could also consider having a pre and post podcast survey to assess any behavior changes. Oftentimes, success metrics are a point in time assessment. But it’s important to understand what are meaningful milestones of intent and engagement. Again, this comes back to what is success? Is it raising awareness on the subject? Or is it changing or educating KOLs? on a specific topic?

Linn Southall   10:31

Thank you, Soroosh. That’s very clear. Now, on to my last question, if we look at the broader picture, or the industry as a whole, are there benchmarks to compare engagements to? Or should organizations compare their engagements and effectiveness internally only?

Christina Gordon   10:50

That’s a really great question, Linn. I think we have to be really careful about comparisons of different strategic environments. Tactically, we have diversity in our targeted audiences that are not delivered in the same way to the same audiences. I would say that from a therapeutic perspective, we couldn’t compare one areas to another because there are nuances to the environment, the practice area. And that might not necessarily be immediately obvious from a measurement perspective, when you’re comparing similar activities. Additionally, when you look at the depth and the breadth of an activity, there’s going to be a variability there, that’s going to be really important to understand. example of that would be like in social media posts, which you’re going to have a very different outcome than, you know, an MSL engagement, we wouldn’t really want to compare the social media impression numbers to the time spent with the direct HCP engagement. Because the context is completely missing. The numbers tell us one story, but the direct impact of the one on one engagement is going to have more depth. So when you look at the target range of the engagement, what the relative engagement is for that activity, and you use that to make comparisons, we may want to get a more in depth understanding of measurable activity by asking was the Social Media Campaign better than the last? What was that Congress attendance last year, and were the numbers up or down and what was different?

Linn Southall   12:12

And that wraps up our panel discussion for today. So thank you, Soroosh, Christina, and Jonathan for sharing your expertise and insights with the listeners today. My takeaways from this conversations is that Medical Affairs is operating in a highly complex environment with so many different channels, audiences and messages. If the purpose of measurement is to show how well we are delivering on our strategy, defining what our core objectives are, and what success looks like, it’s absolutely critical. Equally important, is it to take the time to analyze the data that we collect, and let those insights guide future planning. Learning that an activity was less impactful than expected is not necessarily a failure, but a first step towards future success. So thanks again to our speakers. This has been the second podcast in a series of three on this topic. In the next podcast, we will discuss how to measure the impact of scientific publications. We encourage you to engage in conversations with a medical communications with other MAPS members via the MAPS Connect on the MAPS website or mobile app. Simply log in with your email address and password associated with your MAPS account and access the global community. Then click on the discuss tab and scroll down to medical communications to post a question or review previous postings. If your MAPS Member, thank you for your support of MAPS. If you’re not yet the MAPS member and would like to access the additional resources in this area, please visit the MAPS website to explore joining today at MedicalAffairs.org/membership. This concludes the podcast.