Garth Sundem   00:00

Welcome to this episode of the Medical Affairs Professional Society podcast series: “Elevate.” I’m your host, Garth Sundem, Communications Director at MAPS. And today, we’re talking about publications impact, with Mike Taylor, Head of Data Insights, and Grainne Farrell, Manager, Corporate Life Science, Central Europe, both at Digital Science. This episode is sponsored by Digital Science, choosing journals and understanding publication impact in the digital age. So Mike, will get to current and future publications approaches. But could you start by telling us a little bit about what metrics we’ve used to play the academic journal game for maybe about the past 50 years,

Mike Taylor   00:49

I don’t think we’ve got 50 years to talk about it. But I could probably talk that long about it, I’m terrible once I get going. So I’ve been working in this area for 2627 years now, I spent 20 years, one of our larger scientific publishers. And for me, it’s been a really interesting couple of decades to be looking back at how data is because for such a long time, research, understanding how well a journalist performing has been dominated by the impact factor that really is the the metric by which journals have lived and died over over the course of the last 20 or 30 years. But it’s a metric that’s been around for a very, very long time, is the original computation for this was back in the back in the I think was the late 50s. Or, or the early 60s, when using Garfield had to collate the citations on pieces of card and run them into a mechanical a card reader. And it took something like a year to do those initial calculations. Obviously, this was a little bit before my time, even before my time. But the thing that that’s meant is that that design decision to make those calculations really hasn’t shifted very much over the course of that time. But still, those single metrics, those single numbers that we have to describe the performance of a journal, really is still embedded in the mindset of how it is that we think about a journal.

Garth Sundem   02:21

Okay, and so just Just to follow up on that, so for Medical Affairs, then the goal of a Medical Affairs department would be to place their publication at the journal with the highest impact factor. And that that that’s as far as it went. So now, are we still what beholden to the Impact Factor? Or are we starting to look at other metrics to choose where to place our publications? Grainne?

Grainne Farrell   02:54

Yes. The short answer is yes, we are starting to look at other metrics. And there’s a lot of information out there. And that’s one of the issues I think, for Medical Affairs professionals, how do you make sense of all all of this data, there’s, there’s so much of it, some of the work that Mike has been doing has been very interesting in terms of pulling together lots of different metrics that can then actually help Medical Affairs people to place, you know that their publications in the right journal, so it’s no longer by looking at sort of, you know, let’s get into the lancet or the New England Journal of Medicine, which can be very difficult to get into. But based on data, that’s my available, you can choose your journal based on your audience. And be confident that you’re, you’re, you know, you can choose a smaller journal, but you know, that you’re going to hit the target audience and get the word out there. So you have confidence behind it, because you’ve got the data helping you to make the decisions.

Garth Sundem  03:59

Oh, that’s interesting. So you know, the impact factor, it seems like it used to be easy. So Mike, I just read a article you wrote about choosing $20 blender, and we’ll link to that from this podcast episode. But your your, your basic point was that we use more data to pick a blender than we do to choose a journal where we want to publish our studies. And, and here’s the thing I came away from that article thinking, god that’s so complicated to choose a blender, I just want an impact factor that helps me choose the journal and move forward. So is it in some way more difficult now that we’re moving beyond the Impact Factor? I mean, this scares me. I like the Impact Factor. What you know, is it harder to choose journals now that we have more data?

Mike Taylor   04:55

Quite possibly, although, you know, I am tempted to offer the first person who can adequate Describe the impact factor to me, perhaps the first person to contact me after this, and you can accurately and fully describe the calculation, I will buy them a very nice drink at the next conference where a very nice drink. Indeed, it is a very, very complicated metric. We think that it’s being simple because it is just a number. So it’s very easy to use, it’s very easy to make comparisons, but a lot of those high level of complexity that belies, if you like it belies the the nature of how we use that metric. And you’re quite right, of course, that that a blog post that I wrote about the $20 blender, I came about, because my son was going to university, and he wanted a blender. And all of a sudden, you’re in this situation of thinking, Well, I know the one I would buy, but the one that I’m going to buy is one that’s going to last me 20 years, it’s not one that’s going to last me a year, I’m not going to have somebody thinks it’s amusing to suddenly pour a hot soup in there and and blitz it with the lid off. That kind of thing is just not going to happen. But let me know, let’s think about what a what a journalist so you know, I’m a publishing, I’m a publishing researcher, as well. So I understand what it is to submit a journal. And of course, I want my articles to go into the best possible journal. But when you’re thinking about that, you’re thinking, well, who’s going to be reading it? And what are they going to be doing with it? And what kind of outcomes do I want to happen as a consequence of me publishing my article, because sometimes, for example, I wrote a paper, rather off the subject, wrote a paper on open access monographs, which is something which is of use to to publishers, and people who work in funding. So really, the journal that I want to place it in is one where people like that are going to be reading it. Now, that’s not a biomedical one. But sometimes I might be writing a paper that I want to be read by other clinicians. Sometimes I want, I might write a paper that needs to be read by researchers, it may be a paper that’s talking about the applicability of a treatment to a certain group of patients. These are very nuanced decisions. And whereas previously, we’ve just not had the data available to us to make the comparisons. The wonderful thing about the last four or five years, has been that we’ve gotten much more data. And it’s much easier to make a calculations. So you know, just thinking back to Eugene Garfield took him, you know, a year or so to collect the data and run and run those first comparisons. When I was working back at Elsevier, and I was working on the citescore metric, which is an equivalent to the impact factor, it was taking us three days to run it. So we will be literally setting up the calculation on the Friday with books and time on a mainframe somewhere, this is just years ago, we’ve got the time in a in a high performance computer, and then go down to the pub, and then come back on Monday morning, not having been in the pebble weekend, but then come back on Monday morning to see how it worked. And frankly, if it failed, it failed. And you’d have to do the same thing all over again, on the next Friday. Since we’ve been a digital science, the world has revolutionized you know, we talk about software as a service, we talk about cloud computing, we talk about big data, we talk about data science, these are all just terms. And you know, we chuck them around, and they impress people and all the rest of it. But the reality of it is that I can sit here, I can probably use my phone to write the same calculation of citescore. And I will see it in my phone seconds later. And this has completely revolutionized the way that we think about data. So that if I have if I’m talking to a client and the client goes, I wonder which journal is doing that getting more coverage on Twitter, for example, I can tell you in about 10 seconds. And that has changed the way that we think about metrics is no very entrepreneurial. Because you don’t have to come to me with a business case you don’t pay me 100,000. To do this, we can we can figure this out really, really quickly. And then after that, think about the data and think about how we want to use it. But all the emphasis is on how do we take those insights and put them in front of people? How do we make a little sharp a graph pie diagram that makes a convincing case to choose one journal over one of those big high impact journals? Because that’s what it’s all about.

Garth Sundem  09:25

You come back to the office on Monday morning after having been in the pub? And the answer is 42. And you’ve got to wait for another 100,000 years for for a new for a new calculation around. So Grainne is what I’m hearing. Or I should say is it true that we are transitioning maybe from a journal centric approach to an audience centric approach? And asking not like, what score I can get by by placing a study in this journal, but but who’s going to read it and looking from an audience centric perspective?

Grainne Farrell   09:58

Yes, that’s exactly what we’re looking at, right, we’re able to look at which journals are being read by, you know, if you’re trying to target an oncologist in a certain area, which journals are most likely to be read by, you know that audience. So we can definitely look at those metrics.

Mike Taylor   10:18

And you can think about it a little bit like choosing that blender, you’ve got on the left hand side of your favorite online retailers thing, you’ve got little sliders, you’ve got little checkboxes. There’s no reason why we can’t be doing the same things with data and with journals. So you might want to say, this is off the top of my head, but you might want to say, look, I’ve got this neurology paper, I want to get it in front of other neurologists. But I don’t want it to be one of like, 10,000 papers published a year. So you can go okay, I want to go for a medium level journal that does a couple 100 papers a year. And that’s mainly read by clinicians, or mainly read by researchers, you can do those things now. Because this is what big data lets you do. It powers those kinds of insights.

Garth Sundem  11:06

How do you know, I’m going to take us into the weeds, Mike, I’m sorry on this, but how do you audience of a journal? I mean, and how do you quantify that in a way that allows you to then go back into that data? How do you know the audience for a journal?

Mike Taylor   11:23

Okay, so Oh, yeah, so this may be a little journey into the weeds. So one thing we can do is to look at the authorship of it. So we can look at the institutions of the people who are writing the journals is such a simple thing to do, in, in, in a sense. So we can look at the demographics of who is submitting, whether they have a corporate background, whether they work for a research lead institution, whether they work for a clinical institution, for example, we can look at people’s job titles as well. Are they indies are they PhDs are the professors so we can make we can make a good understanding of who the authorship is. And that’s not the same as a readership. We can also do the same thing for the articles that are citing that journal. Of course, we can look at the credentials, the affiliations of people to say, Oh, look, they’re they’re citing, you know, lots of clinicians are citing this journal, which is really good evidence that clinicians are reading it. We can look at Twitter, even now, post musk, we have a we’re attached to the fire hose of Twitter. So we get a live stream of tweets where people are sharing research. We can look at who’s doing that. So we can say, hey, look, there’s there are oncologists exchanging links, making recommendations about this journal. So for example, we use and I don’t want to come across as being all salesy, but we use a sentiment analysis to look at the tweets, so that when an oncologist when someone we think as long columnist on Twitter says, this is a really great article. We can set we can identify who’s, who’s making recommendations to read and access to the journal. One of the amazing things about open access is that it is revolutionizing the way that people engage with research. Because for so much research, it’s open. And that means patients can read it and that means carers and families can read it as well. I mentioned to Grainne earlier a couple of years ago, I did an experiment for a clinical establishment in the US. They have a particular focus, and I’m going to pronounce this wrong, but they have a particular focus on chronic traumatic and carefully, CTE, as it’s known as a Netflix series, the parent of an American football player who, who suffered very badly from CTE and spent some time in prison as a consequence of his behavior, which was just came back from repeated content, concussion, which leads to this thing called CTE. Now, the reason it was so interesting for me is that research into CTE was was largely stimulated by the patients, or the family, rather, the families of the patients. So young men playing American football, getting this awful concussion, developing CTE, and there was a great pressure from families to take this seriously. And to just not write it off as being, you know, a brain, you know, a thing that happens to people who play American football or for that matter in Europe. And what was really interesting when we looked at the tweets around the research, so these people are going on to Twitter, and they’re sharing links to clinical trials, and research and that’s what part of Digital Science Altmetric identifies was that we partition the research. So this university that we were working the sorry, this hospital that were working with, they were publishing very clinical research, but also preventative research, and we can clearly divide that and identify that neurologists were sharing During the the research about the science of it, but there was a subset of that, that looked at preventive prevention and cure and other things that was being shared very heavily by the patients. And it’s kind of quite interesting, then to be able to reflect on Well, what is the article that we’re publishing? And who do we want to engage it and take a more intelligent, intelligent view of where we’re investing our time and money in, in publishing an article?

Garth Sundem  15:27

Well, so here’s another, here’s another maybe interesting transition. And that is that it used to be that we would play the Impact Factor game we would publish in the in the journal. And that was it. That was the finish line. And it sounds like what what you’re saying is that not only is it now important to take an audience centric approach to choosing a journal, but then after publication, that’s not that’s not the finish line. That’s the start line, then you’re looking at continued engagement. With with that article, there’s a whole nother half of the story here. It’s not journaling, you’re done. It’s journal, and then watching for how that research or opinion or knowledge is received by the audience. Is that true?

Mike Taylor   16:23

It is absolutely true. And people are doing some amazing stuff. So this isn’t work that I’m working on. But we’ve come across clients who are doing things like as well as doing plain language summaries doing plain language podcasts, for example. Yeah. And then because the data is available, we can track that down to whether people are then following up by looking at the looking at the article, what we’re exchanging or the exchanging the article or the exchange of the podcast, Ken, this isn’t work that I’m doing, that this is work that other people might my peers in other parts of, in other parts of the of the of our business are working on. There is no end of interesting experiments that’s going on there. Yeah, that probably for the next five years.

Garth Sundem  17:05

Well, okay, so then you’re talking about new engagement strategies at this point? You know, you’ve got the journal article. Well, okay. So here’s another thing in the landscape, it used to be only the journal article, right? And now, yeah, okay, we would play the Impact Factor game, we’d get in the journal. And that was it, you know, maybe you present it to Congress or something like that, or you’d already presented it, and then it goes to the journal. And now it sounds like you’re saying the journal article is only one touch point in an ecosystem of possible engagement opportunities.

Grainne Farrell   17:41

So now, it’s possible to also, you know, you get you’ve published your journal, but now it’s also possible to measure the impact of that journal. So, you know, what are people saying about it? Who are people? Who are the people that are saying it? You know, what is important? With all within all of that? Who should you be speaking to? Who can you collaborate with? Who’s having the most impact around that particular subject? So there’s, there’s a lot of follow on from just the publication i

Mike Taylor   18:10

Yeah. And I think, you know, we’re really just at the beginning of trying to understand this data as well. So you’ve got all of these different innovation, innovations in communicating research and trials to people, and different things are working, this is a time for experimentation, it should be a really exciting time. One of the things I think that COVID has taught us is that there is an enormous hunger amongst many, many communities going far beyond the clinical communities and the research communities, a real hunger to understand science and to and to query it. Again, Ronnie, and I were just having a chat as we were walking our dogs, and I mentioned a website of an important cancer funder that I’d come across. And there was a discussion on there. And they were saying something like, you know, send us $100. And we’ll, we’ll fix your pancreas. They were not quite as crude as that, but something like that. And there was there were people on there saying, what, where’s the evidence that you’re translating my money into into research? What are you doing for it? What publications Who are you sponsoring? There’s a, there is an appetite for understanding how we’re doing work and what work we’re doing. And that comes apart comes along, partly because people have moved moving on to online moving digitally, partly because of the Open Access revolution that we’ve seen across the last 10 years. And partly because COVID has lent an urgency to our online communications that we’ve not experienced before. I mean, who would have thought two years ago that preprints would have been taken so seriously in biomedicine.

Garth Sundem  19:51

So it’s not only an audience, it’s not only a shift to an audience centered approach, but we’re talking about new audiences as well. It’s not just the clinicians and researchers that we’ve learned to communicate with again over the past 50 years. It’s It’s It’s patients, caregivers, you know others in the healthcare ecosystem, who we’re communicating with, in new ways. Oh man, we’re gonna go for an hour, but let’s leave it there. So, Mike, and Grainne thanks for joining us today. MAPS members, please don’t forget to subscribe. To learn more about how your organization can partner with Digital Science Visit digital-science.com and we hope you enjoyed this episode of the Medical Affairs Professional Society podcast series: “Elevate”.