Qualitative vs Quantitative
A central challenge in the use of metrics in MA is quantifying outcomes that are inherently qualitative, such as stakeholder education or customer awareness of product messages. Unfortunately, it can be difficult to calculate and communicate the success of initiatives measured with qualitative metrics (eg, the value of insights gained across customer channels). Historically, this problem was addressed by focusing on quantitative measurements as proxies for desired outcomes, for example, the number of KOL interactions as proxy for the success of a team’s external engagement strategy. Current strategies seek to both formalize qualitative metrics and quantify the impact of MA actions. This concept of impact measurement can be understood as looking a layer deeper than traditional quantitative metrics to ask not only “What was done?” but “What was the impact of what was done?”
Scalar vs Vector
Related to the previous discussion of qualitative vs quantitative metrics, scalar metrics show size, whereas vector metrics show both size and direction. For example, measuring the annual number of publications or KOL interactions are scalar metrics demonstrating the quantity of actions but not their impact. Capturing the impact of a publication (eg, through altmetrics) or the depth of KOL interactions (eg, through qualitative survey) could help transform these scalar metrics into vector metrics.
Organizational vs Strategic
Where does a metric sit in the hierarchy of the MA Strategic Plan? For example, top-line goals like increased adoption of a product might sit at the organizational level, while the different groups within MA including Publications, Field Medical, and Health Economics and Outcomes Research (HEOR) may each use strategic metrics to support adoption of that product through initiatives such as guideline inclusion, key data dissemination, or medical education.
Reach, Relevance, and Resonance
Reach, relevance, and resonance combine a KOL’s reach with their relevance to the target market to predict their resonance. This can be seen as combining scalar and vector metrics such that how many times something is accomplished is multiplied by the impact of each iteration to determine an overall measure of success. For example, if this concept is applied to MA, a KOL with a significant publication history in neurology may have impressive reach, but if the Medical group is focused on oncology, the KOL may have little relevance, decreasing this KOL’s resonance in the oncology space.
Value vs Impact
MA teams often speak of metrics to demonstrate the “value” of the function. As MA evolves into increased prominence within the organization, it has the opportunity to shift from the language of justifying value to demonstrating impact, thereby shifting from a deficit perspective (“Is MA really worth it?”) to an asset perspective (“What opportunities does MA present?”). This shift can also influence the design of an MA team’s metrics framework, with teams focusing on metrics to drive opportunity and innovation rather than metrics meant to justify value.
Can vs Should
Eventually a strategic planning process with metrics at its core will generate more metrics than are feasible to implement. Keep in mind that every measurement comes with an associated cost not only in time and budgetary resources but in the danger that less useful metrics may obscure the learnings from more central and essential metrics. Just because something can be measured does not mean that it should be measured. Likewise, the choice to measure assigns value to the actions or outcomes being measured, meaning that metrics have the potential to incentivize behaviors, and it becomes essential to consider whether a specific metric may unintentionally incentivize negative actions.
Measuring vs Monitoring
Some metrics require no context—they stand alone as absolute measures. However, many metrics are only relevant as measures of change. This latter category of monitoring metrics can show a group or organization whether it is moving toward or away from its goals. Measuring and monitoring metrics can be cyclical and even periodic, with feedback from monitoring influencing the next iteration of metrics designed to measure outcomes. However, monitoring metrics present special challenges in implementation and interpretation. Who will do the measuring and how often? What types or magnitudes of changes over time are meaningful, and what actions should be taken based on these changes? Does a baseline exist against which to compare current and future measurements and, if not, how will baselines be established and at what point will the metric return meaningful insights? These questions must be answered in the Strategic Plan.
Internal vs External
Internal metrics demonstrate impact to other stakeholders within the organization and must be designed from the perspectives of these stakeholders; for example, Commercial may prioritize reporting on HEOR value, whereas Research & Development may prefer insights generated by MA leading to additional clinical studies. External metrics describe MA’s impact on stakeholders outside the organization and are likewise designed from the audience’s perspective, for example, measuring the increase in knowledge in a patient advocacy community around the effectiveness of investigational agents due to an MA external education program. When aligning metrics with the Strategic Plan, it is useful to keep these internal and external audiences in mind so that MA can choose metrics to match motivations of various internal and external audiences.