Extending the Conversation on Return-On-Investment (ROI) Analysis

Posted in Blog What's New

…Designing Evaluations for the Decisions They Must Inform.

Return on Investment (ROI) analysis is having a moment in health services research. As healthcare organizations increasingly address the drivers of health, policymakers and health system leaders are seeking stronger financial evidence to guide resource allocation decisions. That growing interest is welcome—but it also highlights the importance of choosing analytic methods that match the decisions the evidence is intended to inform. As we work to make sense of transformations in primary care to integrate and evaluate health-related social needs, selecting measures and metrics to best evaluate the impact and potential of creative extensions of primary care is a pivotal choice.

In a recent Health Affairs Forefront article, “To Evaluate Interventions That Address the Drivers of Health, Look Beyond Return on Investment,” Seth Berkowitz argues that ROI is often not the most appropriate evaluation approach and that cost-effectiveness analysis (CEA) is frequently the better analytic method. In our recent research projects, we have wrestled with this choice, and I’d like to extend the conversation.

Mini-Glossary: Three Ways to Relate Costs, Benefits, and Outcomes

Many published research evaluations are driven by data availability or by practitioners’ research or implementation questions about program outcomes, to the exclusion of methods aligned with specific intent resource allocation questions from the program sponsors’ perspectives. My take is that the growing interest in ROI is a well-intended response to that gap, if the first question that starts the evaluation process is: how is this evaluation result to be used, by whom and for whose benefit? What evidence will inform that purpose? In the case of our HCFI projects, we often find the need for CEA in addition to ROI analyses.

Consider an enhanced perinatal care program that provides additional care coordination, home visiting support, and postpartum follow-up for pregnant individuals at high risk of preterm birth or other adverse outcomes. Importantly, the program also includes non-medical, health-related expenditures—such as transportation assistance, housing stabilization referrals, nutrition support, and connections to community-based services that address social and environmental risk factors for adverse birth outcomes. These components are not clinical services, but they are integral to improving health by addressing upstream drivers of health risk. An expiring research grant to the health system funded the novel service. Health systems, payers, and the grant sponsor all have decisions to make about how to continue financial support for the program in the years ahead. 

In this example, the program design, target population, and core service components remain fixed; similarly, the underlying clinical and economic evidence base used to evaluate its effects is held constant. What changes is the resource allocation question—and, critically, the decision context in which that question is being asked.

Several features define this decision environment:

• Novelty: the intervention represents a relatively new approach to addressing perinatal risk through integrated medical and non-medical supports, with evolving evidence on which components drive outcomes.
• Disparate alternatives: decision-makers are comparing this program not only to clinical care alternatives, but also to other social, public health, and health plan-covered priorities that compete for the same fixed resources.
• Finite time horizon: both budget cycles and political or administrative planning windows constrain how long costs and benefits are relevant for decision-making.
• Domain-specific realization of value: the benefits of the program must materialize within the healthcare and Medicaid financing context (even when some of the value accrues in other sectors), shaping which outcomes are considered decision-relevant.

In this example, CEA, CBA, and ROI assemble overlapping elements of management information to answer related, but different questions:

• How efficiently does the program improve health? CEA estimates the cost per preterm birth avoided within distinct subpopulations of patients. That helped us estimate how to scale the program most efficiently- including which patient groups to lead expansion in a way that would likely show short-term results and enable the most benefits for the greatest number of patients.
• Do the total societal benefits exceed the total costs? CBA quantifies avoided healthcare expenditures, healthier child development, and future productivity to estimate net social value. We first calculated the aggregate benefits and costs to determine whether the program was beneficial. Then we had to deconstruct the benefits and costs by stakeholder to understand impact from each perspective (health plans, health system).
• How much should health plans replace funding from the expiring grant? ROI asks whether downstream savings and other financial benefits are sufficient to justify continued support from the sponsor or payer’s perspective. Stakeholder specific CBA results, expressed as ROI are useful to compare across unrelated programs – will funding this program today generate financial returns that allow even more services to be provided later?

These are not competing analyses. They are complementary approaches designed to inform different decisions.

I also agree with Berkowitz’s cautions against the imprecise use of the word investment. In healthcare, resource allocation decisions are rarely about maximizing profit. They are about determining the best use of scarce resources, comparing competing opportunities, and understanding whether an intervention can become financially sustainable without requiring continuing subsidies that displace other priorities. Yet, the term remains useful in its temporal sense: committing resources today with the expectation of creating future value. ROI and CEA are methods which can serve our normative goals towards equity and sustainability when we cannot do everything indefinitely or all at once.

Perhaps the larger methodological challenge is not choosing between ROI, CEA, and CBA at all. All three depend on the same foundational inputs and awareness of perspective: whose costs or investments are included, whose benefits count, which outcomes matter, whose frame of reference the analysis presents, and over what time horizon value is expected to emerge. Those framing decisions often shape the conclusions more than the analytic method itself.

Primary care interventions addressing the drivers of health make this especially clear. These programs invest in patients’ health and human capital, yet the same stakeholders rarely bear the costs and benefits. Health systems, payers, families, schools, employers, and society all experience different impacts, often over very different time horizons. Decisions to participate or support such interventions are influenced not only by the magnitude of expected benefits, but also by their timing. CEA, CBA, and ROI analysts must therefore contend with the incidence (for whom) and timing (by when) of costs and benefits—not simply their magnitude (how much).

Rather than competing evaluation approaches, CEA, CBA, and ROI should be viewed as complementary components of a decision-support toolkit. CEA identifies efficient ways to improve health. CBA provides a common decision currency for comparing fundamentally different interventions. ROI answers a different but equally important practical question: Can this intervention generate sufficient financial value to justify continued funding and achieve financial sustainability? The goal is to assemble evidence with agility to answer all the necessary questions—ROI can be a necessary, if not sufficient, part of that evidence.

Carol Davis is an Assistant Research Professor at Georgetown’s McCourt School of Public Policy, focusing on the economics and policy issues in the delivery of health services.

#HealthServicesResearchMethods #ROI #CEA #SDOH