 Methodology
 Open access
 Published:
Stochastic costeffectiveness analysis on population benefits
Cost Effectiveness and Resource Allocation volume 21, Article number: 78 (2023)
Abstract
Dealing with randomness is a crucial aspect that costeffectiveness analysis (CEA) tools need to address, but existing stochastic CEA tools have rarely examined risk and return from the perspective of population benefits, concerning the benefits of a group of individuals but not just a typical one. This paper proposes a stochastic CEA tool that supports medical decisionmaking from the perspective of population benefits of risk and return, the riskadjusted incremental costeffectiveness ratio (ICER). The tool has a traditional form of ICER but uses the cost under a riskadjusted expectation. Theoretically, we prove that the tool can provide medical decisions trimming that promote the riskreturn level on population benefits within any intervention structure and can also serve as a criterion for the optimal intervention structure. Numerical simulations within a framework of mean–variance support the conclusions in this paper.
Key points for decision makers

The typical assumption in classical CEA that all get the new intervention versus standard of care may not be the best to achieve the best outcome to population, a mixed structure can be better

The intervention structure should be modified using a criterion considering slight changes on the structure of the treatment mix.

Use a risk adjustment concerning cost and outcome uncertainties in taking expectation in ICER calculation gives the optimal treatment mix for population benefits.
Introduction
The classical costeffectiveness analysis (CEA) framework is of high importance in deterministic medical decisionmaking, and it has been widely applied in both theoretical and practical problems. Moreover, accounting for uncertainty has gained much attention. Some tools are developed in the framework probabilistic dominance analysis, using the CEA result under different random scenarios to get a judgement with uncertainty. For example, the costeffectiveness acceptability curve (CEAC, see Stinnett and Mullahy [1]) uses different monetary standard in judging the probability of a new intervention to be costeffective, and finally propose the curve describing the probability of the intervention to be accepted under different monetary standard. Another kind of tools use riskadjusted performance measurement (RAPM) to deal with the net monetary benefit (NMB), to determinate if the NMB is positive when taking the risk of expenses and outcomes as a kind of cost. For example, the cost‑effectiveness riskaversion curve (CERAC, see Sendi [2]) uses the downside variance as a measure of risk and measuring the benefittorisk ratio as a RAPM to generate the curve. These tools present effective approaches for making medical decisions under uncertainty (see Al [3]). However, Sendi et al. [4] show that the most frequently used CEACs still have many weak points, as it is not sensitive to the changes of radial shifts of the joint distribution of incremental costs and effects in the NorthEast and SouthWest quadrants of the costeffectiveness plane. Also, the level of risk aversion is not fully collaborated into these existing tools.Article structure: Kindly check and confirm whether the "Key points for decision makers" section is correctly processed. Yes，I have checked the section of key points for decision markers, I make sure that it is correctly processed.
Besides, other challenges confront the existing stochastic CEA methods. Correlation structure problems are noted by Barton et al. [5], showing that the correlation between the costs and outcomes will weaken the reliability of the conclusions of existing stochastic CEA tools. Also, it is mentioned that correlations of outcomes among different opinions from different individuals will also influence the judgment. Asymmetric uncertainty issues, as highlighted by Jakubczyk et al. [6], which can stem from smallsized samples, also could make troubles. This issue could cause the intervention with higher costeffective probability to get a negative incremental net benefit. Moreover, the suboptimality of a unique intervention, as mentioned in Sendi et al. [7], exists because of the heterogeneity of patients. The population of patients may consist of different etiologies or genotypes, and some are more suitable to the new intervention but some are not.
The challenges arising from correlation structures and the constraints from unique interventions can be addressed through the application of population benefits analysis tools. A population benefits analysis is to analysis the costs and outcomes, with consideration about risks and uncertainties, for a group of individuals instead of unique or typical ones. For the correlation structure problem, population benefits analysis can effectively differentiate the correlation of benefits for different patients within the same intervention and the correlation between different interventions’ outcomes and costs. Also, for the problem of the constraint of the unique intervention, a patient cannot choose to receive two interventions simultaneously, but a population of patients can choose to mix different interventions proportionally.
Furthermore, specific choices and assumptions in some methods have also been challenged in practical applications. Elbasha [8] compared the riskadjusted performance measure of CERAC with other methods and found that choosing the Sortino ratio as a RAPM implies a highly riskaverse assumption, which causes the method to fail when the real level of risk aversion is not high. Kim et al. [9] also show that most studies lack proper considerations when selecting broader evaluation perspectives. Lomas [10] directly points out the nonmarginality issue of existing methods, which may cause seemingly costeffective interventions to be rejected in actual applications. A large proportion of the judgements are made under the current situation, which will only be reliable if the change is tiny enough. However, the suggestion of these judgements directly aims at further changes, which is out of the margin of the present one and hard to be reasonable. Paulden [11] challenges the function of traditional ICER as a “distance” by pointing out its impossibility on intervention ranking. These findings collectively suggest that a riskadjusted ICER concerning on the benefits of the population with marginal judgements needs to be developed.
To this end, this paper proposes a stochastic costeffectiveness analysis tool aiming at enhancing population benefits. We also present a discriminant tool for judging whether a medical decision changing an intervention to another for a small part of the population would contribute to this goal. Furthermore, we propose the analytical expressions for the discriminant measure in a traditional ICER form with good economic interpretation. Also, the properties and superiority of the proposed approach are shown with numerical simulations with some general assumptions on mean and covariance matrix. At last, the feasibility and excellence of the mixed interventions is verified by an empirical example.
The optimization problem of population benefits
Population benefits refer to the total level of benefits for a specific group of patients. Different from the classical individual benefits problem, population benefits assess the total benefit level for all patients in the group. A group of patients with a specific condition, namely a patient population, may have different sociodemographic characteristics, may have different concomitant conditions, may have a different treatment history for the specific condition and may respond in different ways to the new intervention if they choose to try it. Therefore, population benefits analysis should be introduced to allow for the coexistence of different medical interventions and also takes the variability and correlation among the patients into account. The stochastic optimization problem aimed at the population benefits forces maximizing the overall population benefits by solving for the optimal structure of medical interventions while accounting for the patient heterogeneity.
Model settings
Consider a specific population of patients, and the number of patients in the population is denoted as M. There are N optional interventions that may be effective for this population, with each intervention possessing potential or existing feasibility. Let M^{i} represent the number of patients in the group who have received the ith intervention, where \(i \in \{ 1,2,\ldots,N\}\). Every patient requires an intervention, hence \(\sum_{i = 1}^N {M^i } = M\). The vector \({{\mathbf{M}}} = \{ M^1 ,M^2 ,\ldots,M^N \}\), composed of all M^{i} values, represents the intervention structure adopted by this patient population. It is worth noting that some patients may not be taking any intervention, and “to take no intervention” is also a kind of intervention, named “Null Intervention” in our settings.
The net monetary benefit (NMB) is a measurement used to determine an individual's benefit level in monetary. This calculation employs a monetization coefficient λ to convert the qualityadjusted lifeyear (QALY) that a patient receives through a given intervention to a monetary value and forms a monetization return measure in summary. For the patient m^{i} who receives the ith intervention, their level of NMB, \(X_{m^i }^i\), is defined as:
As the costs incurred by patients during the intervention and their QALY outputs are stochastic, each \(X_{m^i }^i\) represents a random variable. These random variables representing NMB levels belong to a random space, \((\Omega ,{{\mathcal{F}}},{{\mathbb{P}}})\). For any i, the expected value of \(X_{m^i }^i\) is denoted as \({{\mathbb{E}}}[X_{m^i }^i ] = \mu^i\). It is generally assumed that \(\{ X_{m^i }^i \}\) has crossmoments of any order, which means that one could calculate the moments of these variables in any orders.
Under a given intervention structure M, the population benefits S(M) of the studied population can be determined by summing up the NMB of each case. Therefore, the following definition is proposed.
Definition 1
(Population Benefits) We define the population benefits for the population of the patients with some specific conditions as a function \(S( \cdot ):\;R^N \mapsto R\) that is Ndimensional. Specifically, given an intervention structure M, we express S(M) as:
Optimization problem
The stochastic optimization problem for population benefits is to find the optimal intervention structure M*, which balances the expected benefits of the population and the risk level of them. This means that we want to get a higher population benefit in average, but do not want it to be too volitive. That could be done by optimizing a riskadjusted performance measure on the population benefits. A riskadjusted performance measure is a functional taking the average benefit and its uncertainty into consideration. It is monotonically increasing with respect to the expectation on benefits, but with risk penalties. Typical stochastic CEA studies use specific riskadjusted performance measures, such as the Sharpe ratio and the Sortino ratio, to measure such tradeoffs by pursuing expectation while controlling variance. This article employs a more generalized family of riskadjusted performance measures, \({{\mathbb{E}}}[ \cdot ]/\rho ( \cdot )\), as the target functional of population benefits in riskreturn tradeoffs, where \(\rho ( \cdot )\) is a onedimensional nonnegative functional used to represent the risk measure applied to the random variable of the population benefits (see Artzner et al. [12]). The risk measures commonly utilized in typical stochastic CEA research include variance (such as in the case of deterministic equivalence under the Constant Absolute Risk Aversion framework), standard deviation (such as in the case of the Sharpe Ratio), and downside deviation (in the case of the Sortino Ratio). Additionally, other risk measures such as value at risk (VaR) and conditional tail expectation (CTE) are employed in more generalized researches of risk.
Under the riskadjusted performance measure, the target function of the stochastic optimization problem in this paper, which represents the riskreturn ratio of population benefits, can be expressed as
With this objective, subject to the constraint \(\sum_{i = 1}^N {M^i } = M\), the stochastic optimization problem can be represented as:
Similar to the discussion in Buch et al. [13], when M is sufficiently large, the risk measure is subadditive, and \(\rho^{ij} \in (0,1),\forall i,j\), the optimization problem (4) has a unique positive inner solution, denoted as M*. This inner solution represents the optimal intervention structure that maximizes the riskreturn ratio of population benefits.
The necessary conditions for the optimal solution of the problem
Assuming that M is sufficiently large, we can assert that \({{\mathbb{U}}}({{\mathbf{M}}} + {{\mathbf{e}}}^i )  {{\mathbb{U}}}({{\mathbf{M}}})\) can be approximated by \(\partial {{\mathbb{U}}}({{\mathbf{M}}})/\partial M^i\), where e^{i} is an Ndimensional basis vector with the ith element being 1 and the remaining elements being 0. Under this situation, the optimization problem (4) can be solved using the Lagrange method. Specifically, the following theorem specifies the necessary conditions that the optimal intervention structure should satisfy.
Theorem 1
Assuming that M is sufficiently large, the inner solution for the optimization problem expressed in formula (4), which is the optimal intervention structure M, must satisfy the following necessary conditions:
where γ is a nonzero constant.
Proof of Theorem 1
For Eq. (4), We construct the Lagrange function,
where γ is the Lagrange parameter which must be nonzero due to the nature of the constraints. Thus, the inner solution must satisfy the condition \(\partial {{\mathcal{L}}}({{\mathbf{M}}},\gamma )/\partial M^i = 0\) for any i, then we can get
which are necessary conditions. Moreover, the properties of risk measures stipulate that \(\rho (S({{\mathbf{M}}})) \ne 0\), then we get Eq. (5). □
Theorem 1 shows that under certain conditions, the optimal intervention structure can be obtained by solving the system of equations composed of Eq. (5) and \(\sum_{i = 1}^N {M^i } = M\). For medical decisionmaking situations with adequate decision independence, selecting the intervention structure derived from solving these equations can lead to the maximum riskreturn ratio of the population benefits.
Stochastic costeffectiveness analysis
In general, the medical decisionmaking is often limited by various internal and external factors. It is challenging for decisionmakers to fully reconstruct the optimal intervention structure of the group within a single period. To face this situation, we introduce the stochastic costeffectiveness criterion for population benefits for that any two interventions could be compared. One can also find the similarities and differences between the tools in this paper and traditional CEA from the criterion.
From a population perspective, the superiority of one intervention over another can be understood in a form of marginal contribution. In other words, if the intervention for a unit of patients switches from one to another and the RAPM of the population benefits increase, then the latter intervention can be considered to be marginally superior to the former. With the model framework and theorem presented earlier, we can obtain the marginal optimization theorem for population benefits, which enables us to determine whether one of any two interventions satisfies the superiority criterion for stochastic CEA over the other under the current situation.
Marginal optimization theorem for population benefits
The marginal criterion for stochastic CEA on population benefits can be expressed through the following theorem.
Theorem 2
(Marginal optimization theorem for population benefits): The equivalence criterion for an intervention to dominate another in a population sense is that the additional riskadjusted expected cost to obtain a unit increase in expected QALY is less than the monetization factor \(\lambda\).
Mathematically, for any \(i,j \in \{ 1,2,\ldots,N\}\), if \({{\mathbb{E}}}[{\text{QALY}}^j ]  {{\mathbb{E}}}[{\text{QALY}}^i ] > 0\), then
where \({{\mathbb{E}}}^Q [{\text{Cost}}^j ]\) is the riskadjusted expected cost under the intervention j, defined as
and \(\alpha ({{\mathbf{M}}}) = {{\mathbb{E}}}(S({{\mathbf{M}}}))/\rho (S({{\mathbf{M}}}))\) is the current level of riskreturn for population benefits.
Proof of Theorem 2
First, the substitution definition, \(D(c) = {{\mathbb{U}}}({{\mathbf{M}}}  c{{\mathbf{e}}}^i + c{{\mathbf{e}}}^j )\), denotes the returntorisk level of population benefits after transferring patients of an amount of c from intervention i to intervention j. By the definition of partial derivatives, \(D( \cdot )\)’s continuity for c and the uniform integrability of S(M), there is
and then only the equivalence condition of \({\mathop {\lim }\limits_{c \to 0^+ }} \frac{\partial D(c)}{{\partial c}} > 0\) needs to be examined.
Since we have
we can obtain
Since \(\rho (S({{\mathbf{M}}})) > 0\), the equivalence condition for \({\mathop {\lim }\limits_{c \to 0^+ }} \frac{\partial D(c)}{{\partial c}} > 0\) can be expressed as
and there is \(\forall i,\quad \mu^i = \lambda {{\mathbb{E}}}[{\text{QALY}}^i ]  {{\mathbb{E}}}[{\text{Cost}}^i ]\), which is substituted to express the equivalence condition as
Noting that
there is
then the theorem is proved with \({{\mathbb{E}}}[{\text{QALY}}^j ]  {{\mathbb{E}}}[{\text{QALY}}^i ] > 0\). □
It can be seen that Theorem 2 provides a way of comparing the marginal costeffectiveness of any two interventions when considering uncertainty. The expression on the lefthand side of Eq. (8) measures the change in the riskreturn ratio of population benefits when switching from intervention i to intervention j on a small unit of patients. The expression on the righthand side of Eq. (8) measures the incremental riskadjusted expected cost required to achieve one unit of incremental QALY, by transforming from intervention i to intervention j. In other words, under the current intervention structure, an intervention switch results in an incremental riskadjusted expected cost ratio for gaining incremental QALY less than \(\lambda\) is beneficial to the riskreturn ratio of population benefits.
It is worth noting that although the criterion in Theorem 2 has a form similar to that of classical ICER, the main difference lies in the measurement of “expected costs”. Here, the incremental expected cost needs to be measured from the perspective of risk adjustment, that is, considering the “implicit” adjustment corresponding to the risk. In the definition formula, Eq. (9), of the riskadjusted expected cost, α(M) is the riskreturn level for population benefits under the current intervention structure, which can be understood as the current returntorisk rate. The \(\partial \rho (S({{\mathbf{M}}}))/\partial M^j\) is a risk allocation method (often called a marginal allocation, see Denault [14]) and can be understood as the risk level allocated to one unit of patients receiving intervention j when the risk measure of the population benefits is \(\rho (S({{\mathbf{M}}}))\). It is worth noting that although it is named as “riskadjusted expected cost”, it does not mean that the adjustment only considers the risk in costs. Instead, all risks and uncertainties are taking into consideration, containing the risks of costs and outcomes. The name is just because the adjustment is made on the term of cost. Furthermore, the riskadjusted expected cost can be understood as the sum of the expected cost (used in classical CEAs) and the risk cost allocated to the intervention. And then, the criteria are constructed using such riskadjusted expected costs.
Riskadjusted ICER
The criterion in Theorem 2 uses a specific form of ICER as a measure to discriminant the superiority of any two interventions measured under the consideration of stochasticity. Such a criterion is defined as the riskadjusted ICER due to the utilization of riskadjusted expected costs.
Definition 2
(Riskadjusted ICER): Under the intervention structure M, if \({{\mathbb{E}}}[{\text{QALY}}^j ]  {{\mathbb{E}}}[{\text{QALY}}^i ] > 0\), the riskadjusted ICER, \({\text{ICER}}_{i,j}^Q ({{\mathbf{M}}})\), of switching intervention from i to j, is defined as:
where \({{\mathbb{E}}}^Q [{\text{Cost}}^j ]\) is the RiskAdjusted Expected Cost under intervention j, defined as:
and \(\alpha ({{\mathbf{M}}}) = {{\mathbb{E}}}(S({{\mathbf{M}}}))/\rho (S({{\mathbf{M}}}))\) is the riskreturn level of the current population benefits.
It can be seen that riskadjusted ICER is an incremental costeffectiveness ratio based on the riskadjusted expected cost. Therefore, riskadjusted ICER is a generalization of classical ICER considering all sources of risks. Compared with other CEA methods that consider stochasticity, riskadjusted ICER has several typical advantages. First, from the perspective of the riskreturn ratio of population benefits, riskadjusted ICER allows different interventions to coexist, solving the “either black or white” challenge faced by classical CEA for individuals. Second, riskadjusted ICER is a marginal discriminant condition that incorporates the current intervention structure into the decision of optimality, allowing medical decisionmakers to gradually adjust interventions. This means that the decision is made under the current situation, which ensured the reliability. Third, the riskadjusted ICER accommodates any degree of risk aversion in the risk measure selection, allowing medical decisionmakers to select any risk measure for CEA analysis without the need for special discriminant tools for each measure. Finally, riskadjusted ICER retains the classical ICER form, and the monetization constant \(\lambda\) remains unchanged from the original setting, ensuring the explicit distinction between cost and QALY in the expressions.
Technically, it is worth adding the condition, \({{\mathbb{E}}}[{\text{QALY}}^j ]  {{\mathbb{E}}}[{\text{QALY}}^i ] > 0\), which actually only requires a specified order of intervention i and intervention j according to their \({{\mathbb{E}}}[{\text{QALY}}]\) values when comparing the two interventions. Indeed, excluding interventions that are exactly equivalent in expected QALY, the set of situations with superiority of j over i and situations of i over j are complementary. The decision result can be obtained by comparing in any determined order. In other word, one can always choose the intervention with larger expected QALY to be the intervention j, and use these two Theorem to may decisions. This means that the condition is without loss of generality, and therefore, both Theorems 1 and 2 are stated under a specified order, and they also have equivalent dual forms in the other order, which will not be repeated here.
Numerical and empirical examples
In order to further clarify the properties of the method proposed in this paper and demonstrate its association and difference with classical CEA tools, we perform numerical simulations to calculate the results, differences, and effectiveness of various types of uncertain CEA tools. Further, we take an empirical example to show that the mixed structure of interventions could get a higher population benefit, and the tool in this paper helps to find it.
Stochastic system settings
We choose the mean–variance system to describe the stochasticity of the simulation system to simply and clarify the simulation processes and results. It is assumed that the joint distribution of the costs and QALY outcomes of individuals receiving various interventions follows an overall joint multinormal distribution. Therefore, characterizing all the stochasticity can be accomplished by setting the means, standard deviations, and the matrix of the correlation coefficient. For each intervention, it is assumed that the costs and QALY outcomes of each patient taking this intervention are identically distributed but not independent. And, there is a certain correlation between the costs and QALY outcomes of individual patients. The correlation between patient costs of different interventions is only related to the intervention they take, but without distinguishing individuals. Same assumption applies to QALY outcomes. In addition, it is assumed that the cost of any patient is independent of the QALY outcome of other patients, which means that the cost of patient A does not correlate with the QALY outcome of any patient B.
Correlation matrix can express the structure of correlation much clearer. Specifically, for each group of patients taking the same intervention, the costs and QALY outcomes of each patient have unique mean and variance values. That is, for any i and m^{i}, we have
And for the correlation coefficient of each cost, there is
and for each QALY there is a similar equation
In addition, the crosssectional relationship between cost and QALY output is assumed as
The structure of each correlation is schematically shown in the following Fig. 1.
Simulation methods and settings
Given a selected intervention structure, the numerical simulation uses Monte Carlo sampling to form a column vector that combines the costs and QALY outputs of each patient under the assumed parameters. A sufficient number of samples are generated and the statistical features of the overall results (such as population benefits and total costs) are calculated. Then, the results of the objective function under the given intervention structure are obtained. Following this, different stochastic CEA methods are applied to this problem to get their own judgements on optimal interventions. Performance of population benefits are compared with these optimal interventions. Moreover, simulations are performed again for each method's recommended intervention structure modification, showing the effect of recommendations for different stochastic CEA methods’ judgements. Some key intermediate results are also recorded and presented.
Simulation process
The purpose of simulation mainly lies in three aspects. Firstly, illustrating through numerical examples that, in terms of population benefits, the mixture of different interventions could have a better returntorisk result than an absolutely unique intervention. Secondly, under different intervention structures, calculating the riskadjusted expected cost and riskadjusted ICER proposed in this paper, and verifying the process of inducing the optimal riskreturn ratio of population benefits in a marginal sense. Thirdly, comparing the decisions generated by different stochastic CEA methods for the same problem, testing the performance of different methods on population benefits, and checking their suggested direction of optimization.
Specifically, we first select two interventions from all optional ones, separately calculate their riskreturn levels of population benefits, and then mixes them in different proportions to form intervention structures. Secondly, under different intervention structures, the riskadjusted expected cost and riskadjusted ICER of the two interventions are calculated, along with the corresponding decisions that the criterion makes for optimizing the target function. Then, to show the effectiveness of each method, a specific intervention structure is selected, and different stochastic CEAs are used for making determinations between any two interventions. The changes in the performance of population benefits are measured after adjusting the intervention structure according to the corresponding determination. These changes can show the effectiveness of different methods.
In the simulation process, it is necessary to simulate the QALY outcomes and costs under the selected intervention and evaluate the population benefits quantitatively. In detail, under each selected intervention structure M, the evaluation of the returntorisk ratio of the population benefits level consists of the following steps in the simulation process:

1)
Divide the M simulated patients into N groups, where the ith group has M^{i} patients according to the structure of M.

2)
Combine the random variables of cost and QALY output of each patient into a random vector and calculate the mean and covariance matrix of the random vector.

3)
Generate 10,000 samples of the group using the calculated mean and covariance matrix as the parameters of a highdimensional multinormal distribution.

4)
Measure the population benefits of each sample of the group.

5)
Summarize the population benefits of each sample and measure their statistical characteristics and returntorisk ratios.
Since the simulation can only generate a limited number of samples, selecting extremely tailed risk measures may cause the simulation results to be unstable. Therefore, we repeat the evaluating steps mentioned above 100 times independently and use the average value as the final result. Finally, we use empirical probability of correctly recommendations to estimate the accuracy of different CEAs.
Parameter settings and reference methods selection
To balance the adequacy of alternative interventions and the interpretability of results, we select a total of N = 5 interventions and assume that the patient population consists of M = 100 individuals. The number of samplings of the outcome results of any patient subgroup is chosen to be 10,000, and the number of repeated experiments for the returntorisk evaluation of population benefits is chosen to be 100 times.
The assumed distribution parameters within intervention subgroups are shown in Table 1:
In addition, the correlation coefficients between different individual outcomes among different groups are set to \(\forall i \ne j,\;\rho_C^{ij} = 0.005,\;\rho_Q^{ij} = 0.002\), and the monetary constant, \(\lambda\), is set to \(\lambda = 2000\;\$ /{\text{QALY}}\) unless specifically mentioned.
Besides the method proposed in this paper, we select four different stochastic CEA methods as reference methods to participate in the evaluation together. The first is the classical ICER method, where the criterion measure is
The second is the Sortino ratio method (see Elbasha [8]) based on NMB, which compares the Sortino ratios (a kind of returntorisk ratio using the expected benefit dividend by the downside variance) of different interventions to determine their priority. The third and fourth methods are the probabilistic methods based on CEAC, which determine the superiority of the interventions by judging whether the α percentile of the ICER distribution, \(Q_\alpha (\frac{{{\text{Cost}}^j  {\text{Cost}}^i }}{{{\text{QALY}}^j  {\text{QALY}}^i }})\), satisfies the requirement of the willingtopay level (\(\lambda\)). The third method is relatively aggressive, using α = 5%, while the fourth method is relatively conservative, using α = 95%. Comparisons are made between the method proposed in this paper and these four reference methods.
Simulation results
The simulation results are mainly presented in three parts. The first part shows the superiority of mixed intervention compared to unique intervention from the perspective of population benefits. The second part displays the relationship between changes in the riskadjusted ICER and the intervention structures, which demonstrates the marginal optimality under different intervention structures. The third part shows the preference decisions of different interventions by different stochastic CEA methods under specific intervention structures, which demonstrates the validity of the decision by the tool of this paper.
Population benefits with mixed intervention
In this subsection, we evaluate the population benefits under different intervention structures through simulation, and the incremental population benefits introduced by intervention mixing. It is verified that mixed interventions can lead to better returntorisk ratios. Specifically, intervention 4 and intervention 5 are chosen as the controlled subgroups of this part, while the first three interventions have proportions of 0 in the structure, i.e., M^{1}, M^{2}, M^{3} = 0 to avoid extra interference. Then, the level of M^{4}/M gradually varies from 0 to 1, and the corresponding returntorisk ratios of the population benefits are calculated. The results are shown in Fig. 2.
From Fig. 2, it can be seen that as the proportion of intervention 4 gradually increases from 0 to 1, the returnofrisk ratios of population benefits shows a concave pattern, which firstly increasing and then decreasing. The target function reaches its optimum at around M^{4}/M = 0.4. The optimum level, 0.0227, is significantly higher than the returnofrisk ratio of 0.0173 at M^{4}/M = 0 and 0.0127 at M^{4}/M = 1. This suggests that if considering the medical decisions with intervention 4 and intervention 5, using a 4:6 structure to mix both interventions is more efficient in a population view than using either intervention alone. This validates that it can be superior to mix interventions for population benefits.
Riskadjusted expected cost and riskadjusted ICER
Following the previous subsection, we calculate riskadjusted expected cost of both interventions and the riskadjusted ICER for switching from intervention 4 to intervention 5 marginally as we gradually adjust the proportion of M^{4}/M in the [0, 1] range. Through the simulation results, it is shown that the riskadjusted ICER changes can lead the intervention structure to achieve the optimum. The simulation results are shown in Fig. 3.
In Fig. 3, the red curve represents the riskadjusted expected cost of intervention 4, and the blue curve represents that of intervention 5. It can be seen that as the proportion of intervention 4 increases, the riskadjusted expected costs’ difference between intervention 5 and intervention 4 changes from extremely positive to extremely negative. This is caused by changes in the diversification structure. Specifically, when the proportion of intervention 4 is small, there are many cases using intervention 5 in the population. At this time, increasing one unit of intervention 5 cases is equivalent to adding a highly positive correlated risk to the total risk pool, resulting in a relatively high level of risk allocated to the newly added one. This makes the riskadjusted expected cost of the new unit of intervention 5 high at this situation. In contrast, the riskadjusted expected cost of intervention 4 is relatively low because it is relatively independent with the risk pool. This pattern validates that the riskadjusted expected cost does take the marginal impact of intervention restructuring on the risk pool into account in the risk adjustment.
In addition, the black curve in Fig. 3 represents the evaluation results of the riskadjusted ICER from intervention 4 to 5. It can be seen that this curve intersects with the \(\lambda = 2000\) threshold at around M^{4}/M = 0.4, the optimal structure point. To the left of the intersection point, the riskadjusted ICER is higher than the monetized constant, indicating that switching one unit of patients from intervention 4 to intervention 5 is not costeffective and has a negative impact on the population benefits. Therefore, the proportion of intervention 5 needs to be reduced. On the contrary, to the right of the intersection point, the opposite is true. It is worth noting that the intersection of the black curve in Fig. 3 with \(\lambda = 2000\) coincides perfectly with the position where the returnofrisk ratio of population benefits gets its optimum in Fig. 2. This also confirms that the riskadjusted ICER can induce changes in the intervention structure to achieve the optimum.
Comparison of methods
In this subsection, we selected a specific intervention structure (equalweighted) as the starting point to examine the decisions of different stochastic CEAs for adjusting interventions and quantified the corresponding impact on the distribution of the population benefits. These results are shown in Fig. 4.
“Traditional ICER” represents the results of the traditional ICER tool, “Riskadjusted ICER” represents our proposed method, “Sortino Ratio” represents the decision made using the Sortino ratio tool, “CEAC at 5%” represents the aggressive CEAC method, and “CEAC at 95%” represents the conservative CEAC method.
In Fig. 4, each subplot presents the information in a 5 × 5 matrix, where the grid in the ith row and jth column of any subplot represents the decision made by the corresponding stochastic CEA tools or its correctness, for changing an unit of patients from intervention i to intervention j. The decisions are shown in the subplots of the first row, where + 1 represents a decision to accept the change, − 1 represents a decision to accept an inverse change, and 0 represents no modification. The correctness of the decision results is displayed in the subplots of the second row, where the number in each grid represents the percentage of scenarios where population benefits are improved after the changes are taken based on the corresponding decision, within the 100 times of simulated scenarios. Additionally, the cells on the diagonal of each subplot do not contain any information.
From this comparison, it can be seen that the decisions of the traditional ICER and CEAC methods differ from our proposed method and the Sortino method to some extent. Specifically, the traditional ICER does not consider the stochasticity and only requires the ICER to be below the monetized constant in an expected view for each accepted intervention change. And, the CEAC method, similar to the traditional ICER, does not use the information contained in the current intervention structure. The Sortino Ratio method obtained similar decisions with respect to our proposed method, which can be attributed to the similarity in riskadjusted measures between the two methods. However, the Sortino Ratio method still does not consider the current intervention structure, so there are still differences. These differences are not very significant due to the low level of correlation assumed in the simulation.
Also, it can be found that only our proposed method can almost perfectly ensure that the decisions to achieve an accuracy over 50%, which also reaches the highest level among all methods in expectation (The absolute value of the accuracy is related to the number of simulated samples, so the relative level of the accuracy of each method is mainly compared.). In addition, except for the judgement^{Footnote 1} between intervention 2 and 1, our proposed method achieved the highest accuracy rate among all methods for any pair of comparisons. This also validates the effectiveness of our proposed method.
An empirical example
In this subsection, we discuss an empirical example using the information extracted from recent research on comparing direct oral anticoagulants compared to lowmolecularweightheparins for treatment of cancer associated venous thromboembolism, see Muñoz et al. [15]. It is worth noting that we just use some of the parameters from the research, but do not try to answer or challenge any problem or conclusion about this health technology assessing problem in real world. For short, we just borrow some parameters from this work to show that the population can benefit from mixed interventions, and we can find the optimal mix structure with our tools.
There are four kinds of interventions introduced in this article, the low molecular weight heparin (LMWH), the Apixaban, the Rivaroxaban and the Edoxaban. All these four interventions could be used for treatment of cancer associated venous thromboembolism. The paper shows the expectation of costs and QALYs (12 months) of all these four interventions, and we can roughly extract the coefficient of variation (CV, standard error divided by mean) form the confidence intervals and simulation figures. With some more assumptions, like \(\lambda = 30000\), the parameters in our example are shown in Table 2.
To show the benefit from mixed interventions and the optimality of our proposed structure, we compare six situations. Four of them are using these four interventions alone, the fifth is an equal weighting structure and the final is the structure generated from our tool. The mean, variance and the returntorisk ratio are calculated for all these six situations, and the results are shown in Table 3.
We can find in Table 3 that none of the four interventions alone could make the returntorisk ratio of population benefit more than 0.100, but just a simple mix with equal weight can make it 0.112, which clearly shows the benefit from mixing interventions. Moreover, simple mix does benefit, but only the optimal structure generated by our tool reaches the highest ratio of 0.148. This is around 0.7 times higher than solely use the intervention suggested by classical CEA tools.
The optimal structure is {0.00%, 32.62%, 33.22%, 34.16%}, which gives zero weight on LMWH and nearly equal weights on Apixaban, Rivaroxaban and Edoxaban. This consistent with the judgment of the original article that the intervention of LMWH is dominated. This also shows the reliably of our tool.
Discussion
The tool proposed in this paper may suggest the population to use mixed interventions instead of a unique one. The population benefit can result in a controlled risk level due to diversification effect. To suggest mixed interventions in the real world has many considerable benefits, and still has kinds of challenges.
Besides risk consideration, mixed interventions could give the population a chance to accept an innovated intervention gradually, to avoid potential longtailed losses, such as some longterm adverse effects that have not yet been discovered by clinical trials (such as the example of Thalidomide). Moreover, unique intervention decisions could result in high concentration risk and the potential monopolies. An intervention provider who won the winnertakesall game can make a much higher price without competitors. Third, there is also heterogeneity in patients' willingness to pay and mixed interventions leaves room for patients to make their own decisions. At last, mixed interventions tend to give chance to innovated interventions, even just a little part of the market, to encourage technological advances.
However, it is necessary to face up to the challenges of mixed interventions. First, mixed interventions may confuse clinical decision makers in making a finally decision for individuals. This challenge could be settled by give soft suggestions with room for choice. Clinical decision makers could select the interventions accepted in the mixed interventions with their own expertise. Indepth differential diagnosis and intervention may yield better outcomes. Second, mixed interventions may force the healthcare settings to prepare the ability to provide all kinds of interventions in the mix, which will cause a lot of waste in resources. This can be solved by an integrated management. We can suggest a big hospital A to provide robotic surgeries and allow a primary care setting B to provide minimally invasive ones. Interventions could be allocated according to their own preference and specialties. The mix should be kept in an integrated view. Third, it can be difficult to keep the structure of mixed interventions at the optimal point. That is also a reason that we proposed a marginal adjustment tool. The real world is dynamic, and we should manage it dynamically. Even we can’t always keep an optimal structure, we can make slight adjustment according to the marginal adjustment criterion proposed in this paper to get a better population benefit.
Also, some statistical tools might be introduced in real world estimation of the parameters used in our method. For the correlation between costs and outcomes for individuals, it is already contained in classical HTA data records, but just neglected in classical assessment processes. For correlation among different patients, retrospective statistical analysis in the real world can help to get the estimations. Moreover, sensitivity tests can examine the reliably of results. More econometric tools can help in estimating systematic uncertainties.
At last, it is worth to mention that the tool we proposed has a good property of generalization. Although the criterion is named as riskadjusted expected costs, we consider all kinds of risks and uncertainties for interventions including but not limited to costs and outcomes. Moreover, this tool support for all types of risk measures, which means that the decision could be made in any level of risk aversion. Also, the criterion, riskadjusted ICER, retains the same expression form as the classical ICER, keeping high comparability with previous conclusions and low tool substitution costs. The riskadjusted expected cost used in the numerator also has clear economic implications. The risk adjustment item added to the expected cost is obtained by multiplying the cost rate of risk by the allocated risk level, which is consistent with the economic logic of risk premium. And finally, it is not sensitive to the measure of life quality and currency.
Conclusion
This paper proposes a stochastic CEA tool that considers uncertainty from the perspective of population benefits. The tool can evaluate the costeffectiveness of different interventions under the current intervention structure, considering all kinds of risks in costs and outcomes. In theory, we present a necessary conditions for achieving the optimal intervention structure that maximizes the returntorisk ratio of population benefits. Furthermore, we introduce the riskadjusted ICER as a criterion for comparing any two interventions under any structure of mixed interventions. Theoretical analysis confirms that the criterion is optimizing population benefits. Results from numerical simulations and empirical examples also support these findings.
The population benefit gives a new perspective on measuring the costefficiency of a population but not individuals. And the tool we proposed helps to find the optimal mixed structure of interventions to maximize the returntorisk ratio of the population benefit. The riskadjusted ICER as the criterion can effectively induce the optimization and has clear economic implications. The conclusions obtained by the method proposed in this paper in the CEA analysis with uncertainty should be provided as decision reference to medical decisionmakers. Acknowledgements: Kindly check and confirm whether the “Preprint Acknowledgements” statement is correctly processed.Yes, the address of the preprint and the statement is correctly processed.
Availability of data and materials
Not applicable.
Notes
The bias in the comparison results of this pair of interventions is a random result with a small probability due to the limited number of samples.
Abbreviations
 CEA:

Costeffectiveness analysis
 ICER:

Incremental costeffectiveness ratio
 CEAC:

Costeffectiveness acceptability curve
 NMB:

Net monetary benefit
 RAPM:

Riskadjusted performance measurement
 CERAC:

Cost‑effectiveness riskaversion curve
 QALY:

Qualityadjusted lifeyear
 VaR:

Value at risk
 CTE:

Conditional tail expectation
References
Stinnett AA, Mullahy J. Net health benefits: a new framework for the analysis of uncertainty in costeffectiveness analysis. Med Decis Mak. 1998;18:S68–80. https://doi.org/10.1177/0272989X98018002S09.
Sendi P. Dealing with bad risk in costeffectiveness analysis: the costeffectiveness riskaversion curve. Pharmacoeconomics. 2021;39:161–9. https://doi.org/10.1007/s40273020009695.
Al MJ. Costeffectiveness acceptability curves revisited. Pharmacoeconomics. 2013;31:93–100. https://doi.org/10.1007/s4027301200118.
Sendi P, MatterWalstra K, Schwenkglenks M. Handling uncertainty in costeffectiveness analysis: budget impact and risk aversion. Healthcare (Basel). 2021;9:1419. https://doi.org/10.3390/healthcare9111419.
Barton GR, Briggs AH, Fenwick EAL. Optimal costeffectiveness decisions: the role of the costeffectiveness acceptability curve (CEAC), the costeffectiveness acceptability frontier (CEAF), and the expected value of perfection information (EVPI). Value Health. 2008;11:886–97. https://doi.org/10.1111/j.15244733.2008.00358.x.
Jakubczyk M, Kamiński B. Costeffectiveness acceptability curves–caveats quantified. Health Econ. 2010;19:955–63. https://doi.org/10.1002/hec.1534.
Sendi P, Al MJ, Zimmermann H. A riskadjusted approach to comparing the return on investment in health care programs. Int J Health Care Finance Econ. 2004;4:199–210. https://doi.org/10.1023/B:IHFE.0000036046.80562.06.
Elbasha EH. Costeffectiveness riskaversion curves: comparison of riskadjusted performance measures and expectedutility approaches. Pharmacoeconomics. 2022;40:497–507. https://doi.org/10.1007/s40273021011235.
Kim DD, Silver MC, Kunst N, Cohen JT, Ollendorf DA, Neumann PJ. Perspective and costing in costeffectiveness analysis, 1974–2018. Pharmacoeconomics. 2020;38:1135–45. https://doi.org/10.1007/s40273020009422.
Lomas JR. Incorporating affordability concerns within costeffectiveness analysis for health technology assessment. Value Health. 2019;22:898–905. https://doi.org/10.1016/j.jval.2019.05.003.
Paulden M. Calculating and interpreting icers and net benefit. Pharmacoeconomics. 2020;38:785–807. https://doi.org/10.1007/s40273020009146.
Artzner P, Delbaen F, Eber JM, Heath D. Coherent measures of risk. Math Financ. 1999;9:203–28. https://doi.org/10.1016/j.insmatheco.2007.02.006.
Buch A, Dorfleitner G, Wimmer M. Risk capital allocation for RORAC optimization. J Bank Finance. 2011;35:3001–9. https://doi.org/10.1016/j.jbankfin.2011.04.001.
Denault M. Coherent allocation of risk capital. J Risk. 2001;4:1–34. https://doi.org/10.21314/JOR.2001.053.
Muñoz A, Gallardo E, Agnelli G, Crespo C, Forghani M, Arumi D, de Cabo SF, Soto J. Costeffectiveness of direct oral anticoagulants compared to lowmolecularweightheparins for treatment of cancer associated venous thromboembolism in Spain. J Med Econ. 2022;25(1):840–7. https://doi.org/10.1080/13696998.2022.2087998.
Acknowledgements
Not applicable.
Preprint Acknowledgements
A preprint version is available: URL: https://www.preprints.org/manuscript/202304.0680/v1, DOI: https://doi.org/10.20944/preprints202304.0680.v1.
Funding
No funding has been received for the conduct of this study.
Author information
Authors and Affiliations
Contributions
EC is the sole author of this article and has read and approved the final version.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Chen, E. Stochastic costeffectiveness analysis on population benefits. Cost Eff Resour Alloc 21, 78 (2023). https://doi.org/10.1186/s1296202300488y
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s1296202300488y