Cost-effectiveness analysis of the diarrhea alleviation through zinc and oral rehydration therapy (DAZT) program in rural Gujarat India: an application of the net-benefit regression framework

Background This study evaluates the cost-effectiveness of the DAZT program for scaling up treatment of acute child diarrhea in Gujarat India using a net-benefit regression framework. Methods Costs were calculated from societal and caregivers’ perspectives and effectiveness was assessed in terms of coverage of zinc and both zinc and Oral Rehydration Salt. Regression models were tested in simple linear regression, with a specified set of covariates, and with a specified set of covariates and interaction terms using linear regression with endogenous treatment effects was used as the reference case. Results The DAZT program was cost-effective with over 95% certainty above $5.50 and $7.50 per appropriately treated child in the unadjusted and adjusted models respectively, with specifications including interaction terms being cost-effective with 85–97% certainty. Discussion Findings from this study should be combined with other evidence when considering decisions to scale up programs such as the DAZT program to promote the use of ORS and zinc to treat child diarrhea. Electronic supplementary material The online version of this article (doi:10.1186/s12962-017-0070-y) contains supplementary material, which is available to authorized users.


Background
Cost effectiveness analysis (CEA) is useful for evaluating productive efficiency in the health sector [1]. Applications of CEA in countries with strong health technology assessment programs often follow a general continuum from modeling studies to evaluations alongside clinical trials, which can inform revisions to make models more comprehensive [2]. Less attention has been paid to CEA applied to quasi-experimental study designs.
Methods have been developed which combine econometric and economic evaluation techniques by transforming incremental cost-effectiveness ratios into net-benefit statistics for use in regression analysis [3]. Net-benefit regression allows analysts to control for variables that may confound cost-effectiveness, particularly for non-identical groups, handle censored data, calculate confidence intervals parametrically, and to evaluate costeffectiveness according to subgroups. This method contrasts with more commonly used methods for estimating CEA, which assume that costs and consequences across population groups, geographic areas, and other factors of interest are homogenous for individual study arms or phases.
The growing number of programs being implemented at scale without a contemporaneous comparison arm represents a trend in evaluation efforts from demonstration of efficacy and effectiveness to delivery of interventions integrated within existing health systems. This trend increases the relevance of cost-effectiveness analyses of before and after studies [4][5][6][7][8][9]; however, these studies are not well established in the literature. The opportunity exists to expand methods for conducting CEAs alongside programs at scale [10] to include approaches which adjust for heterogeneity across study phases and populations and provide a more unbiased measure of cost-effectiveness.
To contribute to the growing body of evidence on the cost-effectiveness of treating child diarrhea with zinc and oral rehydration salts (ORS) in low-and middleincome countries (LMICs), we conducted a cost effectiveness analysis of the diarrhea alleviation through zinc and oral rehydration Therapy (DAZT) program in Gujarat India using net-benefit regression. The objective of the DAZT program was to scale-up coverage of these interventions, and reduce the irrational prescription of more expensive antibiotics and antidiarrheal medicines. Zinc and ORS were delivered through the public and private sectors through a wide variety of channels to maximize market impact. Program evaluation took a before and after study design across 2010-2013, which allowed for the evaluation of zinc itself as an intervention, as well as evaluation of the health systems delivery strategy.

Study hypothesis and rationale
This study complimented findings from another analysis which explored the economic costs to caregivers of diarrhea management [11]. The hypothesis of the present study was that the DAZT program was cost-effective in delivering zinc and ORS to treat child diarrhea episodes relative to status quo conditions in rural Gujarat existing before the program was initiated when very few children with diarrhea were given zinc. This comparator was chosen to represent the decision about scaling up the program to other districts faced by decision makers. Data suggested that the coverage of zinc and ORS to treat diarrhea among children increased in the intervention districts [12], in addition to lowering the economic burden borne by caregivers across the span of the program [11]. Consistent with other interventions delivered at the community level [13,14], we expected that program costs were distributed thinly enough across beneficiaries of the program to make economic costs per person good value for money when considered alongside health benefits.

Study setting
The DAZT program introduced zinc and promoted ORS in Gujarat, India for treatment of child diarrhea through both the public and private sectors, utilizing a variety of community and facility based strategies (Additional file 1: Table S1) [12,15]. The study area consisted of six districts in northeastern Gujarat (Banas Kantha, Dohad, Panch Mahals, Patan, Sabar Kantha, and Surendranagar, population 13 million [16]) and was chosen for its low coverage of zinc, availability of non-governmental organization partners, and priorities set by the Indian Government.

Program evaluation
Evaluation of the DAZT program consisted of household surveys spanning 2 years to represent the program at maturity, comparing 4200-5080 caregivers and their 2-59 month old children between the start and finish of the study. These surveys assessed only each household's youngest child, and the latest episode per child occurring in the last 2 weeks. Data from 613 surveys with complete data on length of diarrhea episode were sufficient to perform costeffectiveness analysis in terms of increases in coverage.

Power calculation
The main trial was powered to detect differences in coverage of ORS across trial phases [17][18][19]. For our NBR analysis, power was computed to detect meaningful differences in between a ceiling ratio (λ), or a policy maker's valuation of a correctly treated episode, and an Incremental Cost-Effectiveness Ratio (ICER) [20], translated to incremental net-benefit (Box1). Participants included in this analysis were restricted to those individuals for whom data on date of recovery were available to allow for consideration of disease duration and recovery during analyses. Power was calculated based on Glick et al. [21] to account for different sample sizes across study phases, and different standard deviations in costs and effects across phases. Alternative hypotheses were tested against a null hypothesis of no difference of cost-effectiveness from λ.
Values for incremental cost and its standard deviations for each study phase were taken from a cluster randomized controlled trial on zinc conducted in Haryana India [22]. Values for incremental coverage were taken from the expected level of scale up of ORS coverage from current levels in the study area. The expected correlation between incremental cost and effect was assumed to be 0.2 based on convention for sociological studies (Ahmed S, personal communication), given an absence of covariance data [21]. The standard Z score for 95% confidence was used for a two-tailed alpha (1.96), with sensitivity analyses on 99, 90, and 80% confidence. Sensitivity analyses were conducted on effectiveness, cost, ceiling ratios (varied between $0.50 and $13.50), and correlation coefficient. Given the available sample size, power calculations indicate that the study had sufficient power to detect a significant difference in incremental cost-effectiveness from λ across major parameterizations including half, twice, and no incremental costs; half, twice, and no incremental effects; correlation coefficients between effects and costs from 0 to 0.7, Z statistics for type 1 and 2 error, and maximum valuations for the treatment of an episode of diarrhea. drawn from program records. Costs to caregivers were assessed according to caregiver report through the household surveys, and included direct medical costs, direct non-medical costs (transportation), and indirect costs (wages lost) [25]. Cost-effectiveness according to a caregiver's perspective was assessed in sensitivity analysis.
Program costs were combined for public sector and private sector activities across starting point, midpoint, and endpoint surveys; and were divided evenly across the caregivers interviewed at the endpoint survey (Table 1). Allocation of program costs across caregivers was scaled to the population of children in the study area (n = 1,188,634), length of the recall period of the survey (cost/child/2 week period). Costs to caregivers were derived from an accompanying study, and declined from $4.04 to $2.49 per child with diarrhea during the program [11]. Program costs components included capital costs (start-up and sustainability, furniture and equipment, and training), and recurrent costs (personnel, vehicles, buildings, zinc and supplies). Annualized capital costs were considered for the number of years that they were in use-2 years for starting point and midpoint costs, and 1 year for endpoint costs. Recurrent costs were considered for starting point, midpoint, and endpoint phases. Salary costs for service providers were derived per year from the provider assessment survey ( Table 2 shows these in more detail).
Capital components were annualized using factors from the WHO-CHOICE study discounted at 3% as the

Costs and effects
Economic costs were assessed from the societal perspective, consisting of those to the caregiver and program costs borne by the non-governmental organizations (NGOs) and the government. Program costs were calculated according to Saving Newborn Lives (SNL) costing guidelines for capital and recurrent costs [24], with data rate for a risk-free investment [26] and recommendations for economic evaluation in LMICs [27]. In the absence of other data, items were assumed to last the duration of the program. Currency was inflated to 2014 base year United States dollars using consumer price indices from the International Monetary Fund (IMF), and converted using mid-year exchange rates from [28].
In the absence of data on mortality and long-term sequelae from diarrhea, outcomes were assessed in terms of episodes correctly treated with zinc and ORS. Episodes were assessed according to caregiver report of those beginning or resolving within the 2 weeks prior to survey.

Cost-effectiveness according to net-benefit regression
All children included in the baseline and endline surveys of the study were included. For each child, the net-benefit statistic was calculated by multiplying the variable representing whether a child with diarrhea was treated with zinc (E i ) by the monetary valuation of an increase in coverage by one individual with diarrhea (λ), then subtracting the economic costs associated with that individual (C i ).
The net-benefit statistic was used as the dependent variable in a series of simple and multivariable linear regressions. In these formulations, the coefficient on the treatment variable (δ) represented incremental net-benefit of the program relative to the initial survey. Simple linear regression on δ alone, representing study phase, was conducted to show that the incremental net-benefit is the same when calculated in this approach as when calculated in the standard deterministic approach.
Multiple linear regression was then conducted to evaluate net-benefit when controlling for covariates (x ij ).
Finally, incremental net-benefit for subgroups was calculated by interacting each parameter with the treatment variable (γ).
Different values of λ were tested in sensitivity analysis, including the extreme case of 0 to test negative costs as the response variable. An effects-only regression to represent the scenario where λ equals infinity was not conducted as the outcome is binary and logistic regression coefficients would not be comparable to other model specifications.

Cost-effectiveness acceptability curves
Cost effectiveness acceptability curves (CEACs) were constructed to depict the probability that the DAZT intervention was cost-effective according to different levels of λ according to established methods [3].

Regression modeling
Models were tested according to assumptions of linear regression to inform the appropriate econometric approach. Despite skewness apparent in the residual in quantile-quantile plots, ordinary least squares (OLS) estimation was deemed appropriate given that it produces unbiased estimates due to asymptotic normality. Breusch Pagan/Cook Weisberg tests indicated that the residuals did not have constant variance at different points along the regression line, and visual inspection of residual versus fitted plots confirmed heteroskedasticity. Therefore robust standard errors were used to ensure accurate measurement of p-values. Variance inflation factors indicated that there was no collinearity between covariates.
The importance of highly influential points, meaning those with the greatest potential for altering the slope of the regression line, was checked by calculating DFBETAs for adjusted models with interaction terms. Individuals with the top 10 DFBETA scores were excluded in sensitivity analysis, as using the 2/√n threshold excluded well over 10% of the data. Comparison of AIC/BIC scores indicated that omitting these points slightly improved model fit, although no difference was seen in the significance of the coefficient on the treatment variable. Since these extreme cases were considered important to the analysis to make results comparable to the CEA according to bootstrapping, reference case calculations were performed on the full dataset of children with diarrhea and known date of recovery. Covariates for the model were chosen according to those collected in the study that were relevant for an adapted form of the Andersen and Newman conceptual framework for treatment seeking [29], with hypotheses about the direction of each variable shown in Additional file 2: Table S2. Three main models were specified; (1) using a linear regression with endogenous treatment effects approach based on a conceptual framework ( Fig. 1-developed from [10], with hypotheses tied to the literature in Additional file 2: Table S2), (2), using the full set of covariates, and (3) using the set of covariates used in a related study [30]. The approach using linear regression with endogenous treatment effects was used as the reference case, which used probit regression in the reduced form equation and linear regression in the structural equation. Rationale for this choice was that treatment seeking from private providers was expected to be endogenous with a bidirectional relationship with costs, and statistical evidence using a Heckman test confirmed this expectation at λ = $0 [31]. Having blood in the stool was chosen as the instrumental variable since it was correlated with seeking care from a private provider, and was not expected to be correlated with the error term of the structural equation. Sensitivity analyses were performed testing education as a continuous variable, and collapsing treatment seeking into a single dichotomous variable. Specification using automatic selection procedures was ruled out since extensive arguments exist against both 'significance testing approaches' [32][33][34][35] and 'change in estimate approaches' [33,36,37]. In addition, automatic selection approaches including bivariable regressions and stepwise backwards selection revealed different sets of explanatory variables for the model at different levels of λ making this approach meaningless in the absence of a consensus value for λ.
To construct a wealth index for assessing the effect of assets on net-benefit, principal components analysis was performed according to standard methods [38,39], using the component with the highest eigenvalue to construct a scale for classifying caregivers into quintiles. Descriptive statistics and regression analyses were calculated in Stata 13, with program costing and acceptability curves calculated in Microsoft Excel.

Descriptive statistics and program costs
Survey participants were 18 and 21 months old at the starting point and endpoint surveys respectively, and had diarrhea for a mean of 3 days at the time of interview. Initially, less than 1% of children received appropriate diarrhea treatment, and only 22% of their caregivers had more than primary education. Among the 39% of caregivers that sought care outside of the home, 78% went to private providers (Table 3).
Program costs totaled $3.56 per child in recurrent costs, and $0.26 per child in annualized capital costs. Capital costs were higher in the public sector, which was responsible for block, district, and state level training workshops to launch the program, and for the initial seed supply of ORS and zinc. The largest component of costs to the private sector were for subcontracts to other NGOs and a pharmaceutical company for training pharmaceutical representatives in zinc use and the short messaging service (SMS) system to monitor sales, and managing of 'DAZT corners' .
Sample statistics for costs and health outcomes are presented in Table 4. Cost per average person in the sample decreased by $0.12, and diarrhea episodes correctly treated with zinc and ORS increased by 17% leading to a tradeoff between higher costs and greater effectiveness.

Regression outputs
The program was cost-effective above λ > $5.50 with 95% certainty for all simple linear specifications tested ( Fig. 2; Table 5). Adjusting the model for the set of covariates retained in the linear regression with endogenous treatment effects approach revealed a threshold of $7.50 per correctly treated episode above which the program was cost-effective with 95% certainty (Fig. 3; Table 6), and $10 for the specification with the full set of covariates (Additional file 3: Table S3). When interaction terms were added to the model using the linear regression with endogenous treatment effects approach, the program was cost-effective with above 85% certainty across levels of λ ( Fig. 4; Table 7). These results were robust to sensitivity analysis using the caregiver's perspective (Additional file 3: Table S3). Since seeking treatment from a private provider was endogenous, the linear regression with endogenous treatment effects model with interaction terms should be considered the reference case and least prone to bias. These results were variable in other approaches to choosing and defining independent variables, although many of the acceptability curves were meaningless. Including education as a continuous variable and categorizing treatment seeking as a single dichotomous variable had little effect on results (Additional file 4: Figure S1, Additional file 5: Table S4, Additional file 6: Figure S2, Additional file 7: Figure S3, Additional file 8: Figure S4, Additional file 9: Figure S5, Additional file 10: Figure S6).

Discussion
Net-benefit regression provides a useful framework for evaluating the cost-effectiveness of the DAZT program while adjusting for confounding variables (Additional file 11: Table S5). Reference case results indicate that the program was cost-effective with 85% certainty relative to conditions at the initial survey across levels of λ. Claxton has argued that 51% certainty about cost-effectiveness is sufficient to maximize net-benefit to society [40], although decision makers may be risk averse about making an incorrect decision, in which case a threshold closer to 95% would be more appropriate. Expected Value of Perfect Information (EVPI) analysis could be used to evaluate the level of uncertainty that should be acceptable to decision makers.
Subgroup effects were observed, but should not be emphasized as hypotheses about subgroups were not specified before this analysis [41], and data dredging is never recommended [42]. In addition, zinc is inexpensive, microbes do not develop resistance to zinc [43], and there are often costs associated with rationing medicine [44]; reducing the risks of, and supporting the case for widespread use. It is unlikely that policy will be made to allocate zinc according to sociodemographic variables [45], and treating sociodemographic groups differently can lead to problems with human rights and discrimination [42]. For these reasons, we did not emphasize results from sub-group analysis, despite this being a methodological functionality of net-benefit regression.
Currently, there is a lack of cost-effectiveness analyses evaluating interventions on PubMed in the Indian context indicating key gaps in the evidence base. Therefore results from this study are likely to be considered in an ad-hoc fashion by program managers and policy makers instead of being used in a more structured sector-wide decision making process [47]. In addition, this analysis is limited to cost-effectiveness, not cost-utility analysis, making it  more useful to the diarrhea community than more general decision makers in the health sector. As the evidence base of economic evaluation strengthens in India, results may be useful for its newly formed health technology assessment organization [48], pharmaceutical pricing negotiations, regulation of insurance, providing bargaining power to public health centers and procurement agencies, promotion of India's generic drug industry, and education and research purposes among other things [49]. A database of pharmacoeconomic studies has been proposed for India, in which this study could be included [49].
The proliferation of large scale public health programs in the developing world evaluated without control groups provides increasing opportunities to conduct net-benefit regression when undertaking economic evaluation. In addition, this methodology is useful for randomized trials that fail to produce a balanced allocation of covariates. However, the application of net-benefit regression in a LMIC context is relatively new, particularly with regard to maternal, newborn and child health programs. Hounton and colleagues have conducted two studies [50,51] to evaluate public health interventions in a LMIC context, including programs for skilled attendance at childbirth and community based insurance in Burkina Faso.
Consistent with WHO recommendations [52], zinc has been found to be cost-effective for treatment of nonsevere child diarrhea based on modeled evidence from Tanzania [53], and an evaluation of a social franchise platform from Myanmar [54]. Findings from hospital based efficacy trials have demonstrated mixed results; indicating a trend towards costs savings and cost-effectiveness while failing to achieve statistical significance [56,57] and in a further trial not achieving cost-effectiveness [55]. However, these studies have been small in scale and in two instances zinc was provided with copper [55,56]. Our study contributes to the evidence base about the use of zinc in real world sector-wide conditions, suggesting that may be cost-effective for scaling up coverage depending on how much decision makers value scaling up coverage of zinc and ORS by one appropriately treated patient. These findings are consistent with previous evidence in favor of the cost-effectiveness of close to client treatment of child diarrhea with zinc and ORS in a community setting [53,54]. However, scaling up requires a variety of supply and demand side factors for effectiveness to be achieved in other areas such as manufacturing capacity, supply chains, absorptive capacity, and measures to stimulate care seeking and reduce demand for antibiotics [58][59][60]. Decisions will also be affected by contextual factors beyond diarrhea considerations.
Evidence on the cost-effectiveness of zinc for child diarrhea is particularly relevant in the Indian context where the government has set policy for a national program to provide zinc to all affected children over 3 months old    [61], and diarrhea was the third leading killer of children under 5 years old nationwide in 2010 [62]. With diarrhea falling from the second to fourth largest killer of children worldwide in 2 years [63,64], this study adds to needed evidence to expedite progress in addressing this solvable problem.

Limitations
As data were not collected describing the date that diarrhea resolved past the administration of the survey, and imputation was not done to estimate these values, the analysis was conducted on only 53% of diarrhea episodes in the sample. It was not possible to determine whether the 47% of episodes for which complete duration was not assessed were acute or persistent, although only one case in the subset evaluated in complete case analysis could be classified as persistent. However, adequate power existed to detect a meaningful difference in cost-effectiveness in complete case analysis, and limiting the analysis to the acute cases clearly defined by the survey was deemed appropriate as it is consistent with other analyses of this program [65]. The evidence base on which we designed the conceptual framework was limited as net-benefit regression has not been done previously on this research question. As a result, rationale for each variable included was based on literature evaluating treatment seeking or cost of treatments. In addition, rationale for hypotheses for each variable was developed after the specifications for the dissertation draft of this model were estimated. Other variables that have been found to be associated with these outcomes but that were not measured in our survey included breastfeeding [66], perceived severity of the diarrhea episode [67,68], distance to health provider [69,70], birth order [70,71], antenatal care of the mother from trained providers [71], expected cost according to perceived diarrhea status [66], timing of careseeking [66], and frequency of episodes [66]. Further research may be conducted to determine the association of these covariates with net-benefit and their influence on incremental results. Specifications were not defined fully in advance of estimating models. While preliminary results did not affect hypotheses we formulated based on literature, the fact that we did not prespecify them or anticipate some of the challenges to specification is not ideal when estimating econometric models, recognizing that both statistical theory and improving model fit are considered in applied econometrics [72]. After analysis on our study was completed, new guidelines were released, which advocate the use of a societal perspective and health systems perspective as the reference case [73] and could be estimated from our data. During this project, methods were identified that could be used to extrapolate deaths averted from coverage levels to which our results could be compared [10]. Other endogeneity tests exist, and further work should be done on a similar dataset that was collected in Uttar Pradesh during the course of this program.
This study's uncontrolled before and after study design limits measurement of impact. Imbalances between before and after phases were controlled for through the regression approach; however, no mathematical adjustment can control for unobserved characteristics in uncontrolled study designs. In the case of this study, secular trends may have influenced results such as patterns of zinc availability that would have occurred in the absence of the program due to the national recommendation. To some extent these effects may have been countered by secular trends in costs, which may have been expected to increase given current trends of economic development in Gujarat. Therefore, we did not present analyses using episodes averted as the effectiveness measure, despite having sufficient power to do so. Authors can provide results showing that the program was cost-effective using this denominator on request. To protect against contamination to the extent possible, the program was monitored for the influence of similar program activities in DAZT areas. However, the Hawthorne effect may have magnified effects relative to the impact of the program without evaluation activities taking place. To some extent these effects may have been countered by secular trends in costs, which may have been expected to increase given current trends of economic development in Gujarat. Given that the Government of India approved zinc for use in pediatric diarrhea in 2006, evidence from cluster randomized trials suggested both effectiveness and cost-effectiveness [22,46], and the objective of this study was to evaluate zinc when delivered at scale, our study design was appropriate for these conditions.   The fact that this analysis evaluates cost-effectiveness analysis defined in natural units limits capacity to compare results to cost-utility analyses of other interventions. However, outcomes measured in terms of natural units are often used in net-benefit regression [3,50,51], and outputs of this analysis are important in other ways. The finding that controlling for confounders and sets of those included had an effect on cost-effectiveness, with different model specifications producing heterogeneous results, challenges the robustness of conclusions from calculations performed according to non-stratified nonregression based methods.

Conclusions
This study evaluated the cost-effectiveness of the DAZT program to treat child diarrhea in Gujarat India. The main finding from this study was that the program was cost-effective with 95% certainty above $5.50 and $7.50 per appropriately treated child in the unadjusted and adjusted models respectively, with specifications Multivariable regression with interaction terms using a linear regression with endogenous treatment effects approach define covariates and ORS and zinc coverage as the effectiveness measure