Skip to content

Advertisement

  • Research
  • Open Access

Cost effectiveness of mHealth intervention by community health workers for reducing maternal and newborn mortality in rural Uttar Pradesh, India

  • 1Email author,
  • 1,
  • 1,
  • 1,
  • 1 and
  • 1
Cost Effectiveness and Resource Allocation201816:25

https://doi.org/10.1186/s12962-018-0110-2

Received: 26 September 2017

Accepted: 13 June 2018

Published: 25 June 2018

Abstract

Background

A variety of mobile-based health technologies (mHealth) have been developed for use by community health workers to augment their performance. One such mHealth intervention—ReMiND program, was implemented in a poor performing district of India. Despite some research on the extent of its effectiveness, there is significant dearth of evidence on cost-effectiveness of such mHealth interventions. In this paper we evaluated the incremental cost per disability adjusted life year (DALY) averted as a result of ReMiND intervention as compared to routine maternal and child health programs without ReMiND.

Methods

A decision tree was parameterized on MS-Excel spreadsheet to estimate the change in DALYs and cost as a result of implementing ReMiND intervention compared with routine care, from both health system and societal perspective. A time horizon of 10 years starting from base year of 2011 was considered appropriate to cover all costs and effects comprehensively. All costs, including those during start-up and implementation phase, besides other costs on the health system or households were estimated. Consequences were measured as part of an impact assessment study which used a quasi-experimental design. Proximal outputs in terms of changes in service coverage were modelled to estimate maternal and infant illnesses and deaths averted, and DALYs averted in Uttar Pradesh state of India. Probabilistic sensitivity analysis was undertaken to account for parameter uncertainties.

Results

Cumulatively, from year 2011 to 2020, implementation of ReMiND intervention in UP would result in a reduction of 312 maternal and 149,468 neonatal deaths. This implies that ReMiND program led to a reduction of 0.2% maternal and 5.3% neonatal deaths. Overall, ReMiND is a cost saving intervention from societal perspective. From health system perspective, ReMiND incurs an incremental cost of INR 12,993 (USD 205) per DALY averted and INR 371,577 (USD 5865) per death averted.

Conclusions

Overall, findings of our study suggest strongly that the mHealth intervention as part of ReMiND program is cost saving from a societal perspective and should be considered for replication elsewhere in other states.

Keywords

  • mHealth
  • Cost effectiveness
  • Maternal and child health
  • DALY

Background

The state of Uttar Pradesh (UP) is one of the major contributors to maternal and child deaths in India with low coverage of key maternal, neonatal and child health (MNCH) services [1]. The coverage of institutional deliveries, full antenatal care (ANC) and full immunization in UP were 45.6, 29.6, 45.3% respectively in year 2011–2012 [2]. Kaushambi is one of the 19 high focus districts in UP and exhibits some of the worst health statistics with maternal mortality ratio and infant mortality rate being 366 per 100,000 live births and 80 deaths per 1000 live births respectively [2, 3].

In 2005, National Rural Health Mission (NRHM)—now called National Health Mission (NHM) was introduced in India with the purpose of improving various health indicators through strengthening of government health care system [1]. A new cadre of community health workers called Accredited Social Health Activist (ASHA) was introduced to generate public demand for health services [4]. ASHAs are the local women who act as community mobilizer and motivator with a prime purpose to generate demand for health care services and to serve as a link between health system and community, primarily for maternal and child health services. An evaluation of ASHAs in year 2011 found that although a 23-day training schedule has been developed by Ministry of Health and Family Welfare (MoHFW), but the quality of training needs to be strengthened in order to improve their performance [5].

To bolster and supplement the knowledge of community health workers in developing countries, mobile technology is being utilized as one of the effective and sustainable method [6]. Several positive effects of use of mHealth interventions have been noted in literature. Knowledge about the number of ANC visits among community showed an increase from 10 to 37% after introduction of text messages for health promotion in southern Indian state of Tamil Nadu [7]. Use of mHealth intervention in Afghanistan resulted in 20% improvement in antenatal attendance and 22.3% improvement in the number of women receiving skilled deliveries at a health facility [8]. In terms of quality of counselling, a study from India found increase in knowledge retention of health workers about identification of danger signs during pregnancy, after delivery, among newborns and children from 48 to 70% [9]. Beneficial effects of mHealth studies are also available from the field of maternal and child health [10], malaria [11, 12], diabetes [13, 14], HIV/AIDS [15], sexual and reproductive health [16], health behaviour change [17] etc.

Against this background, ReMiND (reducing maternal and newborn deaths) program was introduced in two blocks of district Kaushambi in state of Uttar Pradesh. As part of this program a mHealth application which runs on an open source platform was introduced as job aid for ASHA workers. This mHealth application tracks and supports clients for the ASHA workers and provides inputs for individualized service and counselling needs [18, 19]. ReMiND intervention resulted in a statistically significant increase in coverage for iron and folic acid (IFA) tablets consumption among pregnant women (12.7%), abdominal examination during ANC (18.7%), identification and self-reporting of complication during pregnancy (13.20%) and after (19.5%) delivery in the intervention area [20].

The need for optimal utilization of available resources with the help empirical data gains importance in resource constrained country like India. Realising this National Health Policy 2017 emphasised on the use of Health Technology Assessments (HTA) as a tool for taking informed decisions in scaling up health interventions in India [21]. The use of HTA is also supported by Disease Control Priority (DCP 3) and experts across globe for getting maximum health benefits out of available pool of resources [2224]. However, a systematic review of economic evaluations of mHealth interventions concluded that there is a lack of concrete evidence to fully assess the economic impact of telemedicine, e-health, and mHealth systems [25]. Deficiencies in design of studies, such as lack of randomized control trials, small sample sizes, and absence of quality data and appropriate measures further limit the relevance of findings. Furthermore, though effectiveness studies from low- and middle-income countries are available in literature, there is no evidence on cost effectiveness of mHealth interventions from India [26]. In this paper, we assessed the incremental cost per disability adjusted life year (DALY) averted as a result of the ReMiND intervention as compared to routine care without the mHealth intervention in Uttar Pradesh state in India.

Methods

Intervention setting: ReMiND intervention and theory of change

The intervention scenario comprised of routine maternal and child health care services plus the ReMiND intervention. Out of Kaushambi district’s eight community development blocks, the ReMiND was implemented in two blocks: Mooratganj and Manjhanpur. In 2012, mHealth application was implemented through 259 ASHAs in two intervention blocks serving a population of about 300,000 individuals. The ASHAs in ReMiND program were provided with basic Java-based mobile phones operating on an open-source Comm-Care software [19]. It had a tailored content, which guided the ASHA through the course of a woman’s pregnancy and newborn child care. More specifically, it was used to register the pregnant woman; update her ANC record during subsequent home visits; track her utilization of services from pregnancy into the postpartum period; and track the health of the newborn and the immunization until 2 years of age. They were given extensive trainings on use of mobile phone which also provided audio-visual support to the ASHA workers in order to counsel the pregnant woman at each of these steps [18, 19].

Data entered about the pregnancy guided ASHAs in providing timely and appropriate health information to pregnant women; and helped them to prioritize home visits. It contained algorithms to assist in the early identification, treatment, and rapid referral for appropriate care of any danger signs among pregnant women or neonates [18, 19]. Data on services which are due and those utilized by pregnant women, recorded by ASHAs through the mHealth application, were pooled on a common server. The sector facilitators used the data to monitor all the ASHAs working in their area. Data were also shared with the health education officer at the primary health centre during monthly meetings. Thus, the implementing NGO partners—Catholic Relief Services (CRS) and Vatsalya worked in coordination with the district health system to monitor the performance of ASHAs using data generated by the mHealth application.

The purpose of the mHealth application was to improve the quality of counselling by ASHA worker, which in turn was aimed to improve the knowledge of pregnant women. Ultimately, this was intended to generate demand for seeking antenatal and natal services; and for timely care of complications during pregnancy, after delivery and during neonatal period [19]. The increased utilization of preventive health services, as a result of demand generation and better supply-side monitoring, is likely to result in lower illnesses and as a result reduction in mortality and disability. Similarly, improved care-seeking can also bring about a reduction in fraction of illnesses which are fatal or which result in long-term complications.

Counterfactual: routine care

The routine care scenario comprised of delivery of preventive and curative maternal and child health services, including implementation of the flagship program—National Health Mission [27]. These comprised of all the set of basic demand and supply side services which are recommended for maternal, newborn, child and adolescent health care as envisaged under the Reproductive, Maternal, Newborn, Child and Adolescent Health Care program (RMNCHA+) [28]. It was launched in year 2013 to provide a continuum of care right from the start of reproductive age of a girl child to the adolescent health of her offspring. It envisages health system strengthening for providing antenatal care, intra-partum, postpartum care to women for safe maternity, essential new born, early identification and referral services in case of any complication, immunization, prevention and treatment of childhood morbidities, family planning along with interventions for improving physical and psychological health in adolescents. The only difference between the intervention and the control area was the rollout of mHealth application which was used by ASHA workers.

General model overview

A decision tree (Additional file 1: Figure S1) was parameterized on MS-Excel spreadsheet to estimate the incremental cost effectiveness of implementing ReMiND. A time horizon of 10 years starting from base year of 2011 was considered appropriate to cover all costs and effects comprehensively on grounds of intervention characteristics and theory of change for effectiveness mediation. This time horizon was justified based on several reasons. Firstly, the m-health software is unlikely to change in this period as the broad nature of services will remain same. Secondly, based on expert opinion, even if the software has to be edited based on revisions in the program package, such changes are unlikely to have any major cost implications. Thirdly, while several costs of capital nature are incurred during the early years of implementation, however, the consequences of those investments i.e. health benefits continue to occur till many years later. These health effects are likely to occur during pregnancy (such as reduction of high-risk pregnancies, and their early detection and appropriate management), childbirth (such as reduction in post-partum haemorrhage), neonatal (reduction of low-birth weight, and prevention and management of neonatal illnesses), infancy and childhood period (such as prevention of vaccine preventable diseases) up to 5–10 years of age. Finally, economic evaluations of similar m-health packages have also relied on a similar 10–12 year time horizon [29, 30].

We analyzed costs and effects from both health system and societal perspective. Health system costs included the resources spent by the department of health and the implementing partners in delivering the intervention. These included resources such as building, space, staff salaries, equipment, software for m-health intervention, medicines, consumables, overheads etc. While measuring the societal costs, in addition to the health system cost, we also measured the out-of-pocket expenditures (OOPE) incurred by households. These OOPE were incurred for purchasing medicines, medical or surgical procedures, boarding, lodging, and transportation as a result of any health care sought during the pregnancy, intra-partum care, or neonatal period. We did not include the measurement of indirect costs in terms of productivity loss to the household as a result of absenteeism due to illness. Effect was measured in terms of illness episodes averted, maternal and neonatal deaths prevented, life years gained and DALYs averted. Both costs and effects were discounted at 3% to account for time preference of cost and utility. The choice of discount rate is justified on the following grounds. First, as per World Health Organization’s Choosing Health Interventions for Cost Effectiveness guidelines (WHO-CHOICE), it has been recommended to discount all future costs and consequences at 3% for international comparability [31]. This is in coherence with the recent guidelines released by Disease Control Priority 3 [22] and the reference case developed for low middle income countries by International Development Support Initiative and the Gates Foundation [32]. Also a recent systematic review of the economic evaluations done in India revealed that 82% of the studies which reported the value of discount rate, 3% rate was used to discount future costs and benefits [26]. To account for uncertainty in value of discount rate, we varied it up to 8% in probabilistic sensitivity analysis. The concept of discounting incorporates both the components i.e. time preference and inflation rate. Time preference represents the opportunity cost of an investment.

We report our findings as incremental cost of implementing ReMiND intervention per DALY averted, per illness episode prevented and per infant death averted as compared to routine care services [33]. An incremental cost-effectiveness ratio (ICER) is a summary measure in economic evaluations to represent the economic value of an intervention in comparison with an alternative or no alternative (comparator). The ICER is expressed as the ratio of the difference in costs between two strategies to the difference in effectiveness. For preference based outcomes, disability rates were taken from the Global Burden of Disease data [34, 35].

There are several thresholds which could be used for decision making in a cost-effectiveness analysis [36]. It could be a supply-side threshold, demand-side threshold or GDP based thresholds. Supply-side threshold is a measure of health benefits forgone due to reduced funding for current interventions as a result of allocating resources for a new intervention from provider’s perspective. A demand-side threshold describes the willingness to pay of an individual to gain additional health benefits in view of other competing demand of his resources. Third, the per capita GDP of a country recommended by several guidelines in the absence of evidence on other threshold measures [31]. The approach suggested by the commission for Macroeconomics on Health (2001) is that interventions with an incremental cost per DALY averted less than the per capita GDP in low middle income countries (LMICs) are “very cost effective”, and those costing less than triple the per capita GDP are “cost- effective”. In India, till date, there is a scarcity of evidence on supply-side and demand-side thresholds. Hence, per capita GDP is the most commonly used threshold in economic evaluations done in India [3739].

The standard guidelines for conducting and reporting an economic evaluation survey (CHEERS) were adhered to and details are available as Additional file 2: Appendix S1.

Costing

We analysed the costs from both health system and societal perspective. The health system costs comprised of four distinct components—firstly, it included the cost of implementing the mHealth application, i.e. development of software, training of ASHA workers, mobile phones and data transmission charges etc. [40]. These costs were obtained in US dollars which were converted into Indian rupees using dollar exchange rates given by Internal Revenue Service for year 2015 (1US$ = INR 63.35) [41]. The converted rates were then inflated from the year of purchase to the current value of product in 2015 by applying Consumer Price Index in India [42]. Secondly, we considered the incremental time spent for monitoring and supervision of this additional activity by implementing partners and state health department. Thirdly, introduction of intervention could have resulted in change in the allocation of time for provision of services by ASHA workers and hence affecting the staff costs. However, ASHA workers are not full-time paid staff, and are instead paid a performance-based incentive. As a result, determining the time allocation was not meaningful in this scenario. Instead, we estimated the total performance based payment paid to ASHA workers in intervention and control area to compute the difference. The mHealth application was intended to increase the counseling skills of ASHAs and better understanding of the beneficiaries; we found that there was no significant increase in the utilization of incentive based services which largely includes institutional deliveries. Hence, the incremental costs related to ASHAs incentive were not included in the analysis.

Finally, the intervention could have brought about change in utilization of heath care services, which entails a cost. The benefits reaped as a result of improved knowledge and treatment-seeking in the intervention population were inherent in overall benefits measurement and therefore, we included its associated increase in cost (i.e. cost of increased utilization of healthcare services). As a result, we estimated the cost of delivering extra services—preventive or curative. We used the unit cost of providing mHealth intervention under ReMiND in two blocks of Kaushambi as estimated in the cost analysis—i.e. INR 31.4 (US $ 0.49) per capita and INR 1294 (US $ 20.5) per pregnant woman [40]. The cost per pregnant woman is most appropriate which incorporate not only population distribution but also fertility levels and thus, allows modeling of costs in a scale up scenario in most appropriate way. The cost per woman of reproductive age captures the age distribution of fertile women in the population but it does not capture the level of fertility and its effect on cost of program implementation. The unit cost capita is also inappropriate for use in the economic evaluation, as it neither captures the effect of population demographics, nor fertility.

The cost of delivering preventive and curative health services at different levels of the health care delivery system in another study from North Indian states was used [43, 44]. The preventive services included in our analysis were maternal and child health services like provisioning of antenatal care (consumption of iron folic acid tablets, tetanus toxoid vaccine, number of ANC visits), postnatal care, essential newborn care and full immunization till 1 year of age. The curative services included institutional deliveries; treatment of complications during pregnancy, after delivery, among newborn and infants in either outpatient or inpatient setting at various levels of health care facilities. These studies had employed bottom up costing methods to comprehensively estimate the cost of delivering services in a representative sample of sub-centers, primary health centre, community health centre and district hospitals. Unit costs for antenatal care, postnatal care, and immunization were INR 525 (USD 10) per full ANC care, INR 767 (USD 14) per PNC case registered, and INR 97 (USD 1.8) per child immunized in routine immunization respectively [43]. Similarly, the cost incurred on per outpatient consultation at PHC and CHC was taken as INR 120 (95% CI 90–151) and 126 (95% CI 92–160) respectively while the unit cost per hospitalization was INR 1156 (95% CI 343–2140) at PHC and INR 1115 (95% CI 400–2188) at CHC level [45]. The cost per OPD consultation and bed day hospitalization for gynaecology (INR 165; 997) and paediatrics (INR 137; 1028) department at district hospital respectively were taken for our model [46]. The detailed cost analysis is provided in Additional file 3: Appendix S2.

In order to assess the change in utilization of health care services, we analysed the care seeking behaviour of pregnant women for illnesses/complications during delivery and after child-birth. This was assessed based on analysis of a household survey—CEAHH (cost effectiveness analysis household survey) survey, which was used to determine care seeking for illnesses reported in pregnancy, after child-birth and during neonatal period [18]. The out of pocket expenditure estimates for seeking outpatient and inpatient care for various maternal and childhood illnesses were given in Table 1. Expenditures by households’ in the form of OOPE were included along with health system costs to estimate cost to the society from a societal perspective. Details for the household survey are available elsewhere in the protocol and impact assessment papers [18, 20].
Table 1

Demographic and epidemiological parameters

Parameter (base year: 2011)

Base value

Lower limit

Upper limit

Distribution

Source

Demographic parameters

 Total population of Uttar Pradesh State

199,812,341

169,840,490

229,784,192

Lognormal

Census 2011

 Birth rate (per 1000 population)

27.8

27.5

29.1

Lognormal

Census 2011

 Annual decline in birth rate (%)

− 0.04

− 0.03

− 0.05

Lognormal

 

 Maternal mortality ratio (per 100,000 live births)

258

241

275

Lognormal

Annual Health Survey (AHS) Report, 2012–2013

 Neonatal mortality rate (per 1000 live births)

40

23

43

Lognormal

Census 2011

Epidemiological parameters

 Prevalence of anaemia in pregnant women

0.51

0.34

0.68

Beta

NFHS-4, 2015–2016

 Risk of anaemia: for women taking IFA during pregnancy

0.25

0.21

0.28

Beta

Haider et al. [47]

 Risk of anaemia: for women not taking IFA during pregnancy

0.75

0.60

0.90

Beta

 Risk of postpartum haemorrhage (PPH) among anaemic pregnant women

0.29

0.19

0.39

Beta

Prata et al. [48]

 Risk of prematurity among anaemic pregnant women

0.63

0.33

1.01

Beta

Rahman et al. [49]

 Risk of low birth weight (LBW) among anaemic pregnant women

0.31

0.13

0.51

Beta

 Probability of maternal mortality with PPH: with treatment

0.00038

0.00029

0.00047

Beta

Prata et al. [48]

 Probability of maternal mortality with PPH: without treatment

0.00051

0.00044

0.00058

Beta

 Probability of neonatal mortality due to prematurity: with treatment

0.102

0.082

0.122

Beta

Bang et al. [50]

 Probability of neonatal mortality due to prematurity: without treatment

0.332

0.266

0.398

Beta

 Probability of neonatal mortality due to LBW: with treatment

0.047

0.038

0.056

Beta

 Probability of neonatal mortality due to LBW: without treatment

0.113

0.090

0.136

Beta

 Prevalence of hypertension (HTN) in pregnancy

0.07

0.047

0.093

Beta

NFHS-4, 2015–2016

 Risk of preeclampsia in hypertensive pregnant women

0.63

0.422

0.838

Beta

Borade et al. [51]

 Risk of eclampsia in preeclampsia pregnant women

0.115

0.077

0.153

Beta

 Risk of perinatal complications due to eclampsia

0.524

0.351

0.697

Beta

The Magpie Trial 2007

 Probability of maternal mortality due to eclampsia among pregnant women: with treatment

0.18

0.144

0.216

Beta

 Probability of maternal mortality due to eclampsia among pregnant women: without treatment

0.4

0.32

0.48

Beta

 Probability of neonatal mortality due to perinatal complications: with treatment

0.102

0.082

0.122

Beta

Bang et al. [50]

 Probability of neonatal mortality due to perinatal complications: without treatment

0.332

0.266

0.398

Beta

Risk of maternal mortality in home deliveries

0.02

0.016

0.024

Beta

Montgomery et al. [52]

 Risk of maternal mortality in institutional deliveries

0.00279

0.00223

0.00335

Beta

Prevalence of sepsis in neonates

0.03

0.02

0.04

Beta

National Neonatal Perinatal Database

 Probability of neonatal deaths due to sepsis: with treatment

0.18

0.14

0.22

Beta

Seale et al. [53]

 Probability of neonatal deaths due to sepsis: without treatment

0.95

0.95

0.95

Beta

 Average length of illness (years): anemia

0.75

0.60

0.90

Lognormal

Expert opinion

 Average length of illness (years): PPH

0.01

0.01

0.01

Lognormal

 Average length of illness (years): HTN/eclampsia

0.75

0.60

0.90

Lognormal

 Average length of illness (years): prematurity

0.03

0.02

0.03

Lognormal

 Average length of illness (years): LBW

0.03

0.02

0.03

Lognormal

 Average length of illness (years): sepsis

0.03

0.02

0.03

Lognormal

 Average length of illness (years): perinatal complications

0.03

0.02

0.03

Lognormal

Impact parameters

 Increase in coverage of 3 ANC visits (%)

0.00

0.00

0.00

Lognormal

ReMiND-impact assessment study

 Increase in coverage of IFA (%)

12.70

8.70

16.70

Lognormal

 Increase in coverage of TT (%)

0.00

0.00

0.00

Lognormal

 Increase in coverage of care seeking (%)

25.7

13.70

41.10

Lognormal

 Increase in coverage of institutional delivery (%)

0.00

0.00

0.00

Lognormal

Cost parameters (INR)

 Health system costs

  Unit cost: ANC

525

456

619

Gamma

Prinja et al. [54]

  Unit cost: PNC

767

538

1092

Gamma

  Unit cost: immunization

97

77

120

Gamma

  Unit cost: institutional delivery

1872

1080

2990

Gamma

Prinja et al. [37] (PLOS one)

  Unit cost: PHC

   OPD

120

90

151

Gamma

   IPD

1156

343

2140

Gamma

  Unit cost: CHC

   OPD

126

92

160

Gamma

   IPD

1115

400

2188

Gamma

  Unit cost: gynaecology and obstetrics

   OPD

165

68

274

Gamma

Prinja et al. [55] (IJMR)

   IPD

997

592

1412

Gamma

  Unit cost: paediatrics

   OPD

137

102

182

Gamma

Prinja et al. [55] (IJMR)

   IPD

1028

444

1703

Gamma

Out of pocket expenditures (INR)

 Control: public

  ANC

406

305

508

Gamma

Primary data analysis (CEAAH)

  Institutional delivery

610

548

672

Gamma

  Postpartum care

1216

912

1520

Gamma

  Neonatal illness

   OPD

283

29

601

Gamma

   IPD

2357

852

6908

Gamma

 Control: private

  ANC

845

634

1056

Gamma

  Institutional delivery

13,000

11,154

14,846

Gamma

  Postpartum care

699

524

874

Gamma

  Neonatal illness

   OPD

769

472

1066

Gamma

   IPD

5164

3039

7289

Gamma

 Intervention: public

  ANC

878

659

1098

Gamma

  Institutional delivery

861

713

1009

Gamma

  Postpartum care

450

338

563

Gamma

  Neonatal illness

   OPD

380

145

615

Gamma

   IPD

1399

1049

1749

Gamma

 Intervention: private

  ANC

1420

1065

1775

Gamma

  Institutional delivery

16,900

11,051

22,749

Gamma

  Postpartum care

1791

1343

2239

Gamma

  Neonatal illness

   OPD

800

500

1100

Gamma

   IPD

1000

750

1250

Gamma

Valuing consequences of ReMiND intervention

ReMiND intervention was intended to improve the quality of counselling of pregnant woman by the ASHA worker. Improved counselling was desired to improve knowledge of pregnant women, and utilization of appropriate maternal and child health care services, during pregnancy, child-birth and during the neonatal period. Secondly, the data entered by the ASHA worker helped in tracking the ASHA worker to track pregnant women and their services utilized; besides being used for supervision of ASHA performance [18].

We undertook a pre and post quasi experimental study to assess the impact of the intervention. Two blocks other than two intervention blocks were selected as controls after matching for coverage of two indicators at baseline—ante natal care and institutional deliveries from the same district. The pre-intervention data was obtained from the Annual Health Survey 2011 conducted by Ministry of Health and Family Welfare. A household survey was carried out in four blocks of Kaushambi district in year 2015 to observe the post intervention coverage. Propensity score matched sample from intervention and control areas in pre-intervention and post-intervention periods were analysed using difference-in-difference method to estimate the impact of ReMiND program. Overall, the ReMiND led to a statistically significant increase in coverage for IFA consumption (12.7%), abdominal examination during ANC care (18.7%), identification and self-reporting of complication during pregnancy (13.20%) and after (19.5%) delivery and care seeking (25.7%) in the intervention area [20, 56]. The coverage of three or more ANC visits, tetanus toxoid vaccination, full ANC care and ambulance usage also increased in intervention area by 10.3, 4.3, 1 and 2.5%; however, the difference between the improvements in the intervention and control area was not statistically significant [20] (Table 1).

The coverage of MNCH services in control area from baseline and end-line surveys was used to interpolate the coverage during intervening years, and extrapolate during the future years from 2015 onwards. Linear change was assumed for the purpose of modelling. In case the coverage for any indicator reached 90%, no further increase was assumed thereafter in the subsequent years. Similarly, the impact estimates of difference in difference for intervention area were used to compute annual rate of change in the intervention area, which was further used to model the coverage of MNCH services in intervention scenario, relative to the counterfactual.

In this paper, we model the effect of increased utilization of health care services on reduction in illnesses or complications during pregnancy and after child-birth. Together, these two contributed to reduction in maternal and neonatal deaths—ultimately resulting in averting years of life lost (YLL) to premature mortality and reduction of disability adjusted life years (DALY). In terms of maternal complications, we primarily modelled the effect of changes in antenatal services on two major illnesses during pregnancy—anaemia and hypertension. These two were particularly considered in view of their prevalence in the targeted population [57], as well as evidence linking reduction in occurrence of these medical conditions with better ANC care [58]. Baseline prevalence of 51.4 and 5.8% was assumed for anaemia and hypertension during pregnancy [59]. Subsequently, we modelled the effect of improvement in coverage of complete IFA supplementation as a result of mHealth intervention—risk of anaemia is 75 and 25% without and with IFA supplementation respectively [47]. Similarly, an 11.5% risk of developing eclampsia was assumed among hypertensive pregnant women [51]. In turn, we assumed that anaemia results in complications during pregnancy and after child-birth, such as post-partum haemorrhage (29%), and is also associated with adverse neonatal health outcomes such as prematurity (63%) and low birth weight (31%) [49]. Finally, reduction of post-partum haemorrhage is associated with reduced maternal mortality [60]; while both prematurity and low birth weight results in a higher risk of neonatal mortality [50] (Table 1).

Secondly, the ReMiND intervention resulted in improved recognition of the danger signs during pregnancy and after child birth. The care seeking for any illness during pregnancy was higher in the intervention area (71.9%) as compared to control are (46.2%). We modelled the impact of improved care seeking on maternal and neonatal survival. For example, the risk of maternal mortality with post-partum haemorrhage is 25% less with treatment than without [60, 48].

Besides an increase in YLL as a result of reduction in mortality, we also estimated the reduction in years of life lived in disability (YLD) as a result of reduced illnesses during pregnancy, after child-birth and during neonatal period. We used the disability weights as provided in the Global Burden of Disease, 2010, for computing YLD [61]. For calculating YLL in case of an infant death, we estimated that the mean age of infant death is 26 days. This estimation was based on the assumption that 60% of infant deaths occur in neonatal period, 60% of neonatal deaths are early neonatal deaths (within first 7 days of birth) [6264]. We also assumed that mean age of early neonatal, late neonatal and post-neonatal death is 3, 20 days and 6 months respectively; i.e. the mid-point of class interval. Similar assumptions have also been used by another study evaluating cost effectiveness of IMNCI program in India [37]. We computed the percentage reduction in maternal and neonatal deaths by comparing the number of deaths in intervention where ReMiND program was implemented relative to control using a decision tree model (Additional file 1: Figure S1). Table 1 cites the estimates of effectiveness on proximal outputs i.e. coverage of services, and assumptions for modeling impact on long term outcomes such as morbidity and mortality derived from the Indian studies.

Sensitivity analysis

We undertook a probabilistic sensitivity analysis to test the effect of parameter uncertainty on the findings of the analysis and to estimate the effect of joint uncertainty in all parameters [6567]. While base analysis is valid for UP state, there is significant variability in values for various parameters from Indian perspective which was important to test in the PSA analysis. For several parameters related to unit cost of health care services, effectiveness estimates of mHealth interventions, some demographic parameters, and service coverage in counterfactual scenario etc., 95% confidence intervals were available from primary analysis as part of the current study or secondary literature [40, 44, 45, 54, 55, 68]. For other parameters, such as demographics and epidemiological parameters such as risk of various morbidities with or without use of preventive interventions etc., we varied the base estimate obtained from literature 20% on either side. For certain parameters, such as risk of mortality with and without treatment we varied the base estimate by 50% on either side, since this is heavily dependent on other supply side inputs which could vary significantly across different parts of the country. In case of prevalence of risk factors such as anaemia and hypertension during pregnancy, or low birth weight babies, we varied the base parameter by 33% on either side. Finally, we varied the cost of mHealth intervention by 50% on lower side and 20% on higher side. The same was done as we expect that implementation of mHealth intervention through support from a donor partner would be higher than when it is implemented through public sector health system which has relatively lower salary structures.

Probability of ReMiND intervention to remain cost effective at a willingness to pay threshold equal to per capita gross domestic product (GDP) was estimated, using a health system perspective. For undertaking PSA analysis, we assumed a lognormal distribution for unit costs. In case of parameters where 95% confidence interval was available, a beta distribution was used; while uniform distribution was applied where an upper and lower bound were available. Monte Carlo method was used for simulating the results 999 times. Median was computed along with 2.5th and 97.5th percentile to estimate 95% confidence interval.

Results

Costs

Overall, we found that the implementation of ReMiND in UP state from 2011 to 2020 would save 4,127,529 DALYs at an incremental cost of USD 982 million (Tables 2, 3). More than 90% of this cost is on account of implementation of the intervention which includes monitoring and supervision of the intervention, cost of increased uptake of preventive services such as ANC care and institutional deliveries; and curative services [40]. Implementing partners’ incurred almost 3/4th share of the implementation cost (Fig. 1). Interestingly, the cost of curative care in the intervention scenario is less than the counterfactual. This could be a result of reduction in illnesses during pregnancy and after child-birth as a result of increase in preventive interventions. This reduction in illnesses was significant enough to offset the inflationary effect of improved care seeking on the health system costs. In terms of the start-up costs, which constituted about 10% of incremental cost of intervention, little over one-third (37%) comprised of the resources spent for training the ASHA workers [40]. Health system spent majority of its costs on institutional deliveries (75%) followed by monitoring and supervision (16%) (Fig. 2). From societal perspective, there was a cost saving of USD 425 million with ReMiND intervention (Table 2). These cost savings were mainly due to two reasons. First, with ReMiND intervention, there was an increased uptake of preventive services like ANC which led to reduction in number of maternal and neonatal illnesses in the intervention scenario and therefore, decreased demand for curative care. Second, with better contact with public health system, more people utilized public health facilities both for preventive and curative care and hence, incurred less amount of OOPE (Tables 1 and 2).
Table 2

Incremental costs (in INR and USD) of ReMiND intervention in Uttar Pradesh state, India

Incremental costs of m-health program (2015)

Base value

Lower limit

Upper limit

INR

USD

INR

USD

INR

USD

Startup costs (mHealth)

 Development of software

17,495,229

276,168

8,747,615

138,084

20,994,275

331,401

 Training of ASHAs

1,826,036,757

28,824,574

913,018,378

14,412,287

2,191,244,108

34,589,489

 Equipments

244,654,336

3,861,947

122,327,168

1,930,973

293,585,203

4,634,336

 Purchase of mobile phones

1,618,284,035

25,545,131

809,142,018

12,772,565

1,941,940,842

30,654,157

 Programmatic expenses

98,490,876

1,554,710

49,245,438

777,355

118,189,051

1,865,652

 Overheads

199,921

3156

99,960

1578

239,905

3787

 Administrative costs

1,361,328,050

21,488,998

680,664,025

10,744,499

1,633,593,660

25,786,798

 Total

5,166,489,205

81,554,684

2,583,244,602

40,777,342

6,199,787,046

97,865,620

 Implementation costs (mHealth)

43,130,264,882

680,825,018

21,565,132,441

340,412,509

51,756,317,858

816,990,021

Health system incremental costs (with mHealth)

 Monitoring and evaluation

2,178,513,501

34,388,532

1,089,256,751

17,194,266

2,614,216,201

41,266,238

Preventive services

 Antenatal care

1,000,908,828

15,799,666

869,360,811

13,723,138

1,180,119,171

18,628,558

 Institutional delivery

10,462,218,682

165,149,466

6,035,895,393

95,278,538

16,710,488,173

263,780,397

Curative services

 Curative care for mothers: OPD

109,167,717

1,723,247

52,210,647

824,162

272,919,293

4,308,118

 Curative care for mothers: IPD

97,857,094

1,544,706

54,078,920

853,653

178,899,733

2,823,989

 Curative care for neonates: OPD

3,828,183

60,429

2,734,416

43,164

30,156,706

476,033

 Curative care for neonates: IPD

3,655,308

57,700

643,294

10,155

11,305,025

178,453

Total (health system)

62,152,903,400

981,103,448

32,252,557,276

509,116,926

78,954,209,206

1,246,317,430

Incremental out of pocket expenditures (with mHealth)

 ANC

− 1,033,319,209

− 16,311,274

− 1,345,597,054

− 21,240,680

− 697,526,728

− 11,010,682

 Institutional delivery

− 89,638,635,385

− 1,414,974,513

− 116,728,192,568

− 1,842,591,832

− 60,509,224,497

− 955,157,451

 Postpartum care

1,392,533,772

21,981,591

1,813,369,309

28,624,614

940,009,163

14,838,345

 Pediatrics: OPD

203,844,023

3,217,743

265,447,419

4,190,172

137,601,869

2,172,089

 Pediatrics: IPD

9,438,499

148,990

12,290,894

194,016

6,371,318

100,573

Total (out of pocket expenditures)

− 89,066,138,301

− 1,405,937,463

− 115,982,682,000

− 1,830,823,710

− 60,122,768,874

− 949,057,125

Grand total

− 26,913,234,901

− 424,834,016

− 59,581,747,680

− 940,516,933

76,787,545,236

1,212,115,947

INR Indian National Rupee, USD United States Dollar

Table 3

Effectiveness and cost effectiveness of m-health program in Uttar Pradesh state, India (2011–2020)

Characteristics

Base case

Lower limit

Upper limit

Health outcomes with routine services

 Maternal illness episodes

34,598,786

21,796,486

50,063,694

 Neonatal illness episodes

1,941,237

1,292,080

2,725,595

 Maternal deaths

96,921

79,833

117,305

 Neonatal deaths

2,168,635

1,833,492

2,548,800

 Life years lost

32,835,472

19,235,215

61,273,469

 DALYs lost

53,028,454

32,271,008

89,078,621

Health outcomes with routine health services and m-health

 Maternal illness episodes

31,444,322

18,681,922

46,116,428

 Neonatal illness episodes

1,903,900

1,270,809

2,677,032

 Maternal deaths

96,609

79,688

116,916

 Neonatal deaths

2,019,167

1,700,694

2,389,956

 Life years lost

30,604,197

17,896,762

57,353,515

 DALYs lost

48,900,926

29,160,101

83,178,589

Incremental benefits with m-health

 Maternal illness episodes averted

− 3,154,464

− 3,114,564

− 3,947,266

 Neonatal illness episodes averted

− 37,337

− 21,271

− 48,564

 Maternal deaths averted

− 312

− 146

− 389

 Neonatal deaths averted

− 149,468

− 132,798

− 158,844

 Life years saved

− 2,231,275

− 1,338,453

− 3,919,955

 DALYs averted

− 4,127,529

− 3,110,907

− 5,900,032

Incremental cost effectiveness ratio, health system perspective

 Cost per illness averted in INR (USD)

15,208 (240)

12,372 (195)

18,958 (299)

 Cost per death averted in INR (USD)

371,577 (5865)

253,502 (4002)

526,418 (8310)

 Cost per life year gained in INR (USD)

25,371 (400)

13,461 (212)

47,428 (749)

 Cost per DALY averted in INR (USD)

12,993 (205)

8570 (135)

18,523 (292)

Incremental cost effectiveness ratio, societal

 Cost per illness averted in INR (USD)

− 6656 (− 105)

− 39,965 (− 631)

2608 (41)

 Cost per death averted in INR (USD)

− 162,634 (− 2567)

− 1,109,743 (− 17,518)

53,433 (843)

 Cost per life year gained in INR (USD)

− 11,105 (− 175)

− 99,982 (− 1578)

2837 (45)

 Cost per DALY averted in INR (USD)

− 5687 (− 90)

− 39,049 (− 616)

1806 (29)

INR Indian National Rupee, USD United States Dollar

Figure 1
Fig. 1

Distribution of incremental costs of mHealth intervention: start up and implementation cost

Figure 2
Fig. 2

Distribution of incremental health system costs

Valuation of consequences

Cumulatively, from 2011 to 2020, implementation of ReMiND intervention in UP would result in a reduction of 312 maternal and 149,468 neonatal deaths during the 10-year period (Table 3). This implies a reduction of 0.2% maternal and 5.3% neonatal deaths (Figs. 3, 4). The reduction in maternal illnesses during pregnancy and neonatal illnesses were 9.11 and 1.9% respectively, between the intervention and counterfactual scenarios. This resulted in increase in 2,231,275 life years and reduction of 4,127,529 DALYs.
Figure 3
Fig. 3

Trend of maternal deaths in Uttar Pradesh, 2011–2020

Figure 4
Fig. 4

Trend of neonatal deaths in Uttar Pradesh, 2011–2020

Cost effectiveness

We found the ReMiND intervention to be cost saving from the societal perspective (Table 3, Additional file 4: Figure S2 and Additional file 5: Figure S3). ReMiND intervention resulted in a cost saving of USD 90 per DALY averted USD 2569 per death averted (Table 3). From health system perspective, t incurs an incremental cost of INR 12,993 (USD 205) per DALY averted and INR 371,577 (USD 5866) per death averted (Table 3). Figure 5 shows results from simulations done as a part of probabilistic sensitivity analysis. The points in scatter plot indicates that considering all the uncertainties in the analysis, majority concentration of simulated results are in quadrant I of cost-effectiveness plane. This implies that ReMiND intervention has additional health benefits at an additional cost (Fig. 5). With a GDP per capita of nearly USD 1500 per capita, the ReMiND intervention for reducing maternal and neonatal mortality is very cost effective from Indian health system perspective. Accounting for all the uncertainties in the analysis, there is a 90% probability of ReMiND intervention to be cost-effective at willingness to pay threshold of USD 354 i.e. INR 22,500 which is only 23.6% of per capita GDP of India in 2016 (Fig. 6).
Figure 5
Fig. 5

Cost effectiveness plane with incremental cost effectiveness ratios, health system perspective

Figure 6
Fig. 6

Cost effectiveness acceptability curve, health system perspective

Discussion

We undertook the present economic evaluation to compare the costs and consequences of implementing an mHealth intervention (ReMiND) in the existing set-up of routine health services, compared to routine maternal and child health services. In our analysis, we report findings from both health system and societal perspectives. We used per capita gross domestic product (GDP) of India as threshold for determining the cost-effectiveness. India had a GDP per capita of INR 88,440 (USD 1451.5) in 2013 [23, 69]. Our analysis shows that ReMiND implementation costs the Government of Uttar Pradesh an additional USD 205 (INR 12,993) per DALY averted. There is 90% probability of ReMiND intervention to be cost-effective at willingness to pay threshold of USD 354 i.e. INR 22,500 which is only 23.6% of per capita GDP of India in 2016. The Disease Control Priority 3 advises against the singular use of per capita GDP as the cost effectiveness threshold and rather recommends the comprehensive application of principles of equity and extended cost-effectiveness analysis while deciding about adoption of an intervention. It also highlights that the most cost effective interventions are the primary or preventive care interventions that prevent people from falling ill and seeking tertiary care. ReMiND program is one such intervention with the focus on enhancing the coverage of preventive care during pregnancy and early identification and referral for the complications [22]. ReMiND program become cost saving from a societal perspective. From a societal perspective, there is 88% probability of ReMiND intervention to be cost-effective if there is no difference in the costs of two scenarios (refer Additional file 4: Figure S2 and Additional file 5: Figure S3). We found that the improvements in counselling as a result of mHealth through ASHA workers, lead to generation of demand for preventive as well as curative care. Both preventive and curative care utilization ultimately leads to an increase in health system cost of health care. However, our modelled findings show that the increase in preventive services lead to a reduction of illness during pregnancy—such as anemia and hypertension, after child-birth and during neonatal period. Overall, from a health system perspective, an incremental cost of USD 982 million was incurred in the intervention setting. However, from societal perspective, there was an overall cost saving of USD 425 million. Despite a difference in increase (25.7%) in care being sought for such illnesses in intervention area [56], a reduction in number of illness episodes and OOPE resulted in these cost savings. A reduction in the occurrence of illnesses was due to increased uptake of preventive services—such as IFA supplementation, health monitoring through ANC care, whereas reduction in OOPE was because of increased utilization of public sector facilities both for preventive and curative care. Together these two reasons offset the increase in costs of increased utilization. As a result, the overall costs for preventive and curative care were higher in the counterfactual scenario of routine MNCH services rather than the mHealth intervention scenario (Table 3). In terms of health gains, ReMiND averted 3.1 million maternal illnesses and 37,337 neonatal illnesses. This translates to reduction of 312 maternal deaths, 0.15 million neonatal deaths and 4.1 million DALYs. In intervention setting, our model estimated more number of neonatal deaths averted corresponding to neonatal illnesses averted. This is due to the fact that we assumed in our model, prematurity which is one of the major cause of neonatal death either leads to death or major/minor disability but does not leads to illness episode.

A number of estimates for cost effectiveness of individual child health interventions are available such as haemophilus influenza type ‘b’ vaccine [70], insecticide treated bednets for malaria [71, 72], HIV preventive interventions for maternal to child transmission etc. [73]. In terms of findings from developing country context, ReMiND is as cost effective as some of the well-known child health interventions such as vitamin A and zinc fortification, measles immunization, case management of pneumonia and oral rehydration therapy (USD 24) [74]. Within the literature from India, ReMiND (USD 205) is less cost effective than measles (USD 13.8), hepatitis B (USD 31), HPV vaccination (USD 1.1) [26, 75] and rotavirus vaccination (USD 139) [76]. However, it is more cost effective than some other vaccines against cholera (USD 595–1310), typhoid (USD 227–621) and haemophilus influenza type ‘b’ (USD 363) [70].

Our study has several methodological strengths. Firstly, our decision model is plausible in terms of program implementation design, care-seeking and health care delivery system. A decision tree was considered appropriate than any other modelling method, such as Markov model, considering the acute nature of most maternal and childhood illnesses. The decision tree models have been used in cost effectiveness studies across the globe specifically to calculate the incremental cost effectiveness ratios of various maternal and newborn programs. For instance, to calculate cost effectiveness of ‘Integrated Management of Newborn and Childhood Illnesses’ program in India [37]; group B streptococcal vaccine immunization to prevent neonatal sepsis and meningitis in Sub Saharan Africa [77]; voucher scheme combined with obstetrical quality improvements as a part of quasi experimental results from Uganda [78] and maternal and Child health voucher scheme in Myanmar [79].

Secondly, almost all the values for parameters were sourced from local Indian context. Most of these values have been drawn from findings of a quasi-experimental study which enhances the internal validity of our estimations [20]. Thirdly, a comprehensive costing analysis was undertaken to analyse all costs involved with intervention and counterfactual scenarios. Some costs were obtained from the published literature from the neighbouring states where not much difference in infrastructure is expected [43, 45].

Limitations

The present analysis relied on modelled estimation of reduction in mortality and morbidity as a result of improvements in uptake of preventive services, based on robust evidence on such downstream benefits. It is very difficult to measure changes in maternal mortality and as a result we had to resort to modelling to document the same. Several analysis based on the LiST tool have shown that the modelled long-term benefits in terms of reduction in mortality are quite similar to those where empirical observations are available [80]. There were several reasons for preferring our own decision model over LiST. First, LiST model uses set of evidence based on systematic reviews about effect of m-health interventions on coverage of various preventive and curative services. While the validity of LiST model assumptions is not to be questioned, the context of a given intervention remains ‘central’ for assessment of its effectiveness. In this regard, while the LiST model uses global evidence on impact of interventions, the current intervention was implemented in Uttar Pradesh state of India—a setting which has much poorer maternal and child health indicators. The potential of any intervention to create an impact is dependent on the baseline situation. Hence, it might be better to rely on estimates from local settings if available and use it to populate the model. Secondly, the data on assumptions to link proximal effectiveness estimates (such as coverage of health services) to long term outcomes (including morbidity and mortality) were derived using studies from India. We also varied these assumptions and undertook a probabilistic sensitivity analysis to check the robustness of our results to variations in parameter uncertainty. Thirdly, LiST provides framework for modelling long term outcomes by populating a model on a set package of interventions. However, this leaves relatively little user flexibility to incorporate the specific aspects of a given intervention which may be different to the package of interventions which were considered while populating the LiST model. We used a previously validated model to assess the cost-effectiveness of Integrated Management of Neonatal and Child Illnesses in public health setting in India [37], and adapted it to incorporate the specifics of ReMiND program so that it becomes much more generalized to Indian settings.

Modelling the effect of multiple interventions which have effect on the same outcome is an area which needs methodological development. It is not clear whether the effect of several interventions will be additive, multiplicative or otherwise. However, in our case we have primarily relied on modelling the impact of IFA supplementation on anaemia during pregnancy; and ANC visits on hypertension during pregnancy. Each of these morbidities is then modelled separately for complications and mortality as a result. Secondly, the effect of improved care-seeking is directly on the case-fatality rate, rather than reduction of any illness. Hence we believe that this limitation could not have confounded our analysis. We do acknowledge that such modelled outcomes will result only if the supply-side of health system is able to meet the demand generated through the interventions like ReMiND. Hence, while the findings of the study recommend up scaling of ReMiND on grounds of efficiency, it is equally important to continue to focus of health system strengthening so that the supply of services match up to the demand generated, both in terms of quantity and quality. Secondly, the cost of scaling up will also need to be assessed. The cost of scaling up ReMiND intervention from two blocks in Kaushambi to 821 blocks in state was estimated using two case scenarios. This included a scenario in which the existing human resource in state health department was used for monitoring and supervisory activities, and second, if an additional supervisor was recruited in every block as was the case in original ReMiND pilot implementation in Kaushambi district. We found that using existing human resources, ReMiND scale up in UP state would cost INR 876 million (US$ 13.8 million) annually, implying a cost per pregnant woman of INR 175.3 (US $2.77). Similarly, in case of additional supervisory cadre was created, the overall annual cost and cost per pregnant woman would be INR 993 million (US $15.7 million) and INR 198.8 (US $ 3.14) respectively [40]. However, these costs were estimated in ideal conditions without considering any bottlenecks in the implementation of programme which may deviate to some extent in the real life situations. We also acknowledge that the scale up of such community health intervention depends on a number of unforeseen social and political factors which are beyond the control of the researchers. We have estimated costs in the ideal conditions without considering any bottlenecks in the implementation of programme.

As per the guidelines of Financial Management Group of National Health Mission—India’s flagship health program, it is recommended to increase the budget in program implementation plan of high priority districts by 10–15% annually [81]. Since the cost for scale up of ReMiND intervention is 6% of the total budget allotted to ‘Maternal and child health’ line item under the NRHM budget of UP state, the intervention appears financially sustainable. However, scale up of such community health interventions is social and political factors which are beyond the scope of present research. Changes in political and administrative structure also affects introduction of newer programs. Besides, scale-up of ReMiND intervention would also involve several rounds of training the health workers in use of technology, as well as their supportive supervision. We acknowledge that these uncertainties could not be accounted for in our analysis. For assessing the impact of the intervention, we have used different datasets of 2 time-periods to represent baseline and end-line period. The endline survey was designed to be similar to the baseline in order to avoid any bias. However, since the intervention was not randomly assigned, there could be possibility of confounding due to individual or community level characteristics. We used two approaches, i.e. matching of control with intervention blocks, and matching of individuals within these two areas in order to avoid confounding which are explained in detail in our paper which reports on impact of ReMiND intervention [20] and is also explained in the Additional file 6: Appendix S3. Hence, we believe that the impact assessment is robust.

Finally, in terms of costing, we relied on collection of cost data retrospectively. This implies that we collected data of resources spent for services delivered around 4 years old. This could have lead to a recall bias in usual scenario. However, since the entire data on resource use and expenditure was digitized, there is little chance of any recall bias. Moreover, since the intervention is still continuing, time-motion observations were made in order to develop statistics for apportioning joint costs, which are explained in the cost-analysis paper [40]. However, we do acknowledge that we could not directly interview a few officials who were involved in designing of the software, and whose inputs could have improved the estimates further. We also were not able to include wage or productivity loss as indirect expenditures while calculation of societal costs. In present study, the value for money for mHealth interventions in high burden settings like Kaushambi was calculated. However, there is a need to take further analysis to assess cost effectiveness in low burden settings. Since the state level estimates were used in model instead of two intervention blocks, the results are more generalizable to entire state.

Conclusions

Overall, the findings of our study suggest strongly that the mHealth intervention as part of the ReMiND intervention is very cost effective from Indian health system’s viewpoint, and cost saving from a societal perspective, and should be considered for replication elsewhere in India. Such interventions for generating demand through community health worker programs would need to be matched with similar strengthening of the health system which is able to meet the increase in demand for services, both in terms of quantity and quality.

Declarations

Authors’ contribution

Conception of study: SP, MG, JST. Study design: SP, PB. Preparation of tool: AG, RN, SP. Data collection: AG, PB, RN. Data analysis: PB, SP. First draft: SP. All authors reviewed the manuscript, provided critical inputs for revision. All authors read and approved the final manuscript.

Acknowledgements

We gratefully acknowledge the implementing agencies of ReMiND program, namely; Catholic Relief Services, Vatsalya and Dimagi Inc., for providing data for undertaking costing of ReMiND program. We also recognize critical inputs from the Study Advisory Committee comprising Rajesh Kumar, Indrani Gupta, Stephen Jan, Krishna Rao, Pavitra Mohan, Meenu Singh, Jitendra Kumar, Rajani Ved, Ashok Jha, Neeta Rao, Suresh Dalpath, Prabhu Dayal, Rajesh Jha and Rajkumar Mishra.

Competing interests

The authors declare that they have no competing interests.

Availability of data and dataset

Data is under the custody of lead author and is the guarantee for data. However, a data sharing agreement was signed prior to the start of study between the implementing agency (Catholic Relief Services) and evaluation agency (Post Graduate Institute of Medical Education and Research, Chandigarh) which prevents any data sharing without prior permission of implementing agency. Any request for data sharing may be sent to lead author at his email address:shankarprinja@gmail.com. Data will be shared to interested researchers subject to approval from Catholic Relief Services.

Consent for publication

Not applicable.

Ethics approval and consent to participate

The Institute Ethics Committee of the Post Graduate Institute of Medical Sciences, PGIMER, Chandigarh approved the study vide letter no. ‘Program No IEC-01/2015-108’.

Funding

The present study was funded by the United States Agency for International Development (USAID), New Delhi, India. The funding agency has no role in the study design, collection of data, analysis or report writing.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
School of Public Health, Post Graduate Institute of Medical Education and Research, Chandigarh, India

References

  1. Paul VK, Sankar MJ, Saini S. Trek to MDG 4: state of Indian States. Indian J Pediatr. 2014;81(10):993–9.View ArticlePubMedGoogle Scholar
  2. Ministry of Home Affairs, Annual Health Survey Fact Sheet: Uttar Pradesh 2012–2013. New Delhi. 2012–2013. http://www.censusindia.gov.in/vital_statistics/AHSBulletins/AHS_Factsheets_2012-13/FACTSHEET-UTTAR_PRADESH.pdf. Accessed 29 Mar 2016.
  3. Press Information Bureau. Government of India. Ministry of Health and Family Welfare. List of high priority districts (HPDs) in the country. http://pib.nic.in/newsite/PrintRelease.aspx?relid=118620. Accessed 29 Mar 2016.
  4. Sharma AK. National rural health mission: time to take stock. Indian J Community Med. 2009;34(3):175.View ArticlePubMedPubMed CentralGoogle Scholar
  5. Bajpai N, Dholakia RH. Improving the performance of accredited social health activists in India. New York: Columbia University; 2011. http://globalcenters.columbia.edu/files/cgc/pictures/Improving_the_Performance_of_ASHAs_in_India_CGCSA_Working_Paper_1.pdf. Accessed 29 Mar 2016.
  6. Kallander K, Tibenderana JK, Akpogheneta OJ, Strachan DL, Hill Z, Ten Asbroek AH, et al. Mobile health (mHealth) approaches and lessons for increased performance and retention of community health workers in low-and middle-income countries: a review. J Med Internet Res. 2013;15(1):e17.View ArticlePubMedPubMed CentralGoogle Scholar
  7. Datta SS, Ranganathan P, Sivakumar KS. A study to assess the feasibility of Text Messaging Service in delivering maternal and child healthcare messages in a rural area of Tamil Nadu, India. Australas Med J. 2014;7(4):175–80.View ArticlePubMedPubMed CentralGoogle Scholar
  8. Vision W. Deploying MoTECH suite to support Global MNCH & nUtrition Programs. 2014. http://www.dimagi.com/wpcontent/uploads/2015/03/MOTECH-World-Vision-Case-Study.pdf. Accessed 31 Mar 2016.
  9. Chatfield A, Javetski G, Lesh N. CommCare evidence base. Dimagi. 2013. http://www.dimagi.com/wp…/02/CommCare-Evidence-Base-March-2015.pdf. Accessed 31 Mar 2016.
  10. Little A, Medhanyie A, Yebyo H, Spigt M, Dinant GJ, Blanco R. Meeting community health worker needs for maternal health care service delivery using appropriate mobile technologies in Ethiopia. PLoS ONE. 2013;8(10):e77563.View ArticlePubMedPubMed CentralGoogle Scholar
  11. Meankaew P, Kaewkungwal J, Khamsiriwatchara A, Khunthong P, Singhasivanon P, Satimai W. Application of mobile-technology for disease and treatment monitoring of malaria in the “Better Border Healthcare Programme”. Malar J. 2010;9:237.View ArticlePubMedPubMed CentralGoogle Scholar
  12. Blanas DA, Ndiaye Y, MacFarlane M, Manga I, Siddiqui A, Velez O, et al. Health worker perceptions of integrating mobile phones into community case management of malaria in Saraya, Senegal. Int Health. 2015;7(3):176–82.View ArticlePubMedGoogle Scholar
  13. Ajay VS, Prabhakaran D. The scope of cell phones in diabetes management in developing country health care settings. J Diabetes Sci Technol. 2011;5(3):778–83.View ArticlePubMedPubMed CentralGoogle Scholar
  14. Surka S, Edirippulige S, Steyn K, Gaziano T, Puoane T, Levitt N. Evaluating the use of mobile phone technology to enhance cardiovascular disease screening by community health workers. Int J Med Inform. 2014;83(9):648–54.View ArticlePubMedPubMed CentralGoogle Scholar
  15. Chang LW, Njie-Carr V, Kalenge S, Kelly JF, Bollinger RC, Alamo-Talisuna S. Perceptions and acceptability of mHealth interventions for improving patient care at a community-based HIV/AIDS clinic in Uganda: a mixed methods study. AIDS Care. 2013;25(7):874–80.View ArticlePubMedPubMed CentralGoogle Scholar
  16. Jennings L, Ong’ech J, Simiyu R, Sirengo M, Kassaye S. Exploring the use of mobile phone technology for the enhancement of the prevention of mother-to-child transmission of HIV program in Nyanza, Kenya: a qualitative study. BMC Public Health. 2013;13:1131.View ArticlePubMedPubMed CentralGoogle Scholar
  17. Hill Z, Curtis KE, Lahiri S. Targeting parents for childhood weight management: development of a theory-driven and user-centered healthy eating app. JMIR mHealth uHealth. 2015;3(2):e69.View ArticleGoogle Scholar
  18. Prinja S, Nimesh R, Gupta A, Bahuguna P, Thakur JS, Gupta M, Singh T. Impact assessment and cost-effectiveness of m-health application used by community health workers for maternal, newborn and child health care services in rural Uttar Pradesh, India: a study protocol. Glob Health Action. 2016;9(1):31473.View ArticlePubMedGoogle Scholar
  19. CRS. Baseline study summary: ReMiND-reducing maternal and newborn deaths, Kaushambi District, December 2011–January 2013. 2013. http://www.crs.org/sites/default/files/tools-research/baseline-study-summary-remind-reducing-maternal-newborn-deaths.pdf. Accessed 6 April 2016.
  20. Prinja S, Nimesh R, Gupta A, Pankaj B, Gupta M, Thakur JS. Impact of m-Health application used by community health volunteers for improving utilization of maternal, newborn and child health care (MNCH) services in a rural area of Uttar Pradesh, India. Trop Med Int Health. 2017. https://doi.org/10.1111/tmi.12895.PubMedGoogle Scholar
  21. Ministry of health and Family Welfare. Government of India. National Health Policy 2017. 2017. http://cdsco.nic.in/writereaddata/National-Health-Policy.pdf. Accessed 6 April 2016.
  22. Horton S. Cost-effectiveness analysis in disease control priorities. 3rd ed. 2018. http://dcp-3.org/sites/default/files/chapters/DCP3%20Volume%209_Ch%207.pdf. Accessed 4 May 2016.
  23. Downey LE, Mehndiratta A, Grover A, Gauba V, Sheikh K, Prinja S, et al. Institutionalising health technology assessment: establishing the Medical Technology Assessment Board in India. BMJ Glob Health. 2017;2(2):e000259.View ArticlePubMedPubMed CentralGoogle Scholar
  24. Prinja S, Downey LE, Gauba VK, Swaminathan S. Health technology assessment for policy making in India: current scenario and way forward. Berlin: Springer; 2017.Google Scholar
  25. de la Torre-Diez IL-C, López-Coronado M, Vaca C, Aguado JS, de Castro C. Cost-utility and cost-effectiveness studies of telemedicine, electronic, and mobile health systems in the literature: a systematic review. Telemed J eHealth. 2015;21(2):81–5.View ArticleGoogle Scholar
  26. Prinja S, Chauhan AK, Angell B, Gupta I, Jan S. A systematic review of the state of economic evaluation for health care in India. Appl Health Econ Health Policy. 2015;13(6):595–613.View ArticlePubMedGoogle Scholar
  27. Ministry of Health & Family Welfare, Government of India. About NHM. http://nhm.gov.in/nhm/about-nhm.html. Accessed 31 Mar 2016.
  28. National Rural Health Mission. Ministry of Health & Family Welfare. NHM components: RMNCH + A. http://nhm.gov.in/nrhm-components/rmnch-a.html. Accessed 31 Mar 2016.
  29. Peels DA, Hoogenveen RR, Feenstra TL, Golsteijn RH, Bolman C, Mudde AN, et al. Long-term health outcomes and cost-effectiveness of a computer-tailored physical activity intervention among people aged over fifty: modelling the results of a randomized controlled trial. BMC Public Health. 2014;14(1):1099.View ArticlePubMedPubMed CentralGoogle Scholar
  30. Graves N, Barnett AG, Halton KA, Veerman JL, Winkler E, Owen N, et al. Cost-effectiveness of a telephone-delivered intervention for physical activity and diet. PLoS ONE. 2009;4(9):e7135.View ArticlePubMedPubMed CentralGoogle Scholar
  31. Organization WH. Making choices in health: WHO guide to cost-effectiveness analysis. 2003. http://apps.who.int/iris/bitstream/handle/10665/42699/9241546018.pdf;jsessionid=6AF3430426FB4D38FFD4FCB7393DBDB5?sequence=1. Accessed 25 June 2016.
  32. Wilkinson T, Sculpher MJ, Claxton K, Revill P, Briggs A, Cairns JA, et al. The international decision support initiative reference case for economic evaluation: an aid to thought. Value Health. 2016;19(8):921–8.View ArticlePubMedGoogle Scholar
  33. Fox-Rushby J, Cairns J. Economic evaluation. London: Oxford University Press; 2006.Google Scholar
  34. WHO. The global burden of disease: 2004 update. Geneva: World Health Organisation. http://www.who.int/healthinfo/global_burden_disease/GBD_report_2004update_full.pdf. Accessed 27 June 2016.
  35. Ng M, Fleming T, Robinson M, Thomson B, Graetz N, Margono C, et al. Global, regional, and national prevalence of overweight and obesity in children and adults during 1980–2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet. 2014;384(9945):766–81.View ArticlePubMedPubMed CentralGoogle Scholar
  36. Bertram MY, Lauer JA, De Joncheere K, Edejer T, Hutubessy R, Kieny M-P, et al. Cost-effectiveness thresholds: pros and cons. Bull World Health Organ. 2016;94(12):925.View ArticlePubMedPubMed CentralGoogle Scholar
  37. Prinja S, Bahuguna P, Mohan P, Mazumder S, Taneja S, Bhandari N, et al. Cost effectiveness of implementing integrated management of neonatal and childhood illnesses program in district Faridabad, India. PLoS ONE. 2016;11(1):e0145043.View ArticlePubMedPubMed CentralGoogle Scholar
  38. Prinja S, Kaur G, Malhotra P, Jyani G, Ramachandran R, Bahuguna P, et al. Cost-effectiveness of autologous stem cell treatment as compared to conventional chemotherapy for treatment of multiple myeloma in India. Indian J Hematol Blood Transfus. 2017;33(1):31–40.View ArticlePubMedPubMed CentralGoogle Scholar
  39. Goldie SJ, Sweet S, Carvalho N, Natchu UCM, Hu D. Alternative strategies to reduce maternal mortality in India: a cost-effectiveness analysis. PLoS Med. 2010;7(4):e1000264.View ArticlePubMedPubMed CentralGoogle Scholar
  40. Prinja S, Gupta A, Bahuguna P, Nimesh R. Cost analysis of implementing m-health intervention for maternal, newborn & child health care through community health workers: assessment of ReMiND Program in Uttar Pradesh, India. Chandigarh: School of Public Health, Post Graduate Institute of Medical Education and Research. 2015. http://www.healtheconomics.pgisph.in/admin/publication/cost_analysis_of_ReMiND_project.pdf. Accessed 6 June 2016.
  41. Yearly average currency exchange rates translating foreign currency into US dollars. https://www.irs.gov/Individuals/International-Taxpayers/Yearly-Average-Currency-Exchange-Rates. Accessed 17 Dec 2015.
  42. Prinja S, Bahuguna P, Gupta R, Sharma A, Rana SK, Kumar R. Coverage and financial risk protection for institutional delivery: how universal is provision of maternal health care in India? PLoS ONE. 2015;10(9):e0137315.View ArticlePubMedPubMed CentralGoogle Scholar
  43. Prinja S, Jeet G, Verma R, Kumar D, Bahuguna P, Kaur M, et al. Economic analysis of delivering primary health care services through community health workers in 3 North Indian states. PLoS ONE. 2014;9(3):e91781.View ArticlePubMedPubMed CentralGoogle Scholar
  44. Prinja S, Mazumder S, Taneja S, Bahuguna P, Bhandari N, Mohan P, et al. Cost of delivering child health care through community level health workers: how much extra does IMNCI program cost? J Trop Pediatr. 2013;59(6):489–95.View ArticlePubMedGoogle Scholar
  45. Prinja S, Gupta A, Verma R, Bahuguna P, Kumar D, Kaur M, et al. Cost of delivering health care services in public sector primary and community health centres in North India. PLoS ONE. 2016;11(8):e0160986.View ArticlePubMedPubMed CentralGoogle Scholar
  46. Prinja S, Balasubramanian D, Jeet G, Verma R, Kumar D, Bahuguna P, et al. Cost of delivering secondary-level health care services through public sector district hospitals in India. Indian J Med Res. 2017;146(3):354.PubMedPubMed CentralGoogle Scholar
  47. Haider BA, Olofin I, Wang M, Spiegelman D, Ezzati M, Fawzi WW. Anaemia, prenatal iron use, and risk of adverse pregnancy outcomes: systematic review and meta-analysis. BMJ. 2013;346:f3443.View ArticlePubMedPubMed CentralGoogle Scholar
  48. Prata N, Bell S, Quaiyum MA. Modeling maternal mortality in Bangladesh: the role of misoprostol in postpartum hemorrhage prevention. BMC Pregnancy Childbirth. 2014;14(1):78.View ArticlePubMedPubMed CentralGoogle Scholar
  49. Rahman MM, Abe SK, Rahman MS, Kanda M, Narita S, Bilano V, et al. Maternal anemia and risk of adverse birth and health outcomes in low- and middle-income countries: systematic review and meta-analysis. Am J Clin Nutr. 2016;103(2):495–504.View ArticlePubMedGoogle Scholar
  50. Bang AT, Reddy HM, Deshmukh MD, Baitule SB, Bang RA. Neonatal and infant mortality in the ten years (1993 to 2003) of the Gadchiroli field trial: effect of home-based neonatal care. J Perinatol. 2005;25:S92–107.View ArticlePubMedGoogle Scholar
  51. Borade PV, Haralkar SJ, Wadagale AV. Hypertensive disorders of pregnancy: an ongoing holocoust. Headache. 2014;39(21):9.Google Scholar
  52. Montgomery AL, Ram U, Kumar R, Jha P, for The Million Death Study Collaborators. Maternal mortality in India: causes and healthcare service use based on a nationally representative survey. PLoS ONE. 2014;9(1):e83331. https://doi.org/10.1371/journal.pone.0083331.View ArticlePubMedPubMed CentralGoogle Scholar
  53. Seale AC, Blencowe H, Zaidi A, Ganatra H, Syed S, Engmann C, Newton CR, Vergnano S, Stoll BJ, Cousens SN, Lawn JE. Neonatal severe bacterial infection impairment estimates in South Asia, sub-Saharan Africa, and Latin America for 2010. Pediatric Res. 2013;74(S1):73.View ArticleGoogle Scholar
  54. Prinja S, Bahuguna P, Lakshmi P, Mokashi T, Aggarwal AK, Kaur M, et al. Evaluation of publicly financed and privately delivered model of emergency referral services for maternal and child health care in India. PLoS ONE. 2014;9(10):e109911.View ArticlePubMedPubMed CentralGoogle Scholar
  55. Prinja S, Balasubramanian D, Jeet G, Verma R, Kumar D, Bahuguna P, et al. Cost of delivering secondary level health care services through public sector District Hospitals in India. Indian J Med Res. 2017;146:354.View ArticlePubMedPubMed CentralGoogle Scholar
  56. Prinja S, Nimesh R, Gupta A. Delivery of maternal, neonatal and child health services in district Kaushambi, Uttar Pradesh (2011–2015): a descriptive report. 2015.Google Scholar
  57. Sharma J, Meenakshi S. Anemia in pregnancy. JIMSA. 2010;23(4):253–60.Google Scholar
  58. Hollowell J, Kurinczuk JJ, Oakley L, Brocklehurst P, Gray R. A systematic review of the effectiveness of antenatal care programmes to reduce infant mortality and its major causes in socially disadvantaged and vulnerable women. Oxford: National Perinatal Epidemiology Unit, University of Oxford; 2009.Google Scholar
  59. IIPS. GOI, MoHFW, National Family Health Survey-4 (2015–16), Uttar Pradesh fact sheet. http://rchiips.org/NFHS/pdf/NFHS4/UP_FactSheet.pdf. Accessed 5 May 2018.
  60. Prata N, Bell S, Weidert K. Prevention of postpartum hemorrhage in low-resource settings: current perspectives. Int J Womens Health. 2013;5:737.View ArticlePubMedPubMed CentralGoogle Scholar
  61. Murray CJ, Vos T, Lozano R, Naghavi M, Flaxman AD, Michaud C, et al. Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2013;380(9859):2197–223.View ArticleGoogle Scholar
  62. Bassani DG, Kumar R, Awasthi S, Morris SK, Paul VK, Shet A, et al. Causes of neonatal and child mortality in India: a nationally representative mortality survey. Lancet. 2011;376(9755):1853–60.Google Scholar
  63. Rajaratnam JK, Marcus JR, Flaxman AD, Wang H, Levin-Rector A, Dwyer L, et al. Neonatal, postneonatal, childhood, and under-5 mortality for 187 countries, 1970–2010: a systematic analysis of progress towards Millennium Development Goal 4. Lancet. 2010;375(9730):1988–2008.View ArticlePubMedGoogle Scholar
  64. WHO. Country profiles on neonatal and child health. Geneva: World Health Organization; 2013.Google Scholar
  65. Andronis L, Barton P, Bryan S. Sensitivity analysis in economic evaluation: an audit of NICE current practice and a review of its use and value in decision-making. Perth: Prepress Projects Limited; 2009.Google Scholar
  66. Briggs AH, Weinstein MC, Fenwick EA, Karnon J, Sculpher MJ, Paltiel AD. Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-6. Med Decis Making. 2012;32(5):722–32.View ArticlePubMedGoogle Scholar
  67. Claxton K. Exploring uncertainty in cost-effectiveness analysis. Pharmacoeconomics. 2008;26(9):781–98.View ArticlePubMedGoogle Scholar
  68. Prinja S, Manchanda N, Aggarwal AK, Kaur M, Jeet G, Kumar R. Cost & efficiency evaluation of a publicly financed & publicly delivered referral transport service model in three districts of Haryana State, India. Indian J Med Res. 2013;138(6):1003.PubMedPubMed CentralGoogle Scholar
  69. The World Bank. World Bank national accounts data, and OECD National Accounts data files. http://data.worldbank.org/indicator/NY.GDP.PCAP.CD?end=2013&start=1960. Accessed 5 May 2018.
  70. Gupta M, Prinja S, Kumar R, Kaur M. Cost-effectiveness of Haemophilus influenzae type b (Hib) vaccine introduction in the universal immunization schedule in Haryana State, India. Health Policy Plan. 2012;28:51–61.View ArticlePubMedGoogle Scholar
  71. Butraporn P, Kamolratanakul P, Prasittisuk M, Prasittisuk C, Indaratna K. Cost-effectiveness analysis of lambdacyhalothrin-treated nets for malaria control: the patients’ perspective. Southeast Asian J Trop Med Public Health. 1999;30:427–31.PubMedGoogle Scholar
  72. Kamolratanakul P, Butraporn P, Prasittisuk M, Prasittisuk C, Indaratna K. Cost-effectiveness and sustainability of lambadacyhalothrin-treated mosquito nets in comparison to DDT spraying for malaria control in western Thailand. Am J Trop Med Hyg. 2001;65:279–84.View ArticlePubMedGoogle Scholar
  73. WHO. Towards universal access: scaling up priority HIV/AIDS interventions in the health sector: progress report 2010. Geneva: World Health Organization; 2010.Google Scholar
  74. Edejer TT, Aikins M, Black R, Wolfson L, Hutubessy R, Evans DB. Cost effectiveness analysis of strategies for child health in developing countries. BMJ. 2005;331(7526):1177.View ArticlePubMedPubMed CentralGoogle Scholar
  75. Prinja S, Bahuguna P, Faujdar DS, Jyani G, Srinivasan R, Ghoshal S, et al. Cost-effectiveness of human papillomavirus vaccination for adolescent girls in Punjab state: implications for India’s universal immunization program. Cancer. 2017;123:3253–60.View ArticlePubMedGoogle Scholar
  76. Megiddo I, Colson AR, Nandi A, Chatterjee S, Prinja S, Khera A, et al. Analysis of the Universal Immunization Programme and introduction of a rotavirus vaccine in India with IndiaSim. Vaccine. 2014;32:A151–61.View ArticlePubMedGoogle Scholar
  77. Russell LB, Kim S-Y, Cosgriff B, Pentakota SR, Schrag SJ, Sobanjo-ter Meulen A, et al. Cost-effectiveness of maternal GBS immunization in low-income sub-Saharan Africa. Vaccine. 2017;35(49):6905–14.View ArticlePubMedPubMed CentralGoogle Scholar
  78. Alfonso YN, Bishai D, Bua J, Mutebi A, Mayora C, Ekirapa-Kiracho E. Cost-effectiveness analysis of a voucher scheme combined with obstetrical quality improvements: quasi experimental results from Uganda. Health Policy Plann. 2013;30(1):88–99.View ArticleGoogle Scholar
  79. Kingkaew P, Werayingyong P, Aye SS, Tin N, Singh A, Myint P, et al. An ex-ante economic evaluation of the Maternal and Child Health Voucher Scheme as a decision-making tool in Myanmar. Health Policy Plann. 2015;31(4):482–92.View ArticleGoogle Scholar
  80. Jo Y, Labrique AB, Lefevre AE, Mehl G, Pfaff T, Walker N, et al. Using the lives saved tool (LiST) to model mHealth impact on neonatal survival in resource-limited settings. PLoS ONE. 2014;9(7):e102224.View ArticlePubMedPubMed CentralGoogle Scholar
  81. National Rural Health Mission 2005–2012. Approval of state program implementation programme: Uttar Pradesh, 2013–2014.Google Scholar

Copyright

© The Author(s) 2018

Advertisement