Skip to main content

Volume 16 Supplement 1

Priority Setting in Global Health

Introduction: priority setting in global health

The process of setting priorities for social spending is inherently complex. It requires, in general, taking account of heterogeneity in preferences and expectations over a wide range of options and outcomes. It may involve multiple stakeholders, each with different and not-necessarily aligned interests, within or across different sectors. Priority setting processes can also incorporate a varying range of subjective/qualitative and objective/quantitative considerations. Resource-allocation decisions can be shaped by institutional requirements or strictures and might also be driven by political expediency and the desire to build popular support.

Multi-criteria decision analysis (MCDA) is an approach that supports priority setting “by taking explicit account of multiple criteria when helping individuals or groups explore decisions that matter” [1]. Researchers at the Harvard T.H. Chan School of Public Health hosted a Priority Setting in Global Health symposium in Cambridge, Massachusetts on October 5–6, 2016, with a special focus on exploring MCDA’s strengths and identifying practical solutions to its limitations. This symposium brought together under one roof some of the world’s leading experts on MCDA and global health. Symposium participants included co-chairs of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) MCDA Emerging Good Practices Task Force, developers and users of MCDA instruments in various contexts (e.g., the EVIDEM framework and the SMART Vaccines tool), academics and researchers from a variety of disciplines (including biotechnology, economics, epidemiology, medical ethics, and medicine), and political representatives from around the world. This special issue is a collection of cutting-edge MCDA research, reviews, and commentaries built on the illuminating presentations and comments offered by symposium participants. (See Tables 1 and 2 for the symposium agenda and the full participant list).

Table 1 Priority setting in global health, October 2016, Cambridge, MA: Agenda
Table 2 Priority setting in global health, October 2016, Cambridge, MA: Participants

Over the past decade, MCDA has increasingly been discussed and adapted to address the challenges of priority setting in global health. MCDA uses health and health technology impact data to rank a variety of decision alternatives in order of priority, based on multiple explicit criteria that are articulated, evaluated/scored for their impact, and weighted by relevant stakeholders. Proponents of MCDA believe that its three core strengths are its pragmatism, its basis in real-world evidence as well as the contextual preferences of the decision makers, and its focus on optimizing the setting of priorities [2, 3].

In response to the widening adoption of MCDA in health care decision making, ISPOR established the MCDA Emerging Good Practices Task Force to provide initial recommendations on how MCDA can best support health care decisions [4, 5]. This Task Force recommends the following steps to combine scientific evidence with stakeholder preferences in any MCDA process: Explicit criteria are selected for alternative courses of action for the health decision problem under consideration. The measured or expected impact of each health care alternative is quantified according to each of the explicitly defined criteria, requiring decision makers to reference, understand, and utilize relevant scientific evidence. To allow meaningful comparisons, the performance metrics are then translated into common-scale scores with uniform increments. For example, quality-adjusted life years (QALYs) and mortality rates by ethnicity might both be converted to a scale from 0 to 100, with higher scores indicating that the health care alternative has a higher impact for that specific criterion. Then, the multiple criteria are weighted based on stakeholder preferences and summed, allowing for a mechanism in which objective health data are melded with subjective values to generate aggregate scores for each health care alternative. Taking explicit account of any uncertainty/limitations in the design and application of the MCDA process, the aggregate scores are interpreted and used to generate a ranking of health priorities that is intended to inform practical and rational priority setting.

One of the preeminent applications of MCDA in global health is the evidence and value: impact on decision making (EVIDEM) framework, created in 2006 to facilitate deliberative and evidence-based multi-criteria health care decision making at the individual and institutional levels [6]. Implementation of the Framework is intended to incorporate accountability for reasonableness (A4R) principles, which state that priority setting should occur in a context of cooperative deliberation and that rationales involved in decision making should be publicly transparent [7]. The EVIDEM framework is designed to raise awareness of the ethical implications of each step of the MCDA process, ranging from the identification of relevant criteria and corresponding evidence to the selection of stakeholders, elicitation of preference weights, and the interpretation of results [8].

Along similar lines, the U.S. National Academies of Sciences, Engineering, and Medicine recently produced a blueprint, a prototype, and use-case scenarios for multi-criteria decision making through the Strategic Multi-Attribute Ranking Tool for Vaccines (SMART Vaccines), to aid priority setting specific to vaccine development, investment, and policy making [9,10,11]. With over two dozen criteria that extend beyond economic considerations, SMART Vaccines allows decision makers to explicitly incorporate indicators and considerations pertaining to health equity, national security, vaccine delivery, operational management, and scientific and business advancement into vaccine priority setting [12].

Multi-criteria decision analysis developers and users have argued that MCDA’s potential rests on its ability to evolve as both a rigorous instrument and a versatile process in response to diverse stakeholder needs [13, 14]. However, if MCDA is to gain legitimacy and traction in the global health priority setting community, limitations in the assumptions and processes inherent to the development and application of MCDA models must be explicitly addressed.

For instance, selecting comprehensive criteria requires a fully transparent and documented process with input from key stakeholders, such as decision makers (e.g., ministers of health and finance, insurance companies, etc.) and public health beneficiaries (e.g., patients with specific health conditions or members of the general public) [15]. Concurrently, MCDA criteria are more meaningful if the selected criteria do not overlap in their scope and definition. A majority of MCDA demonstrations to date have been criticized for using linear, first-order weighted sums of multiple criteria to generate the final output ranking scores. This process relies on an often-unsupported assumption that the underlying criteria do not overlap and that they are orthogonal and preferentially independent [16, 17]. Even if MCDA models are designed with strictly non-redundant criteria, a simultaneous limitation of such an approach is that it might not extend beyond purely academic/mathematical rationale to take account of more practical and ethical considerations. For example, while MCDA developers and users often attempt to be as comprehensive as possible, they might consider limiting the criteria to an appropriate number based on the availability of impact data and the feasibility of obtaining complete and meaningful stakeholder preferences [17, 18].

Though there is no dominant method for eliciting individual or collective stakeholder preferences, MCDA models and processes are more likely to be incorporated in priority setting activities if they have been developed by researchers and decision makers in close partnership [19, 20]. Similarly, MCDA will be more acceptable if perspectives from the general public are elicited in a representative and meaningful manner [21, 22]. Preference elicitation surveys must be carefully constructed to minimize the cognitive burden on the respondent, while still presenting meaningful questions that are consistent with the full range of criteria and alternatives in a given MCDA framework [23, 24]. To enhance the legitimacy and fairness of these value-driven aspects of MCDA, diverse stakeholder participation will ideally extend beyond a solitary opportunity for input to ongoing “evidence-informed deliberative processes” that facilitate iterative discourse and greater stakeholder understanding throughout every phase of MCDA development and application [25, 26].

Multi-criteria decision analysis can impart greater structure and transparency to priority setting, but effectively leveraging its strengths largely depends on the context in which it is implemented. Developing countries that tend to have implicit and ad hoc priority setting processes are faced with practical barriers such as the dearth of extensive, meaningful data to measure the performance of each alternative according to each criterion [27, 28]. Notwithstanding such limitations, recent evidence suggests that MCDA can provide a structured, objective, and value-based framework in low- and middle-income countries, especially in combination with other approaches such as health technology assessment [15, 29]. The World Health Organization (WHO) recently demonstrated an MCDA application of the WHO-CHOICE methodology, in which key stakeholders from around the world prioritized an extensive list of interventions for the prevention and control of non-communicable diseases, using criteria of cost-effectiveness, feasibility, and equity, as well as health system considerations [30].

How MCDA outputs are interpreted for policy decisions also remains an open question. By definition, MCDA frameworks employ multiple criteria, often with differing units that do not lend themselves to obvious, comparable value improvement thresholds (similar to incremental cost-effectiveness ratios in cost-effectiveness analyses) to account for the opportunity costs in funding decisions [31]. To this end, it has been proposed that existing dollar estimates of willingness to pay for QALYs might be used to create comparable cutoffs for multi-criteria value measures in resource allocation decisions [32]. It has also been suggested that policy entrepreneurs or institutions might be leveraged to outline and oversee the specific goals, designs, rules, ethics, and processes that govern MCDA applications in health [33].

In summary, priority setting in global health typically requires tradeoffs among a variety of clinical, economic, ethical, political, scientific, and social factors that vary across relevant stakeholders. While there remain ethical, conceptual, and empirical challenges to MCDA’s widespread implementation, MCDA has the potential to explicitly identify and account for each of these competing factors in a comprehensive, systematic, and value-driven manner [34].

We would like to thank the editors and managers of the journal Cost Effectiveness and Resource Allocation for hosting this Priority Setting in Global Health special issue. We would also like to thank the symposium participants for their insightful contributions and commentaries. We would especially like to thank the referees for their detailed review of all the manuscripts. In addition, we are grateful to Mark O’Friel, the Brinson Foundation, and the Payne Family Foundation for their generous financial support for the publication of this special issue. Finally, we would like to thank Mireille Goetghebeur, Guru Madhavan, and Praveen Thokala for their helpful comments on this Introduction.


  1. Belton V, Stewart TJ. Introduction. In: Belton V, Stewart TJ, editors. Multiple criteria decision analysis: an integrated approach. Boston: Kluwer Academic Publishers; 2002. p. 2.

  2. Marsh KD, Goetghebeur M, Thokala P, Baltussen R. Multi-criteria decision analysis to support healthcare decisions. Cham: Springer International Publishing; 2017.

    Book  Google Scholar 

  3. Goetghebeur M, Castro-Jaramillo H, Baltussen R, Daniels N. The art of priority setting. Lancet [Internet]. 2017;389(10087):2368–9.

    Article  PubMed  Google Scholar 

  4. Thokala P, Devlin N, Marsh KD, Baltussen R, Boysen M, Kalo Z, et al. Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA emerging good practices task force. Value Health. 2016;19(1):1–13.

    Article  PubMed  Google Scholar 

  5. Marsh KD, Ijzerman M, Thokala P, Baltussen R, Boysen M, Kaló Z, et al. Multiple criteria decision analysis for health care decision making—emerging good practices: report 2 of the ISPOR MCDA emerging good practices task force. Value Health. 2016;19(2):125–37.

    Article  PubMed  Google Scholar 

  6. EVIDEM | A short history. EVIDEM. Accessed 11 May 2018.

  7. Daniels N, Sabin JE. Accountability for reasonableness. In: Daniels N, Sabin JE, editors. Setting limits fairly: can we learn to share medical resources? 2nd ed. New York: Oxford University Press; 2008. p. 45.

    Google Scholar 

  8. Goetghebeur MM, Cellier MS. Can reflective multicriteria be the new paradigm for healthcare decision-making? The EVIDEM journey. Cost Eff Resour Alloc. 2018.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Madhavan G, Sangha K, Phelps C, Lieu T, Martinez RM, King L. Ranking vaccines: a prioritization framework. Washington, D.C.: The National Academies Press; 2012.

    Google Scholar 

  10. Madhavan G, Sangha K, Phelps C, Rappuoli R, Martinez RM, King L. Ranking vaccines: a prioritization software tool. Washington, D.C.: The National Academies Press; 2013.

    Google Scholar 

  11. Madhavan G, Phelps C, Rappuoli R, Marie R, King L. Ranking vaccines: applications of a prioritization software tool. Washington, D.C.: The National Academies Press; 2015.

    Google Scholar 

  12. Madhavan G, Phelps C. Strategic planning tools for preparedness. In: Symposium—priority setting in global health. Cambridge; 2016.

  13. Norheim OF, Marsh KD, Baeza C, Edejer TTT, Jit M. Considerations for development of MCDA tools. In: Symposium—priority setting in global health. Cambridge; 2016.

  14. Watson M, Chalkidou K, SteelFisher GK, Habtemariam MK. Considerations for applications of MCDA tools. In: Symposium—priority setting in global health. Cambridge; 2016.

  15. Habtemariam MK, Semegn ST. Setting health sector priorities: a brief overview of ethiopia’s experience. Cost Eff Resour Alloc. 2018.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Jit M. MCDA from a health economics perspective: opportunities and pitfalls of extending economic evaluation to incorporate broader outcomes. Cost Eff Resour Alloc. 2018.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Norheim OF. How can MCDA tools improve priority setting? Four critical questions. Cost Eff Resour Alloc. 2018.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Gellin B, Knobler S. SMART Vaccines 2.0: Piloting further development of a multi-criteria decision analysis tool. In: Symposium—priority setting in global health. Cambridge; 2016.

  19. Madhavan G, Charles P, Rappuoli R. Compare voting systems to improve them. Nature. 2017;541:151–3.

    Article  CAS  PubMed  Google Scholar 

  20. Thokala P, Madhavan G. Stakeholder involvement in Multi-Criteria Decision Analysis. Cost Eff Resour Alloc. 2018.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Dayalu R, Cafiero-Fonseca ET, Fan VY, Schofield H, Bloom DE. Priority setting in health: development and application of a multi-criteria algorithm for the population of New Zealand’s Waikato region. Cost Eff Resour Alloc. 2018.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Sevilla JP. MCDA or preference-based social welfare functions? Cost Eff Resour Alloc. 2018.

    Article  PubMed  PubMed Central  Google Scholar 

  23. SteelFisher GK. Including the public perspective in health-related MCDA: ideas from the field of public opinion research and polling. Cost Eff Resour Alloc. 2018.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Marsh KD, Sculpher M, Caro JJ, Tervonen T. The Use of MCDA in HTA: great potential, but more effort needed. Value Health. 2018;21(4):394–7.

    Article  PubMed  Google Scholar 

  25. Daniels N. Combining A4R and MCDA in priority setting for health. Cost Eff Resour Alloc. 2018.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Baltussen R, Paul Maria Jansen M, Bijlmakers L, Grutters J, Kluytmans A, Reuzel RP, et al. Value assessment frameworks for HTA agencies: the organization of evidence-informed deliberative processes. Value Health. 2017;20(2):256–60.

    Article  PubMed  Google Scholar 

  27. Marsh K, Thokala P, Youngkong S, Chalkidou K. Incorporating MCDA into HTA: challenges and potential solutions, with a focus on lower income settings. Cost Eff Resour Alloc. 2018.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Glass RI. Priority setting for the introduction of rotavirus vaccine: what evidence was essential? Cost Eff Resour Alloc. 2018.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Castro-Jaramillo HE, Moreno-Mattar O, Rivillas JC. HTA and MCDA solely or combined? The case of priority-setting in Colombia. Cost Eff Resour Alloc. 2018.

    Article  Google Scholar 

  30. Luciani S, Hennis A. Commentary: setting priorities in NCD prevention and control. Cost Eff Resour Alloc. 2018.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Sculpher M, Claxton K, Pearson SD. Developing a value framework: the need to reflect the opportunity costs of funding decisions. Value Health. 2017;20(2):234–9.

    Article  PubMed  Google Scholar 

  32. Phelps C, Madhavan G. Resource allocation in decision support frameworks. Cost Eff Resour Alloc. 2018.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Fan VY. Multi-criteria decision analysis: of politics, policy entrepreneurs, and processes. Cost Eff Resour Alloc. 2018.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Watson M. Panel Discussion on the application of MCDA tools. Cost Eff Resour Alloc. 2018.

    Article  PubMed  PubMed Central  Google Scholar 

Download references


Authors’ contributions

All authors read and approved the final manuscript.


The Guest Co-Editors dedicate this special issue of Cost Effectiveness and Resource Allocation to the memory of Dr. Aakanksha H. Pande, who passed away on July 21, 2018 at the age of 37. Dr. Pande, who earned degrees from Yale University and Harvard University, served as Senior Health Economist at the World Bank. In addition to being a visionary thinker and a creative, hard-working, collegial, and accomplished professional, Aaka was a fine and loving person and a good friend to many. She will be missed.

Competing interests

The authors declare that they have no competing interests.

Availability of data and materials

Not applicable.

Consent for publication

All authors consent to the publication of this manuscript.

Ethics approval and consent to participate

Not applicable.


The publication costs for this article were funded by Mark O’Friel, the Brinson Foundation, and the Payne Family Foundation.

About this supplement

This article has been published as part of Cost Effectiveness and Resource Allocation Volume 16 Supplement 1, 2018: Priority Setting in Global Health. The full contents of the supplement are available online at

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and Affiliations


Corresponding author

Correspondence to David E. Bloom.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bloom, D.E., Cadarette, D., Dayalu, R. et al. Introduction: priority setting in global health. Cost Eff Resour Alloc 16 (Suppl 1), 49 (2018).

Download citation

  • Published:

  • DOI: