Skip to main content

Volume 16 Supplement 1

Priority Setting in Global Health

How can MCDA tools improve priority setting? Four critical questions


This comment argues that four critical questions needs to be resolved before MCDA tools can improve priority setting in health: how to merge the quantitative and deliberative elements of MCDA; how to select criteria; how to weigh them, and whom to bring to the table.


Baltussen and others have proposed that multi-criteria decision analysis (MCDA) can improve priority setting processes [1,2,3]. Other approaches include cost-effectiveness analysis [4, 5], criteria-based systems that include cost-effectiveness and other concerns [6,7,8,9], programme budgeting and marginal analysis (PBMA) [10], procedural frameworks such as accountability for reasonableness [11], and rights-based approaches [12].

For example, the World Health Organization and the World Bank have championed cost-effectiveness as a key criterion for global and national priority setting [4, 5]. In UK, the National Institute for Health and Care Excellence (NICE) identifies the most cost-effective services through health technology assessment, with input on key values from the Citizen’s Council. In Thailand, the Health Intervention and Technology Assessment Program (HITAP) appraises a wide range of health technologies and public health programs by six criteria: size of population affected, severity of disease, effectiveness of health intervention, variation in practice, economic impact on household expenditure, and equity and social implications [8, 9].

Common for all priority setting approaches is that they seek to provide a ranking of services delivered at different levels of the health system by linking evidence on needs and outcomes with values, principles, and criteria that have support in the population. The approaches differ along at least two axes: (1) technical versus deliberative, and (2) narrow focus versus comprehensive.

MCDA as discussed in this issue of the journal have moved from a more technical framework with a narrow focus towards a deliberative framework allowing for discussion and use of a wide range of criteria and concerns [3]. In my view, MCDA has moved too far from its origin. In the further development of MCDA tools, four critical questions therefore need further scrutiny.

Technical or deliberative approaches?

Already in 1998, the ethicist Soren Holm said “goodbye to the simple solutions”, arguing that the false dichotomy between substantive and deliberative approaches to priority setting would be replaced by a third phase of integration [6]. A similar development may be seen for MCDA. If both quantification and deliberation is key, how can the two approaches be integrated? Enthusiasm for deliberation should not lead to the abandonment of quantification.

How many criteria can a decision-maker handle?

The trend of embracing all possible criteria for priority-setting analysis is, in my view, unfortunate. While moving beyond cost-effectiveness as a single criterion is appropriate, quantitative models may not be able to handle an unlimited number of additional concerns without ending up as another black-box approach. Likewise, stakeholders discussing priority setting decisions may not be able to handle and balance 10–15 different considerations for a large set of interventions or services at the same time. Handling such large amount of data is simply not possible in a comprehensive way. The field needs to set priorities among the criteria included in the analytical framework. I am prepared to argue that, at the most general level, we need only four criteria: level and distribution of health outcomes; level and distribution of non-health outcomes.

Quantitative approaches: additive or multiplicative weights?

Current practice in MCDA for priority setting is to ask key stakeholders which criteria they find relevant and ask them (through discrete choice experiments) to assign weights to each criterion. The MCDA analyst then rank services based on stakeholder values. Often additive weights are used; and sometimes 10–15 criteria are identified in this way. If, say cost-effectiveness is assigned a weight of “20%” though this process (and the remaining 80% is distributed to the other criteria), the result strikes me as problematic. What does “20%” mean? I believe the use of additive weights is at risk of forcing key stakeholders to end up with a rank-order that is different from what they would end up with simply through individual judgment or a purely deliberative process. One possible explanation is the choice of weighing scheme.

Consider this example (Table 1). Three interventions have the same cost, different benefits, and target groups with different incomes. Let us assume that stakeholders care about only two concerns: equity and benefit maximization.

Table 1 Three interventions (a–c) with the same cost, different benefits, and target groups with different incomes (PW poverty-weighted)

In A, stakeholders think that benefits to the poor have three times the value of benefits to the rich; in B, stakeholders also think that benefits to the poor are more important than benefits to the rich, and they assign equal weight to ‘equity’ (50%) and ‘benefit’ (50%).

As can be seen from the table, using multiplicative weights would clearly rank the three interventions, while using additive weights would not—in this example. The reason is that additive weights, as used in many forms of MCDA, are applied to ordinal ranks. There is loss of information compared to the multiplicative weighing scheme. Methods for assigning weights to different concerns and their rationale need to be further discussed and explored in MCDA.

Deliberative approaches: who are invited to the table?

For deliberative approaches to MCDA, issues related to representation and power asymmetries require further clarification [13]. The outcome of MCDA depends crucially on who the participants are. Current practice tends to favor health policy decision-makers and health planners, with some patient or user-participation. Given the technical nature of priority setting, there may be feasibility constraints on who can participate. Yet, this foundational issue for approaches that aim to be more democratic is insufficiently explored. Who is invited to the table matters.


In summary, recent developments of MCDA tools for priority setting are detached from their theoretical foundations: deliberation without proper quantification can mislead; choosing too may criteria can confuse; using additive weights causes loss of information, and the choice of participants is arbitrary.


  1. Baltussen R, et al. Value assessment frameworks for HTA agencies: the organization of evidence-informed deliberative processes. Value Health. 2017;20(2):256–60.

    Article  Google Scholar 

  2. Marsh K, et al. Multiple criteria decision analysis for health care decision making-emerging good practices: report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health. 2016;19(2):125–37.

    Article  Google Scholar 

  3. Goetghebeur M, et al. The art of priority setting. Lancet. 2017;389(10087):2368–9.

    Article  Google Scholar 

  4. World Bank, Investing in Health. World development report 1993. Oxford: World Bank, Oxford University Press; 1993.

    Book  Google Scholar 

  5. WHO. CHOosing interventions that are cost effective (WHO-CHOICE). 2012. Accessed 15 Aug 2012.

  6. Norheim OF, et al. Guidance on priority setting in health care (GPS-Health): the inclusion of equity criteria not captured by cost-effectiveness analysis. Cost Eff Resour Alloc. 2014;12(1):18.

    Article  Google Scholar 

  7. World Health Organization. Making fair choices on the path to universal health coverage. Geneva: World Health Organization; 2014.

    Google Scholar 

  8. Glassman A, Chalkidou K. Priority-setting in health: building institutions for smarter public spending. Washington, DC: Center for Global Development; 2012.

    Google Scholar 

  9. Culyer AJ, Podhisita C, Santatiwongchai B. A star in the east. A short history of HITAP. Bangkok: Amarin Printing and Publishing Public Co., Ltd.; 2016.

    Google Scholar 

  10. Mitton C, Donaldson C. The priority setting toolkit: a practical guide to the use of economics in health care decision making. London: BMJ Books; 2004.

    Google Scholar 

  11. Daniels N, Sabin JE. Setting limits fairly: learning to share resources for health. 2nd ed. Oxford: Oxford University Press; 2008.

    Google Scholar 

  12. Yamin A, Gloppen S, editors. Health rights in comparative perspective. Cambridge: Harvard University Press; 2011.

    Google Scholar 

  13. Baerøe K. Priority-setting in healthcare: a framework for reasonable clinical judgements. J Med Ethics. 2009;35(8):488–96.

    Article  Google Scholar 

Download references


Authors’ contributions OFN is the sole author of this manuscript. The author read and approved the final manuscript

Authors’ information

OFN is a physician and professor in medical ethics and philosophy of science, Department of Global Public Health and Primary Care, University of Bergen, and adjunct Professor at the Department of Global Health and Population, Harvard TH Chan School of Public Health. Norheim’s research interests include theories of justice, inequality in health, the ethics of priority setting in health systems, and how to achieve Universal Health Coverage and the Sustainable Development Goal for health. He is currently leading, with Stéphane Verguet, the research project Disease Control PrioritiesEthiopia (2017–2020). Dr. Norheim has chaired the World Health Organization’s Consultative Group on Equity and Universal Health Coverage (2012–2014) and the third Norwegian National Committee on Priority Setting in Health Care (2013–2014).


OFN wants to thank the participants of the Harvard meeting for useful comments to the first draft of the paper and for fruitful discussions with Rob Baltussen over the years.

Competing interests

The authors declare no competing interests.

Availability of data and materials

Not applicable.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Not applicable.

Publication funding

The publication costs for this article were funded by Mark O’Friel, the Brinson Foundation, and the Payne Family Foundation.

About this supplement

This article has been published as part of Cost Effectiveness and Resource Allocation Volume 16 Supplement 1, 2018: Priority Setting in Global Health. The full contents of the supplement are available online at

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Ole F. Norheim.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Norheim, O.F. How can MCDA tools improve priority setting? Four critical questions. Cost Eff Resour Alloc 16 (Suppl 1), 44 (2018).

Download citation

  • Published:

  • DOI: