Is cost-effectiveness analysis preferred to severity of disease as the main guiding principle in priority setting in resource poor settings? The case of Uganda

Abstract

Introduction

Several studies carried out to establish the relative preference of cost-effectiveness of interventions and severity of disease as criteria for priority setting in health have shown a strong preference for severity of disease. These preferences may differ in contexts of resource scarcity, as in developing countries, yet information is limited on such preferences in this context.

Objective

This study was carried out to identify the key players in priority setting in health and explore their relative preference regarding cost-effectiveness of interventions and severity of disease as criteria for setting priorities in Uganda.

Design

610 self-administered questionnaires were sent to respondents at national, district, health sub-district and facility levels. Respondents included mainly health workers. We used three different simulations, assuming same patient characteristics and same treatment outcome but with varying either severity of disease or cost-effectiveness of treatment, to explore respondents' preferences regarding cost-effectiveness and severity.

Results

Actual main actors were identified to be health workers, development partners or donors and politicians. This was different from what respondents perceived as ideal. Above 90% of the respondents recognised the importance of both severity of disease and cost-effectiveness of intervention. In the three scenarios where they were made to choose between the two, a majority of the survey respondents assigned highest weight to treating the most severely ill patient with a less cost-effective intervention compared to the one with a more cost-effective intervention for a less severely ill patient. However, international development partners in in-depth interviews preferred the consideration of cost-effectiveness of intervention.

Conclusions

In a survey among health workers and other actors in priority setting in Uganda, we found that donors are considered to have more say than the survey respondents found ideal. Survey respondents considered both severity of disease and cost-effectiveness important criteria for setting priorities, with severity of disease as the leading principle. This pattern of preferences is similar to findings in context with relatively more resources. In-depth interviews with international development partners, showed that this group put relatively more emphasis on cost-effectiveness of interventions compared to severity of disease. These discrepancies in attitudes between national health workers and representatives from the donors require more investigation. The different attitudes should be openly debated to ensure legitimate decisions.

Introduction

Priority setting in health occurs at different levels and can be defined as distribution decisions involving clear and direct limitations of access to beneficial care or as a process of determining how health care resources should be allocated among competing programmes or people [1–3]. This can be implicit whereby the decisions and reasons for those decisions are not clearly expressed or explicit where they are clearly stated [4–6]. In the latter case, criteria can be used to facilitate the process. Several criteria for priority setting in health have been developed [7]. However, cost-effectiveness of interventions and severity of disease are some of the most widely discussed criteria. They are also some of the cardinal principles laid down in the Ugandan national health policy [8].

Cost-effectiveness compares cost per outcome of different interventions [9, 10]. Net economic cost is used as a numerator and improved health as a denominator and the lower the ratio the more preferred the intervention. Effects can be evaluated in terms of the impact of an intervention on mortality, morbidity, or quality of life. Cost-effectiveness analysis allows for comparison between interventions and makes allocation of resources explicit. Application of this procedure ensures that the maximum possible expected health benefit is realised, subject to whatever resource constraint is in effect [10, 8, 11].

Conversely, severity of disease has a variety of interpretations. We use the concept of the degree to which a condition affects a person's or population's health by causing death, handicap, disability, any kind of suffering or pain. Others use the concept of burden of disease measured in terms of Disability Adjusted Life Years (DALYs) which is a composite measure that combines both morbidity, mortality and other values in one single outcome measure [12, 11]. Severity of disease is an important concern in egalitarian approaches to priority setting [13].

Which one of the two criteria should be the most important criterion when setting priorities? In the extreme, a system that considers only cost-effectiveness would channel all its resources to people who happen to have the best potential to benefit from treatment in order to ensure efficient use of meagre resources [9, 14]. However, some studies have shown that using cost-effectiveness as a major criterion may not respond to what people want or expect [11, 13, 15, 16]. Conversely, a system that considers only severity may satisfy societal concern for the severely ill, but may lead to inefficient use of resources by overlooking the potential for patients to benefit from the interventions and ignoring costs. These factors are even more crucial in contexts of extreme resource scarcity [17–20].

Attempts to prioritise solely on the basis of cost-effectiveness as the major guiding principle, like the first plan in the state of Oregon, have not had much success in practice [21]. This is partly because of the ethical tensions between the maximisation of health benefit and societal concerns for the severity of disease [22]. Studies done in Australia and Norway have also shown societal preference for severity of disease as opposed to cost-effectiveness of intervention, as main criteria for priority setting [23–25].

Preferences may vary depending on culturally constructed values and norms in each population. Patterns of preferences may also be different in deprived settings where decision-makers are accustomed to having insufficient resources to treat everybody and having to exclude some beneficial treatments [8, 26]. Preferences may also differ with the level of priority setting [27, 28]. It is thought that people far removed from patients may have different values compared to those held by physicians, patients and their families [29, 30].

Most of the studies examining the relative importance of cost-effectiveness of intervention compared to severity of disease have been carried out in developed countries. This debate may seem far removed from the developing countries' contexts, such as Uganda where most severe diseases also have interventions that are cost-effective [30]. Still, the upsurge of non-communicable diseases, the resource demands due to the HIV/AIDS epidemic and the limited budgets for the health sector, makes this a relevant discussion.

Study Objectives

• 1. To establish the relative preferences regarding cost-effectiveness of interventions, and severity of disease as main criteria for setting priorities in Uganda.

• 2. To identify the perceived actual and ideal actors in priority setting in health.

Methods

We carried out a qualitative pre-survey study where eight group discussions were convened. Participants were homogeneous and included health workers, district planners, patient groups and the general population. These were asked what values they thought were important in priority setting in health. After a brain storming and deliberations, the values mentioned were ranked in order of perceived importance by consensus. All the groups, ranked severity of disease as the most important criteria, with the exception of the district planners' group, who ranked costs and effectiveness of care as the most important (Table 1).

Table 1 Relative importance of cost-effectiveness and severity of disease: sample results from the qualitative study

We also carried out in-depth interviews with international development partners and national level government officers who identified health workers as the main actors in priority setting and we therefore let this group form the biggest proportion of our survey respondents.

The survey was carried out in Uganda. We included respondents at national level and from four districts (out of the 49 districts), namely Kampala, Adjumani, Hoima and Kamuli. These were purposefully selected to represent the northern, eastern, western and central geographical regions. A sample size of 610 was calculated (assuming a response rate of 50%). Respondents included health planners and workers involved in priority setting at national, district, health sub-district and facility levels, and representatives from the general population. At national level, a list of all health workers was obtained and the heads of the different directorates included. In case the selected person was not available, the questionnaire was given to the person next in charge. At the national teaching hospital, respondents included senior house officers and student nurses. In the district hospitals, questionnaires were given to health workers working in each of the four major departments (medicine, paediatrics, surgery and obstetrics and gynaecology), with instructions to distribute them randomly between the doctors and nurses. In the health centres, all health workers were included. All members of the district local council and district health team were included. The rest of the questionnaires were strategically by virtue of the respondents' having some knowledge of the subject we were exploring.

Respondents were reminded three times, at intervals of one week, after which non-response was registered.

Study Focus

To identify the key actors, survey respondents were provided with the list of key actors that had been mentioned by the informants in the in-depth interviews, and were first asked to indicate the degree of importance attached to the different actors using ranks. The respondent could distinguish between their perception of the actual and ideal situation with the following statements:

• 1. In my opinion, the following are (at present) the main actors in priority setting in health.

• 2. In my opinion, the following should, ideally, be the main actors in priority setting in health.

The second question was asked in order to identify those people the respondents perceived to be legitimate actors in priority setting. The mean rank derived for a category of actors was taken as the overall rank for the category in the whole study population.

We then explored the general view of the consideration of cost-effectiveness of interventions or severity of disease by asking whether or not the respondents felt they were important for priority setting. Respondents indicated their degree of agreement on a six-point scale.

We further examined the respondents' preferences using three different scenarios based on a study by Nord [31]. We assumed that both patients, A and B, in the scenarios had similar personal characteristics and that with their given treatment, both patients would completely recover to their full health (Figure 1).

Figure 1

Schematic presentation of the three scenarios. Cost-effectiveness of A=B in Scenario 1, A>B in Scenario 2 and 3.

Scenario 1: Imagine you are a medical officer in a health unit and you receive two patients A and B. Patient B is severely ill while patient A is not so ill. Both patients require treatment that is equally cost-effective. They both can be restored to full health with their treatment. If you are only able to treat one of the two patients, which one would you treat, A or B?

Scenario 2: Imagine you are a medical officer in a health unit and you receive two equally severely ill patients A and B. Patient A requires a more cost-effective treatment, while patient B requires treatment which is less cost-effective. They both can be restored to full health with their treatment. If you are only able to treat one of the two patients, which one would you to treat, A or B?

Scenario 3: Imagine you are a medical officer in a health unit and you receive two patients. Patient B is severely ill but needs treatment that is less cost-effective, while patient A is not severely ill but requires treatment, which is more cost-effective. They both can be restored to full health with their treatment. If you are only able to treat one of the two patients, which one would you to treat, A or B?

Analysis

Survey data were analysed using SPSS. The mean rank was used for the question about actors in priority setting. For the relative importance of cost-effectiveness and severity, we first derived the frequencies of respondents' degree of acceptance of the two criteria for priority setting in Uganda.

We then dichotomised the six-point response scale such that respondents who strongly agreed or agreed with a criterion were re-coded as agree and those that disagreed or strongly disagreed were re-coded as disagree. The rest of the responses were re-coded as missing. For the three scenarios, after analysing the frequencies, we considered only specific responses: Patient A, Patient B. Non-respondents and respondents who preferred treatment of both patients (about 30%) were re-coded as missing because of small numbers. We also dichotomised the demographic characteristics. Age was re-coded as <= 35 years; > 35 years, designation as health worker; Non-health worker, Respondent consider priority setting as part of their daily duties: Yes; No, and Level of work as District and national planning level; Health facility level. Respondents who did not fit in these categories were re-coded as missing. We tested for associations between the characteristics of the respondents using the chi-square test. Interactions between these were adjusted for (using multiple logistic regression analysis), when testing for associations between the respondents' characteristics and their degree of agreement with the consideration of the different criterion for priority setting and their choices in the three scenarios. Odds ratios and confidence intervals are reported.

Results

Of the 610 questionnaires sent out, 413 (67.7%) were retrieved. The lowest response rate was from the northern region, in which there are ongoing acts of war. Fifty one percent of the survey respondents were health workers at health sub-district level. Respondents not directly employed in the health sector (politicians, administrators, other civil servants and the general population) accounted for 28% of the study population. The mean age of the respondents was 30.4 years (Table 2).

Table 2 Demographic characteristics of the survey respondents (n= 413)

Using the mean rank given to the actors by the 413 survey respondents, the current actors in priority setting in health, in order of perceived role played included health professionals, donors and government officers respectively (Fig. 2). Similar results were found in the in-depth interviews. However, in response to which the ideal actors should be, health professionals maintained their first position, while donors and politicians exchanged ranks with patients and the public respectively.

Figure 2

The actual and ideal rank given to various stakeholders' role in priority setting.

Relative importance of cost-effectiveness and severity of disease

In response to the question whether cost-effectiveness and/or severity should be considered when setting priorities in health, most of the respondents (63%) strongly agreed that both were important criteria (Table 3). About 8% either disagreed or were undecided.

Table 3 Respondents' opinion about considering cost-effectiveness or disease severity in priority setting (n = 413).

There were significant associations between some of the respondents' characteristics. Respondents above 35 years were less likely to be involved in priority setting at national or district levels than those below 35 years. The former were also less likely to consider priority setting to be part of their duties. Non-health workers were less likely to consider priority setting in health as part of their duty (Table 4). We controlled for these associations in the further analyses.

Table 4 Test for associations between the respondent's characteristics

The regression analysis (Table 5), showed that significantly more people involved in priority setting supported the consideration of cost-effectiveness as compared to those not involved in priority setting (OR= 8.9, p < 0.001).

Table 5 Test for associations between responses to the three scenarios and the respondents' characteristics

The in-depth interviewees, who were mainly donors and national level government officers, also expressed a preference for cost-effectiveness of interventions over severity of disease when asked about their guiding principle in priority setting.

In response to the three different scenarios where we tried to directly compare cost-effectiveness and severity through simulated situations, the majority of the survey respondents (66% of the total sample and 75% of the people who responded to this question) preferred to treat the severely ill patient (B), in the first scenario where both patients required equally cost-effective treatment. In the second scenario, where both patients had conditions of equal severity, about 60% preferred to treat the patient whose intervention was more cost-effective (patient A). In the last scenario, however, where the two values were directly compared against one another, a vast majority (83%) of those who responded to this question, opted to treat patient B, who was severely ill but required a less cost-effective intervention.

About 25% of the respondents declined to respond to both the second and third scenario questions, while a small proportion (1–3 %) preferred strict equality (treat both cases as equal) under the different scenarios (Figure 3).

Figure 3

Respondent's choices in the three different scenarios.

Some respondents, both from the in-depth interviews and the survey, gave reasons for their choices. These are presented in Table 6. Most of the survey respondents' favoured severity of disease as the guiding principle for setting priorities, as opposed to the respondents in the in-depth interviews who, to a larger extent, favoured cost-effectiveness.

Table 6 Reasons and comments by the development partners (from the in-depth study) and survey respondents.

Discussion

The survey respondents strongly supported the importance of considering both cost-effectiveness of interventions and the severity of the condition as criteria for priority setting. This is in line with the current literature on criteria for priority setting, which indicates the importance of both criteria. However, the finding that significantly more people involved in priority setting supported the consideration of cost-effectiveness may be a reflection of their experiences with priority setting at their levels.

The findings in the first two scenarios may be regarded as reasonable and the choices may not have been particularly difficult. In the third scenario, where respondents were forced to make a definite choice between cost-effectiveness and severity, there was a clear preference for the severity of the disease over the cost-effectiveness of intervention. This is surprising, given that Uganda is a context of extreme scarcity of resources; one may have expected a preference for cost-effectiveness. The lack of statistical differences in the responses to this critical question, in relation to age, designation, level of work, and whether or not the respondent considered priority setting to be part of their work, was also surprising. Some differences might have been expected, especially between actors at different levels of priority setting, and health workers and non-health workers; since it is generally believed that people far removed from patients may hold different values.

Our findings should, however, be interpreted with caution. Since the sample was strategic, with a majority of the respondents being health workers, the findings may not be representative for the Ugandan population as such. However, several studies exploring public values in priority setting indicate that the public regards health workers to be their legitimate representatives [32, 33, 2, 34, 4]. Given the weakness of civil society in Uganda, we regard the health workers' preferences as a first proxy for better understanding the public's values. Furthermore, being self-administered, we are unable to rule out the limitation of poor interpretation of the questions. The hypothetical situations may be too simplistic. This was intended since the study was very exploratory in the context. Moreover, given respondents' sensitivity to the phrasing of the questions, [15] we can only cautiously compare our findings with those from other studies, as we did not use exactly the same wording. In the third scenario, for example, the response might have been different if the scenario had consisted in treating one severely ill patient with an expensive treatment or many less severely ill patients with less costly treatments. We are also aware that these responses could differ in different circumstances and may be dependent on the type of respondents [23]. We also recognise the limitations to empirical ethics and the fact that there are additional criteria and values of relevance to priority setting which are not presented in this paper [35]. However we maintain that the findings, provide an initial step to the understanding some of the values held by people from a low-income context, more so since similar results were found in the pre-survey group discussions and in another study in similar settings [36].

Although the questions were different, our findings that the respondents preferred severity might be comparable to the studies carried out in Norway and Australia, both of which are high-income countries [23–25]. This may to some appear unreasonable in this context. However, societal concerns for equality and solidarity, seem to be as relevant in resource-poor settings as they are in resource-rich settings [19, 22, 33].

On the other hand, these results may be explained by lack of respondents' familiarity with these concepts or the way the scenarios were formulated in the number of beneficiaries and the budget limits were not specified. These issues require more exploration since such findings may have implications for the designing of the essential health care package.

The survey respondents' strong preference for severity of disease did not fully comply with what the other actors expressed in the in-depth interviews. In particular, the international development partners, were more positive to setting priorities according to cost-effectiveness of interventions as compared to the survey respondents. This may reflect that it is easier for donors to consider cost-effectiveness of treatments, than it is for health workers who meet the patients directly. In such instances, where the key actors and those representing the interests of the public do not agree on values guiding health policy, explicit negotiation, deliberation and open debates about values are required.

The persistent number of respondents who either insisted on strict equality (3%) (treating both patients as equal), or declined to respond to the questions, may reflect respondents' escape from making difficult decisions, aversion to hypothetical choices or indeed a strong preference for equality [14].

Actors in health

The actors identified in this study are similar to those identified in other countries, apart from the international development partners (donors) and politicians [34]. Donors play an important role in providing resources for the health sector and may therefore influence the resource allocation process [32, 14]. Politicians, especially if democratically elected, may have an important role in representing the public in priority setting [4]. This is becoming increasingly evident in Uganda, which enjoys a democratic system [37]. Although the survey respondents this was not the ideal. The rank given to the judiciary, consumer organisations and insurance companies may be a reflection of the actual role played by these actors. Their role, although recognised as important, remains limited [38, 34].

Although it may be premature to draw policy implications from this preliminary study, there are some general implications. Given the concern for severity of disease shown in this and other studies and the WHO recommendation that services should be responsive to the needs of the people (within limits) [39, 11], decisions not to fund relatively less cost-effective treatments, such as HAART for severely ill patients, may need to be re-examined, especially if severity of disease is indeed a strongly held value in the Ugandan context.

However, if such a decision is to be taken, there'd be need for clear definitions and good evidence. To this end, information from the WHO project Choosing Interventions that are Cost-effective (CHOICE) and the severity of disease data, would be indispensable resources, provided this information is reliable and can openly be discussed [40]. At the theoretical level, we might add that our study suggests the use of a concept discussed by Amartya Sen – that "extra-welfarist" information about severity of disease is a necessary supplement to the "welfarist" framework currently employed in standard cost-effectiveness analysis [41].

In addition, more information on different actors' values is necessary. This information can then be mapped out for the development and definition of essential health care interventions, as illustrated in Fig. 4. With additional resources, government can choose to take either path (a) to include more cost-effective interventions, or path (b) to include more severe (but not necessarily cost-effective) interventions. Any choices made on the trade-off between efficiency and severity should be openly debated to ensure legitimacy.

Figure 4

Potential policy implications for the trade off between cost-effectiveness of intervention and severity of disease. The essential national health package is based on the most cost-effective interventions (line parallel to the x-severity axis) against the leading causes of severity of disease (line parallel to the Y-cost-effective axis). Transparent criteria need to be developed in case of expansion of the package to include other interventions. In case cost-effectiveness is the criteria, then expansion should be in the direction of arrow (a), in case it is severity, then one should consider (b).

Conclusions

To the best of our knowledge, this is one of the first studies conducted in a developing country to establish the relative importance placed on cost-effectiveness and disease severity as criteria for priority setting.

The current main actors in priority setting in Uganda include, in order of influence, development partners, the Ministries of Finance and Health, health professionals and politicians. Our survey respondents indicated that health professionals, patients and the public should play a leading role ahead of the development partners.

In this study, the national survey respondents' preferences differed from those elicited in in-depth interviews with the development partners. Whereas the vast majority of the survey respondents preferred severity of disease to cost-effectiveness of intervention as the guiding principle, most of the development partners preferred cost-effectiveness of intervention. The results of the national survey are surprisingly similar to those from contexts with relatively more resources and different cultures.

To ensure legitimacy in decision-making, it is necessary that actors are transparent about their values and encourage an open debate on difficult choices [42]. A schema such as Fig. 4 could facilitate open discussions. These and other additional values and criteria would inform decision-making processes and debate on criteria for priority setting in contexts of extreme resource constraints.