Table 1 Initial criteria including Likert scores and important comments given in the Delphi study

Effectiveness

4.75

5

4-5

Effectiveness is covered by its components. Effectiveness should therefore be removed and its components should be independent criteria, otherwise they will overlap.

Size of effectiveness

4.70

5

3-5

No important comments.

Certainty of the evidence

4.35

5

1-5

Not related to effectiveness only. The strength of the evidence varies by criterion for any given intervention. Much simpler and effective to include considerations of certainty of evidence in assigning scores for all given criterion.

Time until the effect emerges

3.09

3

1-5

Time preference for immediate effects goes against principles of intergenerational equity, and is especially inappropriate for preventive services. Therefore this criterion should be removed.

Cost-effectiveness

4.25

4.5

1-5

MCDA might replace C/E. We can have costs but “effectiveness” is defined by the sum of the criteria so adding this criterion introduces double-counting.

Efficiency cannot be replaced by costs since higher costs do not per se mean lower efficiency as the effectiveness may be higher.

Feasibility

4.23

4

2-5

This should be four different criteria, otherwise they will overlap each other.

Reach

4.46

5

2-5

See comments accessibility.

Technical complexity

3.5

3.5

1-5

No important comments.

Capital intensity

3.75

4

1-5

This criterion should not be limited to capital costs but also explicitly include operating costs required from the health system.

Cultural acceptability

4.13

4.5

1-5

No important comments.

Safety

4

4

2-5

The importance of safety may vary with respect to whose safety (provider vs. patient) and what is at stake, while the level of acceptability may remain the same. Therefore acceptability and safety should be kept separated.

Accessibility

4.33

4.5

1-5

Accessibility due to geographical coverage of an intervention (‘Reach’) is not the same as accessibility due to socio-economic status. Therefore this criterion should be about equal access for patients with different socio-economic status, while geographical coverage should be covered by another criterion (‘Reach’)

Severity of breast cancer

3.26

3

1-5

Of course I think that palliative care is very important. On the other hand, if you do nothing for all the people with earlier stage cancer, the cancer will progress and they will all need palliative care. So you could treat people with stage 1 or 2 cancer and most of them will not experience late stage cancer, therefore will not need palliative care. I guess I don’t find this a useful way to think about breast cancer.

Age

3.29

3.5

1-5

Ages of patients with breast cancer don’t seem appropriate even if one wanted to create prioritized age groups, which I wouldn’t.

Magnitude of individual health impact

3.83

4

1-5

No important comments.

Catastrophic health expenditures

4.17

5

1-5

Affordability is about whether the health system can afford an intervention and catastrophic health expenditures is about whether patients can afford it. Extreme health expenditures might however be covered by accessibility, because patients with lower socio-economic status cannot afford high health expenditures.