Table 2 Overview of substantive criteria, sub-criteria, and decision aspects

Health related balance of benefits and harms

General considerations surrounding benefit/effect

Benefits/effect/efficacy/effectiveness/impact [1, 2, 4–10, 12–15, 17–27, 29, 33–36]

Health related Benefits/effect/efficacy/effectiveness/impact [1, 2, 4, 8, 11–15, 21, 23, 26, 31, 34, 35]

Uptake of intervention [15, 20, 34, 35]

Magnitude of benefit/effect/impact [2, 4, 11, 14, 18, 31, 33, 36]

Additional or indirect effects [2, 6, 33, 34]

Type and composition of effect/benefit/impact [2]

Impact on mortality, survival, longevity and life expectancy [1, 2, 4, 11, 16, 19, 21, 24–26, 28, 34–36]

Last chance therapies [23, 24]

Impact on morbidity and disability [1, 2, 16, 35]

Potential changes in health consequences [24, 25]

Impact on (health-related) quality of life [2, 8, 11, 12, 14, 19, 20, 22, 25, 26, 28, 29, 31, 33, 35, 36]

Impact on patient-reported outcomes [2, 12, 16, 21, 26]

Valuation of health outcomes by patients and desirability of the effects [2]

Preventive benefits/effects or preventive approaches [1, 2, 4, 5, 16, 21, 25, 26, 31]

Character of benefit or effect

Onset of effect and time until benefit [2, 11, 13]

Duration, sustainability and lasting effect [2, 11, 13, 15, 27, 31]

Individual and population level of benefit

Clinical benefits/effectiveness/impact [2, 3, 5, 11, 14, 17–22, 24, 25, 28, 31–33, 36]

Individual level benefit, effectiveness or impact [1, 2, 4, 18, 21, 22, 25, 26, 36]

Marginal benefits (for every patient) [20, 26]

Population level benefit, effectiveness or impact [1–5, 9, 12, 16, 18, 19, 24–27, 31, 34, 35]

Threshold effectiveness on populations (herd immunity) [15, 34]

Balance of benefits and harms

Balance of (health) benefits and harms [5, 14, 18, 19, 23, 24, 34]

General considerations surrounding harm/risk

General safety, risk and tolerability of intervention [2, 4, 5, 7–9, 11–17, 19–26, 29, 31, 33–36]

Magnitude and likelihood of adverse events [26, 33–35]

Valuation of health outcomes by patients and desirability of the effects regarding harms [34]

Short and long term risk and safety profile [26]

Over diagnosis and over treatment [16, 26]

Stigmatization [2, 26]

Risk of failure of intervention [15, 34, 35]

Burden of treatment [2, 26, 33]

Risk of inappropriate use [8, 16, 25, 26]

Impact on disease patterns and reduced long-term effectiveness [15, 34]

Other or additional adverse events [2]

Health-related need and priority

Health-related needs: in general [1, 2, 11, 14, 22, 23, 24]

Burden and impact of disease: in general [1, 2, 4, 7–10, 13–16, 21, 23–26, 33, 34, 36]

Magnitude of the problem [4, 10, 25, 34]

Burden of disease measured through epidemiological indicators [1, 2, 4, 5, 8, 11, 13–15, 21, 23–26, 28, 33–36]

Size of affected population and number of potential beneficiaries [1, 2, 4, 5, 8, 10, 11, 13–17, 20–26, 28, 31–34, 36]

Maximum potential for disease burden reduction [10]

Severity of disease/condition: in general [1, 2, 4, 5, 9–12, 14, 16–19, 21–26, 28, 29, 31, 33–36]

Severity of disease/condition: long term outcomes [34, 35]

Severity of disease/condition:

life threatening disease/condition and prognosis without treatment [1, 5, 11, 16, 21, 28, 29, 34]

Severity of disease/condition:

late stage or end-of life status of disease/condition [5, 21, 24, 25, 28, 29, 36]

Outbreaks and epidemic potential [15, 34]

Urgency and emergencies [1, 2, 13, 25]

Human and

individual

rights

Human rights considerations [2, 11, 12, 20, 36]

Autonomy and informed consent [2, 11, 35]

Privacy and confidentiality [26, 35]

Intrusiveness of intervention [22]

Acceptability considerations

Perceived priority of the problem

Public perception of disease burden, disease risk or severity [15, 34, 35]

Acceptability

Acceptability in general [2, 4, 6, 14, 15, 24, 26, 27, 34–36]

Acceptability of cost and financial outcomes [2, 14, 25, 26, 34]

Acceptability by beneficiaries

Acceptability by beneficiaries: in general [2, 6, 8, 11, 15, 16, 26, 35]

Comfort, convenience and user experience [2, 11, 12, 14, 15, 20, 24–26, 31, 33, 35]

Acceptability by those providing intervention

Acceptability by those providing intervention [15, 16, 34, 35]

Social and cultural acceptability

Social and cultural acceptability [2, 4, 8, 9, 11, 15, 18, 21, 22, 25, 26, 31–34, 36]

Ethical/moral acceptability [2, 6, 11, 12, 15, 18, 22, 25, 26, 32–34, 36]

Stakeholder demand, interests and pressures

Advocacy and stakeholder (in general) interests and pressures [1, 2, 6, 14, 15, 21, 25, 33, 36]

Demands, interest and pressures of the public [1, 2, 8, 14, 24–26, 36]

Demands, interest and pressures by industry [2]

Pressures, demand and interest of beneficiaries and patient representatives [2, 17, 20, 22, 25, 32, 33]

Pressures, demand and interest of those providing intervention [2, 17, 20, 25, 26, 32, 33]

Media attention and coverage [1, 32]

Societal considerations

Societal needs and priority

Social burden of disease (individual/population) [8, 18, 21, 29, 34, 35]

Social needs [24]

Economic burden of disease on society (in general) [32]

Social and societal impact

Social impact or benefits [2, 3, 5, 6, 14, 21, 23–26, 31, 33, 35]

Impact on non-health outcomes (in general) [15, 18, 26, 34]

Impact on poverty [1, 2, 4, 14, 25, 26, 36]

Relevance to social development of the country [10]

Value of hope [11]

Raise profile of condition [14]

Impact on economy

Impact on economy [2, 25, 26]

Impact on productivity and population in productive age [2, 12, 16, 25]

Relevance to economic development of the country

Innovativeness (potential to encourage innovation) [2, 9, 10, 14, 21, 23–25, 31]

Environmental impact

Environmental and/or ecological impact of intervention (in general) [14, 16, 21, 29–31, 33]

Impact on future generations

Impact on future generations [19]

Considerations of equity, equality and fairness

Equity and equality

Equity/equality considerations:

in general [1–4, 9–11, 14, 15, 18, 19, 22–24, 26, 27, 29, 33–36]

Fairness considerations:

in general [2, 3, 6, 11, 16, 26, 36]

Impact on (health) (in-)equity/(in-)equality [2, 6, 10, 14, 26, 31, 34, 35]

Distribution of benefits and harms [10, 26]

Accessibility

Accessibility in general [2–4, 6, 9, 14–17, 19, 21, 24, 29, 31, 33–35]

Equity in accessibility [1, 2, 4, 6, 7, 11, 25, 26, 35]

Physical and spacial accessibility [2, 4, 6, 16, 26]

Timeliness of access (time spent waiting for treatment) [2, 3, 22, 36]

Informational accessibility [11]

Financial accessibility of intervention/Affordability [4, 9, 10, 12, 14, 19, 25, 26, 34–36]

Affordability: risk of catastrophic health expenditure [11, 16, 25]

Affordability: cost and Financial impact on beneficiaries [11, 13, 14, 16, 22, 25, 26, 31, 34, 35]

Availability

Availability/lack of suitable alternatives [1, 2, 5, 11, 12, 14, 17, 19–25, 28, 32, 34–36]

Limitations of alternative interventions [2, 11, 14, 15, 25, 26, 34, 35]

Rare diseases/orphan disease [12, 14, 16, 21, 24, 28]

Unmet needs [11, 12, 16, 21, 24, 29]

Responsibility

Ability to reduce own health risk and conditions arising from patient behavior [4, 5, 11, 16, 24, 25]

Non-discrimination

Non-discrimination [11, 23, 24]

Consideration regarding specific populations

Consideration of high-risk populations [2, 11, 14, 26]

Consideration of vulnerable populations [1, 2, 4, 11, 21, 25, 26]

Consideration of Socio-economic status [4, 16, 25, 26, 36]

Consideration of sex, gender, and/or sexual orientation [2, 6, 16, 25, 26, 34]

Consideration of race/ethnicity [25, 36]

Consideration of care giver responsibilities [29]

Consideration of age-groups [1, 2, 4, 11, 14, 16, 26, 28]

Consideration of place of living [11, 16, 25]

Consideration of identity and ideology [25]

Other group related considerations [11]

Cost and financial considerations

Financial burden of disease

Financial burden of disease or current intervention on health system [2, 15, 16, 25, 32, 34, 35]

Cost and budget impact of intervention

Cost/budget impact:

in general [1, 2, 4–6, 8, 9, 11–19, 21–23, 25–31, 33–36]

Cost per unit/usage [1, 2, 5, 11, 15, 17, 20, 23–26, 33, 34, 36]

Cost over time [20, 21, 25, 32, 34]

Long term cost/budget impact [25]

Overall cost/budget impact [1, 2, 11, 22, 26, 36]

Direct cost [11, 25, 32, 34, 35]

Indirect/additional/hidden cost [11, 34–36]

Marginal cost [11, 25]

Opportunity cost [2, 11, 21, 31]

Impact on other spending/investments [2, 6, 14, 25]

Investment/start-up cost [2, 22, 30]

Operating cost [2, 30]

Lifecycle cost of intervention/technology [30]

Economic/financial benefits and cost-minimization potential [1, 3, 8, 11, 13, 22, 25, 27, 28, 34–36]

Cost to/budget impact on government or society [1, 2, 5, 18, 25, 26, 29, 33, 36]

Relation of cost and benefits

Relation of cost and benefits [1–7, 9–16, 18–26, 28, 31–36]

Financial context

Appropriateness [2, 14]

Financial feasibility

Affordability to health system

Availability or lack of funds/funding [1–4, 6, 10–12, 14, 15, 21–25, 31, 34, 36]

Financial sustainability

Financial sustainability of intervention and consistency of funding [4, 11, 15, 31, 34]

Feasibility and health system considerations

Health-system related needs and priority

Burden of disease on health system [2, 25, 26, 31, 33, 34]

Needs of the health-system

Feasibility and capacity to implement

General feasibility considerations [2–4, 7, 9, 10, 14–16, 22–27, 31, 33, 34, 36]

Technical feasibility considerations [6, 26, 33]

Practical feasibility considerations [4, 6, 26]

Capacity to implement [1, 2, 6, 14, 21, 25, 29]

Considerations of management and organization of health system

Availability of, capacity of and need for management and organizational structure [1, 2, 16, 22, 26, 34]

Impact on management and organizational structure [2, 8, 11, 25]

Logistical considerations [2, 15, 34]

Availability of, capacity of and need for monitoring, surveillance and information system [34, 35]

Resource considerations

Availability of, capacity of and need for resources (in general) [1, 2, 10, 14, 30, 36]

Impact on resources (in general) [2, 22–24]

Efficiency of resource use [18, 24, 31]

Considerations of human resources and their skills

Availability of, capacity of and need for human resources [2, 10, 13, 14, 18, 22, 26, 30]

Availability of, capacity of and need for skill levels/knowledge of human resources [2, 3, 14, 16, 22, 25, 30, 35]

Impact on human resources and skill levels [2, 14, 18, 25, 31]

Considerations of Non-financial physical resources (equipment, infrastructure)

Capacity of, availability of need for of physical resources and infrastructure [16, 18, 22, 25, 26, 34–36]

Impact on non-financial physical resources and infrastructure [18]

Interaction with and impact on health system

Impact on performance of health system

and impact on other services [1, 2, 4, 6, 8, 10, 11, 14, 16, 18, 20, 22, 26, 31, 33–35]

Interaction and compatibility with other health system components [1–3, 6, 14, 22, 25, 33–35]

Ease of use, application and burden of intervention [13, 15, 22, 26, 35]

Appropriateness within health system

Appropriateness (in general) [2, 3, 11, 21, 25, 26, 31, 36]

Appropriateness of intervention for specific context [2, 16, 30]

Legislative and regulatory considerations

Adherence to legal requirements, constrains and implications [2, 8, 10, 15, 16, 18, 25, 26, 31–35]

Adherence to other directives, standards and requirements [2, 6, 14, 22, 30, 34]

Political considerations

Political acceptability, interests and pressures [1–4, 6, 10, 11, 14, 18, 21, 25, 26, 33–36]

Donor and global interests and pressures [2, 3]

Political impact [2, 25, 33, 35]

Alignment with priorities [1–4, 6, 13–15, 21, 23, 24, 26, 29, 30, 34, 35]

Mission, mandate and goals of health system [2, 13, 14, 21, 25]

Strategic considerations

Strategic planning and considerations [1, 2, 13, 19, 25, 31]

Existing cooperation [2, 10, 14, 22]

Decisions and practice of other institutions or stakeholders [2, 11, 17, 24, 29, 34]

Alignment with recommendations, guidelines and standards [1–3, 7, 11, 14, 16, 21, 24, 26, 29, 34]

Historical context and past decisions [1–3, 11, 13, 14, 17, 19–22, 25]

Availability of incentives [1, 2]

Impact on future decisions [6, 14]

Keeping promises and commitments [2, 13]

Characteristic of intervention

(technical) Complexity of intervention [2, 4, 6, 16, 21]

Scalability of intervention [9, 27]

Ability to evaluate intervention [2, 33–35]

Reversibility of intervention [2]

Flexibility of implementation [2]

Uniqueness [21]

Frequency of use and expected level of usage/activity [1, 2, 6, 8, 14, 25, 26, 32]

Dependence on maintenance [26, 30]

Sustainability of intervention utilization

(e.g. Supply of parts, vaccines) [2, 14, 15, 27, 29, 34]

Position of intervention in care pathway [5, 12, 20, 24]

Additional uses of intervention [5, 21, 23–25]