A number of challenges arise when applying the simple HsW model described above. First, a lack of objective evidence on effectiveness for the full range of intervention options (existing and potential) across settings, sub-populations and the care continuum may leave a potentially large number of interventions excluded from explicit performance measurement. Secondly, converting clinical and other outcomes into a common unit of benefit to support comparison of interventions across disease stages and across diseases is problematic due to data gaps and unresolved methodological questions. Thirdly, decision-makers frequently prefer to leave political, ethical or distributional objectives implicit rather than support explicit decision-rules based on objective evidence.
The extent to which these challenges impede implementation of the HsW model has been explored in applications to osteoarthritis , Type 2 Diabetes [10, 11], hypertension and harmful life style behaviours [12, 13]. These applications have shown that the HsW model is not distinguished from other priority setting models by its ability to overcome these problems. Rather, it is the sort of questions that the HsW model emphasises that distinguishes it from priority setting models that are predominantly concerned with the narrower question of allocation within a program budget. The disease/health problem framework of the HsW model supports explicit consideration of allocation and reallocation of funding across all mutually exclusive and complementary interventions for the prevention and treatment of a disease/health problem under consideration. The published applications demonstrate the robustness of the framework in this task and in developing recommended resource shifts across disparate modalities and across disease stages, which, if followed, should contribute to social objectives and that application across the entire health sector is feasible. (Recommendations have covered for instance, exercise, patient education, surgery, complementary medicines, prescription medicines, public health campaigns, primary care advice etc.).
That said, the translation of recommendations to actual resource shifts is dependent on the incentives in the health system and in particular whether funding arrangements are compatible with the health sector wide context and disease/health problem framework. The most compatible funding framework is a global budget for the (sub) population, managed by a single fundholder. The alignment of budgetary silos with the disease/health problem framework might also be successful. A mismatch between the funding framework and the priority setting framework requires a mechanism for shifting resources between budgets. This could plausibly be achieved by permitting one fundholder to 'buy-out' the budget of another fundholder provided suitable accountability mechanisms are in place. More generally, the very funding models that give rise to the observed distortions in the health service mix discourages any redirection of resources to address them. Whilst the HsW model is not distinguished from other priority setting models by its ability to overcome this problem, it does identify the cost of retaining funding silos. Setting priorities at the level of the program budget will typically fail to identify this major source of inefficiency.
In relation to consideration of interventions for which the evidence is extremely poor, the solution in the medium to longer term has to lie in the gathering of additional evidence. The alternatives are to adopt the status quo – to continue to fund unproven interventions, which in effect means adding to but not deleting interventions from the formulary (requiring an ever expanding program budget), or rely on the black-box of committee or working-group decision-making, allowing decision-maker values or expert opinion to determine the final list of priorities. Neither alternative is likely to promote efficiency. In the short term, a lack of data should not be allowed to paralyze the decision-making process. The HsW model avoids decision paralysis by retaining all competing and complementary interventions of interest in the choice set but differentiating interventions according to the level of supporting evidence. The extent of reliance on opinion and the sway of political imperatives in modifying the set of priorities is therefore made quite explicit when implementing the HsW model. The value of the HsW model is to identify data gaps, highlight the need for objective evidence and avoid any confusion between opinion-base and evidence-based decision-making.
Comparison with other priority setting models
The HsW model has elements in common with other economic-based approaches such as QALY League Tables [2, 14], PBMA [1, 15–17] and generalised cost-effectiveness analysis [18, 19]. A full description and critique of all competing priority setting models is beyond the scope of the current paper but the key features of three alternative economic models, together with a brief consideration of their strengths and weakness relative to the HsW model, are outlined below. See also the critique by Coast and colleagues , the report by Segal and Chen  and the discussion paper by Hauck and colleagues .
Typically in health economic evaluation a small number of interventions are compared, often drawing on the results of a single clinical trial or a meta-analysis of similar trials. The process for the listing of pharmaceuticals on government formularies, such as the Australian Pharmaceutical Benefits Schedule (PBS) is a well-known example of a restricted approach to priority setting. In order for drugs to be listed on the PBS (a mechanism to subsidise pharmaceuticals) pharmaceutical companies must submit an economic evaluation to support listing. Evaluations are prepared according to a set of published guidelines  and submissions are subject to independent review. Other treatment modalities are ineligible for listing.
The process ensures consistency in methods and high standard of analysis. But the guidelines define the main comparator narrowly, as another drug, normally of the same pharmacological analogue – if such a drug is listed on the PBS, or otherwise of the same therapeutic class. Comparison with 'standard medical management' occurs only on the rare occasion when there is no suitable comparator listed on the PBS. This restriction on scope means that drug treatment is rarely compared with alternative modalities and never with modalities that do not represent usual care. In the context of an open ended pharmaceutical budget (where supply responds without limit to demand, as applies in Australia), this process supports the increasing use of pharmaceuticals relative to other modalities. This will be the result unless the performance threshold, in terms of incremental cost-effectiveness ratio, that supports listing is set to reflect decision making elsewhere in the health sector. But it is more likely that decisions on pharmaceuticals will be made independently, without regard to the incremental cost-effectiveness ratios of competing modalities that could be obtained through a broader health sector wide approach to priority setting. In the absence of that broader context, limited cost-effectiveness comparisons are at clear risk of entrenching resource silos within program boundaries and modalities.
QALY league tables
QALY league tables in contrast attempt to be comprehensive, with the aim of comparing all current and possible health interventions in a single priority setting exercise. The most commonly cited example of QALY League Tables in action is the Oregon Experiment of the early 1990s – an attempt to identify a set of health services to be funded under the US Medicaid program for the people of Oregon . The research program and subsequent recommendations ran into difficulties, due to lack of confidence in the cost and QALY estimates, but also a fundamental mistrust of the allocation of resources on the basis of cost/QALY. The difficulty posed by the sheer scope of the exercise is substantial, especially if the analysis is to be truly at the margin and to consider all possible target populations and intervention options .
Of the four alternative economic models considered here, QALY league tables share the greatest commonality with the HsW model. It is therefore worth highlighting exactly how the HsW model addresses some of the difficulties experienced when using QALY league tables for priority setting. The HsW model addresses issues related to the lack of confidence in cost/QALY estimates by permitting the measure of benefit to capture ethical or distributional concerns, by a reliance on objective evidence with respect to all arguments in the objective function for interventions included in performance measurement tasks, and by making explicit the lack of evidence associated with interventions excluded from the performance measurement but included in the priority setting exercise. The HsW model addresses the vastness in scope and size of task, by dividing the priority setting exercise into a succession of manageable research activities, covering all mutually exclusive and complementary interventions for the prevention and treatment of a single disease/health problem. That is, by adopting the disease/health problem framework described above.
Generalised Cost-Effectiveness (G-CE) analysis
was developed to resolve issues with regards the transferability and policy applicability of cost-effectiveness analysis (CEA) that might limit or preclude its use and re-use for sector-wide priority setting [18, 19]. More specifically, Murray and colleagues  provide a set of guidelines for G-CEA with two potential situations in mind: (i) "CEA of a wide-range of interventions ...to inform a specific decision-maker facing a known budget constraint, a set of options for using the budget and a series of other (ethical or political) constraints" (p237), and (ii) "CEA of a wide-range of interventions ...to provide general information on the relative costs and health benefits of different technologies that are meant to contribute through multiple channels to a more informed debate on resource allocation priorities" (p237). Given the alignment of budgetary silos with the disease/health problem framework or the existence of a global budget managed by a single fundholder for the (sub) population under study, the application of the HsW model would be equivalent to the first situation. However, in the presence of a mismatch between the funding framework and the priority setting framework, the HsW model would have more in common with the second situation – informing debate by identifying the cost of retaining budgetary silos and a narrow focus on resource shifts within a program budget. In either case, addressing the issues raised in the development of G-CEA might also be considered a pre-requisite for the use and re-use of the HsW model.
Several of these issues have been discussed above including: the availability of a suitable composite outcome measure that can support comparisons across disparate interventions and identifying political constraints on the set of possible resource shifts. However, the pivotal issue raised in development of the G-CEA model is the limited transferability of cost-effectiveness ratios in the presence of a complex series of dependencies between the costs and effects of related interventions. If cost-effectiveness is calculated relative to a comparator that is available in one population but not in another, then the resulting cost-effectiveness ratio will be applicable and policy-relevant in only one of those populations. In principle, evaluating sets of related interventions with respect to the 'null set' of related interventions (rather than with respect to the current mix of available interventions) would yield cost-effectiveness ratios that are independent of any such dependencies between related interventions.
After evaluating related interventions with respect to the null set of related interventions, the G-CEA model requires that independent sets of mutually exclusive interventions be ordered in increasing order of cost per unit of outcome, before applying standard decision rules for constrained maximisation. The G-CEA model can therefore be viewed as an adaptation of the QALY League Table approach, designed to enhance the transferability of results from one population to another. The G-CEA model de-contextualises CEA analyses, eschewing context-specific distributional concerns and taking only limited account of political constraints such that enhanced transferability may come at the price of policy-relevance. Decision-makers attempting to derive a context-specific set of priorities would then have to perform the usual adjustments to the price and quantity of inputs and to the coverage, efficacy and adherence of interventions , as well as conducting multiple indirect comparisons (when the null set is not politically feasible in the relevant context and to recover incremental comparisons against current practice) and re-weighting outcomes to reflect context-specific distributional concerns [22, 23].
Program Budgeting and Marginal Analysis (PBMA)
is a relatively common approach to priority setting in the health sector, developed in the context of health agency decision making. The key features are; i) the establishment of a Working Group (from the health organisation and possibly other constituencies) to determine program objectives and to identify a set of services to be expanded and another set to be contracted, based on consideration of possible benefits (gained or lost) from service expansion or contraction, ii) estimation of budgets for sub-programs, and iii) calculation of cost-effectiveness ratios for each intervention in the expansion and contraction lists (optional).
The strength of this approach derives from the engagement of the key players in the task through the primary role of the Working Group, arguably increasing the likelihood that recommendations will be implemented. It is also possible to complete such an exercise at relatively low cost and with few data inputs. The key limitation of PBMA relates to the subjectivity of the process and consequent lack of confidence in the rankings and specifically in the expansion and contraction set identified. Peacock and colleagues , for example, report only a weak relationship between recommended Panel rankings and cost-effectiveness estimates prepared by the research team (even when measuring effectiveness to reflect objectives developed by the Panel). While it is possible that cost-effectiveness ratios do not adequately capture the range of social objectives that influencing rankings, it is also possible that rankings that rely heavily on subjective assessments will fail to reflect arguments in the social welfare function and will instead reflect decision-maker values. The latter is a particular concern given that the process has few requirements around the quality of the evidence-base and may be vulnerable to capture by vested interest.
Partly to address the limitations of a traditional PBMA analysis, Carter and colleagues have emphasised the importance of incorporating objective evidence into priority setting, whilst maintaining a core role for the Working Group. This work has progressed under the acronym of ACE: Assessment of Cost-Effectiveness (24, 25). Under ACE the incremental cost per DALY is adopted as the primary measure of performance, but 'second stage filters' such as 'acceptability', 'implementability', budget impact and equity are considered through a subjective process, possibly changing recommendations. The Working Group selects the intervention options and assists in the application of the 'second stage filters'. The research team prepares the cost/DALY estimates based on the clinical trial literature supported by economic and epidemiological modelling.
The ACE model has much in common with the HsW model, notably the disease focus and the greater reliance on objective evidence. Where it differs from the HsW model is in the role of the Working Group in selecting intervention options, rather than the use of objective criteria for this purpose, the adoption of the DALY as the primary measure of outcome, and the role for 'second stage filters' in the ranking of interventions to account for distributional concerns and to bring selected implementation issues into the priority setting exercise. While the ACE model does not allow the primary measure of outcome to be adjusted to account for non-health dimensions of benefit, the use of second-stage filters allows for other issues, including barriers to implementation that might modify the list of priorities to be made explicit. The selection of interventions by the Working Group remains as a source of inefficiency.
To date, the majority of PBMA exercises have set priorities at the level of relatively narrow program budgets that rarely include all mutually exclusive and complementary interventions for the relevant (sub) population . Recently, the term macro marginal analysis (MMA) has been coined to describe the application of PBMA methods across disease-stages, disease-areas, modalities and program budgets administered by major integrated health care organisations . The rationale for the development of MMA PBMA is much the same as for the development of the HsW model – to eliminate artificial constraints that entrench allocative inefficiency. However, the choice set is delimited firstly, by the ambit of the budget-holder (a constraint not present in the HsW model) and secondly, by the expansion and contraction lists compiled by the Working Group. It is worth noting that reluctance among program managers to nominate interventions for contraction has been identified as a key challenge in the implementation of MMA. Mitton and colleagues  argue that a recent implementation of MMA to identify desirable resource shifts for one regional health authority (Alberta, Canada) confirms that it is possible for a PBMA working group to pragmatically compare re-allocation options at the level of a regional health authority and achieve resource release of a significant magnitude without formal measurement of health benefits and without attaching explicit weights to competing objectives. While a comparison of re-allocation options without measurement and without a well-defined objective function is certainly possible, the more pertinent question would seem to be whether or not the resulting trade-offs are likely to leave people better-off or worse-off? The fact that resource shifts are acceptable to the Working Group provides little in the way of reassurance.