Skip to main content
Download PDF
Download ePub

Decision uncertainty and value of further research: a case-study in fenestrated endovascular aneurysm repair for complex abdominal aortic aneurysms

Cost Effectiveness and Resource Allocation volume 16, Article number: 15 (2018) Cite this article

Abstract

Background

Fenestrated endovascular aneurysm repair (fEVAR) is a new approach for complex abdominal aortic aneurysms, limited to a few specialist centers, with limited evidence base. We developed a cost-effectiveness decision model of fEVAR compared to open surgical repair (OSR) to investigate the likely direction of costs and benefits and inform further research projects on this technology.

Methods

A systematic review with meta-analysis and a four-state Markov model were used to estimate the cost-effectiveness of fEVAR versus OSR. We used a recent coverage with evidence development framework to characterize the main sources of uncertainty and inform decisions about the type of further research that would be most worthwhile and feasible.

Results

Seven observational comparative studies were identified, of which four presented odds ratios adjusted for confounders. The odds ratios for operative mortality varied widely between studies. Assuming a central estimate of the odds ratio of 0.54 (95% CI 0.05–6.24), the decision model estimated that the incremental cost per quality adjusted life year (QALY) was £74,580/QALY with a probability of 9 and 16% of being cost-effective at standard cost-effectiveness thresholds of £20,000/QALY and £30,000/QALY, respectively. The Expected Value of Perfect Information over 10 years at a threshold of £20,000/QALY was £11.2 million. Operative mortality contributed to most of the uncertainty in the decision model.

Conclusions

In the case of “maturing technologies”, decision modelling indicates the likely direction of costs and benefits and guides the development of further research projects. In our analysis of fEVAR versus OSR, decision uncertainty, particularly around operative mortality, might be effectively resolved by a short-term RCT, or possibly a well-conducted comparative observational study. Decision makers may consider that a conditional coverage decision is warranted with assessments required to make this type of recommendation depending on local priorities and circumstances.

Background

Health technology assessment (HTA) agencies are charged with providing guidance to practitioners and patients based on a systematic and comprehensive assessment of the clinical and cost-effectiveness profile of health technologies. Assessing medical devices (MDs), in particular, raises several challenges which require careful consideration [1]. For a range of reasons, evidence on costs and effects relating to medical devices may be limited [2]. This situation, usually but not exclusively associated to MD evaluation, leads to a dilemma that typically has proved challenging for HTA and policy makers. Making an early decision based on poor evidence carries a high risk of error that may be difficult to reverse, but delaying a decision until more evidence becomes available may leave current patients without effective treatment or health systems with inefficient technology.

Fenestrated endovascular repair fEVAR is an example of such dilemma. Fenestrated EVAR might be considered a maturing innovation in the typology of Sculpher et al. [3], that is a technology early in its life cycle, usually limited to a few specialist centers, with an evidence base comprising case series and small RCTs. Decision modelling in these situations is unlikely to provide a definitive statement about whether a technology is cost-effective, but rather indicate the likely direction of costs and benefits, the circumstances under which the new technology might represent good value for money, and to guide the development of further research projects [3]. The aim of the study is to illustrate this approach through the clinical and cost-effectiveness evaluation of fEVAR compared to open surgical repair and to estimate  the value of further research for the same decision problem.

Abdominal aortic aneurysms (AAAs) occur when the main artery in the body develops a balloon-like bulge of diameter exceeding 3 cm. The prevalence of AAAs in the UK has been estimated at between 1.2 and 7.6% in those over 50 years [4]. The benefits of endovascular aneurysm repair (EVAR) compared to open surgical repair (OSR) of AAAs are well established by large multicenter randomized trials assessing all-cause or aneurysm related mortality and postoperative complications [5,6,7]. However, about 15% of AAAs are considered “complex” (cAAAs), i.e. aneurysms involving renal or visceral arteries or ‘juxta-renal AAAs’ [8]. Surgical treatment options for cAAAs include OSR, usually requiring suprarenal aortic cross-clamping, or fEVAR [8,9,10]. In fEVAR the stent-graft fabric extends over the renal arteries, but perfusion to these arteries is preserved via accurately placed windows (fenestrations) within the stent-graft fabric. The fEVAR procedure is more challenging than standard EVAR as graft positioning requires both longitudinal and rotational alignment of the fenestrations with the target vessels [11].

Although a small number of systematic reviews have sought to compare efficacy and safety outcomes of fEVAR compared to OSR, their conclusions have been limited by lack of available robust evidence [12,13,14]. In particular, these reviews noted that there were no randomised controlled trials (RCTs), but there were several registries and case series, some of which were large and long-running.

This study evaluates the effectiveness, safety and cost-effectiveness of fEVAR versus OSR for elective repair of cAAA in patients suitable for either procedure. Another solution for more complicated aneurysms is branched EVAR (bEVAR). The term ‘branched’ refers to the need to bridge the gap, created by increased diameter of aorta, between the main body of aortic stent graft and target vessels and not to any actual branch from the graft itself [11]. Based on the distinction between bEVAR and fEVAR in accordance with the Society of Vascular Surgery reporting standards on thoracic endovascular aortic repair [10], branched cAAAs endovascular procedures were excluded from this analysis.

Methods

Systematic review and meta-analysis

This updated systematic review was conducted and reported according to the Meta-analysis of Observational Studies in Epidemiology (MOOSE) reporting guidelines [17].

Data sources and search strategy

MEDLINE (via Ovid) was searched for previous systematic reviews or meta-analyses of fEVAR for the treatment of cAAAs. We also sought new primary studies published since the end date of the literature searches of the most recently undertaken systematic review (i.e. October 2013) [18] and up to 14th January 2015. A copy of the search strategy is available as an Additional file 1: Appendix S1. As this recent review found no RCTs, this updated study also includes observational evidence. Two reviewers (OC, RST) independently examined titles and abstracts of primary studies.

Study selection

Relevant articles were obtained in full, and assessed against the inclusion/exclusion criteria described in Additional file 1: Table S1. Any disagreement was resolved through discussion.

Data extraction and risk of bias assessment

Details were extracted for each study including: year and country of publication, sample size, age and gender distribution, duration of follow up, anatomical location of aneurysms, and details of implant used. Outcomes of interest were: operative mortality, late mortality, complications, re-interventions, and resource use during the primary admission (i.e. length of stay, operative time, blood, and intensive care). Study quality was assessed using a modified list of criteria developed for case series [19] and the addition of three criteria for comparative studies. Data extraction and risk of bias assessment was undertaken by one author (OC) and verified by another (RST).

Data analysis

Where outcome data were consistently reported, results were pooled across studies using Der Simonian and Laird random-effects meta-analyses [20]. For operative mortality, we considered that treatment effects should only be pooled where authors made statistical adjustment for relevant confounding factors. For resource use, we considered that unadjusted differences were adequate as these variables are relatively less influenced by factors such as body mass index, leukocyte count or other biomarkers. Moreover, no studies reported between-groups adjusted comparison of these measures. Recommendations from the Cochrane Handbook for systematic reviews of interventions were followed to deal with missing data [20]. Heterogeneity was quantified using the I2 statistic and results displayed in a forest plot. All statistical analyses were performed using STATA® 13.1 StataCorp US.

Cost-effectiveness model

This economic evaluation was undertaken and reported according to the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement [21].

Model structure

A decision analytic model was developed in Excel®2012 (Microsoft Corporation, US) to estimate the lifetime cost and QALY of each procedure. The structure of the model was adapted from a previously published economic evaluation of conventional EVAR for AAA repair, using essentially the same structure, with parameters estimated from the systematic review [22]. The model consists of a lead-in period lasting 6 months, during which the cAAA repair is undertaken with fEVAR or OSR (Fig. 1). Perioperative survivors then enter a long-term three state Markov model, which extrapolates costs and outcomes over a lifetime horizon using 6-monthly cycles. During each cycle, patients may die of cAAA-related causes, such as rupture of the aneurysm, or die of other causes. Patients undergo surveillance with an outpatient visit and ultrasound every year after endovascular repair and every 5 years after open repair [23]. Survivors have an ongoing risk of re-intervention, e.g. to correct complications of the device such as endoleaks which might be detected by surveillance. Patients can have only one re-intervention per cycle, but can have more than one re-intervention over their lifetime. The risk of re-intervention depends on initial procedure (OSR or fEVAR) and time from initial cAAA repair, but is assumed independent between cycles. Re-interventions reduce health-related quality of life (HRQOL) and incur a hospital cost. Mortality risks related to re-intervention are captured in the estimate of aneurysm-related deaths.

Fig. 1
figure 1

Decision analytic model structure. cAAA complex abdominal aortic aneurysms. Patients are at risk of re-intervention in each cycle, with hospital cost and diminished health-related quality of life. Includes deaths related to aneurysm rupture, complications of the grafts, and re-interventions

Full size image

Perspective of the decision model and inputs required

Health outcomes were expressed in terms of quality-adjusted life years (QALYs). The perspective adopted was that of a publicly funded health system using an explicit cost-effectiveness threshold for decision making. The UK National Health Service (NHS) [24] is used here as an example, using thresholds in the range £20,000-£30,000 per QALY. The price year was 2016/17, and costs and benefits were discounted at an annual rate of 3.5% [24].

Table 1 summarises the main input variables required by the decision model and their distributions used to represent uncertainty. Parameter estimates for the model were obtained from the updated systematic literature review, personal communication with the device manufacturer, and routine UK National Health Service (NHS) sources. Where specific estimates for fEVAR were unavailable, the literature on conventional EVAR was considered. The mean age at which cAAAs repair is undertaken was 73 years [25]. Mortality risks for non-AAA cause deaths were based on UK male population life tables [26].

Table 1 Mortality, re-intervention and HRQoL in the cost-effectiveness decision model
Full size table

Expected value of perfect information

Fenestrated EVAR is a relatively novel technology and the evidence relating to its costs and benefits is limited. In this regard, further research might be worthwhile if it reduces uncertainty and avoids the consequences of making a wrong decision about the use of the intervention. The expected value of perfect information (EVPI) estimates an upper bound for the value of a new study, if it could eliminate all uncertainty about which treatment was more cost-effective at the given cost-per-QALY threshold. Partial EVPI (EVPPI) is also calculated using web-based emulator software [27]. EVPPI enables identification of those parameters that contribute particularly highly to decision uncertainty.

A decision timeframe of 10 years is used, to allow for obsolescence and product innovation by competing technologies. We assume that 1782 cAAA patients would be eligible for repair each year in the UK, based on German health insurance data [25] adjusted for UK population size. Net population benefit was calculated at cost-effectiveness thresholds used in the UK.

Decision uncertainty was estimated through probabilistic sensitivity analysis using 1000 Monte-Carlo simulations of the model [28].

Decision making and the value of further research

We used a recently developed framework to characterize the sources of uncertainty and to identify the most appropriate decision option for fEVAR given the available evidence [15, 29]. Four decision options were considered: accept fEVAR for immediate and general use in the national heath service in indicated patients (accept fEVAR), reject fEVAR (i.e. accept OSR), accept fEVAR provisionally but conditioned upon the collection of further evidence (accept with research, AWR), or reject fEVAR for general use until further evidence becomes available (only in research, OIR).

Results

Systematic review and meta-analysis

The updated systematic review study selection process is illustrated in Fig. 2. A total of 22 studies involving 1641 fEVAR patients were included (see Additional file 1: Appendix S2 for list of included studies). Whilst we did not find any randomised trials comparing fEVAR and OSR, we did include seven comparative observational studies, reporting outcomes for both fEVAR and OSR patients [30,31,32,33,34,35,36].

Fig. 2
figure 2

Flow diagram—study selection process

Full size image

Study characteristics

A summary of the study and patient population characteristics and the interventions is reported in Additional file 1: Table S2. Study sample sizes were small, with a median of 78 fEVAR treated patients, a median patient age of 74 years and a median follow-up of 18.4 months. All studies showed high to moderate risk of bias as reported in Additional file 1: Table S3.

Relative risk of operative mortality

Seven observational studies compared perioperative [30, 36] or 30-day [31,32,33,34,35] mortality between fEVAR and OSR (Additional file 1: Table S4). The pooled odds ratio for mortality across all seven studies was 0.60 (95% CI 0.23–1.60, p = 0.014) (Additional file 1: Figure S1). However, there was a high degree of heterogeneity between studies (I2 statistic: 62.6%, p = 0.014).

Risk of bias for estimate of relative risk of operative mortality

In a context of poor available evidence, it is even more important to explore the heterogeneity and evaluate the risk of bias in order to reliably use the information for decision-making purposes. One source of heterogeneity was the method used by the study to account for possible confounding. Three studies did not make any adjustment for confounding variables [31,32,33]. Of the four that took account of confounding variables in some way, one used multivariate regression [35], one used propensity score matching [34], and two used a published risk score [37] to estimate what operative mortality hypothetically would have been in the fEVAR patients had they had undergone open repair [30, 36]. More on the risk of bias assessment performed for estimate of relative risk of operative mortality is reported in Additional file 1: Appendix S3.

Relative risk of operative mortality stratified by study methodology

To further explore the heterogeneity, we stratified the pooled estimate of effect size according to the methodology in the study employed to adjust for confounders. The Raux study [34] (propensity score matching) found an OR of 5.1 (95% CI 1.1–24). The Tsilimparis study [35] (multivariate regression) found an OR of 0.19 (95% CI 0.04–0.83). The two studies that used the V-POSSUM risk score 26,32 found a pooled OR of 0.37 (95% CI 0.20–0.69). The remaining studies that did not adjust for confounders found a pooled OR of 0.98 (95% CI 0.26–3.7).

Relative risk of operative mortality used as input to the decision model

The pooled estimate of the odds ratio over the four studies that performed some form of risk adjustment is 0.54 (95% CI 0.16–1.88) (Fig. 3). The decision model uses the OR estimate of 0.54 as the base-case (log(OR) = μ = − 0.61, SE(μ) = 0.644). However, this SE from random-effects meta-analysis does not take account of the full distribution of effect sizes across the heterogeneous studies [38]. In order to make predictions about the likely effect size that might occur in a future new study or in the general practice use, we considered the estimate of τ2 from the four adjusted studies in the random-effects meta-analysis [30, 34,35,36] (τ2 = 1.15). This suggests a mean and confidence interval for a prediction for a new study of 0.54 (95% CI 0.05–6.24).

Fig. 3
figure 3

Pooled odds ratio of adjusted operative mortality fEVAR vs OSR

Full size image

Baseline risk of operative mortality used as input to the decision model

Twenty-two studies reported operative mortality after fEVAR, with 43 deaths in 1670 patients (mean rate 2.6%) [30,31,32,33,34,35,36, 39,40,41,42,43,44,45,46,47,48,49,50,51,52,53]. However, there was considerable heterogeneity, with operative mortality varying from 0% to 9.5% (Additional file 1: Table S5). Analysis of UK national hospital episode statistics indicates the mean operative mortality after OSR in clinical practice, as opposed to selected case series in specialist centres, is 14%, with considerable heterogeneity [54]. In the model, we use the mean operative mortality after fEVAR from the systematic review as the probability of AAA death during the first model cycle in the base-case and vary this from 0.5% to 9.5% in sensitivity analyses, assuming the baseline rate and the relative risk are independent.

Late aneurysm-related mortality

One study included in the review compared late aneurysm related mortality with about 1 year of follow-up between fEVAR and OSR [31] and found no late AAA-related deaths in either group.

Thirteen studies [31, 32, 36, 39,40,41,42, 44,45,46, 49, 51, 52] reported late AAA mortality after fEVAR. The overall rate of late AAA mortality was 6 deaths in 1558 patient-years (i.e. an average of 0.004 late AAA deaths per person-year) although 10 studies [36, 39,40,41,42,43,44,45,46,47] reported no late AAA-related deaths (Additional file 1: Table S5). The decision model assumes this 0.004 rate of late AAA mortality after fEVAR from > 6 months to 4 years (as found by this review), but that this rate would increase by 50% between 4 years and 8 years and doubles after 8 years (as found after conventional EVAR by the EVAR-1 trial [22]). Furthermore, the base-case model assumes relative risk of late AAA mortality would be greater after fEVAR compared with OSR (as found by the EVAR-1 trial [22]). Sensitivity analyses explore a scenario where the rate of late AAA mortality does not increase over time and is the same as OSR (as found by Chisci et al. [31] study at 1 year) over the patient’s lifetime.

Early and late re-interventions

Eleven studies reported a total of 81 early re-interventions (in-hospital or 30-day) in 1198 patients after fEVAR, an average probability of 0.07 [30, 31, 34, 39, 42, 43, 47, 50, 52]. Three studies compared the risk of early (in-hospital or 30-day) re-interventions for fEVAR compared to open repair [30, 31, 35]. All found more re-interventions after open repair with a pooled OR across the three trials of 0.42 (95% CI 0.26–0.68) (Additional file 1: Figure S4). These results were not adjusted for case-mix. Eleven studies reported a total of 132 late re-interventions in 1507 patient-years of follow-up after fEVAR, an average rate of 8.8 per 100 patient-years. No trial reported late re-interventions after open repair [32, 36, 41, 47, 48, 51, 52].

As the studies did not provide sufficient information on how the re-interventions rate might change over time, the decision model uses the rates of early re-interventions from the literature review, and the relative risk of late re-intervention (fEVAR vs OSR) from the EVAR-1 RCT [22] that compared EVAR versus open repair for conventional AAA, which was 6.29 (95% CI 3.09–12.78) from 6 months to 4 years, 1.60 (95% CI 0.81–3.15) from 4 to 8 years, and 1.51 (1.71–3.19) after 8 years [55].

Resource consumption

Five studies provided comparative information on one or more healthcare resource use outcomes (Table 2, Table 3) [30,31,32,33, 35]. There was considerable heterogeneity across most of the resource use data. The decision model uses the average estimates of procedure resource use (i.e. length of stay, operating theatre, blood units and Intensive Care Unit stay) costed at UK prices in 2016/17 [55, 56]. The list price of the fEVAR device was £16,500 including extra stent parts and accessories (personal communication with Cook Medical, at 2015 prices). Sensitivity analyses were conducted by varying the price of the stent to reflect possible discounts. A reduction in cost might also be achieved if use of fEVAR was associated with lower length of stay or less use of other hospital resources, or if surveillance and re-interventions could be accurately targeted at patients with higher risk of complications. Other unit costs associated with the index procedure are given in Table 3. The cost of a AAA-related re-intervention was taken as £8670 (SE 831) [55].

Table 2 Resource consumption in fEVAR and OSR patients
Full size table
Table 3 Costs of procedures in the economic model
Full size table

Health-related quality of life

No study reported health related quality of life (HRQOL) after fEVAR or OSR. The EVAR-1 trial found that HRQOL was better after conventional EVAR than OSR for the first 6 months (mean difference in EQ-5D-3L 0.060, SE(0.017)) but there was no difference by 1 year [22]. HRQOL is considerably diminished in the 6 months after a re-intervention and in the 6 months preceding death [57].

Complications

Peri-operative complications were poorly and heterogeneously reported, especially after open repair. The studies did not give a clear indication of the seriousness of the complication or its duration. Therefore complications were not explicitly costed or associated with QALYs in the model. However, some measure of their impact is included in the cost-effectiveness analysis, because serious complications and adverse events during the surgical procedure and hospital stay will be reflected in HRQOL, length of stay, operative theatre time, blood use, and use of ICU.

Cost-effectiveness model

The mean difference in lifetime cost between fEVAR and OSR was £15,606 (95% CI 8390–22,512), and the lifetime difference in QALY was 0.209 (95% CI − 0.124–1.311), (Table 4, Additional file 1: Figure S2). The incremental cost-per-QALY is estimated to be £74,579/QALY with a probability of 0.090 of being cost-effective at a threshold of £20,000 per QALY, and a probability of 0.158 at £30,000 per QALY [24]. Almost all of the additional cost is incurred in the price of the fEVAR device (Additional file 1: Figure S3).

Table 4 Mean costs and QALYs estimated by the model and sensitivity analyses
Full size table

External validation of model results

The predicted overall survival in the model after OSR at 5 years is 58% (Additional file 1: Figure S2), the same as reported from a study linking Hospital Episode Statistics to national mortality records in the UK [54].

Sensitivity analyses

The following univariate sensitivity analyses were conducted:

  1. 1.

    Base-case;

  2. 2.

    High relative risk of operative mortality (OR 5.1, 95% CI 1.10–24) [34];

  3. 3.

    Low relative risk of operative mortality (OR 0.19, 95% CI 0.05–0.83) [35];

  4. 4.

    High baseline risk of operative mortality after fEVAR of 9.5% [34];

  5. 5.

    Low baseline risk of operative mortality after fEVAR of 1% [35];

  6. 6.

    No difference in late (> 30 day) mortality between fEVAR and open repair [31];

  7. 7.

    No difference in late re-interventions between fEVAR and open repair;

  8. 8.

    Price of fEVAR device of £7500.

The sensitivity analyses show the circumstances under which fEVAR plausibly might be cost-effective (Table 4). If the OR of operative mortality were that observed in the most favourable study, assuming that baseline risk remains unchanged, then fEVAR comes close to the £30,000/QALY threshold. fEVAR would also be close to this threshold if the cost of the device were £7500, rather than £16,500.

Expected value of perfect information

The EVPI over 10 years at a threshold of £20,000 per QALY was £11.2 million. EVPPI analysis showed that 92% of this value could be attributed to the two operative mortality parameters (baseline and odds ratio), and the remainder to the cost of the procedure.

The value of further research

Under the base-case model, fEVAR is unlikely to be cost-effective. This does not necessarily mean that it should be rejected, if further evidence might lead to different conclusions. Claxton et al. provide a checklist of the sequence of assessments that would be required for an HTA body considering whether further research should be mandated [58]. We provide a structured discussion of the issues as they might apply to fEVAR given the results of this study. However, each decision maker would need to review and interpret the data as they apply to health services in their jurisdiction.

Is the technology expected to be cost-effective?

This would depend on the decision criteria used (e.g. the cost-per-QALY threshold) as well as the decision-maker’s interpretation of the appropriate inputs for the model. We assume it is likely not cost-effective on current evidence and from the perspective of the NHS in the UK [24].

Are there significant irrecoverable costs?

Implementing FEVAR would not require significant irrecoverable costs in NHS infrastructure, if most of the up-front investment in operating facilities and interventional radiography has already been made. However, on-going training for surgical teams would be required, and the procedure is of course irreversible for individual patients.

Does more research seem worthwhile?

There are no RCTs, there is considerable decision uncertainty from the model, and more research might plausibly reduce that uncertainty. Some assessment is therefore needed of the type of evidence and whether it can be conducted without approval.

Is the research possible with (without) approval?

Generally, most types of research are possible without approval, including RCTs. However, the patient population who might potentially benefit from fEVAR is small and hence population EVPI is only about £11 million at a threshold of £20,000/QALY in the UK. This means a large-scale, expensive, lengthy RCT would not be cost-effective. Nevertheless, in our EVPPI analysis, most of the uncertainty was around operative mortality, which might indicate that a RCT with short follow-up, or even a well-designed observational study would provide sufficient evidence for definitive approval or rejection. These factors would indicate OIR would be an appropriate decision.

Will other sources of uncertainty resolve over time?

A further source of uncertainty, which has not been reflected in our analysis, is the possibility of a learning curve for fEVAR. Fully understanding the learning curve might require the technology to be in widespread use. If this knowledge is important to the decision maker, it would argue in favour of AWR even though the technology is not thought to be cost-effective on current evidence. However, a recent analysis based on over 40,000 patients administrative data from year 2006 to 2013 in Germany using hierarchical regression models showed the learning curve for fEVAR on in-hospital mortality and length of stay is actually negligible [25].

Discussion

In this study we investigated the clinical and cost-effectiveness evaluation of fEVAR compared to open surgical repair as an example of a technology with limited evidence base and diffusion across a few specialized sites where the decision modelling is intended to identify the circumstances under which the new technology might represent a worthwhile investment and to inform future evidence generation projects.

This updated systematic review and meta-analysis found no RCTs that compared fEVAR and OSR. A recent NIHR funded project has updated a similar review up to March 2017 confirming no RCTs is available to date and the most recent comparative FEVAR vs OSR study was Raux et al. 2014 as per our results [59]. Seven observational studies compared these procedures, with varying methodological quality and substantial heterogeneity across all the outcomes.

Decision-makers responsible for setting prices of medical technologies or deciding re-imbursement policy need to make an assessment of whether the technology appears cost-effective on current evidence. In cases where the evidence base is highly uncertain and of poor methodological quality, decision makers may also need to assess what type of further research might be of value.

The usefulness of a decision model in these cases is not to make a definitive statement about cost-effectiveness, but rather to serve as a framework to organise the available data, identify the most important gaps in the evidence, and assess the plausible circumstances under which the technology might be cost-effective. In the fEVAR case, we were reliant on non-randomised comparative studies that are prone to bias and confounding. We assessed the risk of bias and focused our meta-analysis on four comparative studies that were judged to have performed adequate adjustment. These data were used in the model to estimate short-term outcomes. There was little data on late outcomes after fEVAR, particularly late AAA-deaths and complications. Long-term data from conventional endovascular devices clearly shows a greater rate of complications and re-interventions than after open repair, though it is difficult to know how generalizable these data are to fEVAR.

Using these data as base-case inputs, the model estimated that fEVAR is associated with greater survival and QALYs over a patient’s lifetime compared with OSR. However, the high acquisition cost of the device means that the cost of fEVAR considerably exceeds OSR. The EVAR-1 trial [22] showed that the frequency of re-interventions over the medium and long term is also likely to be greater under endovascular repair, though this is not a decisive driver of overall cost in the model. The overall treatment costs make fEVAR not cost-effective at conventional National Institute for Health and Care Excellence (NICE) thresholds, with an ICER exceeding £74,000/QALY. The cost-effectiveness threshold seeks to reflect the magnitude of health that other types of patients forgo when additional costs are incurred on a new technology rather than treatments from which they will benefit [24]. NICE’s threshold range of £20,000 to £30,000 has no empirical link to such opportunity costs, and recent research estimates the threshold to be lower at around £13,000 per QALY [60]. This would suggest that fEVAR is not cost-effective in the UK NHS based on current evidence. The model takes into account resource consumption estimates from the literature review of international studies, hence transferability of the model to other jurisdictions is possible, subject to availability of costs data from other countries. Criteria for decision-making rarely rely on a formal cost-effectiveness threshold outside UK [61], hence each decision maker would need to review and interpret these results in the context of the health systems in place in their jurisdiction. Sensitivity analyses showed that fEVAR would be close to being cost-effective at the upper range of NICE’s threshold if the OR of operative mortality were 0.19, as estimated by one of the studies [35]. fEVAR would also be cost-effective at this threshold if the price of the stent was about £7500 instead of £16,500. High-volume centres may already be able to negotiate discounts with suppliers, although these are commercially confidential.

Expected value of further information analysis showed that the population EVPI is not especially great, because this is a specialist procedure with a limited target population. This would increase if one was willing to accept that research is a global public good that would benefit patients worldwide [62]. Nevertheless, most of the decision uncertainty is around operative mortality. This might be effectively resolved by a short-term RCT, or possibly a well-conducted comparative observational study [59]. Decision makers may consider that a conditional coverage decision is warranted with assessments required to make this type of recommendation depending on local priorities and circumstances.

The model we have used in this paper is very simple, as we have tried to remain faithful to the evidence. There may be several other considerations which would warrant further evidence collection. For example, the model assumes the OR of operative mortality is independent of the base-line risk. In practice there may be sub-group effects. The literature review does not discriminate between male and female patients. There is some evidence from conventional EVAR that outcomes for females are different, given the different anatomical morphology. Over 90% of incident AAA patients between 55 and 84 years age are male, and outcomes and costs in women are insufficiently described in the literature [63]. There are some other important complications arising from cAAA repair, such as renal impairment or spinal cord injury that have not been taken into account in the model [8].

Nevertheless, the situation described in this study is typical of a medical-device procedure whose adoption and diffusion occur without strong clinical evidence supporting its marketing authorization, particularly in Europe. Based on current evidence, our model predicts that at conventional decision-making thresholds in the UK, fEVAR is not cost-effective and more research might plausibly reduce the decision uncertainty from the model. Staged approaches to evidence collection [64] or innovative and structured decision-making processes [16] linking the gathering of additional information with policies on approval for reimbursement around devices-related surgical procedures are recommended.

References

  1. Tarricone R, Torbica A, Drummond M. Challenges in the assessment of medical devices: the MedtecHTA project. Health Econ. 2017;26:5–12.

    Article  PubMed  Google Scholar 

  2. Tarricone R, Torbica A, Drummond M. Key recommendations from the MedtecHTA Project. Health Econ. 2017;26:145–52.

    Article  PubMed  Google Scholar 

  3. Sculpher M, Drummond M, Buxton M. The iterative use of economic evaluation as part of the process of health technology assessment. J Health Serv Res Pol. 1997;2(1):26–30.

    Article  CAS  Google Scholar 

  4. Paravastu SC, Jayarajasingam R, Cottam R, Palfreyman SJ, Michaels JA, Thomas SM. Endovascular repair of abdominal aortic aneurysm. Cochrane Database Syst Rev. 2014;1:004178.

    Google Scholar 

  5. Prinssen M, Verhoeven EL, Buth J, et al. A randomized trial comparing conventional and endovascular repair of abdominal aortic aneurysms. N Engl J Med. 2004;351:1607–18.

    Article  CAS  PubMed  Google Scholar 

  6. EVAR Trial Participants. Endovascular aneurysm repair and outcome in patients unfit for open repair of abdominal aortic aneurysm (EVAR trial 2): randomised controlled trial. Lancet. 2005;365:2187–92.

    Article  Google Scholar 

  7. EVAR Trial Participants. Endovascular aneurysm repair versus open repair in patients with abdominal aortic aneurysm (EVAR trial 1): randomised controlled trial. Lancet. 2005;365:2179–86.

    Article  Google Scholar 

  8. Jongkind V, Yeung KK, Akkersdijk GJ, et al. Juxtarenal aortic aneurysm repair. J Vasc Surg. 2010;52:760–7.

    Article  PubMed  Google Scholar 

  9. Cross J, Richards T. Current practice of aneurysm management in the UK. Ann R Coll Surg Engl. 2014;96:27–31.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Cross J, Raine R, Harris P, Richards T. Indications for fenestrated endovascular aneurysm repair. Br J Surg. 2012;99:217–24.

    Article  CAS  PubMed  Google Scholar 

  11. Greenberg RK, Lytle B. Endovascular repair of thoracoabdominal aneurysms. Circulation. 2008;117:2288–96.

    Article  PubMed  Google Scholar 

  12. Sun Z, Mwipatayi BP, Semmens JB, Lawrence-Brown MM. Short to midterm outcomes of fenestrated endovascular grafts in the treatment of abdominal aortic aneurysms: a systematic review. J Endovasc Ther. 2006;13:747–53.

    Article  PubMed  Google Scholar 

  13. Nordon IM, Hinchliffe RJ, Holt PJ, Loftus IM, Thompson MM. Modern treatment of juxtarenal abdominal aortic aneurysms with fenestrated endografting and open repair–a systematic review. Eur J Vasc Endovasc Surg. 2009;38:35–41.

    Article  CAS  PubMed  Google Scholar 

  14. Di X, Ye W, Liu CW, Jiang J, Han W, Liu B. Fenestrated endovascular repair for pararenal abdominal aortic aneurysms: a systematic review and meta-analysis. Ann Vasc Surg. 2013;27:1190–200.

    Article  PubMed  Google Scholar 

  15. Claxton K, Palmer S, Longworth L, et al. Informing a decision framework for when NICE should recommend the use of health technologies only in the context of an appropriately designed programme of evidence development. Health Technol Assess. 2012;16:1–323.

    Article  CAS  PubMed  Google Scholar 

  16. Rothery C, Claxton K, Palmer S, Epstein D, Tarricone R, Sculpher M. Characterising uncertainty in the assessment of medical devices and determining future research needs. Health Econ. 2017;26:109–23.

    Article  PubMed  Google Scholar 

  17. Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA. 2000;283:2008–12.

    Article  CAS  PubMed  Google Scholar 

  18. Armstrong N, Burgers L, Deshpande S, et al. The use of fenestrated and branched endovascular aneurysm repair for juxtarenal and thoracoabdominal aneurysms: a systematic review and cost-effectiveness analysis. Health Technol Assess. 2014;18:1–66.

    Article  Google Scholar 

  19. Moga C, Guo B, Schopflocher D, Harstall C. Development of a quality appraisal tool for case series studies using a modified Delphi technique. 2012. http://www.ihe.ca/publications/development-of-a-quality-appraisal-tool-for-case-series-studies-using-a-modified-delphi-technique. Accessed January 2015.

  20. Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. The Cochrane Collaboration. 2011. www.handbook.cochrane.org.

  21. Husereau D, Drummond M, Petrou S, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. Value Health. 2013;16:e1–5.

    Article  PubMed  Google Scholar 

  22. Brown LC, Powell JT, Thompson SG, Epstein DM, Sculpher MJ, Greenhalgh RM. The UK EndoVascular Aneurysm Repair (EVAR) trials: randomised trials of EVAR versus standard therapy. Health Technol Assess. 2012;16:1–218.

    Article  CAS  PubMed  Google Scholar 

  23. Moll FL, Powell JT, Fraedrich G, et al. Management of abdominal aortic aneurysms clinical practice guidelines of the European society for vascular surgery. Eur J Vasc Endovasc Surg. 2011;41(Suppl 1):S1–58.

    Article  PubMed  Google Scholar 

  24. National Institute of Health and Care Excellence. Guide to the methods of technology appraisal 2013. 2013. https://www.nice.org.uk/article/pmg9/chapter/Foreword. Accessed Mar 15 2016.

  25. Varabyova Y, Blankart CR, Schreyögg J. The Role of Learning in Health Technology Assessments: An Empirical Assessment of Endovascular Aneurysm Repairs in German Hospitals. Health Econ. 2017;26 Suppl 1:93–108.

    Article  PubMed  Google Scholar 

  26. Office for National Statistics. National life tables: United Kingdom. 2017. https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/lifeexpectancies/datasets/nationallifetablesunitedkingdomreferencetables. Accessed 5 July 2017.

  27. Strong M, Oakley JE, Brennan A. Estimating multiparameter partial expected value of perfect information from a probabilistic sensitivity analysis sample: a nonparametric regression approach. Med Decis Making. 2014;34:311–26.

    Article  PubMed  Google Scholar 

  28. Claxton K, Sculpher M, McCabe C, et al. Probabilistic sensitivity analysis for NICE technology assessment: not an optional extra. Health Econ. 2005;14:339–47.

    Article  PubMed  Google Scholar 

  29. Rothery C, Claxton K, Palmer S, Epstein D, Tarricone R, Sculpher M. Characterising uncertainty in the assessment of medical devices and determining future research needs. In: University of York, 2016.

  30. Canavati R, Millen A, Brennan J, et al. Comparison of fenestrated endovascular and open repair of abdominal aortic aneurysms not suitable for standard endovascular repair. J Vasc Surg. 2013;57:362–7.

    Article  PubMed  Google Scholar 

  31. Chisci E, Kristmundsson T, de Donato G, et al. The AAA with a challenging neck: outcome of open versus endovascular repair with standard and fenestrated stent-grafts. J Endovasc Ther. 2009;16:137–46.

    Article  PubMed  Google Scholar 

  32. Donas KP, Eisenack M, Panuccio G, Austermann M, Osada N, Torsello G. The role of open and endovascular treatment with fenestrated and chimney endografts for patients with juxtarenal aortic aneurysms. J Vasc Surg. 2012;56:285–90.

    Article  PubMed  Google Scholar 

  33. Moore Randy D, Macierewicz J, Cina C, Abuznadah W, Nutley M, Motamedi M. Fenestrated endovascular vs open repair for juxtarenal aortic aneurysmal disease. Vascular. 2006;14:S122–3.

    Google Scholar 

  34. Raux M, Patel VI, Cochennec F, et al. A propensity-matched comparison of outcomes for fenestrated endovascular aneurysm repair and open surgical repair of complex abdominal aortic aneurysms. J Vasc Surg. 2014;60:858–64.

    Article  PubMed  Google Scholar 

  35. Tsilimparis N, Perez S, Dayama A, Ricotta JJ 2nd. Endovascular repair with fenestrated-branched stent grafts improves 30-day outcomes for complex aortic aneurysms compared with open repair. Ann Vasc Surg. 2013;27:267–73.

    Article  PubMed  Google Scholar 

  36. British Society for Endovascular Therapy and the Global Collaborators on Advanced Stent-Graft Techniques for Aneurysm Repair (GLOBALSTAR) Registry. Early results of fenestrated endovascular repair of juxtarenal aortic aneurysms in the United Kingdom. Circulation. 2012;125:2707–15.

    Article  Google Scholar 

  37. Mosquera D, Chiang N, Gibberd R. Evaluation of surgical performance using V-POSSUM risk-adjusted mortality rates. J Surg. 2008;78:535–9.

    Google Scholar 

  38. Higgins J, Thompson S, Spiegelhalter D. A re-evaluation of random-effects meta-analysis. J R Stat Soc Ser A Stat Soc. 2009;172:137–59.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Lee JT, Lee GK, Chandra V, Dalman RL. Comparison of fenestrated endografts and the snorkel/chimney technique. J Vasc Surg. 2014;60:849–56 (discussion 56-7).

    Article  PubMed  Google Scholar 

  40. O’Neill S, Greenberg RK, Haddad F, Resch T, Sereika J, Katz E. A prospective analysis of fenestrated endovascular grafting: intermediate-term outcomes. Euro J Vasc Endovasc Surg. 2006;32:115–23.

    Article  Google Scholar 

  41. Tambyraja AL, Fishwick NG, Bown MJ, Nasim A, McCarthy MJ, Sayers RD. Fenestrated aortic endografts for juxtarenal aortic aneurysm: medium term outcomes. Euro J Vasc Endovasc Surg. 2011;42:54–8.

    Article  CAS  Google Scholar 

  42. Verhoeven ELG, Vourliotakis G, Bos WTGJ, et al. Fenestrated stent grafting for short-necked and juxtarenal abdominal aortic aneurysm: an 8-year single-centre experience. Euro J Vascu Endovasc Surg. 2010;39:529–36.

    Article  CAS  Google Scholar 

  43. Amiot S, Haulon S, Becquemin JP, et al. Fenestrated endovascular grafting: the French multicentre experience. Euro J Vasc Endovasc Surg. 2010;39:537–44.

    Article  CAS  Google Scholar 

  44. Dijkstra ML, Tielliu IF, Meerwaldt R, et al. Dutch experience with the fenestrated Anaconda endograft for short-neck infrarenal and juxtarenal abdominal aortic aneurysm repair. J Vasc Surg. 2014;60:301–7.

    Article  PubMed  Google Scholar 

  45. Greenberg RK, Sternbergh WC 3rd, Makaroun M, et al. Intermediate results of a United States multicenter trial of fenestrated endograft repair for juxtarenal abdominal aortic aneurysms. J Vasc Surg. 2009;50(730–7):e1.

    Google Scholar 

  46. Quinones-Baldrich WJ, Holden A, Mertens R, et al. Prospective, multicenter experience with the Ventana Fenestrated System for juxtarenal and pararenal aortic aneurysm endovascular repair. J Vasc Surg. 2013;58:1–9.

    Article  PubMed  Google Scholar 

  47. Semmens JB, Lawrence-Brown MM, Hartley DE, Allen YB, Green R, Nadkarni S. Outcomes of fenestrated endografts in the treatment of abdominal aortic aneurysm in Western Australia (1997–2004). J Endovasc Ther. 2006;13:320–9.

    Article  PubMed  Google Scholar 

  48. Kristmundsson T, Sonesson B, Dias N, Tornqvist P, Malina M, Resch T. Outcomes of fenestrated endovascular repair of juxtarenal aortic aneurysm. J Vasc Surg. 2014;59:115–20.

    Article  PubMed  Google Scholar 

  49. Kristmundsson T, Sonesson B, Malina M, Malina M, Bjorses K, Bjorses K, Dias N, Dias N, Resch T. Fenestrated endovascular repair for juxtarenal aortic pathology. J Vasc Surg. 2009;49:568–75.

    Article  PubMed  Google Scholar 

  50. Vemuri C, Oderich GS, Lee JT, et al. Postapproval outcomes of juxtarenal aortic aneurysms treated with the Zenith fenestrated endovascular graft. J Vasc Surg. 2014;60:295–300.

    Article  PubMed  Google Scholar 

  51. Ziegler P, Perdikides TP, Avgerinos ED, Avgerinos ED, Umscheid T, Umscheid T, Stelter WJ. Fenestrated and branched grafts for para-anastomotic aortic aneurysm repair. J Endovasc Surg. 2007;14:513–9.

    Google Scholar 

  52. Banno H, Cochennec F, Marzelle J, Becquemin JP. Comparison of fenestrated endovascular aneurysm repair and chimney graft techniques for pararenal aortic aneurysm. J Vasc Surg. 2014;60:31–9.

    Article  PubMed  Google Scholar 

  53. Rolls AE, Jenkins M, Bicknell CD, et al. Experience with a novel custom-made fenestrated stent graft in the repair of juxtarenal and type IV thoracoabdominal aneurysms. J Vasc Surg. 2014;59:615–22.

    Article  PubMed  Google Scholar 

  54. Karthikesalingam A, Holt PJ, Patterson BO, et al. Elective open suprarenal aneurysm repair in England from 2000 to 2010 an observational study of hospital episode statistics. PLoS ONE. 2013;8:e64163.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  55. Patel R, Powell JT, Sweeting MJ, Epstein DM, Barrett JK, Greenhalgh RM. The UK EndoVascular Aneurysm Repair (EVAR) randomised controlled trials: long-term follow-up and cost-effectiveness analysis. Health technol Assess. 2018;22:1–32.

    Article  PubMed  PubMed Central  Google Scholar 

  56. Curtis L, Burns A. Unit costs of health and social care 2016. In: PSSRU, editor. Canterbury; 2017.

  57. Epstein D. Health Related Quality of Life after Surgical Repair of Asymptomatic Abdominal Aortic Aneurysms. FEG Working Paper Series 02/16, Faculty of Economics and Business, University of Granada. 2016. https://ideas.repec.org/p/gra/fegper/02-16.html.

  58. Claxton K, Palmer S, Longworth L, et al. A comprehensive algorithm for approval of health technologies with, without, or only in research: the key principles for informing coverage decisions EAES recommendations on methodology of innovation management in endoscopic surgery. Value Health. 2016;19:885–91.

    Article  PubMed  Google Scholar 

  59. Vallabhaneni S, Haycox A, Cook A, et al. A Risk-adjusted and anatomically stratified cohort comparison study of open surgery, endovascular techniques and medical management for juxtarenal aortic aneurysms: The UK COMplex AneurySm Study (UK-COMPASS). 2017. Accessed Feb 2018.

  60. Claxton K, Martin S, Soares M, et al. Methods for the estimation of the National Institute for Health and Care Excellence cost-effectiveness threshold. Health Technol Assess. 2015;19:1.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Torbica A, Tarricone R, Drummond M. Does the approach to economic evaluation in health care depend on culture, values, and institutional context? https://doi.org/10.1007/s10198-017-0943-1. FAU - Torbica, Aleksandra.

  62. Eckermann S, Willan AR. Globally optimal trial design for local decision making. Health Econ. 2009;18:203–16.

    Article  PubMed  Google Scholar 

  63. Howard DP, Banerjee A, Fairhead JF, Handa A, Silver LE, Rothwell PM. Age-specific incidence, risk factors and outcome of acute abdominal aortic aneurysms in a defined population. Br J Surg. 2015;102:907–15.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  64. Sedrakyan A, Campbell B, Merino JG, Kuntz R, Hirst A, McCulloch P. IDEAL-D: a rational framework for evaluating and regulating the use of medical devices. BMJ. 2016;353:i2372.

    Article  PubMed  Google Scholar 

Download references

Authors’ contributions

OC lead the systematic review of the clinical evidence and meta-analyses. DE developed the decision analytic model. All authors read and approved the final manuscript.

Acknowledgements

The authors would like to thank the whole MedTecHTA consortium.

Competing interests

The authors declare that they have no competing interests.

Availability of data and materials

The datasets analysed during the current study are available from the corresponding author on reasonable request.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Not applicable.

Funding

This paper is based on research funded by the European Union Seventh Framework Programme under grant agreement HEALTH-F3-2012-305694 (Project MedtecHTA). The funder had no role in the conduct of the research or in the decision to submit the paper for publication. The views and opinions expressed therein are those of the authors.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and Affiliations

  1. Evidence Synthesis and Modeling for Health Improvement, Institute of Health Research, University of Exeter Medical School, South Cloisters, St Luke’s Campus, Exeter, EX1 2LU, UK

    Oriana Ciani & Rod S. Taylor

  2. Center for Research on Health and Social Care Management, SDA Bocconi University, via Roentgen 1, 20136, Milan, Italy

    Oriana Ciani

  3. Centre for Health Economics, University of York, Heslington, Alcuin ‘A’ Block, York, YO10 5DD, UK

    David Epstein, Claire Rothery & Mark Sculpher

  4. Department of Applied Economics, University of Granada, Campus Universitario de Cartuja, 18071, Granada, Spain

    David Epstein

Authors
  1. Oriana Ciani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Epstein
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Claire Rothery
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rod S. Taylor
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mark Sculpher
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Oriana Ciani.

Additional file

Additional file 1.

Appendix S1. Search strategy. Appendix S2. List of included studies. Appendix S3. More on the Risk of bias for estimate of relative risk of operative mortality. Table S1. Inclusion and exclusion criteria. Table S2. Characteristics of included studies. Table S3. Risk of bias assessment. Table S4. Thirty-day mortality or peri-operative mortality in comparative studies. Table S5. Operative mortality and late mortality after fenestrated EVAR. Figure S1. Odds ratios of operative mortality fEVAR vs OSR, all comparative studies. Figure S2. Predicted survival curves from the base-case model. Figure S3. Predicted costs (£) per patient over time (undiscounted). Figure S4. Early re-interventions (in-hospital or 30 day), unadjusted odds ratio.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ciani, O., Epstein, D., Rothery, C. et al. Decision uncertainty and value of further research: a case-study in fenestrated endovascular aneurysm repair for complex abdominal aortic aneurysms. Cost Eff Resour Alloc 16, 15 (2018). https://doi.org/10.1186/s12962-018-0098-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12962-018-0098-7

Keywords

Cost Effectiveness and Resource Allocation

ISSN: 1478-7547

Contact us