- Open Access
Economic evaluations of vaccines in Canada: a scoping review
Cost Effectiveness and Resource Allocationvolume 15, Article number: 7 (2017)
This study aims to summarise and describe the evolution of published economic evaluations of vaccines in Canada, thereby outlining the current state of this expanding and meaningful research.
Using Arksey and O’Malley’s scoping review framework we assembled relevant research from both academic and grey literature. Following abstract and full-text review we identified 60 articles to be included in the final analysis.
We found that since 1988 there has been a steady increase in the number of economic evaluations on vaccines in Canada. Many of these studies focus on the more recently licensed vaccines, such as influenza (16.7%), human papillomavirus (15.0%) and pneumococcal disease (15.0%). Since 2010 economic evaluations of vaccines have shown increased adherence to economic evaluation guidelines (OR = 4.6, CI 1.33, 18.7), suggesting there has been improvement in the consistency and transparency of these studies. However, there remains room for improvement, for instance, we found evidence that studies who stated a conflict of interest are more likely to assert the vaccine of interest was cost-effective (OR = 7.4; CI 1.04, 17.8). Furthermore, most reports use static models that do not consider herd immunity, and only a few evaluate vaccines post-implementation (ex-post) and traveller’s vaccinations.
Researchers should examine identified research gaps and continue to improve standardization and transparency when reporting to ensure economic evaluations of vaccines best meet the needs of policy-makers, other researchers and the public.
Vaccination is lauded as one of public health’s most significant achievements, contributing to improvements in morbidity, mortality and quality of life worldwide [1, 2]. Originally, one of the major advantages of vaccination as a public health intervention was their low cost, as many of the classic Expanded Program on Immunization vaccines only cost a few cents per dose and were cost-saving to the healthcare system [1, 3]. However, in recent years more technologically complicated vaccines and more strict licensing regulations for vaccine safety and efficacy have led to more expensive vaccine development and manufacturing, which raises the question of vaccine cost-effectiveness .
With ongoing debates in many countries concerning the implementation and funding of these new vaccines, especially within the context of growing financial strain on healthcare systems, economic evaluations are becoming increasingly important to policy-makers . Economic evaluations can improve the quality and consistency of decision making by providing a systematic way of comparing whether a specific vaccination program should be adopted compared with doing nothing or implementing an alternative intervention (e.g. treatment, other vaccines). Economic evaluations can also help evaluate scheduling and target population(s) [4, 5].
The number of vaccine-related economic evaluations has substantially increased in the last few years. However, we have very little knowledge of the coverage and quality of this research, as well as the potential gaps and limitations of these studies . Systematic reviews on economic evaluations of immunizations generally focus on the results from a particular vaccine; recent examples include varicella  and influenza . Meanwhile, many countries have yet to review and synthesize their own research in this area. One of the few examples is Spain, who recently released two comprehensive systematic reviews on Spanish economic evaluations of vaccinations completed between 1990 and 2012 [9, 10].
In comparison, researchers in Canada have never fully synthesized the extent and characteristics of research on economic evaluations of vaccines in Canada. As the number of licensed vaccines in Canada and the number of economic evaluations on those vaccines continue to increase, it is essential that Canada begins to critically analyse and summarise this work . This research is important to ensure improvement in the usefulness, quality and applicability of vaccine-related economic evaluations and the decisions they inform. Furthermore, as countries produce similar evidence, comparisons of reporting, methods and results between countries could contribute to the international discussion on the major gaps in the literature and the quality, standardization and transparency of these studies.
This study aims to systematically gather, review and summarise Canadian economic evaluations on vaccines using a scoping review methodology, with the goal of describing general trends and gaps in the literature.
Scoping review methodology
We based our methodology on Arksey and O’Malley’s  five step framework for conducting a scoping review, including (1) forming the research question, (2) identifying relevant studies, (3) study selection, (4) charting the data and (5) summarising and reporting the results. Each section is described in more detail below. Scoping reviews are designed to summarise rich and complex areas of research that have not be synthesized in the past . A scoping review also offers policy-makers easily accessible and comprehensive cost-effectiveness information and evidence regarding vaccines. Moreover, a review helps identify gaps in the vaccine economic evaluation literature and aids policy-makers spend their limited research funds more effectively and efficiently. We chose to conduct a scoping review rather than a systematic review because it allowed us to methodologically examine the breadth and depth of the work on economic evaluations of vaccines in a Canada, a highly multi-disciplinary area, while allowing for more flexibility than a systematic review . The aims of our study were more consistent with the goals of a scoping review (e.g. to map the current state of the literature and summarize the breadth and depth of the research) in comparison to the goals of a systematic review (e.g. to summarize the evidence on the effectiveness of an intervention or treatment) . The scoping review methodology allowed us to summarize a wide range of evidence including both grey (e.g. government reports) and peer-reviewed literature. Furthermore, rather than having a very specific and narrow research question, we asked a broad research question that encompassed all types of diseases, populations (target vs. universal) and interventions (i.e. types of vaccine) .
Identifying the research question
To further focus our research question—“What has been published regarding economic evaluations of vaccines in Canada?”—we decided to only consider active immunization, vaccines that impact human health, and studies that focused on a Canadian population. To ensure comparability we chose to include only full economic evaluations, which we defined based on Drummond et al., and therefore, partial economic evaluations such as cost-outcome description studies and cost-minimization studies were not included [5, 13]. Furthermore, to ensure the comprehensiveness of the review, we included both peer-reviewed and grey literature in our search. Only English-language articles were part of the final analysis.
Identifying relevant studies
To identify peer-reviewed articles we searched five databases relevant to both public health and economic evaluations: Embase, Medline, Global Health, Cochrane Library—specifically the Health Technology Assessment Database—and the NHS Economic Evaluation Database. No limitations were placed on the date and all databases were searched up until March 17th, 2015. We used a report-based expansion strategy centred on three essential keywords (‘immunization’ AND ‘economic evaluation’ AND ‘Canada’). See Additional file 1 for a full list of search terms used. After title, abstract and full-text review, the reference lists of all included articles were searched for relevant citations. We validated each database search for its efficiency and accuracy using ten articles that the reviewers identified as being highly relevant to the topic area. We compared the ten articles to the citations we identified during each database search.
To identify grey literature we searched both the ProQuest Dissertation and Theses database along with key organizational websites related to health technology assessment, vaccination and economics in Canada, such as Canadian Agency for Drugs and Technologies in Health’s (CADTH), the Health Quality Council from multiple provinces and Institute of Health Economics. We chose these organizations based on advice from a health sciences librarian, the opinion of experts in the field and the CADTH grey literature checklist . In total we searched 25 potentially relevant websites and organizations.
All eligible articles were imported into Microsoft Access 2010 for relevance screening and duplicates were removed. Independently, two reviewers evaluated the titles, abstracts and full-text of peer-reviewed articles by answering ‘yes,’ ‘no,’ or ‘unsure’ to each of the following questions:
Was the research conducted on a Canadian population? (i.e. presented Canadian-specific data or results?)
Was a full economic evaluation conducted (measured both costs and benefits)?
Did the topic area relate to active vaccination or immunizations? (e.g. different types of vaccines, different schedules)?
Is a full-text available?
Is the article in English?
Were the results relevant to human health?
During title, abstract and full-text screening of an article if the answer to any of the above questions was ‘no’ then the article was excluded; otherwise, the article was included in the next stage of analysis. We calculated Cohen’s Kappa Coefficient at both the title, abstract and full-text screening stages to measure agreement between the two reviewers . The reviewers resolved any disagreements through discussion and consensus. One reviewer screened the grey literature for inclusion or exclusion using the same process outlined above; however, if the first reviewer was uncertain about the inclusion of an article it was given to the second reviewer. All the grey literature articles included in the final analysis were read by both reviewers.
Charting the data
Following full-text screening, the two reviewers charted each study chosen for inclusion using a standardized form designed to gather common and comparable information on each study. Data extracted included year of publication, region, targeted disease, type of economic evaluation, modelling type, herd immunity, target population, study perspective, time horizon, type of sensitivity analysis, comparator, the consideration of equity issues, general cost-effectiveness findings and stated conflict of interest(s). These variables were chosen based on CADTH’s Guidelines for the Economic Evaluation of Health Technologies: Canada  and the World Health Organization’s (WHO) Guide for Standardization of Economic Evaluations of Immunization Programmes  that outline information to include in an economic evaluation. The main focus of our scoping review was to summarize the range of evidence and identify any gaps in the literature, rather than compare the results of the economic evaluation; therefore, we did not to compare the articles based on quality. We charted each study’s adherence to these guidelines, and evaluated each study on whether they adequately (according to guideline protocols) presented six key principles that are considered essential for a well conducted economic evaluation. These six key principles included: time horizon, perspective, comparator, model-type and choice of economic evaluation (including justification for choice of model and economic evaluation), and if uncertainty in the results was fully accounted for through sensitivity analysis. This analysis gives policy-makers not only an idea of how economic evaluations are generally being conducted and presented in Canada, but also the effectiveness of their guidelines to inform research and whether the guidelines could be applied consistently across articles.
Summarising and reporting the results
Following data extraction we used thematic analysis and descriptive statistics to analyze general trends and patterns in the data. We used Chi square tests to ascertain whether there was an association between stated conflicts of interests (i.e. at least one authors reported they were either currently working for the pharmaceutical industry or currently receiving funding) and their final cost-effectiveness recommendations (i.e. concluded the vaccine of interest was cost-effective or not). Furthermore, we studied the association between the publication year (i.e. <2010 or ≥2010) and the studies adherence to the six key principles of an economic evaluation discussed in the WHO and CADTH guidelines (i.e. adhered to guidelines or not). We also examined the association between publication year (i.e. <2010 or ≥2010) and choice of modelling type (i.e. static vs. dynamic). We chose 2010 as the cut off year as it provided adequate time after the publication of the WHO  and CADTH  guidelines for authors to incorporate the recommendations in their research. Finally, we calculated the number of studies that took place prior to licensure of the vaccine in Canada, between licensure and the implementation of the vaccination program being studied, and those that occurred after the implementation of the vaccine in the region considered in the study. We were particularly interested in economic evaluations that aimed to re-evaluate the cost-effectiveness of the vaccine post-implementation and therefore included retrospective data (ex-post).
Following the database search and the removal of duplicates, 908 peer-reviewed articles remained to be screened. Based on title and abstract screening we eliminated 792 articles, leaving 116 articles for full-text review, of which 55 were included in the final analysis (Fig. 1). All ten ‘original’ studies chosen as being highly relevant to the research and used to validate the search methods were identified in each of the database searches. There was substantial agreement between the two reviewers; Cohen’s Kappa Coefficient was 0.81 for the title and abstract screening and 0.85 for the full-text screening. The reviewers reached consensus in all cases of disagreement.
Following the grey literature search and title screening we retrieved 38 potentially relevant studies and dissertations from 25 websites and the ProQuest Theses database, of which five were retained in the final analysis. Sixty peer-reviewed and grey-literature studies were included in the final scoping review analysis (Fig. 1).
Summary of studies included in the review
The number of articles by year of publication, region, disease, type of economic evaluation, modelling type, consideration of herd immunity, perspective, time horizon and sensitivity analysis were summarized (Table 1). The number of economic evaluations of vaccines steadily increased between 1988 and March 2015; 66.7% of economic evaluations were published within the last ten years (i.e. since 2005) (Fig. 2). Most studies focused on Canada in general (58.3%), with Quebec (13.3%), Ontario (10.0%) and British Columbia (8.3%) producing the most province-specific vaccine economic evaluations. Economic evaluation of vaccines in Canada considered 20 different diseases including, from most common to least common—influenza (pandemic-2 and seasonal-8) [17,18,19,20,21,22,23,24,25,26], human papillomavirus (HPV) [27,28,29,30,31,32,33,34,35], pneumococcal [36,37,38,39,40,41,42,43,44], pertussis [45,46,47,48,49], meningitis A, C, Y and/or W135 [50,51,52,53,54], hepatitis B [55,56,57,58,59], varicella [60,61,62], measles, mumps and/or rubella [63,64,65], herpes zoster [66, 67], rotavirus [68, 69], rabies , hepatitis A , tetanus , hepatitis C , meningitis B , group B streptococcus , and Escherichia coli  (Fig. 2).
The most popular type of economic evaluation used in Canadian studies was cost-utility analysis (50.0%); although, it was often combined with cost-effectiveness analysis. More than half of the studies used static modelling (e.g. simple tree or static cohort) (63.3%). Most studies used a life-time horizon (i.e. 80+ years) (41.6%), with a healthcare payer (58.3%) or societal perspective (46.7%), often combining the two perspectives. Researchers employed a wide range of comparators (Table 2). The vaccine of interest was compared to no vaccine in 66.7% of studies. The evaluation compared different ways of administering the vaccine program (e.g. different schedules or target populations) in 25.0% of studies. Different types of vaccines (brands or antigen formulations) for the same disease were compared in 26.7% of studies, with some studies employing more than one type of comparator. Most studies (86.7%) evaluated publically-funded vaccines that were included on at least one provincial immunization schedule (Table 2). Finally, only six studies in our analysis had a discussion of equity, which mainly focused on the fact that they were unable to address issues of equity within their analysis.
Factors associated with reporting practices and study findings
In reports where there was a stated conflict of interest, the authors were more likely to assert the vaccine of interest was cost-effective relative to the comparator (OR = 7.36; CI 1.04, 17.8; p value = 0.04) than in reports with no reported conflict. Furthermore, studies published from 2010 to 2015 were more likely to follow the six key principles in the WHO and CADTH reporting guidelines compared to those studies published before 2010 (OR = 4.58, CI 1.33, 18.7, p value = 0.01) (Fig. 3). However, there was no difference in the type of model employed before 2010 compared to 2010–2015 (OR = 3.05; CI 0.66, 16.8; p value = 0.16). All of the five individual-based models were published in the last five years [21, 28, 30, 31, 70].
Most studies (66.7%) were completed before the vaccine program of interest was implemented in Canada with 8.3% before the vaccine was licensed. One in four studies (25.0%) were completed after the vaccine program was implemented. Only five post-implementation studies (8.3%) were ex-post and evaluated past vaccination strategies [21, 22, 43, 48, 52].
In recent years, the number of economic evaluations on health interventions have increased in Canada and worldwide [16, 77]. The results of our scoping review, are consistent with these trends, with 50.0% of the economic evaluations of vaccines in Canada being produced in the last five years [16, 77]. There are many explanations for this increase in economic evaluations, including increasing pressures on the budgets of healthcare systems, as well as the advent of new, complex and more expensive vaccinations [77, 78]. Therefore, it is perhaps unsurprising that the influenza, pneumococcal and HPV vaccines represented almost 50.0% of the economic evaluations, as the immunization programs for these diseases, including the schedules, vaccine-type, administration techniques and target populations have all changed over the last few years. For instance, recently decision-makers have raised questions surrounding the cost-effectiveness of male HPV vaccination, using the modified live intranasal influenza vaccine in children between two and 17, and which pneumococcal vaccine (i.e. PCV13, PCV10 and PCV7) would give the best value for the money, particularly considering the impact of serotype replacement and otitis media [26, 28, 30, 31, 36, 38, 39, 44].
Most economic evaluations focussed on publically-funded vaccines that were part of at least one provincial immunization schedule. A very small proportion of the research was done on privately-available vaccines, such as travel (e.g. yellow fever, Japanese encephalitis) and workplace vaccines (e.g. rabies) [70, 76]. Furthermore, there are only a small amount of research on vaccines not yet available or licensed in Canada (e.g. chlamydia, herpes simplex), with some notable exceptions (i.e. hepatitis C and meningococcal B) [73, 74]. Anticipatory economic evaluations can provide essential information on the needs for additional research and investment.
In Canada, most of economic evaluations occur after the vaccine has been licensed but before the implementation of the vaccine program in the population (ex-ante). These ex-ante studies are important as they provide decision-makers with an estimate of the value of instituting a new vaccine and the costs and benefits of different vaccination programs (e.g., private, targeted, universal). However, ex-ante evaluations are difficult to perform as country-specific effectiveness and cost data is often lacking before implementation, and the ability to predict the vaccination impact, particularly herd effects, is limited . For instance, prior to vaccine implementation it is difficult to predict the real market price of a vaccine, which can vary dramatically in space and time. Therefore, there is a need to evaluate whether or not existing vaccination programs are good value for their cost, especially because effectiveness and cost data can change drastically over time . These ex-post studies allow researchers to validate estimates of costs and outcomes from pre-implementation evaluations, and potentially improve future studies .
However, we saw little evidence of ex-post economic evaluations. For instance, the costs and outcomes of the varicella vaccine ex-post have not been compared with the two inconclusive studies conducted ex-ante [68, 69]. In fact, only five studies were identified where the goal was to determine the efficiency of a mass immunization after its implementation in the population [21, 22, 43, 48, 52]. This lack of retrospective research has left many unanswered questions, such as ‘How good are Canadian economic evaluations at predicting the cost-effectiveness of vaccines and should these studies inform our policy-making?’ This gap in the literature may be partially due to a lack of industry willingness to fund ex-post studies or because of noted methodological limitations, including issues with estimating the ‘no program’ scenario, attributing decrease in disease to vaccination and predicting future benefits . However, Newall et al.  has identified a variety of approaches that help address these issues, noting that as retrospective studies become more important and frequent, it would be beneficial to increase the guidance available to researchers undertaking such analyses. In the future, a clearer understanding of the accuracy of economic evaluation predictions could help researchers improve research techniques and potentially increase the efficiency of current vaccination programs.
Most of the studies in this review compared the vaccine of interest to no vaccination. While this is an essential initial comparison, once a vaccine is well-established, there are other questions that economic evaluations can help answer, including questions about vaccine programming and scheduling. In Canada, healthcare is primarily under provincial/territorial (not federal) jurisdiction and as a result there is a diversity of vaccine programs, technologies and schedules throughout the country. To date there have been no reported studies of which vaccination programs are most efficient or cost-effective, or whether these differences are justifiable due to the epidemiology and costs in different Canadian regions.
The use of modelling in economic evaluation of vaccines has increased in the last few years. Most studies continue to employ static models (e.g. decision tree, Markov processes) as compared to dynamic models (e.g. system dynamics, individual-based models). In dynamic models, the risk of infection can change over time. There has been no apparent increase in the application of dynamic modelling and no relationship between modelling type and year of publication . Although dynamic models are not always necessary to represent vaccine preventable diseases (e.g. tetanus, herpes zoster), they allow for the intrinsic consideration of vaccine externalities (herd immunity, shifts in age of infection and serotype replacement). Consideration of externalities is limited in the Canadian literature, with fewer than half of studies in our review considering herd immunity. Failure to include the impact of herd immunity can significantly underestimate the effectiveness of a vaccine . An increase in dynamic modelling, where relevant, could improve the accuracy of economic evaluations of vaccines and should be an area of future research.
Individual-based models are, however, becoming increasingly popular in economic evaluations [21, 28, 30, 31, 70]. These models have the advantage of being able to replicate individual-level behaviour and interactions (e.g. transmission, risk behaviours, disease history), which may have substantial impacts on the economic evaluation. Individual-based models not only intrinsically account for the impact of herd immunity but also help researchers study the influence of key subsets of the population. Three separate economic evaluations on HPV used a common individual-based model called HPV-ADVISE to estimate the costs and benefits of different vaccination strategies and scenarios in Canada [28, 30, 31]. The HPV-ADVISE model informed HPV vaccine policy-making regarding the cost-effectiveness of various vaccine types (bivalent versus quadrivalent versus nine-valent), catch-up programs, type replacement, schedules (ages, number of doses), as well as male-vaccination. HPV-ADVISE is an example of the power and flexibility of individual-based modelling and how information sharing can improve economic evaluation research and therefore the policy-informing power of these types of studies. However, there are also trade-offs to consider with dynamic models, specifically individual-based models, as they are often more complex, time consuming and can require more fine grained data, potentially impacting the reliability and timeliness of the results.
Authors’ choice of the time horizon and discount rate can significantly impact the outcomes of an economic evaluation. As is noted in many guidelines for economic evaluations, for accurate analysis the chosen time horizons must encompass all costs and benefits of a policy decision . However, as we saw in this analysis many economic evaluations adopt a shorter than necessary time horizon (e.g. economic evaluations adopting the same time horizon as a clinical trials), creating the possibility of a time horizon bias. Another important element of economic evaluations that can significantly impact the outcomes of a cost-effectiveness analysis is the perspective and therefore, the consideration of costs and benefits not directly related to health. It is essential that economic evaluations clearly justify their choice of perspective and discuss the costs not considered, however a complete discussion of choice of perspective was often missing in the studies of our analysis. Studies are starting to adopt a two -perspective approach, where they not only consider the economic benefits and costs from a narrower view point (e.g. healthcare payer) but also using the most comprehensive approach (i.e. societal perspective). Reporting of time horizon and study perspectives in economic evaluation of vaccines must improve to ensure the transparency of complex analyses and to provide the details necessary for informed decision-making.
In our review only six articles included some assessment or discussion of equity in their analysis. Lack of equity considerations and inability to account for equality in resource allocation analyses are common criticisms of economic evaluations . Debate continues around whether or not economic evaluations have the responsibility to consider equity; however, the lack of information on the differential impact of interventions often mean policy-makers are hesitant to use the results of economic evaluations . In recent years there have been a number of methodological advancements to help economists account for equity as part of, or as an extension to, their cost-effectiveness analysis (e.g. extended or distributional cost-effectiveness); however, the use of these tools is still limited . As the field of economic evaluations of vaccines continues to grow it is important that these studies consider and discuss how their findings may have an impact on the equity of health care delivery and health outcomes.
Research shows that guidelines for economic evaluations from non-governmental organizations, governments and journals can help increase the transparency and reliability of these studies . In fact, one of the main reasons the WHO created guidelines for economic evaluations of vaccines in 2008  was to address the limitations they observed in evaluations reported prior to 2005, and to enhance standardization and comparability between studies . Both Baladi et al. and Neumann et al. discovered a growing adherence to recommended practices in cost-effectiveness and cost-utility analyses worldwide and in Canada, which they attributed to stricter journal protocols and guidelines in publishing these studies [77, 83]. Our results further support these findings indicating that since the publication of WHO and CADTH guidelines there has been an increase in the standardization of the methods used in economic evaluation and in the transparency and reporting of the research in Canada [3, 16]. Although, we noted a change in the consistency of reporting following the introduction of the guidelines, there remained certain study elements commonly omitted, such as the study perspective, modelling technique description and time-horizon. The large number of assumptions and parameters used in economic evaluations make reporting guidelines essential, as simple changes to the perspective adopted, discount rate or time horizon chosen can have a major impact on the results . Therefore, the transparency and reliability of these studies should continue to improve and the guidelines must be continually updated to ensure that the results are both useable and translatable.
The use of national guidelines and publishing protocols is particularly relevant and important to our sample of articles, as just under half the studies had reported conflicts of interest and these studies were significantly more likely to find the vaccine of interest cost-effective. Therefore, transparency and consistency in economic evaluations are essential to ensure policy-makers can use the results to make accurate and unbiased decisions.
This study had a number of limitations. First, we were unable to include articles in French, and therefore may have missed contributions to the economic evaluation literature, especially as Quebec was the most prolific region in the production of economic evaluations of vaccines. Second, only one reviewer was able to screen the grey literature articles for inclusion. Nevertheless, all final decisions were discussed between the two reviewers to ensure agreement about all studies included in the analysis. Finally, we did not compare the results of the economic evaluations because we had included articles for a variety of diseases and with a wide range of quality.
To date economic evaluations of vaccines in Canada have not been summarised or reviewed, potentially limiting the influence these studies have on policy-making. This scoping review acts as a guiding document for policy-makers and researchers, to provide easy access to essential information about vaccines. Our scoping review outlined many gaps in the literature, including the lack of studies on various privately-available vaccines, different vaccination schedules and programs, and the shortage of ex-post implementation economic evaluations. Furthermore, we identified important trends in the literature, including an increasing number of economic evaluations on vaccines and a focus on newly-available vaccines.
We found some weaknesses in the literature, including the limited use of dynamic modelling and consideration of herd immunity, the significant association between declared conflicts of interest and a positive cost-effectiveness result, as well as the under-reporting of time-horizon, perspective and modelling type. However, the scoping review outlined some key strengths of the Canadian literature, including an increase in the application of individual-based modelling techniques. Furthermore, the implementation of national guidelines appears to have had an impact on the standardization and transparency of economic evaluations in Canada, and remains an important consideration for countries without similar guidelines. As more countries start to map this literature and analyse the trends and limitations, then comparisons between countries, their economic evaluation literature and their vaccination programs, could provide valuable input for improving the quality and usefulness of economic evaluations of vaccines.
Levine R. Global burden of disease. Vaccine B. San Diego: Academic Press; 2003, p. 23–6.
Stern A, Markel H. The history of vaccines and immunization: familiar patterns, new challenges. Health Aff. 2005;24:611–21.
Initiative for Vaccine Research. WHO guide for standardization of economic evaluations of immunization programmes. Geneva: WHO Press; 2008.
Ess SM, Szucs TD. Economic evaluation of immunization strategies. Clin Infect Dis. 2002;35:294–7. doi:10.1086/341419.
Drummond MF, Sculpher MJ, Torrance GW, O’Brien BJ. Methods for economic evaluation of healthcare programs. Oxford: Oxford University Press; 2005.
Szucs TD. Health economic research on vaccinations and immunisation practices–an introductory primer. Vaccine. 2005;23:2095–103. doi:10.1016/j.vaccine.2005.01.064.
Unim B, Saulle R, Boccalini S, Taddei C, Ceccherini V, Boccia A, et al. Economic evaluation of Varicella vaccination: results of a systematic review. Hum Vaccin Immunother. 2014;9:1932–42. doi:10.4161/hv.25228.
Ott JJ, Klein Breteler J, Tam JS, Hutubessy RCW, Jit M, de Boer MR. Influenza vaccines in low and middle income countries: a systematic review of economic evaluations. Hum Vaccin Immunother. 2013;9:1500–11. doi:10.4161/hv.24704.
Cortés I, Pérez-Camarero S, del Llano J, Peña LM, Hidalgo-Vega Á. Systematic review of economic evaluation analyses of available vaccines in Spain from 1990 to 2012. Vaccine. 2013;31:3473–84. doi:10.1016/j.vaccine.2013.05.097.
García-Altés A. Systematic review of economic evaluation studies: are vaccination programs efficient in Spain? Vaccine. 2013;31:1656–65. doi:10.1016/j.vaccine.2013.01.029.
Arksey H, Malley LO. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;. doi:10.1080/1364557032000119616.
Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5:69. doi:10.1186/1748-5908-5-69.
Dakin H, Wordsworth S. Cost-minimisation analysis versus cost-effectiveness analysis, revisited. Health Econ. 2010;22:22–34. doi:10.1002/hec.1812.
CADTH information services. a practical deep web search for evidence-based medicine: grey matters. Ottawa: 2014.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74. doi:10.2307/2529310.
Canadian Agency for Drugs and Technologies in Health. Guidelines for the economic evaluation of health technologies: Canada. 3rd ed. Ottawa: 2006.
Fisman DN, Tuite AR. Estimation of the health impact and cost-effectiveness of influenza vaccination with enhanced effectiveness in Canada. PLoS ONE. 2011;6:e27420. doi:10.1371/journal.pone.0027420.
Fitzgerald N. Cost-effectiveness of pharmacy-based influenza immunization programs: A case study in the Capital District Health Authority. Halifax Nova Scotia: Dalhousie University; 2006.
Gregg M. Economic Evaluation of an Influenza Immunization Program. Ontario: McMaster University; 2012.
Nosyk B, Sharif B, Sun H, Cooper C, Anis AH. The cost-effectiveness and value of information of three influenza vaccination dosing strategies for individuals with human immunodeficiency virus. PLoS ONE. 2011;6:e27059. doi:10.1371/journal.pone.0027059.
Sander B, Bauch CT, Fisman D, Fowler R, Kwong JC, Maetzel A, et al. Is a mass immunization program for pandemic (H1N1), good value for money? Evidence from the Canadian Experience. Vaccine. 2009;2010(28):6210–20. doi:10.1016/j.vaccine.2010.07.010.
Sander B, Kwong JC, Bauch CT, Maetzel A, McGeer A, Raboud JM, et al. Economic appraisal of Ontario’s universal influenza immunization program: a cost-utility analysis. PLoS Med. 2010;7:e1000256. doi:10.1371/journal.pmed.1000256.
Skedgel C, Langley JM, Macdonald NE, Scott J, Skedgel C, Macdonald NE. An incremental economic evaluation of targeted and Universal influenza vaccination in pregnant women. Can J Public Heal. 2014;102:445–50.
Skowronski DM, Woolcott JC, Tweed SA, Brunham RC, Marra F. Potential cost-effectiveness of annual influenza immunization for infants and toddlers: experience from Canada. Vaccine. 2006;24:4222–32. doi:10.1016/j.vaccine.2005.12.036.
Smetanin P, Stiff D, Kumar A, Bolvin G, Oxford JS. Evaluation of Pandemic H1N1 Interventions in Canada. Toronto: RiskAnalytica; 2009.
Tarride JE, Burke N, Von Keyserlingk C, O’Reilly D, Xie F, Goeree R. Cost-effectiveness analysis of intranasal live attenuated vaccine (LAIV) versus injectable inactivated influenza vaccine (TIV) for Canadian children and adolescents. Clin Outcomes Res. 2012;4:287–98. doi:10.2147/CEOR.S33444.
Anonychuk AM, Bauch CT, Merid MF, Van Kriekinge G, Demarteau N. A cost-utility analysis of cervical cancer vaccination in preadolescent Canadian females. BMC Public Health. 2009;9:401. doi:10.1186/1471-2458-9-401.
Brisson M, Laprise JF, Drolet M, Van de Velde N, Franco EL, Kliewer EV, et al. Comparative cost-effectiveness of the quadrivalent and bivalent human papillomavirus vaccines: a transmission-dynamic modeling study. Vaccine. 2013;31:3863–71. doi:10.1016/j.vaccine.2013.06.064.
Debicki D, Ferko N, Demarteau N, Gallivan S, Bauch C, Anonychuk A, et al. Comparison of detailed and succinct cohort modelling approaches in a multi-regional evaluation of cervical cancer vaccination. Vaccine. 2008;26:16–28. doi:10.1016/j.vaccine.2008.02.040.
Drolet M, Laprise JF, Boily MC, Franco EL, Brisson M. Potential cost-effectiveness of the nonavalent human papillomavirus (HPV) vaccine. Int J Cancer. 2014;134:2264–8. doi:10.1002/ijc.28541.
Laprise JF, Drolet M, Boily MC, Jit M, Sauvageau C, Franco EL, et al. Comparing the cost-effectiveness of two- and three-dose schedules of human papillomavirus vaccination: a transmission-dynamic modelling study. Vaccine. 2014;32:5845–53. doi:10.1016/j.vaccine.2014.07.099.
Kohli M, Lawrence D, Haig J, Anonychuk A, Demarteau N. Modeling the impact of the difference in cross-protection data between a human papillomavirus (HPV)-16/18 AS04-adjuvanted vaccine and a human papillomavirus (HPV)-6/11/16/18 vaccine in Canada. BMC Public Health. 2012;12:872. doi:10.1186/1471-2458-12-872.
Rogoza RM, Ferko N, Bentley J, Meijer CJLM, Berkhof J, Wang KL, et al. Optimization of primary and secondary cervical cancer prevention strategies in an era of cervical cancer vaccination: a multi-regional health economic analysis. Vaccine. 2008;26:46–58. doi:10.1016/j.vaccine.2008.02.039.
Tully SP, Anonychuk AM, Maria Sanchez D, Galvani AP, Bauch CT. Time for change? An economic evaluation of integrated cervical screening and HPV immunization programs in Canada. Vaccine. 2012;30:425–35. doi:10.1016/j.vaccine.2011.10.067.
Brisson M, Van de Velde N, De Wals P, Boily MC. The potential cost-effectiveness of prophylactic human papillomavirus vaccines in Canada. Vaccine. 2007;25:5399–408. doi:10.1016/j.vaccine.2007.04.086.
Chuck AW, Jacobs P, Tyrell G, Kellner JD. Pharmacoeconomic evaluation of 10- and 13-valent pneumococcal conjugate vaccines. Vaccine. 2010;28:5485–90. doi:10.1016/j.vaccine.2010.05.058.
De Wals P, Petit G, Erickson LJ, Guay M, Tam T, Law B, et al. Benefits and costs of immunization of children with pneumococcal conjugate vaccine in Canada. Vaccine. 2003;21:3757–64. doi:10.1016/S0264-410X(03)00361-X.
Earnshaw SR, McDade CL, Zanotti G, Farkouh RA, Strutton D. Cost-effectiveness of 2 + 1 dosing of 13-valent and 10-valent pneumococcal conjugate vaccines in Canada. BMC Infect Dis. 2012;12:101. doi:10.1186/1471-2334-12-101.
Knerer G, Ismaila A, Pearce D. Health and economic impact of PHiD-CV in Canada and the UK: a Markov modelling exercise. J Med Econ. 2012;15:61–76. doi:10.3111/13696998.2011.622323.
Lebel MH, Kellner JD, Ford-Jones EL, Hvidsten K, Wang ECY, Ciuryla V, et al. A pharmacoeconomic evaluation of 7-valent pneumococcal conjugate vaccine in Canada. Clin Infect Dis. 2003;36:259–68. doi:10.1086/345833.
Marra CM, Patrick DM, Pharm CAM. A Cost-effectiveness analysis of pneumococcal vaccination in street-involved, HIV-infected patients. Can J Public Health. 2015;91:334–9.
Moore D, Bigham M, Patrick D. Modelling the costs and effects of a universal infant immunization program using conjugated pneumococcal vaccine in British Columbia. Can Commun Dis Rep. 2003;29:97–104.
Poirier B, De Wals P, Petit G, Erickson LJ, Pépin J. Cost-effectiveness of a 3-dose pneumococcal conjugate vaccine program in the province of Quebec, Canada. Vaccine. 2009;27:7105–9. doi:10.1016/j.vaccine.2009.09.057.
Talbird SE, Taylor TN, Frostad CR, Marti SG. Outcomes and costs associated with PHiD-CV, a new protein D conjugate pneumococcal vaccine, in four countries. Vaccine. 2010;28S:G23–9. doi:10.1016/j.vaccine.2011.02.103.
Greer AL, Fisman DN. Use of models to identify cost-effective interventions: pertussis vaccination for pediatric health care workers. Pediatrics. 2011;128:e591–9. doi:10.1542/peds.2010-0796.
Iskedjian M, Einarson TR, O’Brien BJ, De Serres JG, Gold R, Gemmill IM, et al. Economic evaluation of a new acellular vaccine for pertussis in Canada. Pharmacoeconomics. 2001;19:551–63. doi:10.2165/00019053-200119050-00009.
Iskedjian M, Walker JH, De Serres G, Einarson TR. Economic evaluation of an extended acellular pertussis vaccine program for adolescents in Quebec, Canada. Paediatr Drugs. 2005;7:123–36.
Iskedjian M, De Serres G, Einarson TR, Walker JH. Economic impact of the introduction of an acellular pertussis vaccine in Canada: a 6-year analysis. Vaccine. 2010;28:714–23. doi:10.1016/j.vaccine.2009.10.079.
Iskedjian M, Walker JH, Hemels MEH. Economic evaluation of an extended acellular pertussis vaccine programme for adolescents in Ontario, Canada. Vaccine. 2004;22:4215–27. doi:10.1016/j.vaccine.2004.04.025.
De Wals P. Should university students be vaccinated against meningococcal disease in Canada? Can J Infect Dis. 2004;15:25–8.
De Wals P, Coudeville L, Trottier P, Chevat C, Erickson LJ, Nguyen VH. Vaccinating adolescents against meningococcal disease in Canada: a cost-effectiveness analysis. Vaccine. 2007;25:5433–40. doi:10.1016/j.vaccine.2007.04.071.
De Wals P, Erickson L. Economic analysis of the 1992-1993 mass immunization campaign against serogroup C meningococcal disease in Quebec. Vaccine. 2002;20:2840–4. doi:10.1016/S0264-410X(02)00161-5.
De Wals P, Nguyen VH, Erickson LJ, Guay M, Drapeau J, St-Laurent J. Cost-effectiveness of immunization strategies for the control of serogroup C meningococcal disease. Vaccine. 2004;22:1233–40. doi:10.1016/j.vaccine.2003.09.022.
Rancourt C, Grégoire JP, Simons WR, Dostie A. Cost-benefit model comparing two alternative immunisation programmes against serogroup C meningococcal disease: for Quebec residents aged 2 months to 20 years. Pharmacoeconomics. 2003;21:429–42. doi:10.2165/00019053-200321060-00006.
Krahn M. Detsky a S. Should Canada and the United States universally vaccinate infants against hepatitis B? A cost-effectiveness analysis. Med Decis Making. 1991;13:4–20.
Krahn M, Guasparini R, Sherman M, Detsky AS. Costs and cost-effectiveness of a universal, school-based hepatitis B vaccination program. Am J Public Health. 1998;88:1638–44. doi:10.2105/AJPH.88.11.1638.
Rossi C, Schwartzman K, Oxlade O, Klein MB, Greenaway C. Hepatitis B screening and vaccination strategies for newly arrived adult canadian immigrants and refugees: a cost-effectiveness analysis. PLoS ONE. 2013;8:e78548. doi:10.1371/journal.pone.0078548.
Wiebe T, Fergusson P, Horne D, Shanahan M, Macdonald A, Heise L, et al. Hepatitis B immunization in a low-incidence province of Canada: comparing alternative strategies. Med Decis Making. 2014;17:472–82. doi:10.1177/0272989X9701700413.
Wong WWL, Woo G, Jenny Heathcote E, Krahn M. Cost effectiveness of screening immigrants for hepatitis B. Liver Int. 2011;31:1179–90. doi:10.1111/j.1478-3231.2011.02559.x.
Merrett P, Schwartzman K, Rivest P, Greenaway C. Strategies to prevent varicella among newly arrived adult immigrants and refugees: a cost-effectiveness analysis. Clin Infect Dis. 2007;44:1040–8. doi:10.1086/512673.
Brisson M, Edmunds WJ. The cost-effectiveness of varicella vaccination in Canada. Vaccine. 2002;20:1113–25. doi:10.1016/S0264-410X(01)00437-6.
Getsios D, Caro JJ, Caro G, De Wals P, Law BJ, Robert Y, et al. Instituting a routine varicella vaccination program in Canada: an economic evaluation. Pediatr Infect Dis J. 2002;21:542–7. doi:10.1097/00006454-200206000-00012.
Falk WA, Buchan K, Dow M, Garson JZ, Hill E, Nosal M, et al. The epidemiology of mumps in southern Alberta 1980–1982. Am J Epidemiol. 1989;130:736–49.
Pelletier L, Chung P, Duclos P, Manga P, Scott J. A benefit-cost analysis of two-dose measles immunization in Canada. Vaccine. 1998;16:989–96. doi:10.1016/S0264-410X(97)00281-8.
Rivière M, Mha RT, Levinton C, Fitzsimon C, Leclerc C. Economic benefits of a routine second dose of combined measles, mumps and rubella vaccine in Canada. Can J Infect Dis Med Microbiol. 1997;8:257–64.
Brisson M, Pellissier JM, Camden S, Quach C, De Wals P. The potential cost-effectiveness of vaccination against herpes zoster and post-herpetic neuralgia. Hum Vaccin. 2008;4:238–45. doi:10.4161/hv.4.3.5686.
Najafzadeh M, Marra CA, Galanis E, Patrick DM. Cost effectiveness of herpes zoster vaccine in Canada. Pharmacoeconomics. 2009;27:991–1004. doi:10.2165/11314010-000000000-00000.
Coyle D, Coyle K, Bettinger JA, Halperin SA, Vaudry W, Scheifele DW, et al. Cost effectiveness of infant vaccination for rotavirus in Canada. Can J Infect Dis Med Microbiol. 2012;23:71–7.
Fisman DN, Chan CH, Lowcock E, Naus M, Lee V. Effectiveness and cost-effectiveness of pediatric rotavirus vaccination in British Columbia: a model-based evaluation. Vaccine. 2012;30:7601–7. doi:10.1016/j.vaccine.2012.10.034.
Shwiff S, Aenishaenslin C, Ludwig, Berthiaume P, Bigras-Poulin M, Kirkpatrick K, et al. Bioeconomic modelling of raccoon rabies spread management impacts in quebec, canada. Transbound Emerg Dis. 2013;60:330–7. doi:10.1111/j.1865-1682.2012.01351.x.
Bauch CT, Anonychuk AM, Pham BZ, Gilca V, Duval B, Krahn MD. Cost-utility of universal hepatitis A vaccination in Canada. Vaccine. 2007;25:8536–48. doi:10.1016/j.vaccine.2007.10.001.
Hutchison B, Stoddart G. Cost-effectiveness analysis of primary tetanus immunization among elderly Canadians. Can Med Assoc J. 1988;139:1143–51.
Krahn MD, John-Baptiste A, Yi Q, Doria A, Remis RS, Ritvo P, et al. Potential cost-effectiveness of a preventive hepatitis C vaccine in high risk and average risk populations in Canada. Vaccine. 2005;23:1549–58. doi:10.1016/j.vaccine.2004.09.023.
Tu HAT, Deeks SL, Morris SK, Strifler L, Crowcroft N, Jamieson FB, et al. Economic evaluation of meningococcal serogroup B childhood vaccination in Ontario, Canada. Vaccine. 2014;32:5436–46. doi:10.1016/j.vaccine.2014.07.096.
Raj S. Economic evaluation of five strategies for the prevention of neonatal group B streptococcal (GBS) disease in Alberta. Calgary: University of Calgary; 2002.
Lundkvist J, Steffen R, Jönsson B. Cost-benefit of WC/rBS oral cholera vaccine for vaccination against ETEC-caused travelers’ diarrhea. J Travel Med. 2009;16:28–34. doi:10.1111/j.1708-8305.2008.00270.x.
Neumann PJ, Greenberg D, Olchanski NV, Stone PW, Rosen AB. Growth and quality of the cost-utility literature, 1976–2001. Value Health. 2005;8:3–9. doi:10.1111/j.1524-4733.2005.04010.x.
Postma MJ, Westra TA, Quilici S, Largeron N. Economic evaluation of vaccines: specificities and future challenges illustrated by recent European examples. Expert Rev Vaccines. 2013;12:555–65. doi:10.1586/erv.13.36.
Newall A, Reyes J, Wood J, McIntyre P, Menzies R, Beutels P. Economic evaluations of implemented vaccination programmes: key methodological challenges in retrospective analyses. Vaccine. 2014;32:759–65. doi:10.1016/j.vaccine.2013.11.067.
Brisson M, Edmunds WJ. Economic evaluation of vaccination programs: the impact of herd-immunity. Med Decis Making. 2003;23:76–82. doi:10.1177/0272989X02239651.
Sassi F, Archard L, le Grand J. Equity and the economic valuation of Healthcare. Health Technol Assess. 2001;5:138. doi:10.3310/hta5030.
Cookson R, Mirelman AJ, Grif S, Asaria M, Dawkins B, Norheim OF, et al. Using cost-effectiveness analysis to address health equity concerns. 2017;20:206–12. doi:10.1016/j.jval.2016.11.027.
Baladi JF, Menon D, Otten N. Use of economic evaluation guidelines: 2 years’ experience in Canada. Health Econ. 1998;7:221–7.
All authors contributed to the conception and design of the review. ER conducted the literature search. ER and HG reviewed potential journals for inclusion/exclusion, ER conducted the analysis and interpretation of the data. MF and CW aided in the interpretation of the findings. ER drafted all versions of the manuscript. MF, CW and HG all critically revised the document. All authors read and approved the final manuscript.
We would like to thank Bruce McDonald for his expertise in reviewing some of the original articles and for his editorial advice throughout.
The authors declare that they have no competing interests.
Availability of data and materials
All data supporting the conclusions of this article were included within the article and list of references.
We received no funding for the development of this manuscript. Graduate student funding for ER was provided by the Natural Sciences and Engineering Research Council (NSERC) Collaborative Research and Training Experience (CREATE) grant to the University of Saskatchewan (U of S) for an Integrated Training Program in Infectious Diseases, Food Safety and Public Policy (ITraP) and the U of S Dean’s Scholarship.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.