Skip to main content

Methodological challenges surrounding QALY estimation for paediatric economic evaluation

Abstract

Cost-utility analysis remains the preferred form of economic evaluation for health technology assessment, pricing and reimbursement authorities in several countries. The results of cost-utility analyses are commonly expressed in terms of incremental cost per quality-adjusted life year (QALY) gained where the QALY combines length of life and health-related quality of life in a single metric. This commentary provides an overview of key methodological challenges surrounding QALY estimation for paediatric economic evaluation. These challenges include issues surrounding the relevant attributes to incorporate into measurement instruments, appropriate respondents for the measurement and valuation tasks, perspectives adopted when completing valuation tasks, potential sources of bias in the description and valuation processes, and the paucity of psychometric evidence for existing measures. In addition, the commentary considers methodological challenges raised by research aimed at assessing whether a QALY gain by a child should be valued equally to a QALY gain by an adult.

Introduction

Economic evaluation involves the comparative analysis of alternative programmes or interventions in terms of their costs and consequences [1]. Cost-utility analysis remains the preferred form of economic evaluation for health technology assessment (HTA), pricing and reimbursement authorities in several countries, including the Pharmaceutical Benefits Advisory Committee (PBAC) in Australia [2], the Canadian Agency for Drugs and Technologies in Health (CADTH) in Canada [3], the National Institute of Health and Care Excellence (NICE) in England and Wales [4], and the Scottish Medicines Consortium (SMC) in Scotland [5]. The results of cost-utility analyses are commonly expressed in terms of incremental cost per quality-adjusted life year (QALY) gained where the QALY combines length of life and health-related quality of life in a single metric [6]. For government agencies, the QALY has the advantage of facilitating comparisons of health outcomes across different health care interventions for disparate health conditions. It offers an additional advantage in that the approaches to valuing health-related quality of life outcomes, typically on a cardinal scale anchored at zero (representing death) and one (representing full health), capture people’s preferences for outcomes beyond those framed by a narrow biomedical perspective.

Health economists have developed a number of approaches for the measurement and valuation of preference-based health-related quality of life outcomes (or health utilities) for inclusion within the QALY metric. These include direct valuation methods using scaling techniques, such as the standard gamble (SG) and time trade-off (TTO) approaches, where the measurement and valuation of preferences occur in a single step [6]; health rating scales, such as the Visual Analogue Scale [1]; multi-attribute health status classification systems with preference scores, such as the EQ-5D [7], Health Utilities Index (HUI) [8], SF-6D [9], Quality of Well-Being Scale [10] and Assessment of Quality of Life (AQoL or AQoL-5D) [11]; mapping from non-preference-based measures onto generic preference-based measures of health [12]; and the development of de novo measures [13]. HTA agencies in several jurisdictions provide guidance on their preferred approach for the measurement and valuation of preference-based health-related quality of life outcomes for QALY estimation. In Canada, for example, CADHT accepts both the HUI Mark 2 and HUI Mark 3 as multi-attribute health status classification systems with preference scores for their reference case despite their differing attributes [3], whilst the EQ-5D is the preferred measure of the preference-based health-related quality of life outcomes of adults in England and Wales [4] and in Scotland [5]. In Australia, PBAC remains broadly agnostic regarding a preferred approach for the measurement and valuation of preference-based health-related quality of life outcomes, although a steer is provided to ensure that utility weights are applicable to the general Australian population [2]. Even in jurisdictions with a clearly preferred approach, there is recognition that a ubiquitous approach to the measurement and valuation of preference-based health-related quality of life outcomes may be inappropriate; for example, if there is qualitative empirical evidence on the lack of content validity for a preferred measure or if the preferred measure performs poorly in tests of construct validity and responsiveness in a particular population [4].

Methodological challenges surrounding measurement and valuation of health utilities in the paediatric context

The measurement and valuation of preference-based health-related quality of life outcomes for QALY estimation raises particularly acute methodological challenges in paediatric populations. Methodological concerns that are specific to childhood and adolescent populations include issues surrounding the relevant attributes to incorporate into measurement instruments, appropriate respondents for the measurement and valuation tasks, potential sources of bias in the description and valuation processes, and the paucity of psychometric evidence for existing measures [14]. These limitations have been mitigated to a degree by the development of generic childhood and adolescent-specific multi-attribute health status classification systems that generate preference-based scores. A previous review article [15] identified nine such measures validated for use across health conditions in mid and/or late childhood or in adolescence, namely the HUI Mark 2 [8], HUI3 Mark 3 [8], Child Health Utility 9D (CHU9D) [16], Assessment of Quality of Life 6 Dimension (AQoL-6D) [17], 16D [18], 17D [19], EQ-5D-Y [20], Quality of Well-Being scale (QWB) [21] and Assessment of Health Utility Measurement (AHUM) [22]. Furthermore, research is ongoing to develop multi-attribute health status classification systems that generate preference-based scores for infancy [23] or targeted at specific childhood conditions, such as excess weight [24] or oral health [25]. A recent review of PBAC public summary documents in Australia that considered funding decisions around medicines used by children found that decision-making uncertainty would have been reduced or potentially reduced for approximately 85% of the medicines considered if generic childhood and adolescent-specific multi-attribute health status classification systems that generate preference-based scores had been available and/or used [26].

Each of the generic childhood and adolescent-specific multi-attribute health status classification systems that generate preference-based scores differ in choice of attributes or domains and their conceptual underpinnings, valuation protocol, choice of informant, perspective adopted by the respondent, appropriateness for each developmental stage of childhood and adolescence, and formatting and mode of administration, which is likely to independently impact on the health utility values that are generated. The domain coverage of the Infant health-related Quality of Life Instrument (IQI) includes sleeping, feeding and breathing [23], whilst that for the AHUM includes concerns facing adolescents such as self-image and health perceptions [22]. The choice of informant for the task of describing the child’s health status should take account of poor inter-rater agreement between children’s self-reports and parent-proxy reports for subjective attributes such as cognition, emotion and pain that have been identified by many studies [27]. Furthermore, the development of generic childhood and adolescent-specific multi-attribute health status classification systems that generate preference-based scores raises the normative question of whose values should underpin the preference weights for paediatric health states. Many health economists argue that representative samples of the general population should be asked to act as the social decision-maker when eliciting preferences for health states that can inform cost-effectiveness based decision-making [28]. Recent research has explored whether adult and adolescent preferences differ when asked to value EQ-5D-Y-3 L health states [29, 30]. In practice, however, samples of adults have largely been used to derive value sets for generic childhood and adolescent-specific multi-attribute health status classification systems that generate preference-based scores [15, 31]. This raises the question of which perspective they should adopt when asked to value a childhood or adolescent health state. Adopting different perspectives, for example that of a hypothetical child, themselves as a child, themselves at the current age but experiencing that state, or that of another adult experiencing that state, can affect their preference structures [30, 32]. Arguably, there is scope for incorporating the preferences of individuals generally excluded from social decision-making deliberations, such as of children, into the processes of generating the value sets for generic childhood and adolescent-specific multi-attribute health status classification systems that generate preference-based scores. However, even if it’s accepted that children’s values are valid, approaches for capturing those values have been stymied by tools that require developed cognitive abilities and linguistic skills and articulation of preferences for health states using complex concepts of gain and loss in health economic terms [33]. Their values might also be influenced by specific design features of the preference elicitation task, for example, the valuation protocol, mode of administration or method of anchoring on the 0–1 (death to full health) utility scale [14]. Recent interest has considered the use of mixed samples of adolescents and adults to value child and adolescent health states [34].

None of the generic childhood and adolescent-specific multi-attribute health status classification systems that generate preference-based scores is validated for use across all stages of childhood and adolescence, which raises concerns over comparability of their value sets. Furthermore, attempts at mapping between these value sets have been limited to a small number of measures applied in narrow age banded groups [35]. Particular concerns arise when utility values generated by these measures act as inputs into economic evaluations with time horizons extending across the life course. Paediatric evaluations that adopt a life-time horizon typically value health states using a common measure or valuation protocol [36], overlooking the inherent methodological limitations of this approach. Ultimately, methodological guidance from health technology assessment agencies may be required to inform preferred measures that should be applied at different stages of life, supplemented by evidence from mapping studies that accurately predict the relationships between those measures.

Should a QALY gain by a child be valued equally to a QALY gain by an adult?

Beyond challenges surrounding methods for measuring and valuing health utilities in or on behalf of children and adolescents, a separate set of challenges surround whether overall measures of health consequence, such as an additional QALY, should be valued equally across childhood, adolescent and adult populations. The revealed preference literature has generated evidence that indicates that the value individuals place on reducing health risks or achieving health gains, usually expressed in terms of a monetary value of statistical life, is higher for children than for adults [37]. However, the values estimated in the revealed preference literature are largely based on choices made by parents on the part of family members and, consequently reflect, at least in part, altruistic concerns. Furthermore, revealed preference studies tend to be based on evidence from individuals who are poorly informed about the differential health risks associated with the choices they face. They also provide limited evidence for decisions made in health care systems where care is provided at zero or subsidised prices at the point of use. Perhaps more pertinent evidence for those concerned with deriving social, as opposed to individual, values for age weights for QALYs is provided by stated preference studies. Notably, stated preference studies in the health context provide some evidence of an inverted-U shaped relation between age and the value placed on health gain [37, 38], consistent with the relationship between age and the value of statistical life identified in labour market studies [39]. A large stated preference study in this area surveyed a nationally representative sample (n = 587) of the population in England using two preference elicitation techniques, a discrete choice experiment and a ‘matching’ (or person trade-off) approach. The former revealed that age did not have a strong impact on respondents’ choices over and above the health (QALY) gains presented, while the latter revealed a general tendency to give greater weight to 20- to 40-year olds over other age groups (0- to 20-, 40- to 60- and 60- to 80-year olds) [40, 41]. More recent research in this area has considered age together with broader distributional concerns, for example socioeconomic status or baseline severity of illness, for priority setting [42]. Notably, however, there is no consensus around a number of methodological features of stated preference studies that aim to generate age weights for QALYs, including source(s) for preferences, appropriate dimensions for the health-related quality of life component of the QALY measure, preferred valuation protocol, and methods for controlling for context and design effects on derived values. Moreover, there are operational concerns surrounding the application of age weights for QALYs within economic evaluations that extend in time horizon across the life course. In particular, systems that rely on a cost-effectiveness threshold to inform decision-making will ultimately need to consider how they apply age-related weights to the health benefits foregone as a result of displaced activities [37].

Conclusions

In conclusion, this paper has provided an overview of key methodological challenges surrounding QALY estimation for paediatric economic evaluation. This is in area of active research enquiry, for example, by the TORCH (https://torch.hykecreative.com.au) and QUOKKA (https://quokkaresearchprogram.org) research programmes in Australia. These research programmes are assessing the relative merits of alternative measurement and valuation approaches, developing new preference-based value sets for existing generic childhood and adolescent-specific measures, and assessing the comparability of QALYs across childhood and adolescent populations. Ultimately, this evidence should generate methodological advancements that inform cost-effectiveness based decision making by health technology assessment, pricing and reimbursement authorities.

Availability of data and materials

Not applicable.

References

  1. Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. 4th ed. Oxford: Oxford University Press; 2015.

    Google Scholar 

  2. PBAC (Pharmaceutical Benefits Advisory Committee). Guidelines for preparing a submissions to the Pharmaceutical Benefits Advisory Committee. 5th ed. Canberra: Australian Government Department of Health; 2016.

    Google Scholar 

  3. CADTH (Canadian Agency for Drugs and Technologies in Health). Guidelines for the economic evaluation of health technologies. 4th ed. Ottowa: CADTH; 2017.

    Google Scholar 

  4. NICE (National Institute of Health and Care Excellence). Guide to the methods of technology appraisal. London: NICE; 2013.

    Google Scholar 

  5. SMC (Scottish Medicines Consortium). Guidance to manufacturers for completion of new product assessment form (NPAF). Glasgow: NHS Scotland; 2017.

    Google Scholar 

  6. Torrance GW, Feeny D. Utilities and quality-adjusted life years. Int J Technol Assess. 1989;5:559–75.

    CAS  Article  Google Scholar 

  7. Brooks R. EQ-5D, the current state of play. Health Policy. 1996;37:53–72.

    CAS  Article  Google Scholar 

  8. Torrance GW, Furlong W, Feeny D, Boyle M. Multi-attribute preference functions Health Utilities Index.  Pharmacoeconomics. 1995;7(6):503–20.

    CAS  Article  Google Scholar 

  9. Brazier J, Roberts J, Deverill M. The estimation of a preference-based measure of health from the SF-36. J Health Econ. 2002;21(2):271–92.

    Article  Google Scholar 

  10. Kaplan RM, Bush JW, Berry CC. Health status: types of validity and the index of well-being. Health Serv Res. 1976;11:478–507.

    CAS  PubMed  PubMed Central  Google Scholar 

  11. Hawthorne G, Richardson J, Osborne R. The Assessment of Quality of Life (AQoL) instrument: a psychometric measure of health-related quality of life. Qual Life Res. 1999;8:209–24.

    CAS  Article  Google Scholar 

  12. Brazier JE, Yang Y, Tsuchiya A, Rowen DL. A review of studies mapping (or cross walking) non-preference based measures of health to generic preference-based measures. Eur J Health Econ. 2010;11(2):215–25.

    Article  Google Scholar 

  13. Brazier J, Ratcliffe J, Salomon JA, Tsuchiya A. Measuring and valuing health benefits for economic evaluation. 2nd ed. Oxford: Oxford University Press; 2017.

    Google Scholar 

  14. Petrou S. Methodological issues raised by preference-based approaches to measuring the health status of children. Health Econ. 2003;12(8):697–702.

    Article  Google Scholar 

  15. Chen G, Ratcliffe J. A review of the development and application of generic multi-attribute utility instruments for paediatric populations. Pharmacoeconomics. 2015;33(10):1013–28.

    Article  Google Scholar 

  16. Stevens K. Developing a descriptive system for a new preference-based measure of health-related quality of life for children. Qual Life Res. 2009;18(8):1105–13.

    Article  Google Scholar 

  17. Richardson J, Peacock S, Hawthorne G, Iezzi A, Elsworth G, Day N. Construction of the descriptive system for the assessment of quality of life AQoL-6D utility instrument. Health Qual Life Out. 2012;10(1):38.

    Article  Google Scholar 

  18. Apajasalo M, Sintonen H, Holmberg C, Sinkkonen J, Aalberg V, Pihko H, et al. Quality of life in early adolesence: a sixteen-dimensional health-related measure (16D). Qual Life Res. 1996;5:205–11.

    CAS  Article  Google Scholar 

  19. Apajasalo M, Rautonen J, Holmberg C, Sinkkonen J, Aalberg V, Pihko H, et al. Quality of life in pre-adolescence: a 17-dimensional health-related measure (17D). Qual Life Res. 1996;5(6):532–8.

    CAS  Article  Google Scholar 

  20. Wille N, Badia X, Bonsel G, Burstrom K, Cavrini G, Devlin N, et al. Development of the EQ-5D-Y: a child-friendly version of the EQ-5D. Qual Life Res. 2010;19(6):875–86.

    Article  Google Scholar 

  21. Seiber WJ, Groessl EJ, David KM, Ganiats TG, Kaplan RM. Quality of Well Being Self-Administered (QWB-SA) Scale User’s Manual. San Diego: Health Services Research Center, University of California; 2008.

    Google Scholar 

  22. Beusterien KM, Yeung JE, Pang F, Brazier J. Development of the multi-attribute Adolescent Health Utility Measure (AHUM). Health Qual Life Out. 2012;10:102.

    Article  Google Scholar 

  23. Jabrayilov R, Vermeulen KM, Detzel P, Dainelli L, van Asselt ADI, Krabbe PFM. Valuing health status in the first year of life: the infant health-related quality of life instrument. Value Health. 2019;22(6):721–7.

    Article  Google Scholar 

  24. Oluboyede Y, Robinson T. Measuring weight-specific quality of life in adolescents: an examination of the concurrent validity and test-retest reliability of the WAItE. Value Health. 2019;22(3):348–54.

    Article  Google Scholar 

  25. Kularatna S, Amilani U, Senanayake S, Tonmukayakul U, Jamieson L, Arrow P. Developing an early childhood oral health impact-specific health-state classification system for a new preference-based instrument, the ECOHIS-4D. Community Dent Oral. 2021. https://doi.org/10.1111/cdoe.12650.

  26. Bailey C, Dalziel K, Cronin P, Devlin N, Viney R. Quality Of Life in Kids: Key Evidence to Strengthen Decisions in Australia (QUOKKA) Project Team. How are child-specific utility instruments used in decision making in Australia? A review of Pharmaceutical Benefits Advisory Committee public summary documents. Pharmacoeconomics. 2022;40(2):157–82.

    Article  Google Scholar 

  27. Khadka J, Kwon J, Petrou S, Lancsar E, Ratcliffe J. Mind the (inter-rater) gap. An empirical investigation of self-reported versus proxy-reported assessments in the derivation of childhood utility values for economic evaluation. Soc Sci Med. 2019;240:112543.

    Article  Google Scholar 

  28. Dolan P, Olsen JA, Menzel P, Richardson J. An inquiry into the different perspectives that can be used when eliciting preferences in health. Health Econ. 2003;12(7):545–51.

    Article  Google Scholar 

  29. Mott DJ, Shah KK, Ramos Goni JM, Devlin NJ, Rivero-Arias O. Valuing EQ-5D-Y-3L health states using a discrete choice experiment: Do adult and adolescent preferences differ? Med Decis Making. 2021;41(5):584–96.

    Article  Google Scholar 

  30. Ramos Goni JM, Oppe M, Stolk E, Shah KK, Kreimeier S, Rivero-Arias O, Devlin NJ. International valuation protocol for the Eq. 5DY3L. Pharmacoeconomics. 2020;38:653–63.

    Article  Google Scholar 

  31. Kwon J, Freijser L, Huynh E, Howell M, Chen G, Khan KA, et al. Systematic review of conceptual, age, measurement and valuation considerations for generic multidimensional childhood patient-reported outcome measures. Pharmacoeconomics. 2022. https://doi.org/10.1007/s40273-021-01128-0.

  32. Powell PA, Rowen D, Rivero-Arias O, Tsuchiya A, Brazier JE. Valuing child and adolescent health: a qualitative study on different perspectives and priorities taken by the adult general public. Health Qual Life Out. 2021;19(1):222. https://doi.org/10.1186/s12955-021-01858-x.

    Article  Google Scholar 

  33. Thorrington D, Eames K. Measuring health utilities in children and adolescents: a systematic review of the literature. PLoS ONE. 2015;10:e0135672.

    Article  Google Scholar 

  34. Rowen D, Mukuria C, Powell P, Wailoo A, Wong R. Valuing child health: exploring the use of a mixed sample of adolescents and adults to value child and adolescent health states. School of Health and Related Research. University of Sheffield: NICE Decision Support Unit Report. 2021.

  35. Dakin H, Abel L, Burns R, Yang Y. Review and critical appraisal of studies mapping from quality of life or clinical measures to EQ-5D: an online database and application of the MAPS statement. Health Qual Life Out. 2018;16(1):31.

    Article  Google Scholar 

  36. Kromm SK, Bethell J, Kraglund F, Edwards SA, Laporte A, Coyte PC, et al. Characteristics and quality of pediatric cost-utility analyses. Qual Life Res. 2012;21(8):1315–25.

    Article  Google Scholar 

  37. Petrou S. Methodological and applied concerns surrounding age-related weighting within health economic evaluation. Expert Rev Pharm Out. 2014;14(5):729–40.

    Google Scholar 

  38. Petrou S, Kandala NB, Robinson A, Baker R. A person trade-off study to estimate age-related weights for health gains in economic evaluation. Pharmacoeconomics. 2013;31(10):893–907.

    Article  Google Scholar 

  39. Aldy JE, Viscusi WK. Age differences in the value of statistical life: revealed preference evidence. Rev Env Econ Policy. 2007;1(2):241–60.

    Article  Google Scholar 

  40. Baker R, Bateman I, Donaldson C, Jones-Lee M, Lancsar E, Loomes G, et al, SVQ Research Team. Weighting and valuing quality-adjusted life-years using stated preference methods: preliminary results from the Social Value of a QALY Project. Health Technol Assess. 2010;14(27):1–162.

    CAS  Article  Google Scholar 

  41. Lancsar E, Wildman J, Donaldson C, Ryan M, Baker R. Deriving distributional weights for QALYs through discrete choice experiments. J Health Econ. 2011;30(2):466–78.

    Article  Google Scholar 

  42. Gu Y, Lancsar E, Ghijben P, Butler JR, Donaldson C. Attributes and weights in health care priority setting: a systematic review of what counts and to what extent. Soc Sci Med. 2015;146:41–52.

    Article  Google Scholar 

Download references

Acknowledgements

SP would like to thank the co-investigators of the TORCH research programme for the discussions that helped formulate his thinking in this area.

Funding

SP receives support as a UK National Institute for Health Research (NIHR) Senior Investigator (NF-SI-0616-10103) and from the UK NIHR Applied Research Collaboration Oxford and Thames Valley. He contributes to the TORCH research programme funded by the Australian Government’s Medical Research Future Fund (Grant Number: APP1199902).

Author information

Authors and Affiliations

Authors

Contributions

SP had sole responsibility for writing this commentary and for the material contained within this commentary. The author read and approved the final manuscript. 

Corresponding author

Correspondence to Stavros Petrou.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The author declares that he has no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Petrou, S. Methodological challenges surrounding QALY estimation for paediatric economic evaluation. Cost Eff Resour Alloc 20, 10 (2022). https://doi.org/10.1186/s12962-022-00345-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12962-022-00345-4

Keywords

  • QALYs
  • Quality-adjusted life years
  • Paediatrics
  • Childhood
  • Methods
  • Valuation