EVIDEM: an adaptable pragmatic open-source MCDA-based framework for accountable healthcare decisionmaking and priority setting
Mireille Goetghebeur, BioMedCom
20 October 2011
We would like to thank Youngkong et al for providing an opportunity to clarify a few important points regarding the EVIDEM framework (1,2). In their commentary on the "usefulness of the EVIDEM framework for priority setting across a broad range of interventions", Youngkong et al report that "the EVIDEM framework ignores the contextual nature of priority setting". The EVIDEM framework is intended to be adapted to local context, and actually includes a Contextual Tool to facilitate this process (www.evidem.org). The Contextual Tool prompts decisionmakers to identify their specific priorities and feasibility issues, and facilitates integration of these specific criteria into the framework. The framework is currently being tested and adapted by decisionmakers in several jurisdictions around the world (e.g, by a district health board in New Zealand, with adaptation, among other things, for specific needs of the Maori population), and some of the work has been published or presented (3-5).
The commentary also states that "the framework requires different expert panel to assess the intervention of every single intervention separately". We do not assert that each decision be made by different panels. The framework is actually intended to be adapted and then applied by standing committees, allowing a consistent approach to evaluate, rank, and prioritize a broad range of healthcare interventions on a scale that reflects the criteria deemed relevant by the decisionmaking body (3). The suggestion of Youngkong et al to develop a meaningful set of criteria by consulting relevant stakeholders is thus in full agreement with what we propose.
The criteria and the simple weight elicitation method of the framework are proposed as a starting point to facilitate the development of a tool tailored to decisionmakers (e.g. testing in Canada [5] and South Africa [4]). Knowledge and understanding of decision criteria is currently being explored by an international research team through a literature review and a survey of criteria in use by healthcare decisionmakers globally (6). A number of other weight elicitation methods can be used and another research team is currently comparing the method proposed for EVIDEM with more standard MCDA methods such as Analytical Hierarchy Process (AHP).
Pragmatism and accountability constitute the basic underlying ethos of the framework. It was designed to reflect the complexity of real-life decisionmaking and support systematic consideration of the numerous criteria upon which healthcare decisionmaking and prioritization is based. The DCE approach proposed by Youngkong et al has some limitation in the number of criteria that can be incorporated, so its use may be limited to decisionmakers who need to incorporate a maximum of six or seven criteria in their evaluations and prioritization processes.
Academic research and feedback from users and testers is integrated into the open access EVIDEM instruments on an ongoing basis to make them more useful to stakeholders for their various applications. The objectives of the EVIDEM collaboration, an independent not-for-profit organization run by an international board of directors, are to enhance public health by development of sound MCDA-based approaches. It is a collaborative endeavour and we appreciate discussions and efforts to facilitate application of MCDA-based frameworks for efficient decisionmaking and priority setting.
References
1 Goetghebeur MM, Wagner M, Khoury H, Levitt RJ, Erickson LJ, Rindress D. Evidence and value: impact on decisionmaking - the EVIDEM framework and potential applications. BMC Health Services Research 2008, 8:270.
2 Goetghebeur MM, Wagner M, Khoury H, Rindress D, Gregoire JP, Deal C. Combining multicriteria decision analysis, ethics and health technology assessment: applying the EVIDEM decisionmaking framework to growth hormone for Turner syndrome patients. Cost Eff Resour Alloc. 2010;8(1):4.
3 Goetghebeur MM, Wagner M, Khoury H, Levitt R, Erickson LJ, Rindress D. Bridging multicriteria decision analysis (MCDA) and health technology assessment (HTA) for efficient healthcare decisionmaking: proof of concept study applying the EVIDEM framework to medicines appraisal. Medical Decision Making - In press. 2011.
4 Miot J, Wagner M, Khoury H, Anderson AN, Rindress D, Goetghebeur M. Field testing of a multi criteria decision analyses (MCDA) framework for coverage of a screening test for cervical cancer in South Africa. ISPOR European Meeting, Paris; 2009 Oct.
5 Tony M, Wagner M, Khoury H, Rindress D, Papastavros T, Oh P, et al. Bridging health technology assessment (HTA) with multicriteria decision analyses (MCDA): field testing of the EVIDEM framework for coverage decisions by a public payer in Canada. BMC Health Services Research - under revision. 2011.
6 Guindo LA, Wagner M, Baltussen R, Rindress D, van Til J, Kind P, Goetghebeur M. Which criteria are used in health care decisionmaking and priority setting? A literature review for an international survey of decisionmakers. ISPOR Annual Meeting, Baltimore; 2011 May.
Competing interests
No competing interest. Mireille Goetghebeur is currently serving on a volunteer basis as President of the EVIDEM Collaboration which is a not-for-profit organization. She is Vice-President at BioMedCom Consultants and Associate Member of the Research Center of the University Hospital Center Ste Justine in Montreal.
EVIDEM: an adaptable pragmatic open-source MCDA-based framework for accountable healthcare decisionmaking and priority setting
20 October 2011
We would like to thank Youngkong et al for providing an opportunity to clarify a few important points regarding the EVIDEM framework (1,2). In their commentary on the "usefulness of the EVIDEM framework for priority setting across a broad range of interventions", Youngkong et al report that "the EVIDEM framework ignores the contextual nature of priority setting". The EVIDEM framework is intended to be adapted to local context, and actually includes a Contextual Tool to facilitate this process (www.evidem.org). The Contextual Tool prompts decisionmakers to identify their specific priorities and feasibility issues, and facilitates integration of these specific criteria into the framework. The framework is currently being tested and adapted by decisionmakers in several jurisdictions around the world (e.g, by a district health board in New Zealand, with adaptation, among other things, for specific needs of the Maori population), and some of the work has been published or presented (3-5).
The commentary also states that "the framework requires different expert panel to assess the intervention of every single intervention separately". We do not assert that each decision be made by different panels. The framework is actually intended to be adapted and then applied by standing committees, allowing a consistent approach to evaluate, rank, and prioritize a broad range of healthcare interventions on a scale that reflects the criteria deemed relevant by the decisionmaking body (3). The suggestion of Youngkong et al to develop a meaningful set of criteria by consulting relevant stakeholders is thus in full agreement with what we propose.
The criteria and the simple weight elicitation method of the framework are proposed as a starting point to facilitate the development of a tool tailored to decisionmakers (e.g. testing in Canada [5] and South Africa [4]). Knowledge and understanding of decision criteria is currently being explored by an international research team through a literature review and a survey of criteria in use by healthcare decisionmakers globally (6). A number of other weight elicitation methods can be used and another research team is currently comparing the method proposed for EVIDEM with more standard MCDA methods such as Analytical Hierarchy Process (AHP).
Pragmatism and accountability constitute the basic underlying ethos of the framework. It was designed to reflect the complexity of real-life decisionmaking and support systematic consideration of the numerous criteria upon which healthcare decisionmaking and prioritization is based. The DCE approach proposed by Youngkong et al has some limitation in the number of criteria that can be incorporated, so its use may be limited to decisionmakers who need to incorporate a maximum of six or seven criteria in their evaluations and prioritization processes.
Academic research and feedback from users and testers is integrated into the open access EVIDEM instruments on an ongoing basis to make them more useful to stakeholders for their various applications. The objectives of the EVIDEM collaboration, an independent not-for-profit organization run by an international board of directors, are to enhance public health by development of sound MCDA-based approaches. It is a collaborative endeavour and we appreciate discussions and efforts to facilitate application of MCDA-based frameworks for efficient decisionmaking and priority setting.
References
1 Goetghebeur MM, Wagner M, Khoury H, Levitt RJ, Erickson LJ, Rindress D. Evidence and value: impact on decisionmaking - the EVIDEM framework and potential applications. BMC Health Services Research 2008, 8:270.
2 Goetghebeur MM, Wagner M, Khoury H, Rindress D, Gregoire JP, Deal C. Combining multicriteria decision analysis, ethics and health technology assessment: applying the EVIDEM decisionmaking framework to growth hormone for Turner syndrome patients. Cost Eff Resour Alloc. 2010;8(1):4.
3 Goetghebeur MM, Wagner M, Khoury H, Levitt R, Erickson LJ, Rindress D. Bridging multicriteria decision analysis (MCDA) and health technology assessment (HTA) for efficient healthcare decisionmaking: proof of concept study applying the EVIDEM framework to medicines appraisal. Medical Decision Making - In press. 2011.
4 Miot J, Wagner M, Khoury H, Anderson AN, Rindress D, Goetghebeur M. Field testing of a multi criteria decision analyses (MCDA) framework for coverage of a screening test for cervical cancer in South Africa. ISPOR European Meeting, Paris; 2009 Oct.
5 Tony M, Wagner M, Khoury H, Rindress D, Papastavros T, Oh P, et al. Bridging health technology assessment (HTA) with multicriteria decision analyses (MCDA): field testing of the EVIDEM framework for coverage decisions by a public payer in Canada. BMC Health Services Research - under revision. 2011.
6 Guindo LA, Wagner M, Baltussen R, Rindress D, van Til J, Kind P, Goetghebeur M. Which criteria are used in health care decisionmaking and priority setting? A literature review for an international survey of decisionmakers. ISPOR Annual Meeting, Baltimore; 2011 May.
Competing interests
No competing interest. Mireille Goetghebeur is currently serving on a volunteer basis as President of the EVIDEM Collaboration which is a not-for-profit organization. She is Vice-President at BioMedCom Consultants and Associate Member of the Research Center of the University Hospital Center Ste Justine in Montreal.