Skip to main content

Topic identification, selection, and prioritization for health technology assessment in selected countries: a mixed study design



There is limited evidence-informed guidance on TISP processes for countries where health technology assessment (HTA) is in a nascent phase. We aimed to explore the range of topic identification, selection and prioritization (TISP) processes and practices for HTA in selected countries and identify aspects relevant to emerging HTA systems.


This mixed design study included a systematic literature review, an electronic survey, and individual interviews. We conducted a systematic literature review with criteria that were developed a priori to identify countries deemed to have a recently formalized HTA system. Based on the literature review, a twenty-three item online survey was shared with the identified countries, we completed follow-up interviews with ten participants who have experience with HTA. We analyzed documents, survey responses and interview transcripts thematically to identify lessons related to TISP processes and practices.


The literature review identified 29 nine candidate countries as having a “potential” recently formalized HTA system. Twenty-one survey responses were analyzed and supplemented with ten individual interviews. We found variation in countries’ approaches to TISP — particularly between pharmaceutical and non-pharmaceutical interventions. Results indicate that TISP is heavily driven by policy makers, expert involvement, and to a lesser extent, relevant stakeholders. The use of horizon-scanning and early warning systems is uncommon. Interviewee participants provided further insight to the survey data, reporting that political awareness and an institutional framework were important to support TISP. TISP can be optimized by stronger national regulations and legislative structures, in addition to education and advocacy about HTA among politicians and decision-makers. In some settings regional networks have been useful, particularly in the development of TISP guidelines and methodologies. Additionally, the technical capacity to conduct TISP, and access to relevant local data were factors limiting TISP in national settings. Increased network collaboration and capacity building were reported as future needs.


This study provides current insights into a topic where there is limited published peer reviewed literature. TISP is an important first step of HTA, and topics should be selected and prioritized based on local need and relevance. The limited capacity for TISP in settings where HTA is emerging may be supported by local and international collaboration to increase capacity and knowledge. To succeed, both TISP and HTA need to be embedded within national health care priority setting and decision-making. More in-depth understanding of where countries are situtated in formalizing the TISP process may help others to overcome factors that facilitate or hinder progress.


Many countries are adopting Health Technology Assessment (HTA) to make evidence-informed decisions about health technologies and interventions, and to support universal health coverage (UHC) [1, 2]. HTA is a multidisciplinary process using “explicit methods to determine the value of a health technology at different points in its lifecycle” [3]. Topic identification, selection, and prioritization (TISP) is the first step of the HTA process.

The EUR-ASSESS project (1997) conceptualized and suggested considerations for HTA topic prioritization [4]. Soon thereafter several established agencies (mainly in high-income countries) provided information online regarding their TISP processes, including criteria and stakeholder involvement [5, 6]. A recent review of the International Network of Agencies for Health Technology Assessment (INAHTA) member agencies’ approaches to topic selection indicated that most well-formalized agencies follow a six-step process: (i) Specification of criteria for topic selection, (ii) Identification of topics, (iii) Shortlisting of potential topics, (iv) Scoping of potential topics, (v) Scoring and ranking of potential topics, and (vi) Deliberation and decision-making on final topics for HTA [5].

For policy makers in resource-limited countries, where HTA is relatively new or where there may not be the necessary supporting institutional mechanisms, or resources, there is limited evidence-informed guidance on the optimal implementation of TISP processes [7]. HTA often reflects different factors within a country, for example, data usage culture, capacities, priorities and organizational arrangements. Consideration of political, economic, social, and cultural factors can be important for the development of evidence-informed processes and could be relevant when deciding what type of TISP approach a country could adopt [8, 9]. HTA is a resource intensive process; and a transparent and rigorous process for the identification, selection and prioritization of topics can result in better utilization of an HTA agencies’ resources [10, 11]. However, standardized approaches to TISP can be underutilized in some countries, potentially undermining the legitimacy of HTA decisions. Additionally, adopting complex TISP approaches may not be a suitable strategy for all HTA systems. Current guidance for countries establishing an HTA program suggests a pragmatic process at the beginning [12], rather than a more intentional approach to TISP, which can minimize its relevance within the program.

Given the importance that the nomination and selection of topics for assessment has as the first step of an HTA process [10, 13], there is value in investigating and describing best practices and lessons that may be transferrable to a variety of HTA systems. We hypothesized that there are common TISP elements across countries, and a focus on TISP processes can provide important insights into how a country’s HTA system is meeting the health sector’s needs. Therefore, the study's objective was to explore the range of TISP processes and practices for HTA in selected countries and identify aspects relevant to emerging HTA systems.


This study comprised a systematic literature review, a survey, and interviews. For the purposes of this study, we defined a formalized HTA system as being one where HTA is set up at the national or regional level to work in a predefined manner, with transparent process steps, and with a clear commission to support decisions applicable to the access, financing, and coverage of health services. In contrast, we defined an emerging HTA system to describe a setting that’s in the early stages of developing and implementing HTA processes and infrastructure. The term "emerging" implies that the system is actively working towards establishing a functional and comprehensive HTA system, but may still be in the process of developing policies and guidelines, building capacity, and conducting assessments.

Framework for the study

We developed a framework to interpret and analyze the data, to guide the formulation of survey questions, and to facilitate the integration of existing knowledge into the study. The framework was influenced by the EuroScan Toolkit, as reported by the European network for HTA (EUnetHTA) [7, 14], and further developed using studies identified from the systematic literature review, combined with the team’s expertise. The framework is not prescriptive, and it focus on common elements that are likely to apply in all countries regardless of the stage of HTA institutionalization within the country. It has three components: Topic identification, Topic selection, and Topic prioritization (Table 1).

Table 1 Detailed TISP framework (adapted from [15])

Systematic literature review

The systematic literature review was conducted to identify and describe TISP options and practices to inform the cross-sectional survey questions, and to identify countries in resource-limited settings (Low and middle income (LMIC)), with a formalized HTA system (as defined above) which we could invite to participate in the survey. Full details of the systematic review methods can be found in the project plan and protocol (Additional file 1: S1), and are reported in a previous publication [15]. We also searched PubMed and Scopus (Additional file 2: S2) (multiple searches were conducted between 6 October 2020 and 14 April 2021) and HTA websites (Additional file 1: S1 and Additional file 2: S2).


We developed a 23-question survey to identify what processes and practices formalized HTA systems use for TISP (Table 2, Additional file S3), as defined above (Additional file 1: S1). The survey used closed, single or multiple select nominal questions. Many questions included an ‘other’ option for respondents to describe their answer in their own words and provide open-ended responses.

Table 2 Measurement outcomes and related survey questions

The target population for both the survey and follow up interviews were individuals who were familiar with their national HTA system. We approached at least one person who we assumed to be familiar with the HTA system, identified through the authors’ network of contacts or contact information on scientific publications. Familiarity with the context was confirmed by the question “Do you consider yourself to have the necessary experience and understanding of the HTA system in the country to respond to questions about TISP processes.” The survey is reported according to the Checklist for Reporting of Survey Studies (CROSS) [16] (Additional file 4: S4).

Survey data collection

The survey was piloted by colleagues who had not been involved in its development. The questionnaire took 20–30 min to complete and was hosted on the online SurveyMonkey platform (Momentive Inc software company, United States) ( All operational definitions (e.g., formalized HTA system) were included in the survey (Additional file 3: S3).

In April 2021, we sent email invitations to 48 individuals associated with 29 countries (Table 3, Additional file 5: S5, Additional file 6: S6) with three follow-up emails to non-responders. Invitees were asked to forward the email to the appropriate person within their organizations if they did not have knowledge in specific areas. No incentives were offered for participation in the survey.

Table 3 Countries invited to participate and response in survey and follow up interview

Survey analysis

We prepared simple descriptive statistics using the quantitative survey data and analyzed the written survey responses thematically, looking for lessons related to TISP processes and practices. Responses were anonymized and reviewed for clarity and completeness. The initial results were presented to survey respondents during a virtual workshop June 2021, the discussion at this meeting was incorporated into the results.

Follow-up interviews

To gain further insight into the survey results we conducted follow-up interviews during April–May 2023. We sent a single email invitation with a summary of the survey results to 27 countries (39 individuals) asking them to join a follow-up meeting (Ukraine was not invited to interview due to the Russian invasion, and we had not secured a respondent from Mexico in the survey, so they too were excluded). Four either declined, did not provide a written response, or agreed after the deadline. Fourteen did not respond (Table 3). Reminder emails were sent to four individuals with the aim of including follow-up interviews from all regions included in the survey. Individuals who accepted the invitation were sent an email with the summary of results and the interview guide (Additional file 7: S7, Additional file S8). JB and EP conducted ten interviews (nine on Microsoft Teams and one on Zoom) in English using open-ended questions. Interviews were 36–58 min in length and participants consented to interviews being recorded. The recordings were transcribed, and thematic analysis was performed by one author (JB) and audited by a second (EP). No incentives were offered to the interviewees.


Systematic literature review results

The literature search retrieved 1094 records. After title and abstract screening, we classified 294 records by region (Africa, Asia, Latin America, and Europe) and country and included 64 records with some information on the status of an HTA system. The only LMICs that met our definition of a formalized HTA system were the Islamic Republic of Iran, Tunisia, and Ukraine. To align with our aim to explore the range of TISP processes and practices, therefore, we expanded the list of eligible countries to include Upper middle-income countries, or those with recently formalized HTA systems (e.g., former members of the Union of Soviet Socialist Republics), as we anticipated that they could have more transferable experience relevant to emerging HTA systems. Twenty-nine candidate countries were identified as having a “potential” recently formalized HTA system. One was from Africa, five from Asia Pacific, 14 from Europe, seven from Latin America, and two from the North Africa & Middle East region (Table 3). For some reports, the information was not detailed enough for the authors to judge whether the HTA system was formalized (Additional file 6: S6).

Survey and interview results

Three of the 29 countries identified for the survey were categorized as middle income, three were lower middle income, 12 were upper middle income, and 14 were high income (according to the 2021 World Bank development indicators [17] (Fig. 1, Table 3).

Fig. 1
figure 1

Geographical distribution of survey respondents

Respondent characteristics

We received 23 survey responses from 21 countries (response rate 72%). One respondent indicated that they did not have experience and understanding of the HTA system in their country and discontinued the survey. Due to emailing several people from the same country, we received four responses from two countries, one respondent requested have their submission removed at the end of the survey. For the other country (that also had two responses), the responses are reported individually as there is no way to verify if one response was more accurate than the other. An additional three respondents were unsure if their country had a formalized HTA system, however we included their responses in the results because the country had been identified as eligible from our literature review (Table 3, Additional file 3: S3).

Respondents for the follow-up interviews

We recruited ten individuals for the follow-up interviews. The participants were from Africa (n = 2), Asia Pacific (n = 2), Europe (n = 4), and Latin America (n = 2).

HTA system background information

Figure 2 shows the reported types of technologies assessed in the HTA systems and Table 4 shows the number of assessments and the prioritization processes. The scope of considered technologies or interventions encompassed specialized care medicinesFootnote 1 (n = 20 countries), primary care medicines (n = 16), high-cost care medicinesFootnote 2 (n = 16), medical devices (n = 11), public health programs (n = 9), and vaccination programs (n = 8). Interviewees noted that the number of technologies included in an agency’s scope depended on several factors, with resources being a significant consideration:

  • “So there are other technologies which are not assessed at all. Uh, because of the limited capacity...”

Fig. 2
figure 2

Types of technologies or interventions in scope for HTA systems

Table 4 Survey results: average number of assessments and topics prioritized by the HTA system(s) each year

The largest volume of pharmaceutical assessments was between twenty-one and fifty products per year (n = 6). Volume was lower for non-pharmaceuticals (range: 0–50 per year). Six respondents said their agency did not initiate non-pharmaceutical assessments and five respondents reported that they did not know. As one participant explained, the ability to complete non-pharmaceutical evaluations may be limited to lack of data. In some cases pharmaceuticals and non-pharmaceutical interventions are processed by different agencies or the choice is influenced by available finances where for example, there is less funding and demand for HTAs related to non-pharmaceutical topics. Our study found that whether the HTA body receives a pharmaceutical dossier from industry can be an influential factor in a country’s approach to TISP. For example, the Republic of Slovakia has the maximum threshold value for assessing the cost-effectiveness of medicines cost-effectiveness threshold explicitly stated in legislation [18, 19], and this means that all medicines under the maximum threshold value are selected as topics for HTA without a prioritisation process.

Survey respondents identified formal decision makers of the HTA systems, including those implementing recommendations, as: (1) policy makers, (2) payers and insurance agencies, and (3) committees. Interviewees reported a lack of awareness and understanding among decision-makers regarding the complexity and nuances of the HTA decision-making process which, in turn, hinders effective and transparent outcomes. Political influence exacerbates these challenges, potentially compromising objectivity, and integrity in decision-making. A lack of knowledge about HTA and evidence-based medicine, and the political nature of decision-making are issues identified that hinder successful HTA and TISP:

  • “Nobody knows how it is and how it should be …even the Ministry

  • There's a lack of understanding in terms of what [HTA] it is and what it's meant to do.”

  • “Even in the Ministry. People are not … aware. People even are not aware of [name] law [and] how it is functioning. So, …everything is, you know, functioning due to inertia.”

An interviewee who described their country’s HTA processes as performed by more than one body reflected on the value of one decision-maker within an HTA system, stating “If you are going to use HTA as a tool for decision-making, then you need to understand that you can't have decision-making pockets everywhere.” Another interviewee describes decision-makers’ lack of awareness and understanding saying:

  • “In my opinion, if a country spends some money for health technology assessment, it needs to improve the decision-making process because if they don’t, then it’s a waste of money.”

When decision-making lacks transparency, survey and interviewees’ responses reported that this compromised the public trust and integrity of the system, potentially leading to dissatisfaction and challenges to the legitimacy of the decisions.

Topic proposals were most typically made by a government department, government officials, or HTA system decision-makers (n = sixteen) (see Table 5). Interviewees remarked that: “also the Ministry of Social Affairs has suggested some topics” another one added “actually the Ministry just recently suggested a topic”.

  • “[identification happens] via negotiations with [the] Ministry of Health, regarding the HTA topics, including the initial topics proposed by the Ministry”.

Table 5 Survey responses: How topics are identified

For the decisions about topic identification participants reported that these decisions were commonly made by Ministries of Health, National Health Services or the Ministry of Research, an interviewee said:

  • There is huge problem, completely uncovered…orphan drugs and rare diseases…orphan drugs are paid in our country since [year] and that achievement was political…the decision from the Ministry of Health was made to pay those drugs.

Some respondents highlighted inconsistency between centralized and regional processes. The terms ‘payers’ or ‘health insurance agencies’ were both used (under the stewardship of Ministries of Health), indicating the impact of differences in health system design on HTA and TISP. The types of committees involved in HTA could be a national board for benefits packages, a drug committee for medicines and vaccines, and HTA committees who reported directly to the Ministry of Health.

How TISP is performed

Survey results for topic identification were heterogeneous. Proactive approaches (horizon scanning or early warning systems) were uncommon (n = five). Eleven (55%) respondents indicated that their TISP was reactive. Interviewees commented: “… in [region name] parts of Europe we are doing more reactive approach” which they associated to a few things one of them being part of a “small market.”

  • “...they [decision-makers] do not see the necessity of investing in scanning activities, and for me it is mind boggling because it is more of a reactive system than a proactive system, right?”

Almost all respondents (n = 18, mean = 4.75, median = 5) reported that more than one stakeholder group was involved in topic identification. In some cases topics are also proposed by health care workers or experts (n = 13) and industry (i.e. manufacturers of drugs/devices) (n = 13). Interviewees commented:

  • what is working, or at least on paper, is of course identifying the topics and the classification … It happens through a committee … this committee is looking at the [World Health Organization] Essential Medicine List and through, you know, any new data that is coming in.”

  • most of the topics come from the health insurance fund as they have had some more complex applications for drugs services”.

On the use of explicit criteria for topic selection and prioritization and the use of ranking systems, survey responses were mixed. Nine respondents (50%) reported that prioritization was performed using explicit processes and criteria. Seven respondents (39%) provided the criteria used in their country, which included estimates of the burden of disease, potential impact on costs, clinical and organizational effects and domestic ethical, equity or mandated considerations as required by national legislation. Interviewees said:

  • We have [an] official criteria for prioritization. Mainly impact, morbidity, mortality, economic burden, clinical tools used in clinical practice sometimes. The criteria are decided by our Ministry of Health, but the process of prioritization includes also the [centre’s name].”

  • “the country has a process to prioritize conditions and treatments called "[name]” in health. This process is enshrined in the law and includes a package of [number] prioritized conditions for treatment and financial protection.”

One respondent reported using the Pritec tool [20]. Two respondents provided links to publicly available criteria [21, 22]. Three respondents reported that they used criteria, but did not elaborate further and another noted that criteria were being piloted. One interviewee suggested topic selection and prioritization may be done on an ad hoc basis by committees appointed without a clear process. Comments included:

  • “We don’t have a formal prioritization process. We don’t have any formal process yet for that [prioritization].”

  • “They [the committee] will say, let's see, which of these drugs should be prioritized… And they have, based on their experience and their “own needs” [emphasis added by authors].

Transparency and the availability of the outcome of TISP decisions

Information on the availability and transparency of TISP outcomes is mixed. Ten respondents (59%) said that TISP outcomes (e.g., a list of topics, alerts, vignettes or short notes, and reports of relevant technologies within specified areas) were not publicly available, and eleven (65%) reported that information on the topics selected was also not publicly available. Seven respondents (41%) stated that neither the outcomes nor the selected topics were publicly available, meaning that in these countries there is no public information on the identification of topics. The survey free text responses shed some light on the process, indicating that information could be made available “on reasonable request,” or it was available for selected groups. Interviewees said:“…it could be more transparent, but I would say that for the regional standards is transparent. “

  • “I think [TISP] it's relatively transparent… [the] health insurance fund, is a public organization, so decisions have to be very transparent…the meeting notes are available to the public, so it's possible to understand how the decisions were made and also when we kind of put together the reports.”

Others suggested that the current system was not transparent and some mentioned that their organization was currently working on improving transparency. An interviewee said “…the decision-making process is opaque and not well-described.” In some cases, the independent agency responsible for HTA assessment was not responsible for or able to influence transparency in other parts of the HTA system, or its related processes (i.e., the Ministry of Health has sole authority):

  • I feel the [TISP] process … could be more transparent, but this part of the process is not doesn't depend on the [agency] it depends on the Minister”.

  • “it's mostly problem of the decision-making process because this process is unclear and fragmented”

Interviewees noted that stakeholders can influence transparency. Patients’ groups may exert significant influence on the HTA process by voicing patients’ needs and preferences, or pharmaceutical manufacturers may exercise influence:

  • [there are] some vocal groups involved putting themselves on the agenda… [the process is] “hijacked” by some groups”. [emphasis added by authors].

  • But in [year], pharma industry said that we are too transparent and so if we are too transparent that they will not deliver drugs to [country’s] reimbursement system because all other countries will see the prices and it can create problems for their businesses in other countries”.

Factors influencing the current TISP processes

The survey indicated several factors relevant to the type of HTA TISP process (Table 6). The process is reported to be limited to policy makers and expert involvement (n = 11). Interviewees agreed that “TISP is a political decision”. Other factors that influenced a country’s choice of TISP were its participatory nature and the involvement of all or most relevant stakeholders (n = eight), or politics and context (i.e., a political decision) (n = six). Some interviewees referred to the appointment of experts by the government for TISP, inferring that they may not be politically neutral.

Table 6 The main factors that have influenced the choice of TISP process

Study participants highlighted the influence of international and regional networks’ published guidance and collaborations on TISP processes, including those published by OSTEBA, INAHTA, HTAi, NICE, EVIDEM, and EUnetHTA. Eight participants (47%) reported that these networks had a significant impact. Networking activities occurred at international and local levels, with international agencies playing a crucial role offering guidance on methods and facilitating mentoring opportunities:

  • “We feel and we need to participate [at] different [periods of time] and with different association[s], like HTA International (HTAi) or INAHTA and for example in the Americas we’re working with the RedETSA (the HTA network f the Americas) and EVIDEM at different points and because we use the methods.”

The EUnetHTA core model was a topic of discussion in several interviews:

  • “…participation within the European Network for Health Technology Assessment (EUnetHTA JA2 and EUnetHTA JA3 projects) have significantly improved the quality of the process of HTA in [country].”

For countries with emerging HTA systems, survey and interview results indicate that those that collaborate with international partners have built capacity, although achieving sufficient levels of education and expertise around HTA continues to be challenge and will take time. Our study’s results provided insights into the links between HTA bodies and countries’ broader education systems, it appears that some interviewees saw the value in establishing relationships and collaborations with local universities. The integration of health and education systems seemed to be essential enabling factors for producing HTAs. Engaging with universities is a factor influencing the TISP process, in terms of supporting capacity building efforts, knowledge innovation, development, dissemination, and in some cases developing the HTA reports. Leveraging universities’ expertise and resources enhances the TISP process by fostering collaboration, facilitating skill development, and promoting the exchange of valuable knowledge within the local innovation ecosystem:

  • The HTA Center in [country] is under the University of [name].”

  • “Increas[ing] capacity is very important. We need the local educational program. Maybe some master programs…this is very important for us, we have to work on capacities”

The interviews identified other important factors influencing the TISP process. Firstly, the upcoming implementation of a new European regulation [23] that will come into effect in 2025, and is anticipated to change TISP and HTA processes within Europe. Secondly, misalignment between procurement and selection timelines was identified as a challenge, leading to inefficiencies and decision-making delays. Interviewees noted the issue of wasteful duplication of HTA efforts by different agencies, sometimes even within the same country, highlighting the need for better coordination and collaboration among stakeholders. Interviewees’ impression of the lack of awareness and understanding of HTA by decision-makers meant that some participants felt it was important to educate Ministry of Health staff and clinicians about the information required for TISP and assessments, since their understanding and involvement are crucial in driving informed decision-making. Despite widespread recommendation, participants reported limited utilization of HTA in the decision-making process, indicating a gap between recommendations and policy implementation.

Factors limiting TISP and future needs

Survey and interview respondents described a range of current issues that require attention, including the lack of topic selection processes, weak regulatory or legislative processes, decisions being irrelevant for policy makers, or incomplete TISP criteria. Some of these issues relate to the HTA system in general, such as the fragmentation of HTA as a decision-making tool in the health sector. Participants widely reported inconsistency between the use of HTA in some contexts. Human resource capacity was flagged many times as a limitation for TISP, and the HTA system more generally. HTA agencies with limited finances may struggle to train and develop their staff, and also experience the migration of skilled workers to health technology and pharmaceutical companies because of higher salaries and enhanced career prospects that industry offers. This loss of expertise poses a significant challenge to the effective functioning of HTA agencies, particularly in resource-limited countries:

  • “I don‘t want to lose any people on my team… to educate them and give the training is very - very complex. It's a long - long process and the people of the pharmacy wants to come here to [the] institute and take the people with the double or three, three times more money.”

Eleven respondents (52%) noted attempts to improve the TISP process within their country, centered around either adjustments to the types of criteria or the revision or weighting of criteria. Other efforts related to strengthening the general HTA processes, such as improving process transparency through publishing information on websites, initiating meetings with stakeholders and international partners, awareness raising and training and capacity building.

Results indicate that HTA agencies need resources, with examples of the duplication of HTA efforts, by public or private entities, suggested to be unproductive. Interviewees reported challenges around data availability and adaptation to local contexts, and certain technologies could lack proper assessment as a result.

The lack of patient and public participation is another factor influencing the TISP process, and identified as a future need in survey and interview. Interviewees agreed there is no established procedure for patient or community involvement in TISP, although some reported seeking input from interest groups, or at the later steps of the HTA process (e.g., analysis, appraisal, or decision-making). They suggested that patient groups can submit proposals to the Ministry of Health or the HTA body, but limited capacity hinders a more active or engaged involvement. Often patient groups are not part of the TISP (or HTA) committees, with healthcare professionals dominating decision-making. One participant said, “…involvement of indigenous populations and communities is important but not adequately addressed.” Overall, efforts are being made to improve patient and public engagement, but further work is required:

  • “We have an understanding of the necessity to involve patients and our citizens in decision-making process, but sometimes our decision-making process is a little closed”.

  • “Uh patient and citizen. It’s well known, you know that the government is, is struggling with that even though you will see in [place] that there is a lot of activity and interventions happening at the community level and it’s so important involve communities.”

TISP insights for countries with emerging HTA systems

Interviewees acknowledged the complexity of offering a universally applicable list of suggestions and emphasized that countries with an emerging HTA system can benefit from other countries experiences. Drawing from the survey and interviews, a set of interconnected key messages were identified:

  • Governance: some participants reported it is important to establish an autonomous and independent agency for HTA that is insulated from political influences. Participants believed the institutional framework is key setting the direction and mechanisms that will support TISP and other HTA decisions being based on scientific evidence and objective analysis rather than political considerations. Critical aspects of governance include transparency and accountability built into the TISP process and the presence of an enabling environment for collaboration and adequate resourcing.

  • Political awareness and support: The awareness, support and understanding of policy makers is another important consideration that was identified through interviews. Participants suggested that information cannot trigger actions if it is not well understood. Education and advocacy about all steps of HTA to politicians and decision-makers can support TISP, and HTA and overall stewardship of health systems.

  • Effective coordination: TISP requires a multi-stakeholder process: an emphasis on fostering a positive culture of collaboration, with clearly defined roles and responsibilities of the various actors involved plus clear lines of accountability is also crucial.

  • Guidelines and Methodologies: although there is currently no universal consensus among countries on this, it would be important to establish agreed guidelines and methodologies for conducting TISP.

  • Regional networks: developing partners can play a critical role helping establish and build TISP/HTA. Regional and international initiatives such as EUnetHTA or RedETSA, seem to have been a useful for sharing best practice, and there may be value in further investment and engagement in more regional networks.

  • Technical Capacity: TISP and HTA processes depend in large part on the availability of a broad range of specialized skills. These processes are inherently multidisciplinary and require the involvement of a variety of disciplines. Technical capacity determines many features that influence TISP. The technical ability of staff can facilitate or inhibit the process. Setting unrealistic expectations of what a HTA body can deliver (due to ineffective TISP), may lead to inadequate coverage and undermine the credibility of the HTA process. A high turnover may weaken the overall technical capacity which should mitigated by strategizing on how retaining current staff.

  • Identification in Data-Scarce Scenarios: selecting the right topic for an HTA depends on accessing data, for example, the issue of access to epidemiological data is necessary to understand the priority of the problem was highlighted. Further, the quality of HTA and their usefulness for decision making is directly linked to the quality and completeness of data. When data are lacking, conducting field research and engaging with patient organizations and academia can provide insights into emerging health technology needs and topics of concern.


This study aimed to understand how TISP is performed in different countries through identifying factors that influence a country’s choice of TISP, particularly those relevant to countries with emerging HTA systems. The need to establish priorities for which technologies are to be assessed is a long-standing important issue for the HTA community [4, 24]. We found variation in TISP approaches, similar to previous findings [5, 25,26,27] with TISP being very closed in some counties (e.g., limited to policy makers and expert involvement), with the potential for being much more inclusive and participatory (e.g., involving all or most relevant stakeholders). At a strategic level, political awareness and governance supported through institutional frameworks were raised as important to TISP. It was suggested by several participants that regional networks have been a useful tool in some settings, particularly in the development of guidelines and methodologies. Factors relevant at a country level included the technical capacity to conduct TISP, and access to relevant local data.

Some survey respondents and interviewees noted that there were both real and perceived political influences in HTA decisions. Stakeholder pressure has been identified as a factor that may influence an HTA body’s priority towards “hot topics” [28], that can sometimes come from people in positions of power. Chinitz (2004) discusses the role of HTA in health policy making, exploring the increased politicalization of HTA. It is suggested that originally these assessments occurred in relatively depoliticized environments where assessments were “politically innocuous” studies of technologies that were considered to be of high importance, although over time these decisions have evolved into more high-profile decisions [29]. Culture, values and institutional context may influence the use of HTA [30], with political buy-in identified as a barrier for HTA in general, and a supportive political environment being relevant for its implementation [31, 32]. The responses from our study suggest that political influence continues to be a factor in the selection of topics, as well as in down-stream HTA decisions and implementation of the technology.

In this study, TISP was characterized as having relatively low patient or public involvement (Table 5). As interviewees mentioned, this may reflect the need for additional resources to recruit and support the involvement of patient and public groups. It may also suggest issues of historical-political trust issues between government institutions and citizens. Patient or public involvement in HTA is reported as increases the credibility and transparency of reimbursement decisions [33], which could also be relevant in the TISP process. A well-recognized example of participatory approaches in health governance is Thailand’s National Health Assembly that was designed to secure greater public participation and consultation in health policy making and has been credited with successfully enabling more meaningful public engagement in health policy decisions [34]. Other examples from participatory budgeting literature highlight the effectiveness of stakeholder engagement in decision-making include Brazil, Italy, and India [35, 36]. A gap that was acknowledged by many interviewees in their TISP process was the recognition that public awareness-raising activities and formal feedback processes should be put in place, or developed further, to increase the engagement of the patient and citizen groups in health policy decisions. However, it was also acknowledged by interviewees that it takes time, and is reported in other studies that it places greater demands on capacity and coordination skills to ensure the necessary representatives are engaged in the process [34].

Transparency is key for a fair priority setting processes in all steps of HTA, including TISP [37]. Transparency considerations include which stakeholders are involved in TISP, documenting the basis of decisions, publishing the decisions, and describing options for appealing decisions [38,39,40,41,42]. The interviews offered insight related to the transparency of TISP decisions. Some interviewees discussed a very open process, where all industry dossiers were reviewed (and publicly available), however other interviewees reported that decisions about topics for HTA were made behind closed doors, or by the Minister. Transparency can have strengths and weaknesses for an HTA system. On the positive side, a transparent process lays bare the decision-making process, and additionally may support the real-world applicability of the HTA recommendations. A transparent process encourages decision-makers to be more systematic and explicit, and by default, publicly accountable [43]. On the negative side, for the system to be very transparent, it requires time and resources to share the information, and as a result may delay the process. The information itself may also be overly complex for the public at large. And there may be instances where secrecy is necessary for legitimate reasons e.g., as a requirement by manufacturers.

Despite countries working to improve their TISP processes by including more explicit criteria and making final decisions more accessible and publicly available, the study suggests that there continues to be challenges with the TISP methodology. Qiu et al. suggested that consensus approach for the development of methods of topic selection would be valuable for the HTA community [44]. However, as TISP needs adapt to different contexts, it is difficult to develop standard methods guidance. Despite this, having explicit criteria and making information on decisions available in the public domain should support better TISP processes, particularly in countries that have several prioritization processes and HTA programs. Many suggestions made during this research on improving TISP processes indicated the need to consider the overarching HTA system, including the need to provide adequate human and financial resources. Success with TISP, and HTA in general, seems to rely on embedding it within the national health care decision-making and priority setting system [45]. HTA can be supported through legislation to regulate and implement the various stages of the HTA process. The more governments and other stakeholders appreciate the usefulness of HTA the greater the likelihood that HTA production process will be sufficiently resourced. Increased public engagement with (and ownership of) the TISP process within both HTA and the decisions arising from it may generate more enthusiasm and demand for such priority-setting processes.

Finally, establishing a TISP mechanism within an emerging HTA system is a process, but not necessarily a linear process, nor a binary one; one cannot say if a country has a full TISP process or not. Instead, it seems is a dynamic, complex and ongoing process with many influences, often happening in parallel involving multiple players, stakeholders, continues learning and changes.

Strengths and limitations of the study

The strengths of this study are that the responses provide a current and unique insight into TISP processes in a wide range of countries, particularly relevant to emerging HTA systems. The responses outline factors to consider when planning to implement HTA in support of UHC. There is little published information about TISP processes for countries where HTA is in a nascent phase, and often the processes are not reported in detail, so this information from actors within HTA systems is particularly valuable. However, despite the relatively high response rate (72%) to our survey, it was necessary to follow up with individual interviews for additional detailed responses. We cannot guarantee that survey respondents and interviewees were the most knowledgeable people on TISP processes within their organizations. Our findings are indicative and require further in-depth research.


This study provides current information on the range of processes and practices influencing TISP in countries with a recently formalized HTA systems. Despite the attention paid to HTA as a catalyst in achieving UHC, TISP in many countries is still developing. TISP is not conducted in any standard manner, lacks transparency in some countries and varies in terms of the degree of collaboration achieved with stakeholders, including the public. Additionally, there are challenges driven by a lack of awareness of the HTA process, few experts and adequate resources for the system were mentioned by many of the individuals in our sample. International and local collaborative approaches to TISP and capacity building were valued by respondents and initiatives such as EUnetHTA, REDESTA, and the EuroScan international toolkit for early awareness. Our findings suggest that TISP reflects national cultures and practices, influenced by each county’s implicit governance structures. Broadly, despite TISP requiring consideration of individual country contexts, including the political setting and available resources, a comprehensive and consistent approach to TISP can support the HTA system.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.


  1. Specialized care medicines: medications specifically designed and tailored for the treatment of specific medical conditions or patient populations, at the specialist level of care (often hospital level).

  2. High-cost medicines: significant financial burden due to their high cost.



European network of Health Technology Assessment

EVIDEM collaboration:

Evidence and Value Impact on Decision Making collaboration


Health Intervention and Technology Assessment Program


Health Technology Assessment


Health Technology Assessment International


International Network of Agencies for Health Technology Assessment


Low-and middle-income country

NICE International:

National Institute of Care and Health Excellence International


Norwegian Institute of Public Health


Office of Basque Health Technology Assessment


Topic identification, selection and prioritization


Universal Health Coverage


  1. World Health Organization. Using health technology assessment for universal health coverage and reimbursement systems. Geneva: WHO; 2015.

    Google Scholar 

  2. World Health Organization. Institutionalizing health technology assessment mechanisms: a how to guide. Geneva: WHO; 2021.

    Google Scholar 

  3. O’Rourke B, Oortwijn W, Schuller T. The new definition of health technology assessment: a milestone in international collaboration. Int J Technol Assess Health Care. 2020;36(3):187–90.

    Article  PubMed  Google Scholar 

  4. Henshall C, Oortwijn W, Stevens A, Granados A, Banta D. Priority setting for health technology assessment: theoretical considerations and practical approaches: a paper produced by the priority setting subgroup of the EUR-ASSESS project. Int J Technol Assess Health Care. 1997;13(2):144–85.

    Article  CAS  PubMed  Google Scholar 

  5. Qiu Y, Thokala P, Dixon S, Marchand R, Xiao Y. Topic selection process in health technology assessment agencies around the world: a systematic review. Int J Technol Assess Health Care. 2022;38(1):e19.

    Article  PubMed  Google Scholar 

  6. Noorani HZ, Husereau DR, Boudreau R, Skidmore B. Priority setting for health technology assessments: a systematic review of current practical approaches. Int J Technol Assess Health Care. 2007;23(3):310–5.

    Article  PubMed  Google Scholar 

  7. Lauvrak V, Arentz-Hansen H, Di Bidino R, Erdos J, Garrett Z, Guilhaume C, et al. Recommendations for Horizon Scanning, Topic Identification, Selection and Prioritisation for European Cooperation on Health Technology Assessment. Norway: EUnetHTA WP4 Deliverable 4.10. 2020.

  8. Organization for Economic Cooperation and Developemnt. building capacity for evidence informed policy making: towards a baseline skill set. OECD; 2020 September 8.

  9. National Collaborating Centre for Methods and Tools. Evidence-informed decision making in public health. 2018. Accessed 2 Aug 2022.

  10. Bertram M, Dhaene G, Tan-Torres Edejer T, Organization WH. Institutionalizing health technology assessment mechanisms: a how to guide. 2021. Report No.: 9240020667.

  11. World Health Assembly. Health intervention and technology assessment in support of universal health coverage. WHA Resolution 6723. 2014.

  12. Jeffrey M, Chi YL, Stewart M. iDSI health technology assessment toolkit. F1000Research. 2019;8(703):703.

    Google Scholar 

  13. Norwegian Institute of Public Health. Supporting implementation of HTA in low-and middle-income countries: NIPH. 2020. Accessed 15 Jan 2024.

  14. Simpson S. A toolkit for the identification and assessment of new and emerging health technologies Birmingham, UK: EuroScan International Network. 2014. Accessed 15 Jan 2024.

  15. Lauvrak V, Bidonde J, Peacocke EF. Topic identification, selection and prioritisation for health technology assessment (HTA). A report to support capacity building for HTA in low and middle income countries. Oslo, Norway: Norwegian Institute of Public Health, Division of Health Services GHD; 2021.

    Google Scholar 

  16. Sharma A, Duc NTM, Thang TLL, Nam NH, Ng SJ, Abbas KS, et al. A consensus-based checklist for reporting of survey studies (CROSS). J Gen Intern Med. 2021;36(10):3179–87.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Word Bank. The World by Income and Region: The World Bank Group. 2021. Accessed 27 June 2023.

  18. National Institute for Value and Technologies in Healthcare (NIHO) Slovakia. How does the system of health technology reimbursement work in Slovakia? : NIHO. 2023 [1 May 2023]. Accessed 15 Jan 2024.

  19. European Observatory on Health Systems and Policies. Changes to entry conditions for medicines to the Slovak market: European Observatory on Health Systems and Policies. 31 August 2022. Accessed 15 Jan 2024.

  20. Varela-Lema LRM, del Carmen M, Preito Yerro I, Arriola Bolado PA-Bea. Herramienta PriTec: adaptación para la selección de tecnologías a evaluar previa entrada en cartera de servicios. 2018. Accessed 15 Jan 2024.

  21. Youngkong S, Baltussen R, Tantivess S, Mohara A, Teerawattananon Y. Multicriteria decision analysis for including health interventions in the universal health coverage benefit package in Thailand. Value in health. 2012;15(6):961–70.

    Article  PubMed  Google Scholar 

  22. Ministry of Health Protection of the Republic of Lithuania. Order: On the approval of the description of the procedure for determining the assessment of healthcare technologies and the assessment organization procedure (Įsakymas: Dėl Sveikatos priežiūros technologijų vertinimo prioritetų nustatymo ir vertinimo organizavimo tvarkos aprašo patvirtinimo): Minister of Health Protection of the Republic of Lithuania (Lietuvos Respublikos Sveitkatos Apsaugos Ministras). 2017. Accessed 16 Aug 2022.

  23. European Commission. Regulation on Health Technology Assessment: European Commission. 2023. Accessed 15 Jan 2024.

  24. Drummond MF, Schwartz JS, Jönsson B, Luce BR, Neumann PJ, Siebert U, et al. Key principles for the improved conduct of health technology assessments for resource allocation decisions. Int J Technol Assess Health Care. 2008;24(3):244–58.

    Article  PubMed  Google Scholar 

  25. Pichon-Riviere A, Augustovski F, García Martí S, Alcaraz A, Alfie V, Sampietro-Colom L. Identification and selection of health technologies for assessment by agencies in support of reimbursement decisions in Latin America. Int J Technol Assess Health Care. 2021;37(1):e80.

    Article  PubMed  Google Scholar 

  26. Teerawattananon Y, Rattanavipapong W, Lin LW, Dabak SV, Gibbons B, Isaranuwatchai W, et al. Landscape analysis of health technology assessment (HTA): systems and practices in Asia. Int J Technol Assess Health Care. 2019;35(6):416–21.

    Article  PubMed  Google Scholar 

  27. Oortwijn W, Jansen M, Baltussen R. Evidence-informed deliberative processes for health benefit package design–part II: a practical guide. Int J Health Policy Manag. 2021.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Specchia ML, Favale M, Di Nardo F, Rotundo G, Favaretti C, Ricciardi W, et al. How to choose health technologies to be assessed by HTA? A review of criteria for priority setting. Epidemiol Prev. 2015;39(4 Suppl 1):39–44.

    PubMed  Google Scholar 

  29. Chinitz D. Health technology assessment in four countries: response from political science. Int J Technol Assess Health Care. 2004;20(1):55–60.

    Article  PubMed  Google Scholar 

  30. Torbica A, Tarricone R, Drummond M. Does the approach to economic evaluation in health care depend on culture, values, and institutional context? Eur J Health Econ. 2018;19(6):769–74.

    Article  PubMed  Google Scholar 

  31. Teerawattananon Y, Painter C, Dabak S, Ottersen T, Gopinathan U, Chola L, et al. Avoiding health technology assessment: a global survey of reasons for not using health technology assessment in decision making. Cost Eff Resour Allocation. 2021;19(1):62.

    Article  Google Scholar 

  32. Essue BM, Kapiriri L. Priority setting for health system strengthening in low income countries. A qualitative case study illustrating the complexities. Health Syst (Basingstoke). 2021;10(3):222–37.

    Article  PubMed  Google Scholar 

  33. Al-Harakeh L, Abbas H, Hassan H, Hallal Z, Hamadeh G, Kurdi M, et al. Survey investigating the knowledge and awareness of payers and patient advocacy groups about the health technology assessment process in Lebanon. Int J Technol Assess Health Care. 2021.

    Article  PubMed  Google Scholar 

  34. Rajan D, Mathurapote N, Putthasri W, Posayanonda T, Pinprateep P, de Courcelles S, et al. Institutionalising participatory health governance: lessons from nine years of the National Health Assembly model in Thailand. BMJ Glob Health. 2019;4(Suppl 7):e001769.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Bartocci L, Grossi G, Mauro SG. Towards a hybrid logic of participatory budgeting. Int J Public Sect Manage. 2019.

    Article  Google Scholar 

  36. Blair H. Accountability through participatory budgeting in India: Only in Kerala? Singapore: Springer; 2020.

    Google Scholar 

  37. Kolasa K, Kalo Z, Zah V, Dolezal T. Role of health technology assessment in the process of implementation of the EU Transparency Directive: relevant experience from Central Eastern European countries. Expert Rev Pharmacoecon Outcomes Res. 2012;12(3):283–7.

    Article  PubMed  Google Scholar 

  38. Stafinski T, Menon D, Philippon DJ, McCabe C. Health technology funding decision-making processes around the world. Pharmacoeconomics. 2011;29(6):475–95.

    Article  PubMed  Google Scholar 

  39. Lauvrak V, Bidonde J, Peacocke E. Topic identification, selection and prioritisation for health technology assessment (HTA). A report to support capacity building for HTA in low and middle income countries. Oslo, Norway: Norwegian Institute of Public Health, Division of Health Services GHD; 2021.

    Google Scholar 

  40. Leopold C, Lu CY, Wagner AK. Integrating public preferences into national reimbursement decisions: a descriptive comparison of approaches in Belgium and New Zealand. BMC Health Serv Res. 2020.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Daniels N, Sabin J. Limits to health care: fair procedures, democratic deliberation, and the legitimacy problem for insurers. Philos Public Aff. 1997;26(4):303–50.

    Article  PubMed  Google Scholar 

  42. Daniels N, Sabin JE. Setting limits fairly: learning to share resources for health. Oxford: Oxford University Press; 2008.

    Google Scholar 

  43. Baltussen R, Jansen MP, Bijlmakers L, Tromp N, Yamin AE, Norheim OF. Progressive realisation of universal health coverage: what are the required processes and evidence? BMJ Glob Health. 2017;2(3):e000342.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Qiu Y, Thokala P, Dixon S, Marchand R, Xiao Y. Topic selection process in health technology assessment agencies around the world: a systematic review. Int J Technol Assess Health Care. 2022.

    Article  PubMed  Google Scholar 

  45. Ananthakrishnan A, Luz ACG, Kc S, Ong L, Oh C, Isaranuwatchai W, et al. How can health technology assessment support our response to public health emergencies? Health Res Policy Syst. 2022;20(1):1–7.

    Article  Google Scholar 

  46. Esandi ME, Gutiérrez-Ibarluzea I, Ibargoyen-Roteta N, Godman B. An evidence-based framework for identifying technologies of no or low-added value (NLVT). Int J Technol Assess Health Care. 2020;36(1):50–7.

    Article  PubMed  Google Scholar 

  47. Oortwijn W, Sampietro-Colom L, Habens F, Trowman R. How can health systems prepare for new and emergin health technologies? The role of horizon scanning revisited. Int J Technol Assess Health Care. 2018;34(3):254–9.

    Article  PubMed  Google Scholar 

  48. European Network for Health Technology Assessment (EUnetHTA). An analysis of HTA and reimbursement procedures in EUnetHTA partner countries: final report. Austria: EUnetHTA WP7; 2017.

Download references


We thank participants for volunteering for this project. Also, we would like to acknowledge the support of Elisabet Hafstad in devising and executing the literature search. Lumbwe Chola, Ingvil Sæterdal, Saudamini Vishwanath, Eia Elena Skjønsberg, Lieke Fleur Heupink and our colleagues at the Norwegian Institute of Public Health and the Health Technology Assessment Program for their feedback on the draft survey. We thank Francis Ruiz and Thomas Wilkinson, who participated as experts in the virtual workshop. Figure 1 was designed with the support of Annique van der Boon. We thank the Norwegian Institute of Public Health for supporting this publication open access fees.


Open access funding provided by Norwegian Institute of Public Health (FHI). This work was partially funded by the Norwegian Agency for Development and Cooperation (Norad), Grant Numbers: QZA-21/0082 and QZA-18/0102 undertaken by the Norwegian Institute for Public Health. The funding body played no role in the design of the study, the collection and analysis, or interpretation of data or in writing the manuscript.

Author information

Authors and Affiliations



Conception: VL, JB, EP; Design of the work: AA, EP, JB, PK, VL; Acquisition, analysis, interpretation of data AA, EP, JB. PK, VL; Drafting the work or substantively revising it: AA, EP, JB. PK, VL.

Corresponding author

Correspondence to Elizabeth F. Peacocke.

Ethics declarations

Ethics approval and consent to participate

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2013. The survey did not require formal ethical approval as the focus was not the individuals’ perspectives, rather the reporting of practices and processes in their institutions. We followed all ethical practices for survey research. For the interviews we obtained a waiver for ethical clearance from the Norwegian Ethics Regional Committees for Medical and Healthcare Professional (application #586503). Participants were given detailed information on the purpose and process of the survey and interviews to enable them to make an informed decision on participation, indicating that any information provided would be used in a report and a scientific publication. We also explicitly informed participants that their responses, including information on their organization, would be identifiable and publicly shared, although we would not link individual responses to a specific country.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests (neither financial nor non-financial interests).

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: S1.

Project plan: Topic identification and Selection for Health Technology Assessment (HTA), options for Low- and Middle- Income Countries (LMICs).

Additional file 2: S2.

Search strategy.

Additional file 3: S3.


Additional file 4: S4.

CROSS Survey reporting checklist.

Additional file 5: S5.

Countries per region meeting the inclusion criteria.

Additional file 6: S6.

Included country list by region.

Additional file 7: S7.

Summary of findings.

Additional file 8: S8.

Interview guide.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bidonde, J., Lauvrak, V., Ananthakrishnan, A. et al. Topic identification, selection, and prioritization for health technology assessment in selected countries: a mixed study design. Cost Eff Resour Alloc 22, 12 (2024).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: