- Open Access
A comprehensive map of the evidence on the performance evaluation indicators of public hospitals: a scoping study and best fit framework synthesis
Cost Effectiveness and Resource Allocation volume 16, Article number: 64 (2018)
Key performance indicators are essential navigation tools for hospitals. They provide managers with valid information enabling them to identify institutional strengths and weaknesses and improve managerial performance. In this study, the synthesis of evidence relating to hospital performance indicators was carried out by means of a field review and the indicators were analyzed through the Best Fit Method.
The five-step approach of Arksey and O’Malley was used as follows: selection of the research question; search for related studies; selection and refinement of the studies; synthesis and tabulation of key information; derivation of the related summary and report. Applying the Best Fit Framework Synthesis Method, the initial themes and subthemes were created and a model of public hospitals performance evaluation finally generated.
Forty-nine studies were considered eligible to form part of the synthesis. The final model included the efficiency/productivity, effectiveness and financial themes. The efficiency/productivity sub-themes incorporated human resources indicators, hospital beds, costs, operating room productivity, emergency rooms, ICU, radiology, labs, technology and equipment productivity. Other sub-themes relate to general indicators such as BOR, ALS, number of outpatients and hospitalized patients. Financial themes included profit, revenue, cash flow, cost, investment, assets, debt and liquidity. Concerning effectiveness, the indicators were categorized in terms of access (equity), safety, quality and responsiveness. The accountability indicators were classified into patient-centeredness, staff orientation, and social responsibility.
Hospital performance management is a multi-dimensional issue, each dimension having its own significance. Based on the evidence, indicators are dependent on the evaluation model employed, the evaluation objective, and the views of executive managers and participants in the study. Selection of the most appropriate indicators is therefore key to a comprehensive performance evaluation system.
Health systems are today one of the largest sectors of the world’s economy and among the most important factors for community development and social welfare . In the World Health Report, hospitals are identified as major health care providers and among the factors determining the equitable distribution of health care and promotion of the justice index in the health system. Furthermore, health systems realize their intermediate and final goals at all levels through enhanced hospital performance [2, 3]. Hospitals are the most essential and, at the same time, most costly part of the health system, so that in developed and developing countries 40% and 80% of the health sector expenses respectively are allocated to hospitals [4, 5]. In line with the rapid growth of expenses, environmental changes cause hospitals to face many political, economic, social and cultural changes over time. These changes include population ageing, advances in health technologies, development of information technology and telemedicine , all of which require rapid and active responses and measures. In this regard, appraisal of hospital performance indicators is an effective strategy for properly managing such changes. Continuous scrutiny of hospital procedures further prepares managers to proactively respond to these changes [7,8,9].
Key performance indicators (KPIs) are considered performance-based decision-making tools for policymakers and managers at national and local levels. These indicators provide valid information for managers, enabling them to identify their strengths and weaknesses and improve their managerial performance. Such information is also a good tool for the development and planning of promotional activities by organizations . However, the paucity of evaluation and control systems in various dimensions such as resources, facilities, staff, goals and strategies means that there is no connection with the environment inside and outside the organization. This is considered to be one of the symptoms of organizations afflicted by disease, leading ultimately to their death .
Assessing clinical and economic performance indicators in hospitals helps policy-makers, managers and doctors to monitor performance and payment systems. It also promotes procedural transparency and individual accountability, resulting in better institutional performance . Paying attention to hospital performance indicators is likewise conducive to achieving the hospital’s internal and external goals [13, 14], making effective and efficient use of available resources, improving service quality , and providing a clear perspective on hospital efficiency and effectiveness . However, given the continuous changes in hospital performance, these indicators should be regularly reviewed on the basis of new evidence . Identifying performance indicators not only helps to promote the responsiveness, efficiency and effectiveness of organizations as well as public trust in them, but also contributes to the planning and development of strategies to deal with complicated environmental changes . The lack of an integrated and universally accepted framework for measuring health service performance has led various studies to examine different dimensions and indicators of hospital performance [19,20,21].
Some studies have employed procedures such as the Balanced Scorecard , Data Envelopment Analysis [22,23,24] and Pabon Lasso [24, 25] models, while others have concentrated on particular aspects of hospital performance. To evaluate and rank hospitals in New Zealand, Davis et al. focused on efficiency, effectiveness, and equity . Pink et al. studied hospital performance in terms of financial performance, employing a review, panel and survey approach to assess the financial indicators of hospitals and reporting on them in terms of the five dimensions of financial sustainability, liquidity, capital, efficiency and human resources . Gu and Itoh evaluated the views of 228 managers and 894 employees, and classified hospital performance indicators into 8 factors: survival and mortality rates, operational efficiency, patient/staff safety, financial effectiveness, quality of work life, staff development, patient-centered care, and patient/staff satisfaction . Xenos et al. appraised the productivity and efficiency of Greek hospitals over a period of financial crisis . Nikjoo et al. conducted a “mix method” study and selected key performance indicators (KPIs) for hospitals in the three areas of quality-effectiveness, financial-efficiency and access-equity . In their study, Khalifa and Khalid identified 58 KPIs for hospitals, and categorized them into patient access, hospitalization utilization, outpatient utilization, operating room utilization, emergency utilization, general utilization, patient safety, infection control, documentation compatibility, and patient satisfaction . There is no consensus regarding an effective approach to evaluating the performance of health services. In this regard, developing a combination of methods, frameworks and indicators for measuring hospital performance can provide a comprehensive perspective on hospital capabilities [2, 17, 23]. Evidence-based management focuses on integrating the findings of management research in the decision-making process of health system managers , preventing or minimizing overuse, underutilization and misuse of managerial activities. Such management further eliminates the gap between research and practice , making it possible to use the experience of other organizations and ameliorate the quality of decision making [32, 33].
Through a comprehensive review and summary of all studies on a given topic, knowledge synthesis interprets the results of those studies within a general evidential framework so as to provide policymakers and managers with assistance in planning and decision-making . Given that summarizing and publishing research results is one of the main objectives of scoping reviews , the evidence about hospital performance indicators was synthesized in this study by means of a scoping review and the indicators were analyzed through the use of the Best Fit method.
In this study, a systematic scoping review carried out in 2018, the Arksey and O’Malley approach and the complementary recommendations of Livak were used to specify the performance indicators of public hospitals. The approach consists of five main stages and one optional stage as follows: selecting the research question, searching for related studies, selecting and refining the studies, synthesizing and tabulating key information, summarizing and reporting, and verifying and validating the results using the expert panel (optional) [35,36,37]. These stages are discussed as follows.
Selecting the research question
The research question is “What effective performance indicators in public hospitals can be observed in the existing studies?”
Data source and search
At this stage of the scoping review, the three main resources included electronic databases, reference lists of articles and a manual search of other resources, such as relevant key journals, networks, organizations and conferences. To ensure that the study was not reiterative, the studies registered at the Cochrane Library were the primary source, where no systematic reviews on the subject were found.
In order to identify the keywords, a pilot study was conducted by the information officer on the PubMed, Web of Science (ISI), Science Direct and SCOPUS databases separately. The pilot study showed that by using different keywords in each database, a higher percentage of related articles could be accessed. Table 1 presents the keywords suitable for each database. The main search on the intended databases was done in 2017, from July 26 to the end of December, without time limitations. In addition, so as to have access to new articles related to the subject, the researcher signed up to the databases and activated the alert option.
To increase sensitivity (i.e. to increase the selection of related articles), the researcher examined several databases, searched with relatively common terms, and used synonym words with the “OR” operator. In addition, in order to increase specificity (i.e. to reduce the selection of unrelated articles), synonyms were used with the “AND” operator. The search strategies are included in Appendix 1: Table 3. To ensure the comprehensiveness of the literature search, references to the selected and related articles were reviewed as well. Furthermore, a manual search was carried out on the resources of networks, organizations and conferences related to the topic, including unpublished studies of national or local organizations. In order to access unpublished information sources, experts in the field of hospital operation were contacted and access to the identified resources was obtained through personal visits or correspondence with the experts.
Inclusion and exclusion criteria
The following criteria were used as a guide for searching and screening the articles. The inclusion criteria were English language studies, studies evaluating public hospital indicators, and original studies and reviews including systematic review, meta-analysis, meta-synthesis, scoping review, narrative review, rapid review, critical review, and integrative review. Studies on the indicators of health centers, the health system at the macro level, clinics and community health indicators, journals that did not have a precise review process, and articles such as book reviews, commentaries and opinion articles were adopted as the criteria for exclusion.
The articles obtained from the search bases were individually reviewed by two people in three stages (title, abstract and full text). The final decision was made on the basis of agreement, which would require the comments of a third party if agreement was not reached. Screening was effected using the EndNote v.8 software. Given that quality assessment is not commonly performed in scoping reviews, the quality of the articles was not investigated in this research .
According to the refined studies, the data were extracted in order to meet the research objectives and questions. To this end, a data extraction form was initially designed and tested on 10 randomly selected papers. Article authors, years, countries, types of study, study objectives, settings, and indices were extracted on this basis. At this stage, one of the authors extracted the data from the selected articles, and the second author examined the data. The form was designed and completed for each article in the Excel software.
The Best Fit Framework Synthesis Method was used to analyze the extracted data. In this way, the most suitable model related to the topic was selected, and the initial themes were created. The codes extracted from the articles were subsequently positioned in front of the themes [39, 40]. In the present study, the framework introduced by Australia was selected as the primary framework for the performance indicators of public hospitals, which were analyzed in terms of equity, effectiveness and efficiency . According to this framework, the equity dimension includes the fair access indicators. Also the three dimensions of access, appropriateness and quality are used to assess service effectiveness. For the evaluation of quality, the model adopts the dimensions of safety, responsiveness and continuity of care. Finally, in order to assess efficiency, the sustainability of serviced was taken into account (Fig. 1).
Based on the Best Fit Method, the selected framework might change during the research and data collection, whereby a new conceptual framework could be generated . Under this method, both deductive and inductive approaches were therefore used for data analysis  (Fig. 1).
Performance indicators were initially coded as semantic units. In the first stage, indicators related to the dimensions of the initial model were inserted deductively through explicit analysis. Specific words including equity, effectiveness, and efficiency were searched and their related indicators were identified and positioned through the closed coding method. In the second stage, indicators that were not included in the initial framework were classified inductively through the open coding method. For this purpose, the articles were studied one or several times for immersion. The indicators were then identified as semantic units through an implicit approach. In the following stage, the codes were grouped on the basis of semantic similarities. After that, the codes of each study were compared with those of other studies and ultimately classified as themes and sub-themes. Finally, the results of these two stages were put together and a new framework was created.
Before using the open access studies, the journals or authors of the articles were contacted and their permission was obtained. In order to prevent bias, all stages of the study such as screening, data extraction and data analysis were carried out by two individuals.
In the initial search, 146,504 English articles were found in scientific databases and by means of manual search, with duplicate and unrelated articles being removed, and 12,163 articles were reviewed. In the second stage, 1136 studies were reviewed based on their abstracts. As a result, 723 articles were excluded because they did not meet the inclusion criteria (413 ones were selected). Finally, after reviewing the full texts of the remaining articles, 49 ones were considered eligible to enter the study (Fig. 2). The features of these studies are summarized in Appendix 2: Table 4. Iran, USA and Brazil had 10, 8 and 5 articles respectively, Australia and Canada had 3, Britain, Turkey, Greece, and New Zealand had 2 articles, and Spain, Romania, Saudi Arabia and Japan had 1 paper; there was 1 article from the OECD countries and 1 from Nordic countries, and the other studies were reviews.
Based on the Best Fit Method, the final model included the efficiency/productivity themes, the effectiveness of the original model and the financial theme identified from the literature review (Fig. 3). The efficiency/productivity sub-themes included human resources indicators, hospital beds, costs, operating room productivity, emergency rooms, ICU, radiology, labs, technology and equipment productivity. Other sub-themes relate to general indicators such as bed occupancy rate, mean length of stay, number of outpatients and hospitalized patients. Financial themes were categorized into eight sub-themes including: profit, revenue, cash flow, cost, investment, asset, debt and liquidity. Concerning effectiveness, the indicators were further categorized into the four sub-themes of access (equity), safety, quality and responsiveness. The accountability indicators were classified into three categories: patient-centeredness, staff orientation, and social responsibility.
The indicators extracted from the studies are shown in Table 2 based on the final model. In this study, 173 indicators of public hospital performance evaluation were identified, most of which were in the effectiveness dimension (100 indicators). Regarding efficiency and financial dimensions, 41 and 32 indicators respectively were identified Best Fit Method.
As demonstrated by this and other studies, there exist various objectives, fields of inquiry and methodological approaches when it comes to evaluating hospital performance; with each study having its specific objective and approach (Appendix 2: Table 4). In any study, performance evaluation frameworks and indicators are selected and evaluated according to the objective of the study. The resulting differences may be due in part to national policies and plans or to technical differences in the health systems of countries . However, the experience of different countries in selecting and using the indicators can be useful to policymakers, health managers and researchers in other countries . The present study seeks to present the indicators used to evaluate hospital performance in the form of a comprehensive package. The indicators concerned have been classified under three main headings (efficiency/utilization, finance and effectiveness), as discussed below.
Analysis of the selected studies shows that the model adopted in this study differs from the original model (Figs. 1, 3). In the original (Fig. 1), equity (access) was considered a major dimension of hospital performance as well as one of the subsets of effectiveness. Given that most studies assigned indicators of equity in access to the effectiveness dimension, and that this dimension was in practice often used in macro-decisions of the Ministry and was less likely to come within the scope of the authority of hospital managers [1,2,3,4,5,6,7], access (equity) in the proposed model was considered one of the subsets of the effectiveness of hospital services, along with other indicators such as safety, quality and responsiveness. In the proposed model, safety and responsiveness were included among the main subsets of effectiveness in view of their importance in hospitals.
Another dimension of the original model was efficiency, which was developed in the proposed model in view of the variety and diversity of the indicators used in previous studies. The indicators of efficiency were organized into ten sub-categories, most of which emphasized utilization of resources and equipment in different parts of the hospital, such as the operating room (OR), emergency room (ER), ICU and laboratory.
The results of this review showed that financial issues were of great importance in hospital performance evaluation studies. Limited financial resources and increased hospital expenses could explain why directors and researchers tend to focus on financial areas. However, new models and frameworks in the field of performance evaluation emphasize the multidimensional aspects of hospital performance and underline that other dimensions, in addition to finance, need to be taken into account . In the proposed model, the effectiveness dimension, including the aspects of quality, safety, access, suitability and responsiveness, also has its place. Service effectiveness and improvement are not only factors of customer satisfaction (including patients, staff and the wider community) but also help to reduce costs and increase hospital income. In what follows, we discuss the dimensions of the proposed model in more detail.
One of the challenges faced by health managers throughout the world is hospital efficiency  given that hospitals represent a large proportion of national health expenditures. In 2012, hospitals accounted for about 30% of total health expenditures in the OECD countries and 37% in the EU countries . In their study, Lotfi et al. described hospitals as “organizations with inefficient resource management, low profitability, and low-quality services” (especially in developing countries). They stated that this poor management entailed a waste of resources and was a barrier to the efficiency of hospitals. Efficiency is therefore one of the most important factors in performance management systems in health-care organizations [23, 24, 44].
In the present study, several indicators were employed to evaluate efficiency as an major dimension of hospital performance. In the framework provided by WHO, efficiency is one of the six main dimensions of hospital performance evaluation . Based on the findings, 17 studies used efficiency indicators in evaluating the performance of hospitals [20, 22,23,24, 26,27,28,29, 45,46,47,48,49,50,51,52,53]. These indicators were categorized under the sub-themes of human resources, hospital beds, costs, operating room productivity, emergency rooms, ICU, radiology, laboratory, technology and facilities productivity. Some of the most important indicators of efficiency are the number of human resources, bed occupancy rate, length of stay, utilization rate of the existing technologies, and the rate of drug prescription [47, 48].
Human resources, are considered important aspects of hospital efficiency evaluation [46, 54]. For instance, the number of hospital staff per bed is a key indicator in evaluating hospital performance and efficiency. The lower this ratio, the more productive and efficient the hospital will be . The quality of care is another major indicator that must be taken into consideration. Additionally, a very low rate of bed occupancy, which represents the rate of hospital bed use, indicates a low level of hospital efficiency, which is highly correlated with the patients’ length of stay and bed turnover .
Another important issue in evaluating hospitals efficiency is cost. In their study, Pink et al. aimed to select key financial indicators for Ontario hospitals, and considered efficiency to be one of the five main dimensions of hospital financial performance. They measured efficiency indicators in terms of the ability to provide services at the level of predicted costs and to minimize management costs. They further selected the cost performance index of departments (units) and the percentage of corporate services as measures for evaluating hospital efficiency .
Operating rooms (ORs) are among the most vital and expensive parts of hospitals since 60% to 80% of hospital admissions involve surgical interventions. This sector accounts for more than 40% of the total hospital costs and a large proportion of hospital income [55, 56]. Utilization of OR affects the outcomes of surgical patients in hospitals so that even a small problem in the OR process can impact on the overall quality and performance of the hospital. Inefficiency of OR lead to delays in service delivery to patients, which can result in dissatisfaction on the part of patients and health care providers . Hence, with the increase in financial pressures, most hospitals are looking for ways to enhance their income and reduce avoidable costs through the evaluation of OR processes. Given the impact of OR performance on hospital productivity, assets and personnel, many hospitals are devoting substantial resources to improving efficiency in this regard [55, 56].
Emergency departments play a major role in hospital performance since they deal with the most numerous, diverse, troubled and sensitive groups of patients, requiring prompt care and service [57, 58]. The number of patients treated and the duration of treatment in the emergency department were identified in the present research as indicators of efficiency and utilization of emergency departments. In the study by Kang et al., the most important emergency performance criteria were the timing of the various stages of emergency processes and the number of patients (admitted, in the waiting queue, and cancelled appointments) . Horwitz et al. introduced the waiting time and length of visit as important indicators of the efficiency, timeliness, safety and patient-centeredness of emergency care .
The DEA and Pabon Lasso approaches are two of the most widely used methods for evaluating hospital efficiency. Using hospital indicators, both methods consider hospital inputs and outputs to measure efficiency. DEA is a linear programming approach that examines the relationship between hospital inputs and outputs, comparing them with the ideal (optimum) process [9, 23, 28, 45, 48]. Although there are limitations in linking inputs to outputs or health care outcomes (such as the lack of activity-based costs), there are also opportunities in measuring efficiency via the optimal use of available and accessible technologies, productivity rate, staff ratios and financial management .
One of the common dimensions of performance evaluation is the financial aspect [20, 60]. In this regard, hospital financial models are unique in terms of their design and application and are affected by a hospital’s mission, goals, financing and accounting methods; the needs of population covered; the form of insurance reimbursement and the type of ownership. Hospital managers can overcome the hospital’s economic problems, make the right decisions, clarify the unit cost of services and create a competitive situation to provide goods and services applying a suitable financial evaluation model .
The results of this study indicated that 15 studies used financial indicators in evaluating hospital performance [19, 22, 27, 29, 30, 45, 50, 62, 63]. Based on the literature review, the different indicators used to evaluate financial performance are categorized into 8 sub-themes including Profit: total marginal profit, medical benefit–cost–per FTE); Revenue: operating revenue per adjusted patient days, non-operating revenue, current ratio, revenue per physician FTE; Cash flow: cash to total debt; Cost: operating costs per adjusted patient days, unit cost performance, cost of outpatient visits, cost of salaries and overtime, emergency services expenses, personnel expenses, goods and services expenses, medicine expenses, average cost per day of hospitalization, pharmacy costs; Investment: return on investment; Asset: total asset turnover, tangible assets, return on assets; Debt: total debt/total assets, long-term debt to capitalization, debt ratio; and Liquidity: current ratio, days revenue in net accounts receivable, days cash in hand, average payment period, replacement viability, acid test ratio, quick ratio, budget flow compared to approved budget) [61,62,63,64,65,66].
Classification of financial indicators focuses on the financial status of a hospital. Since the evaluation of each dimension of financial performance by itself may lead to a wrong decisions and plans, it is necessary to review them simultaneously. For instance, the evaluation of profitability indicators demonstrate the financial gain of a hospital, but liquidity indicators may suggest the inability of the hospital to pay off debts (bills) [27, 61]. Indicators of net profit or loss and operating profit or loss only represent and analyze the balance between income and expenses .
Along with what has been discussed and per the current environment in Iran, the poor economic condition and political sanctions have a detrimental influence on Iranian hospital financial performance and cause financial distress. Early detection of this condition by hospital manager is critically important. Many studies mentioned that the most effective and operational index in this regard is the cost/revenue ratio in public governmental hospitals [19, 59, 84].
Failure to provide effective health services reduces the quality of life, increases the burden of disease and disability and finally prevents the promotion of productivity in other economic, social and political areas . The need to provide effective services has therefore always been a major issue. Performance measurement is a tool for evaluating the effectiveness of any organizational activity . Thus the studies of Braithwaite et al. on eleven identified frameworks found that the effectiveness dimension had the most frequent replication in the performance evaluation frameworks .
Based on data extracted from the literature, 20 studies used indicators related to the effectiveness of hospital services [19, 20, 22, 26, 29, 30, 47, 48, 52, 53, 60, 63,64,65,66,67,68,69,70,71], categorized in the four sub-themes of access (equity), safety, quality and responsiveness. Although hospitals have tended to concentrate on improving efficiency (until the 1990s), recent efforts have addressed the issues of safety, quality, responsiveness and equity [26, 71].
First of all, the effectiveness of health services depends on the fair access of people to health services . Access to medical care is a relatively complex multidimensional issue. From the perspective of a behavioral model, access includes six dimensions: potential access, achieved access, fair access, unfair access, efficient access and effective access . In the Australian health performance framework, access to services was mentioned as part of the hospital performance evaluation. For instance, waiting times for elective surgeries and waiting times in emergency rooms were indicators of access to hospital services. The waiting time for surgery is indicative of the timeliness of the provision of services based on need . In the study of Khalifa et al., patient access indicators included the number of referred patients, admitted patients and those waiting in line for admission . Nerenz et al. considered easy access and waiting time as factors affecting patient satisfaction . Ioan et al. also considered access and equity as aspects of hospital responsiveness . In their study, Davis et al. used ethnic, social, and economic diversities to evaluate equity .
Another factor influencing the effectiveness of hospital activities is the quality of the services provided . Quality of care refers to the clinical content of the care provided for a specific group of patients. However, it also includes certain quality indicators such as hospital infection or satisfaction of all patients admitted to the hospital . Quality influences the effectiveness of activities as well as financial performance through its impact on profitability, cost, customer loyalty, and customer attraction . Thus, quality is a key determinant of market share, return on investment, and cost reduction . So, the need for evidence-based decision-making, measurable improvement, and useful information for comparison has led to an increasing emphasis on quality assessment in the health system . However, the existence of unrestricted indicators related to the quality of services has rendered this dimension of performance evaluation heterogeneous. In the presented frameworks, quality indicators were categorized in different ways. For example, in the Donabedian model, quality was represented by the three concepts of structure, process, and output . The SERVQUAL model also classified service quality into five categories: tangibles, reliability, accountability, service assurance, and empathy [77, 78]. Thus, the vital position of performance quality for all health beneficiaries (specialists, policymakers, service providers and service recipients) has led several studies to focus on the quality of hospital services and various indicators to be used in relation to their objectives.
Another factor influencing the effectiveness of hospital activities is the safety of the services provided. Although safety is one of the basic principles and elements of quality, it has recently been studied separately in certain cases . Patient safety is focused on treatment effectiveness, and its indicators directly reflect treatment effectiveness [30, 68]. In various studies, safety has been considered a dimension of hospital performance evaluation, including the safety of patients, personnel and environment [17, 63]. The framework presented in the study by Veillard et al. highlighted the central role of safety in the governance of health systems and hospital management. Patient safety includes issues such as the development and use of standard guidelines, quality monitoring, issuance of prescriptions and drug delivery, infection control mechanisms, continuing care and professional qualifications . McLoughlin et al. selected 21 indicators for countries and classified them into five categories: hospital infections, operation and postoperative complications, sentinel events, midwifery, and other care-related incidents .
Responsiveness indicators, based on patient feedback, are of great importance in evaluating hospital performance. In certain studies, responsiveness has been regarded as a separate dimension of hospital performance [30, 48]. Based on the analyses conducted in this study, responsiveness encompasses three fields:
Patient centeredness is defined in terms of patient feedback management, patient satisfaction, personnel and hospital environment, patient autonomy (meaning explanation of procedures and informed selection of treatment by the patient), dignity of patients, confidentiality, prompt attention, basic amenities and a social support network;
Staff orientation covering staff burnout, absenteeism, overtime worked, satisfaction with working environment, clearly defined responsibilities, average remuneration, diversity, working hours, frequency of night duty/shift work, position occupied, average experience in current department, personnel safety, number of work-related injuries, paid leave, number of staff per bed, continuous education for health professionals, number of training hours against total number of working hours, training budget against total budget dedicated to staff and vacancy;
Social responsibility is described by leadership and inner processes (including mission and vision), policies and procedures, ethical codes, regulations and procedures, marketing in terms of suppliers and contractors, supply chain, consumer rights, responsibilities and liability management services (including responsible purchasing) and the workplace environment (including staff safety and health and issues of sustainable development, pollution and waste) [75,76,77,78].
This approach is in accordance with Simou et al. who classified responsiveness indicators under the two categories of patient centeredness and staff orientation . These various indicators show the wide compass of this dimension and the importance of this aspect in hospital performance evaluation.
The foregoing indicators in the field of hospital management are extracted from the entire range of existing literature and derived from various countries with a diversity of policies, cultures and rules. It is claimed that careful and comprehensive consideration and categorization of these indicators yield a conceptual framework that can be used as a basic theory and model synthesis worldwide, while remaining subject to adjustment and customization according to each country`s culture, rules and policies and the structure of the health system concerned.
Hospital performance management is a multi-dimensional issue, with each dimension having its own significance. One-dimensional performance evaluation can lead to incorrect policy-making and decisions. On the other hand, several indicators of diversity in the literature highlight the scope and complexity of hospital performance. Based on the evidence, indicators are dependent on the evaluation model employed, the evaluation objective and the views of executive managers and participants in the study. It follows that a comprehensive and complete performance evaluation system is conditional upon the selection of the most appropriate indicators as a first step.
Key performance indicators (KPIs) are considered essential decision-making tools for policymakers and managers at national and local hospitals.
Developing a comprehensive framework to provide the indicators used to evaluate hospital performance.
The synthesis of evidence on hospital performance indicators was carried out through a scoping review and the indicators were analyzed using the Best Fit Method.
Based on the Best Fit Method, the final model included the topics of efficiency/productivity, the effectiveness of the original model and the financial aspects as identified from the literature review.
Through a comprehensive review and summarization of all studies related to the same research question, knowledge synthesis interprets the results of those studies within a general framework of evidence, ultimately helping policymakers and managers with planning and decision making.
Hospital performance management is a multi-dimensional issue, with each dimension having its own significance. One-dimensional performance evaluation leads to incorrect policy making and decisions. On the other hand, several indicators of diversity in the literature highlight the scope and complexity of hospital performance. Based on the evidence, indicators are dependent on the evaluation model employed, the evaluation objective, the views of executive managers, and the study participants. It follows that a comprehensive and complete performance evaluation system is conditional upon the selection of the most appropriate indicators as a first step.
Pourmohammadi K, Shojaei P, Rahimi H, Bastani P. Evaluating the health system financing of the Eastern Mediterranean Region (EMR) countries using grey relation analysis and shannon entropy. Cost Eff Resour Alloc. 2018;16(1):31.
World Health O. How can hospital performance be measured and monitored? How can hospital performance be measured and monitored? 2003; p. 17.
Masoumpour SM, Rahimi SH, Kharazmi E, Kavousi Z, Mosalah NH, Abedi Z. Assessing waiting time in emergency department of Shahid Faghihi hospital, Shiraz and presenting appropriate strategies using quality function deployment (QFD) method, 2011–2012. Hakim Res J. 2013;16(2):159–68.
Zarchi MR, Jabbari A, Rahimi SH, Shafaghat T, Abbasi S. Preparation and designing a checklist for health care marketing mix, with medical tourism approach. Int J Travel Med Glob Health. 2013;1:103–8.
Bastani P, Vatankhah S, Salehi M. Performance ratio analysis: a national study on Iranian hospitals affiliated to ministry of Health and Medical Education. Iran J Public Health. 2013;42(8):876.
Shadpour K. Health sector reform in Islamic Republic of Iran. Hakim Res J. 2006;9(3):1–18.
Gapenski LC, Pink GH. Understanding healthcare financial management. Chicago: Health Administration Press; 2007.
Parkinson J, Tsasis P, Porporato M. A critical review of financial measures as reported in the Ontario hospital balanced scorecard. J Health Care Finance. 2007;34(2):48–56.
Suarez V, Lesneski C, Denison D. Making the case for using financial indicators in local public health agencies. Am J Public Health. 2011;101(3):419–25.
Novick LF, Morrow CB. A framework for public health administration and practice. Public Health Admin. 2008;2:35–68.
Chen F-H, Hsu T-S, Tzeng G-H. A balanced scorecard approach to establish a performance evaluation and relationship model for hot spring hotels based on a hybrid MCDM model combining DEMATEL and ANP. Int J Hosp Manag. 2011;30(4):908–32.
B GWG. Role of analytical tools in airline flight safety management systems. Global Aviation Information Network; 2004.
Watkins AL. Hospital financial ratio classification patterns revisited: upon considering nonfinancial information. J Acc Public Pol. 2000;19(1):73–95.
Rahimi H, Khammar-nia M, Kavosi Z, Eslahi M. Indicators of hospital performance evaluation: a systematic review. Int J Hosp Res. 2014;3(4):199–208.
Pizzini MJ. The relation between cost-system design, managers’ evaluations of the relevance and usefulness of cost data, and financial performance: an empirical study of US hospitals. Acc Organ Soc. 2006;31(2):179–210.
Pro DD, Con BM. Should key performance indicators for clinical services be mandatory? Can J Hosp Pharm. 2011;6(1):55.
Veillard J, Guisset A-L, Garcia-Barbero M. Selection of indicators for Hospital Performance Measurement: A report on the 3rd and 4th Workshop. Regional Office for Europe of the World Health Organization; 2003.
Hilderink H, Hilderink HBM, editors. Scenario development methodology. 2013.
Rahimi H, Kavosi Z, Shojaei P, Kharazmi E. Key performance indicators in hospital based on balanced scorecard model. J Health Manag Inform. 2016;4(1):17–24.
Gu X, Itoh K. Performance indicators: healthcare professionals’ views. Int J Health Care Qual Assur. 2016;29(7):801–15.
Rahimi H, Bahmaei J, Shojaei P, Kavosi Z, Khavasi M. Developing a strategy map to improve public hospitals performance with balanced scorecard and dematel approach. Shiraz E Med J. 2018;19(7):e64056.
Valdmanis VG, Rosko MD, Mutter RL. Hospital quality, efficiency, and input slack differentials. Health Serv Res. 2008;43(5p2):1830–48.
Mehrtak M, Yusefzadeh H, Jaafaripooyan E. Pabon Lasso and data envelopment analysis: a complementary approach to hospital performance measurement. Global J Health Sci. 2014;6(4):107.
Lotfi F, Kalhor R, Bastani P, Zadeh NS, Eslamian M, Dehghani MR, et al. Various indicators for the assessment of hospitals’ performance status: differences and similarities. Iran Red Cresc Med J. 2014;16(4):e12950.
Goshtasebi A, Vahdaninia M, Gorgipour R, Samanpour A, Maftoon F, Farzadi F, et al. Assessing hospital performance by the Pabon Lasso Model. Iran J Public Health. 2009;38(2):119–24.
Davis P, Milne B, Parker K, Hider P, Lay-Yee R, Cumming J, et al. Efficiency, effectiveness, equity (E3). Evaluating hospital performance in three dimensions. Health Policy. 2013;112(1):19–27.
Pink GH, Daniel I, Hall LM, McKillop I. Selection of key financial indicators: A literature, panel and survey approach. Law Governance. 2007;11(3):87.
Xenos P, Yfantopoulos J, Nektarios M, Polyzos N, Tinios P, Constantopoulos A. Efficiency and productivity assessment of public hospitals in Greece during the crisis period 2009–2012. Cost Effect Resour Alloc. 2017;15(1):6.
Gholamzadeh Nikjoo R, Jabbari Beyrami H, Jannati A, Asghari Jaafarabadi M. Selecting hospital’s key performance indicators, using analytic hierarchy process technique. J Commun Health Res. 2013;2(1):30–8.
Khalifa M, Khalid P. Developing strategic health care key performance indicators: a case study on a tertiary care hospital. Procedia Comput Sci. 2015;63:459–66.
McAlearney AS, Hefner JL, Sieck C, Rizer M, Huerta TR. Evidence-based management of ambulatory electronic health record system implementation: an assessment of conceptual support and qualitative evidence. Int J Med Inform. 2014;83(7):484–94.
Arndt M, Bigelow B. Evidence-based management in health care organizations: a cautionary note. Health Care Manage Rev. 2009;34(3):206–13.
Head BW. Reconsidering evidence-based policy: key issues and challenges. Policy Soc. 2010;29(2):77–94.
Tricco AC, Cardoso R, Thomas SM, Motiwala S, Sullivan S, Kealey MR, et al. Barriers and facilitators to uptake of systematic reviews by policy makers and health care managers: a scoping review. Implem Sci. 2015;11(1):4.
Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.
Brien SE, Lorenzetti DL, Lewis S, Kennedy J, Ghali WA. Overview of a formal scoping review on health system report cards. Implement Sci. 2010;5(1):2.
Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5(1):69.
Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Inform Libr J. 2009;26(2):91–108.
Carroll C, Booth A, Cooper K. A worked example of” best fit” framework synthesis: a systematic review of views concerning the taking of some potential chemopreventive agents. BMC Med Res Methodol. 2011;11(1):29.
Dixon-Woods M. Using framework-based synthesis for conducting reviews of qualitative studies. BMC Med. 2011;9(1):39.
Report on Government Services; PART E, CHAPTER 12: public hospitals. Australia: Australian Government productivity commission; 2018.
Carroll C, Booth A, Leaviss J, Rick J. “Best fit” framework synthesis: refining the method. BMC Med Res Methodol. 2013;13(1):37.
Braithwaite J, Hibbert P, Blakely B, Plumb J, Hannaford N, Long JC, et al. Health system frameworks and performance indicators in eight countries: a comparative international analysis. SAGE Open Med. 2017;5:1–10.
Mesabbah M, Arisha A. Performance management of the public healthcare services in Ireland: a review. Int J Health Care Qual Assur. 2016;29(2):209–35.
Büchner VA, Hinz V, Schreyögg J. Health systems: changes in hospital efficiency and profitability. Health Care Manag Sci. 2016;19(2):130–43.
Ramos MCA, Cruz LP, Kishima VC, Pollara WM, Lira ACO, Couttolenc BF. Performance evaluation of hospitals that provide care in the public health system, Brazil. Revista de saude publica. 2015;49:43.
Basu A, Howell R, Gopinath D. Clinical performance indicators: intolerance for variety? Int J Health Care Qual Assur. 2010;23(4):436–49.
Simou E, Pliatsika P, Koutsogeorgou E, Roumeliotou A. Developing a national framework of quality indicators for public hospitals. Int J Health Plan Manag. 2014;29(3):e187.
Nabilou B, Yusefzadeh H, Rezapour A, Azar FEF, Safi PS, Asiabar AS, et al. The productivity and its barriers in public hospitals: case study of ran. Med J Islam Repub Iran. 2016;30:316.
Chirikos TN, Sear AM. Measuring hospital efficiency: a comparison of two approaches. Health Serv Res. 2000;34(6):1389.
Schneider JE, Ohsfeldt RL, Morrisey MA, Li P, Miller TR, Zelner BA. Effects of specialty hospitals on the financial performance of general hospitals, 1997–2004. Inquiry. 2007;44(3):321–34.
Øvretveit J. Quality evaluation and indicator comparison in health care. Int J Health Plan Manag. 2001;16(3):229–41.
Toprak DK, Sahin B. The impacts of ISO 9000 quality management on the performance of public hospitals. Amme Idaresi Dergisi. 2013;46(3):113–40.
Anand S, Bärnighausen T. Human resources and health outcomes: cross-country econometric study. Lancet. 2004;364(9445):1603–9.
Lin Q-L, Liu L, Liu H-C, Wang D-J. Integrating hierarchical balanced scorecard with fuzzy linguistic for evaluating operating room performance in hospitals. Expert Syst Appl. 2013;40(6):1917–24.
Attaallah AF, Elzamzamy OM, Phelps AL, Ranganthan P, Vallejo MC. Increasing operating room efficiency through electronic medical record analysis. J Perioper Pract. 2016;26(5):106–13.
Rahimi SH, Masoumpour M, Kharazmi E, Kavousi Z. Designing the Quality of Emergency Ward Services in Shirazs Shahid Faghihi Hospital Based on Quality Function Deployment Method (QFD) in 2011–2012. J Hosp. 2013;12(3):9–17.
Kang H, Nembhard H, DeFlitch C, Pasupathy K. Assessment of emergency department efficiency using data envelopment analysis. IISE Trans Healthc Syst Eng. 2017;7(4):236–46.
Horwitz LI, Green J, Bradley EH. US emergency department performance on wait time and length of visit. Ann Emerg Med. 2010;55(2):133–41.
Nerenz D, Neil N. Performance measures for health care systems. Commissioned paper for the center for health management research. 2001.
Janati A, Valizadeh S, Jafarabadi MA. Development of financial indicators of hospital performance. J Clin Res Govern. 2014;3:92–8.
Epané JP, Weech-Maldonado R, Hearld L, Menachemi N, Sen B, O’Connor S, et al. Hospitals’ use of hospitalists: implications for financial performance. Health Care Manag Rev. 2017. https://doi.org/10.1097/HMR.0000000000000170.
Ioan B, Nestian AS, Tita S-M. Relevance of key performance indicators (KPIs) in a hospital performance management model. J East Eur Res Business Econ. 2012;2012:1.
Aghaei Hashjin A, Kringos DS, Manoochehri J, Aryankhesal A, Klazinga NS. Development and impact of the Iranian hospital performance measurement program. BMC Health Serv Res. 2014;14:448.
Oliveira DF, Arieta CEL, Temporini ER, Kara-José N. Quality of health care: patient satisfaction in a university hospital. Arquivos brasileiros de oftalmologia. 2006;69(5):731–6.
Gabriel CS, da Melo MRA, Rocha FLR, Bernardes A, Miguelaci T, de Silva MD. Use of performance indicators in the nursing service of a public hospital. Revista latino-americana de enfermagem. 2011;19(5):1247–54.
Keyvanara M, Sajadi HS. Social responsibility of the hospitals in Isfahan city, Iran: results from a cross-sectional survey. Int J Health Pol Manag. 2015;4(8):517.
McLoughlin V, Millar J, Mattke S, Franca M, Jonsson PM, Somekh D, et al. Selecting indicators for patient safety at the health system level in OECD countries. Int J Qual Health Care. 2006;18(suppl_1):14–20.
Fugaca NP, Cubas MR, Carvalho DR. Use of balanced indicators as a management tool in nursing. Revista latino-americana de enfermagem. 2015;23(6):1049–56.
Ozgulbas N, Koyuncugil AS. Financial profiling of public hospitals: an application by data mining. Int J Health Plan Manag. 2009;24(1):69–83.
Hefford M, Crampton P, Foley J. Reducing health disparities through primary care reform: the New Zealand experiment. Health Policy. 2005;72(1):9–23.
Andersen RM, Davidson PL, Baumeister SE. Improving access to care. Changing the US health care system: Key issues in health services policy and management. 2013. p. 33.
Australian Institute of Health and Welfare. Australia’s health 2010. Canberra; 2010.
Bahmei J, Rahimi H, Rahgoshay I, Kavosi Z. Quality evaluation of emergency department services of Nemazee Hospital from the Patients’ Viewpoint. Taṣvīr-i salāmat. 2016;7(1):18–26.
Raju PS, Lonial SC. The impact of service quality and marketing on financial performance in the hospital industry: an empirical examination. J Retail Consum Serv. 2002;9(6):335–48.
Nekoei-Moghadam M, Amiresmaili M. Hospital services quality assessment: hospitals of Kerman University of Medical Sciences, as a tangible example of a developing country. Int J Health Care Qual Assur. 2011;24(1):57–66.
Calisir F, Bayraktaroglu AE, Gumussoy CA, Kaya B. Effects of service quality dimensions including usability on perceived overall quality, customer satisfaction, and return intention in different hospital types. Int J Adv Oper Manag. 2014;6(4):309–23.
Melo AI, Santinha G, Lima R. Measuring the Quality of Health Services Using SERVQUAL: Evidence From Portugal. Handbook of Research on Modernization and Accountability in Public Sector Management. Hershey: IGI Global; 2018. p. 300–18.
Xenos P, Yfantopoulos J, Nektarios M, Polyzos N, Tinios P, Constantopoulos A. Efficiency and productivity assessment of public hospitals in Greece during the crisis period 2009–2012. Cost Eff Resour Alloc. 2017;15:6.
Nabilou B, Yusefzadeh H, Rezapour A, Azar FE, Safi PS, Asiabar AS, et al. The productivity and its barriers in public hospitals: case study of Iran. Med J Islam Rep Iran. 2016;30(1):36–43.
Zwanziger J, Khan N, Bamezai A. The relationship between safety net activities and hospital financial performance. BMC Health Serv Res. 2010;10:15.
Culica D, Prezio E. Hospital board infrastructure and functions: the role of governance in financial performance. Int J Environ Res Public Health. 2009;6(3):862–73.
Alexander JA, Weiner BJ, Griffith J. Quality improvement and hospital financial performance. J Org Behav. 2006;27(7):1003–29.
Reiter KL, Sandoval GA, Brown AD, Pink GH. CEO compensation and hospital financial performance. Med Care Res Rev. 2009;66(6):725–38.
PB designed the study and its overall methodology; she also finalized the data synthesis and the article itself. KP searched all the databases, retrieved the sources and prepared the initial draft of the article. PS contributed to data analysis and edited the article. The study was supervised by NH. All authors read and approved the final manuscript.
This research, derived from Proposal No. 95-01-07-13769, was conducted by Mrs. Kimia Pourmohammadi as part of the activities required for a Ph.D. degree in health care management at the Shiraz University of Medical Sciences. The authors wish to express their sincere gratitude to the research administration of Shiraz University of Medical Sciences for its administrative support and to the English Editor Mr. Kamyar Pourmohammadi and improving native English Mr. John David Corbett as the United Naions and UNESCO reviser/editor.
The author declares that they have no competing interests.
Availability of data and materials
Data is available in an endnote library.
Consent for publication
There was no difficulty in publishing the results. All the included databases and materials are available for public use.
Ethics approval and consent to participate
This study is approved by Shiraz University of Medical Sciences ethics committee with the ID Number of IR.SUMS.REC.1396.S274.
There is no funding.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Pourmohammadi, K., Hatam, N., Shojaei, P. et al. A comprehensive map of the evidence on the performance evaluation indicators of public hospitals: a scoping study and best fit framework synthesis. Cost Eff Resour Alloc 16, 64 (2018) doi:10.1186/s12962-018-0166-z
- Key performance indicators
- Scoping review
- Best Fit Framework Synthesis