As demonstrated by this and other studies, there exist various objectives, fields of inquiry and methodological approaches when it comes to evaluating hospital performance; with each study having its specific objective and approach (Appendix 2: Table 4). In any study, performance evaluation frameworks and indicators are selected and evaluated according to the objective of the study. The resulting differences may be due in part to national policies and plans or to technical differences in the health systems of countries [26]. However, the experience of different countries in selecting and using the indicators can be useful to policymakers, health managers and researchers in other countries [43]. The present study seeks to present the indicators used to evaluate hospital performance in the form of a comprehensive package. The indicators concerned have been classified under three main headings (efficiency/utilization, finance and effectiveness), as discussed below.
Analysis of the selected studies shows that the model adopted in this study differs from the original model (Figs. 1, 3). In the original (Fig. 1), equity (access) was considered a major dimension of hospital performance as well as one of the subsets of effectiveness. Given that most studies assigned indicators of equity in access to the effectiveness dimension, and that this dimension was in practice often used in macro-decisions of the Ministry and was less likely to come within the scope of the authority of hospital managers [1,2,3,4,5,6,7], access (equity) in the proposed model was considered one of the subsets of the effectiveness of hospital services, along with other indicators such as safety, quality and responsiveness. In the proposed model, safety and responsiveness were included among the main subsets of effectiveness in view of their importance in hospitals.
Another dimension of the original model was efficiency, which was developed in the proposed model in view of the variety and diversity of the indicators used in previous studies. The indicators of efficiency were organized into ten sub-categories, most of which emphasized utilization of resources and equipment in different parts of the hospital, such as the operating room (OR), emergency room (ER), ICU and laboratory.
The results of this review showed that financial issues were of great importance in hospital performance evaluation studies. Limited financial resources and increased hospital expenses could explain why directors and researchers tend to focus on financial areas. However, new models and frameworks in the field of performance evaluation emphasize the multidimensional aspects of hospital performance and underline that other dimensions, in addition to finance, need to be taken into account [8]. In the proposed model, the effectiveness dimension, including the aspects of quality, safety, access, suitability and responsiveness, also has its place. Service effectiveness and improvement are not only factors of customer satisfaction (including patients, staff and the wider community) but also help to reduce costs and increase hospital income. In what follows, we discuss the dimensions of the proposed model in more detail.
Efficiency/utilization
One of the challenges faced by health managers throughout the world is hospital efficiency [26] given that hospitals represent a large proportion of national health expenditures. In 2012, hospitals accounted for about 30% of total health expenditures in the OECD countries and 37% in the EU countries [28]. In their study, Lotfi et al. described hospitals as “organizations with inefficient resource management, low profitability, and low-quality services” (especially in developing countries). They stated that this poor management entailed a waste of resources and was a barrier to the efficiency of hospitals. Efficiency is therefore one of the most important factors in performance management systems in health-care organizations [23, 24, 44].
In the present study, several indicators were employed to evaluate efficiency as an major dimension of hospital performance. In the framework provided by WHO, efficiency is one of the six main dimensions of hospital performance evaluation [17]. Based on the findings, 17 studies used efficiency indicators in evaluating the performance of hospitals [20, 22,23,24, 26,27,28,29, 45,46,47,48,49,50,51,52,53]. These indicators were categorized under the sub-themes of human resources, hospital beds, costs, operating room productivity, emergency rooms, ICU, radiology, laboratory, technology and facilities productivity. Some of the most important indicators of efficiency are the number of human resources, bed occupancy rate, length of stay, utilization rate of the existing technologies, and the rate of drug prescription [47, 48].
Human resources, are considered important aspects of hospital efficiency evaluation [46, 54]. For instance, the number of hospital staff per bed is a key indicator in evaluating hospital performance and efficiency. The lower this ratio, the more productive and efficient the hospital will be [50]. The quality of care is another major indicator that must be taken into consideration. Additionally, a very low rate of bed occupancy, which represents the rate of hospital bed use, indicates a low level of hospital efficiency, which is highly correlated with the patients’ length of stay and bed turnover [46].
Another important issue in evaluating hospitals efficiency is cost. In their study, Pink et al. aimed to select key financial indicators for Ontario hospitals, and considered efficiency to be one of the five main dimensions of hospital financial performance. They measured efficiency indicators in terms of the ability to provide services at the level of predicted costs and to minimize management costs. They further selected the cost performance index of departments (units) and the percentage of corporate services as measures for evaluating hospital efficiency [27].
Operating rooms (ORs) are among the most vital and expensive parts of hospitals since 60% to 80% of hospital admissions involve surgical interventions. This sector accounts for more than 40% of the total hospital costs and a large proportion of hospital income [55, 56]. Utilization of OR affects the outcomes of surgical patients in hospitals so that even a small problem in the OR process can impact on the overall quality and performance of the hospital. Inefficiency of OR lead to delays in service delivery to patients, which can result in dissatisfaction on the part of patients and health care providers [55]. Hence, with the increase in financial pressures, most hospitals are looking for ways to enhance their income and reduce avoidable costs through the evaluation of OR processes. Given the impact of OR performance on hospital productivity, assets and personnel, many hospitals are devoting substantial resources to improving efficiency in this regard [55, 56].
Emergency departments play a major role in hospital performance since they deal with the most numerous, diverse, troubled and sensitive groups of patients, requiring prompt care and service [57, 58]. The number of patients treated and the duration of treatment in the emergency department were identified in the present research as indicators of efficiency and utilization of emergency departments. In the study by Kang et al., the most important emergency performance criteria were the timing of the various stages of emergency processes and the number of patients (admitted, in the waiting queue, and cancelled appointments) [58]. Horwitz et al. introduced the waiting time and length of visit as important indicators of the efficiency, timeliness, safety and patient-centeredness of emergency care [59].
The DEA and Pabon Lasso approaches are two of the most widely used methods for evaluating hospital efficiency. Using hospital indicators, both methods consider hospital inputs and outputs to measure efficiency. DEA is a linear programming approach that examines the relationship between hospital inputs and outputs, comparing them with the ideal (optimum) process [9, 23, 28, 45, 48]. Although there are limitations in linking inputs to outputs or health care outcomes (such as the lack of activity-based costs), there are also opportunities in measuring efficiency via the optimal use of available and accessible technologies, productivity rate, staff ratios and financial management [17].
Finances
One of the common dimensions of performance evaluation is the financial aspect [20, 60]. In this regard, hospital financial models are unique in terms of their design and application and are affected by a hospital’s mission, goals, financing and accounting methods; the needs of population covered; the form of insurance reimbursement and the type of ownership. Hospital managers can overcome the hospital’s economic problems, make the right decisions, clarify the unit cost of services and create a competitive situation to provide goods and services applying a suitable financial evaluation model [61].
The results of this study indicated that 15 studies used financial indicators in evaluating hospital performance [19, 22, 27, 29, 30, 45, 50, 62, 63]. Based on the literature review, the different indicators used to evaluate financial performance are categorized into 8 sub-themes including Profit: total marginal profit, medical benefit–cost–per FTE); Revenue: operating revenue per adjusted patient days, non-operating revenue, current ratio, revenue per physician FTE; Cash flow: cash to total debt; Cost: operating costs per adjusted patient days, unit cost performance, cost of outpatient visits, cost of salaries and overtime, emergency services expenses, personnel expenses, goods and services expenses, medicine expenses, average cost per day of hospitalization, pharmacy costs; Investment: return on investment; Asset: total asset turnover, tangible assets, return on assets; Debt: total debt/total assets, long-term debt to capitalization, debt ratio; and Liquidity: current ratio, days revenue in net accounts receivable, days cash in hand, average payment period, replacement viability, acid test ratio, quick ratio, budget flow compared to approved budget) [61,62,63,64,65,66].
Classification of financial indicators focuses on the financial status of a hospital. Since the evaluation of each dimension of financial performance by itself may lead to a wrong decisions and plans, it is necessary to review them simultaneously. For instance, the evaluation of profitability indicators demonstrate the financial gain of a hospital, but liquidity indicators may suggest the inability of the hospital to pay off debts (bills) [27, 61]. Indicators of net profit or loss and operating profit or loss only represent and analyze the balance between income and expenses [60].
Along with what has been discussed and per the current environment in Iran, the poor economic condition and political sanctions have a detrimental influence on Iranian hospital financial performance and cause financial distress. Early detection of this condition by hospital manager is critically important. Many studies mentioned that the most effective and operational index in this regard is the cost/revenue ratio in public governmental hospitals [19, 59, 84].
Effectiveness
Failure to provide effective health services reduces the quality of life, increases the burden of disease and disability and finally prevents the promotion of productivity in other economic, social and political areas [49]. The need to provide effective services has therefore always been a major issue. Performance measurement is a tool for evaluating the effectiveness of any organizational activity [47]. Thus the studies of Braithwaite et al. on eleven identified frameworks found that the effectiveness dimension had the most frequent replication in the performance evaluation frameworks [43].
Based on data extracted from the literature, 20 studies used indicators related to the effectiveness of hospital services [19, 20, 22, 26, 29, 30, 47, 48, 52, 53, 60, 63,64,65,66,67,68,69,70,71], categorized in the four sub-themes of access (equity), safety, quality and responsiveness. Although hospitals have tended to concentrate on improving efficiency (until the 1990s), recent efforts have addressed the issues of safety, quality, responsiveness and equity [26, 71].
First of all, the effectiveness of health services depends on the fair access of people to health services [26]. Access to medical care is a relatively complex multidimensional issue. From the perspective of a behavioral model, access includes six dimensions: potential access, achieved access, fair access, unfair access, efficient access and effective access [72]. In the Australian health performance framework, access to services was mentioned as part of the hospital performance evaluation. For instance, waiting times for elective surgeries and waiting times in emergency rooms were indicators of access to hospital services. The waiting time for surgery is indicative of the timeliness of the provision of services based on need [73]. In the study of Khalifa et al., patient access indicators included the number of referred patients, admitted patients and those waiting in line for admission [30]. Nerenz et al. considered easy access and waiting time as factors affecting patient satisfaction [60]. Ioan et al. also considered access and equity as aspects of hospital responsiveness [63]. In their study, Davis et al. used ethnic, social, and economic diversities to evaluate equity [26].
Another factor influencing the effectiveness of hospital activities is the quality of the services provided [74]. Quality of care refers to the clinical content of the care provided for a specific group of patients. However, it also includes certain quality indicators such as hospital infection or satisfaction of all patients admitted to the hospital [60]. Quality influences the effectiveness of activities as well as financial performance through its impact on profitability, cost, customer loyalty, and customer attraction [75]. Thus, quality is a key determinant of market share, return on investment, and cost reduction [76]. So, the need for evidence-based decision-making, measurable improvement, and useful information for comparison has led to an increasing emphasis on quality assessment in the health system [48]. However, the existence of unrestricted indicators related to the quality of services has rendered this dimension of performance evaluation heterogeneous. In the presented frameworks, quality indicators were categorized in different ways. For example, in the Donabedian model, quality was represented by the three concepts of structure, process, and output [60]. The SERVQUAL model also classified service quality into five categories: tangibles, reliability, accountability, service assurance, and empathy [77, 78]. Thus, the vital position of performance quality for all health beneficiaries (specialists, policymakers, service providers and service recipients) has led several studies to focus on the quality of hospital services and various indicators to be used in relation to their objectives.
Another factor influencing the effectiveness of hospital activities is the safety of the services provided. Although safety is one of the basic principles and elements of quality, it has recently been studied separately in certain cases [68]. Patient safety is focused on treatment effectiveness, and its indicators directly reflect treatment effectiveness [30, 68]. In various studies, safety has been considered a dimension of hospital performance evaluation, including the safety of patients, personnel and environment [17, 63]. The framework presented in the study by Veillard et al. highlighted the central role of safety in the governance of health systems and hospital management. Patient safety includes issues such as the development and use of standard guidelines, quality monitoring, issuance of prescriptions and drug delivery, infection control mechanisms, continuing care and professional qualifications [17]. McLoughlin et al. selected 21 indicators for countries and classified them into five categories: hospital infections, operation and postoperative complications, sentinel events, midwifery, and other care-related incidents [68].
Responsiveness indicators, based on patient feedback, are of great importance in evaluating hospital performance. In certain studies, responsiveness has been regarded as a separate dimension of hospital performance [30, 48]. Based on the analyses conducted in this study, responsiveness encompasses three fields:
Patient centeredness is defined in terms of patient feedback management, patient satisfaction, personnel and hospital environment, patient autonomy (meaning explanation of procedures and informed selection of treatment by the patient), dignity of patients, confidentiality, prompt attention, basic amenities and a social support network;
Staff orientation covering staff burnout, absenteeism, overtime worked, satisfaction with working environment, clearly defined responsibilities, average remuneration, diversity, working hours, frequency of night duty/shift work, position occupied, average experience in current department, personnel safety, number of work-related injuries, paid leave, number of staff per bed, continuous education for health professionals, number of training hours against total number of working hours, training budget against total budget dedicated to staff and vacancy;
Social responsibility is described by leadership and inner processes (including mission and vision), policies and procedures, ethical codes, regulations and procedures, marketing in terms of suppliers and contractors, supply chain, consumer rights, responsibilities and liability management services (including responsible purchasing) and the workplace environment (including staff safety and health and issues of sustainable development, pollution and waste) [75,76,77,78].
This approach is in accordance with Simou et al. who classified responsiveness indicators under the two categories of patient centeredness and staff orientation [48]. These various indicators show the wide compass of this dimension and the importance of this aspect in hospital performance evaluation.
The foregoing indicators in the field of hospital management are extracted from the entire range of existing literature and derived from various countries with a diversity of policies, cultures and rules. It is claimed that careful and comprehensive consideration and categorization of these indicators yield a conceptual framework that can be used as a basic theory and model synthesis worldwide, while remaining subject to adjustment and customization according to each country`s culture, rules and policies and the structure of the health system concerned.