Performance evaluation of hospitals: the proposal of Agenas and Sant’Anna School of Advanced Studies

The National Agency for Regional Health Services (Agenas), in collaboration with the Management and Health Laboratory (MeS) of the Scuola Superiore Sant’Anna in Pisa, presented on July 8 in Naples the experiment with the measurement and evaluation system for the performance of hospitals. .

“The initiative was launched on impulse from Agenas, which has the institutional task of monitoring and evaluating the results of both health companies and regional health services – he explains. Milena Vainieri, head of the Mes Laboratory of the Sant’Anna School of Advanced Studies -. The experiment, inspired by the dimensions of Donabedian quality measurement – i.e. structure, process and result – is based on two consolidated measurement systems – National results programme and Evaluation system for regional health systemsadds a new analysis perspective, which photographs the hospital’s organizational structure and some financial-management choices”.

“Evaluation of health services is one of the institutional tasks of the Board of Regional Health Services – he declares Domenico Mantoan, General Manager Agenas – Law 145/2018 requires the agency to implement a system for analyzing and monitoring the performance of healthcare companies, which includes the economic-management, organizational, financial and accounting, clinical assistance dimensions in addition to the clinical and the diagnostic-therapeutic processes, of the quality, of the safety and of the results of the treatments. The experiment presented on July 8 by Agenas in collaboration with the Management and Health Laboratory (MeS) at the Scuola Superiore Sant’Anna in Pisa, is an important starting point to support regions and companies in the integrated and multidimensional analysis of health services. services”.

“The opportunity that PNRR offers – he comments Sabina Nuti, Rector of the Scuola Superiore Sant’Anna – is asking our national health service and our regional and corporate health systems to make further efforts towards transparency and accountability. Only the public debate on the results can make it possible to direct investments towards the projects that will be able to return the greatest added value and contribute to reducing the citizenship gap, which is one of the cross-cutting priorities of the plan itself.”

The prototype was applied to the year 2019 to have the pre-pandemic starting point and to verify the reliability and relevance of the new indicators introduced with this work. The analysis available for the Regions and the Hospitals concerns around 70 indicators calculated for 52 Hospitals, companies for which all information and economic-management and organizational flows are available.

The aim and perspectives of the trial

The ratio therefore represents one methodological proposal calculation and production of results. This proposal will be developed based on the feedback collected and updated. The paper presents a guide for consulting the indicators and summary representations as well as reporting the methods for calculating the indicators regarding structure and process.

“The goal of the project is to provide tools that help politicians make decisions, understand the strengths and weaknesses of each company in a multidimensional perspective. The model is based on consolidated measurement and evaluation systems on the national stage. We considered in particular the National Outcomes Program ( better known by the acronym PNE) and a number of process indicators related to the clinical assistance, but also management and organizational areas such as indicators related to budget balance and to personnel management, included in the performance evaluation system of the regional network (better known as the target) – explains Vainieri, head of Mes Laboratory -.

“We started by measuring the hospitals’ performance because the administrative flows around hospital treatment are consolidated; focusing on one level of care (instead of three), it is “easier” to launch and validate the method”

We started by measuring the hospitals’ performance because the administrative flows regarding hospital care are consolidated; focusing on one level of care (instead of three), it is more “simple” to start and validate the method. We have collected information that is already present and available at a national level and transform it into a dashboard of indicators that allows you to compare your company’s performance with other hospitals for each area and indicator, finally returning an overall picture through summary representations”.

hospital benefit

The theoretical framework is the one used internationally by most countries looking to measure performance in the round, with a tool that provides guidance on critical issues

How did you operate? “The theoretical framework is the one used internationally by most countries that want to measure performance in the round, with a tool that provides guidance on critical issues. This approach involves three areas: structure, process and outcome. We understood the structure as a snapshot of the companies’ structural choices, even in the past: the size of the staff, the degree of outsourcing, the investment policies such as the use of owned machines or in leasing or renting – says the professor -. Then we continued to evaluate the process through a selection of indicators already present in the system of the network of regions, added others of a management nature that represent the overall performance with the objective and finally, for the result, we made use of the 21 indicators selected using the overall evaluation expressed by tree map of PNE. The result is a really useful dashboard, where the hospitals can read at a glance what performance has been achieved in the results and in the processes and put it in relation to the starting conditions and the setup choices from a comparative point of view.” .

The dashboard can therefore be used by companies and regions to understand where improvement measures must be implemented

The dashboard can therefore be used by companies and regions to understand where they need to implement improvement actions: “Without quantitative documentation and without comparison, it is sometimes difficult to understand whether the result achieved by your company is good or can be improved – says Milena Vainieri – . The system highlights that for some indicators, such as voluntary discharge or discharge against the doctor’s opinion, there is a clear difference between North and South, while on other indicators different performances are recorded between companies, independent of the geographical area, such as the percentage of staff absenteeism. in this case, within the same region, for example Campania, there are companies with absenteeism rates below 8% and those with absenteeism rates two or three times higher.It should be noted, however, that indicating the color as in a traffic light should not be considered a assessment, but a support to understand the areas that need to be worked on more immediately, which also ö gives a kind of prioritization. It should also be emphasized that there are no “all green” or “all red” companies: it is always a mix, and you will not have a single number with a “ranking” of which companies are doing better or worse. Since they are complex organizations with thousands of professionals at work and a context with as many restrictions and rules to respect as the public administration, we analyze how each company manages to produce the health result and also highlight the aspects that must be respected. reward or to be proud of”.


Why 2019? “A year that now seems so distant to us is ideal to test the method and think about the instrument, otherwise there would have been a risk of resistance from the companies – says Vainieri -. It is also a year of normality: to test a certain years like 2020 would have been too challenging. Now the aim is to replicate it over the following years and ensure that a systematic monitoring, updated over time, is a tool available to decision makers and company management.

A further step will be to expand the panel of companies involved: we are thinking of the private sector and local healthcare companies

Another step will be to try to expand the panel of companies involved: so far 52 non-specialist, public companies have entered. The reason lies in the availability of data. Since most of the indicators are based on the flows in the income statement and the balance sheet, or the balance sheet, available to Agenas, we were able to test only the complete model for public companies. It is also desirable to extend it to private companies to try to get a complete picture of how hospitals work by region. Another step will be to try to translate the model to local companies as well”.

Finally, the teacher explains, the next report will be available in an interactive way, making it even more usable by healthcare companies.

“An important step, as commented by Director Mantoan and Professor Nuti, is to be able to transparently report the performance results achieved by each company to support national, regional and corporate planning – concludes the teacher -. Further progress towards the use of ​​information assets, such as the information systems of the NHS and public administrations, and the development of tools to help make sense of the jungle of data and information available to businesses and the NHS”.

Leave a Comment