Evaluating the Reform of the Healthcare System in Sicily : variations of efficiency and appropriateness between 2008 and 2010

Background: Sicilian government has developed a very ambitious Reform through Regional Law n. 5 (14th April 2009). Hospitals were requested to ensure the quality of care through monitoring of appropriateness and quality of service. The aim of this study was to assess variations of efficiency and organizational appropriateness of healthcare delivery before and after this Reform, and to show patterns associated to different types of healthcare delivery organizations. Methods: This study was based on repeated cross-sectional data for 118 (out of 129) short-term, acute-care, non teaching-and-research Sicilian hospitals, in 2008 and 2010. Congestion and slacks analysis was used, with four inputs, two desirable outputs and two undesirable outputs of healthcare delivery. Results: The loss of desirable output increased between 2008 (23%) and 2010 (31%). Most of the variation between the two years in the measured inefficiency could be attributed to congestion due to inappropriate care (p=0.009) and scale inefficiency (p=0.028). Hospitals that have undergone an organizational transformation did not show congestion in the study period. Conversely, hospitals with no variations in their organization were congested in association to the shortfall in the ODs (p=0.019) and in DHs (p=0.018). Conclusion: This study has shown the general worsening of efficiency of acute-care Sicilian hospitals from 2008 to 2010 and, in particular, has suggested that the reduction of efficiency was due to hospitals that have not undergone an organizational transformation. They are medium-low sized and low-complexity public hospitals and for-profits, while larger and high-complexity organizations were shown to be the least congested.


INTRODUCTION
The Italian National Healthcare System (INHS) was introduced in 1978 and has undergone several reforms since 1992, which were intended to increase institutional and financial autonomy while reducing inefficiencies, with the direct assignment of income tax revenue to Regions.
Public healthcare in Italy is organized into Hospital Trusts (HTs) and Local Public Hospitals (LPHs), which were defined according to the Legislative Decree 502/92 (Article 4).LPHs are public hospitals which are managed by Local Health Units (LHUs), with economic-financial autonomy and separate accountability within the budget of each LHU.Following the act of the Ministry of Health on the proposal of the Regions, hospitals that are equipped with special functional technical requirements are considered to be HTs.
At the end of the nineties, an approach based on Essential Levels of Care (LEAs -Livelli Essenziali di Assistenza) was introduced.These LEAs marked the passage from universalized healthcare to a system based on selected benefits that the State, the Regions and the LHUs must compulsorily deliver to the citizen while respecting the principles of human dignity, quality of care and equity in accessing care.The LEA Decree (2001, 29th November) established the main areas of healthcare services to be guaranteed by the INHS (positive list), those completely excluded by public coverage (negative list), and those partially covered (only available for specific clinical conditions).The positive list was based on the recognition and systematization of current legislation (other decrees, laws, guidelines, etc.), by including all the services that must be ensured to all citizens, categorized in three macrolevels of care: public health services, community care, and hospital care [1].
In the early noughties, the process of regionalization of healthcare delivery was reinforced and the Regions with excess healthcare expenditure have committed to reducing their deficit, by signing an agreement with the central State, through financial recovery plans (Law 296/2006) [2].
Dissatisfaction with the Regional Healthcare System (RHS) and its high costs led Sicilian government to develop a very ambitious Reform (RHS Reform from now on), which was established through Regional Law n. 5, dated 14 th April 2009.Hospitals were requested to ensure the quality of care through monitoring of appropriateness, adequacy and quality of service, rate of bed occupancy, treatment and accessibility.
The main interventions introduced by the RHS Reform were: 1. the reduction of beds in accredited private structures (for-profits) and the transformation of 550 beds from acute-care to long-term care and rehabilitation; 2. the redefinition of the hospital network.The RHS Reform established the merger or change of organizational structure from HT to LPH for 15 of the 20 HTs existing in Sicily in 2008 (Table 1).Merger of two or more HTs implies the institution of a main HT, with a significant reduction of beds and the redefinition of the care pathways and the redistribution of medical specialties among the hospitals that make up the main HT.Moreover, in order to increase territorial healthcare delivery, so-called Territorial Outpatient Services (PTA-Punti Territoriali di Assistenza) were introduced in addition to LPHs.These are open 24 hours and enable access to first aid, emergency care, outpatient care, general practice, vaccinations and home visits.
3. the integration of day-hospital provision and territorial outpatient services, the reduction of ordinary hospital admissions and day-hospital admissions, the strengthening of outpatient care, in lieu of inpatient day-hospital admissions, the reduction of admission and first aid which are either inappropriate or at high risk of being inappropriate, and finally, the introduction of outpatient day-service.4. the implementation of an integrated clinical network based on the "hub and spoke model", which consists of transferring cases with high complexity from minor hospitals (spoke) widespread throughout an area to a small number of major hospitals (hub).There are several studies aimed at assessing the impact that changes in the Italian health system have had on healthcare efficiency.The review [3] has analysed how different organizational models adopted in Italy's healthcare services and patient mobility may affect healthcare efficiency at a regional level.The literature focuses particularly on the negative effects of public policies aimed at reducing hospitalization rates [4], on the effects of organizational structure and level of specialization of hospitals [5], of administrative decentralization [6][7], of variation in the Beveridge healthcare system [8] and of society and the third sector [9].Other literature focuses on the role of healthcare organization on efficiency evaluation of OECD countries, taking into account the effect of institutional arrangements [10], of policy instruments that directly target patient behaviours, compared to Beveridgian or Bismarckian financing arrangements or gatekeeping [11][12].
The main scope of this paper was to assess variations of efficiency and organizational appropriateness of healthcare delivery in Sicily between 2008 (before the Reform) and 2010 (soon after the Reform) and to show and explain patterns associated to different types of healthcare delivery organization.
The analysis was intended to compare (i) profit vs. non-profit hospitals, (ii) hospitals that have undergone an organizational transformation and hospitals that have not and (iii) acute care hospitals vs. low-complexity territorial services.The specific part of the RHS Reform, which is analysed in this work, is the redefinition of the hospital network and the reduction and re-functioning of beds and personnel.Congestion analysis, a non-parametric, multi-input and multi-output statistical model, was used to incorporate appropriateness indicators as quality measures in the evaluation of technical efficiency.

Data
This study was based on repeated cross-sectional data for 118 Sicilian hospitals relating the years 2008 and 2010.Hospitals included in the study were short-term, acute-care, non teaching-and-research hospitals and they had to be homogeneous with regards to the employed inputs and the delivered outputs.Of the 125 acute-care hospitals in Sicily in 2008, seven hospitals were excluded because they were teaching-and-research hospitals.The data on personnel used in this study have been extracted from the Health System Database of the Italian Health Department, while the data on hospital care delivery have been extracted from the Hospital Discharge Records (HDR) Database of the Sicilian Region for the years 2008 and 2010.

Variables
The empirical literature on the estimation of technical efficiency in the healthcare sector strongly suggests the number of discharged patients to be the most reliable measure of output, since the number of inpatient days could reflect a productive choice of hospitals [4].Therefore, this study was based on ordinary discharges (medical plus surgical) (ODs) adjusted for case-mix and day hospital admissions (medical plus surgical) (DHs).The following services were excluded as they refer to activities which are outside the field of this study: hospital emergency; home care; rehabilitation; long-term care; neurologicalrehabilitation; the collection, manufacture, testing and distribution of blood components; transfusion services and tissue and organ transplants.
In line with other literature [3][4], four inputs were included as inpatient beds, medical staff (physicians, surgeons and dentists), nursing staff, and other personnel.Input data on global resources, such as drugs, diagnostic exams and instruments, were not available for hospitals included in the analysis.
As measures of organizational inappropriateness, the analysis included inappropriate ODs and inappropriate In order to assess how hospital ownership and the redefinition of the hospitals' networks affect efficiency and appropriateness, two variables were included in the analysis.It is well established that organizations with different ownership structures could respond to incentives and adjust their behavior in different ways and at different speeds.In this study the variable ownership was included, categorized as public vs for-profits, as it was found there is evidence of convergence in the mean level of efficiency between public and not-for-profit hospitals [4].Furthermore, organizational transformation, categorized as variation vs no-variation, was defined as a binary variable where variation occurred if the hospital merged with other hospitals or changed its structure from HT to LPH.

Statistical methods
Efficiency and organizational appropriateness of Sicilian hospitals was estimated for the years 2008 and 2010 through Congestion analysis.The concept of outputcongestion was considered because it allows evaluation of the loss of desirable outputs of the healthcare delivery process, which is caused by the simultaneous occurrence of undesirable outputs.Furthermore, an output-oriented Data Envelopment Analysis (DEA) model was employed, because hospitals are required to optimize the delivery of appropriated health care procedures using the amount of resources available.
As we described in more detail in a previous paper [13], congestion analysis is obtained as a modification of the well-known non-parametric DEA [14][15], by relaxing the so-called strong disposability of outputs assumption into the weak disposability of outputs assumption [16][17].The output measure of total efficiency decomposes into the product Total Efficiency (TE)=Pure Technical Efficiency (PTE)*Scale Efficiency (SE)*Congestion (CO) 13,18].All components of efficiency and total efficiency itself must not be less than unity, so that a unity score indicates that the hospital is operating on the best practice frontier (i.e., that it is output-efficient), while a score more than unity indicates inefficiency.
A binary variable was used to detect congested hospitals, assuming unity if the loss of desirable output (e.g.medical and surgical discharges) was related to the simultaneous occurrence of inappropriate outputs, and zero otherwise.
In a second phase, to assess the effect of including inappropriateness indicators in DEA analysis, we divided desirable outputs by congestion scores, we repeated DEA using the resulting adjusted outputs and the same inputs, and we calculated the slack values using the input-based approach [19].If the difference in the input slacks between congestion-unadjusted and -adjusted DEA is positive, it means that congestion causes the hospital to employ excess input that leads to inefficiency.Conversely, a negative difference in the output slacks indicates that congestion causes the hospital to have a deficit of some outputs that leads to inefficiency.To check the sensitivity of efficiency scores for outlier hospitals, we used the Data cloud method [20].We performed congestion analysis and the Data cloud method using the package FEAR [21] and the slacks analysis using the Benchmarking package, all running in the R environment (http://cran.r-project.org).

RESULTS
The data-set included 60 public hospitals (5 HTs and 55 LPHs), contributing the 69.8% and 62.8% of OD and DH, respectively, of all public hospitals in 2008 and the 68.8% and 61.6% of OD and DH, respectively, of all public hospitals in 2010; 58 for-profits, contributing the 92.6% of OD and the 97.0% of DH of all for-profits, in the two years.Of the 118 hospitals considered in this study, 13(11.0%)had a variation of their organizational structure either for change (9 HTs became LPHs) or for a merger (4 HTs were coupled into 2).
The number of physicians decreased statistically significantly from 2008 to 2010 (p=0.020) and this trend was due to public hospitals, changed from 101.57±121.59 in 2008 to 105.3.5±121.51 in 2010 (p<0.001,data not in table), without any difference among hospitals based on the variation in their organizational structure (Table 2).
From 2008 to 2010 healthcare delivery reduced significantly, both appropriate (p=0.003 for ODs) and inappropriate (p<0.001 for both ODs and DHs).This trend was observed for both hospitals with variation (p=0.042 for appropriate ODs and p=0.002 for appropriate DHs and inappropriate ODs and DHs) and no-variation of organizational structure (p=0.004 for appropriate ODs and p≤0.001 for inappropriate ODs and DHs) (Table 2).Analogously, it was observed for both public hospitals (p=0.001 for appropriate ODs and p=0.022 for appropriate DHs and p<0.001 for inappropriate ODs and DHs) and for-profits (p=0.045 for appropriate DHs and p<0.001 for inappropriate ODs and DHs) (data not in table).
In 2010, 49% of inappropriate ODs were related to medical cases discharged from surgical wards for all hospital types (49% also in 2008), followed by 27% of admissions with DRG at risk of being inappropriate (23% in 2008), followed by 19% of medical cases with length of stay 0-1 days for both HTs (24% in 2008) (Figure 1).
In Table 3, we show the output-based efficiency scores and the output enlargement needed because of inefficiency by year and by organizational transformation.On average, Sicilian hospitals in 2010 could have produced 31% (=(1.45-1)/1.45))more outputs, without using any additional inputs, if they had operated on the best practice frontier.This loss of desirable output was significantly higher than that observed in 2008 (23.1%).Most of the variation between the two years in the measured inefficiency could be attributed to congestion due to inappropriate care (p=0.009) and SE (p=0.028).It was statistically significant only for hospitals with no-variation in their organization (p=0.019 for TE and p=0.014 for SE).
The number of congested hospitals increased significantly from 54 (45.8%) in 2008 to 71 (60.2%) in 2010 (p=0.027).For congested hospitals, there was Evaluating the Reform of Sicilian Health System a significant increase of the average length of stay (p=0.036) and of the turnover index (p=0.003),which was associated to a significant reduction of the occupancy rate (p=0.037) and of the rotation index (p=0.002).This trend could be attributed to the hospitals with no-variation in their organizational structure, and in particular to the
No statistically significant difference between two years was found in either the components of efficiency, or in the performance indicators for uncongested hospitals (Table 4).
In Table 5, we show differences in the slack values between DEA analysis adjusted for congestion and unadjusted DEA, by organizational transformation.In 2008, hospitals with no-variation could improve their efficiency significantly if they were to have used fewer nurses (p=0.031) and other personnel (p=0.047) while   e12593-7 providing more ODs (p=0.032).In 2010, congestion of these hospitals seems to be associated to the shortfall in the ODs (p=0.019) and in DHs (p=0.018),notwithstanding the reduced excess in beds (p=0.046) and physicians (p=0.012).For hospitals that have undergone an organizational transformation, slacks analysis did not show any significant output shortfall or input excess either in 2008 or in 2010.

DISCUSSION
Results of this repeated cross-sectional study for Sicilian acute-care hospitals from 2008 to 2010 showed that there was a statistically significant reduction of total efficiency, which could be attributed to scale and congestion components.The RHS Reform occurred in 2009 and has mitigated the overall observed reduction of efficiency in a different way depending on the different types of healthcare delivery organizations.In fact, the deterioration in the components of efficiency could be attributed to the observed increase in scale diseconomies and was evident especially for hospitals with no-variation of their organizational structure, that are medium-low sized and low-complexity public hospitals and for-profits.
In 2010, as the introduction of PTAs had not yet been completed, these hospitals were tasked with delivering lowcomplexity territorial services, with negative consequences on their efficiency and It was even more marked in hospitals situated in small municipalities, where social and health care, residential and home care was less widespread [22].It would be advisable to extend the merging process to other hospitals in the region, as it has been shown that inefficiency is associated to scale diseconomies [23][24].The observed deterioration in performance indicators for congested hospitals could suggest to policy makers to focus further efforts of renewal and restructuring of hospitals characterized by critical values of the performance indicators.In fact, many of the factors that contribute poor quality are also potential causes of technical inefficiency.This is because poorly performing organizations are typically inefficient in the production of everything: quantitatively as well as qualitatively [25].Slack analysis confirms that in order to contrast inefficiency and inappropriateness, it is advisable to counteract the use of excess personnel and to increase appropriate ODs.
The process of incorporation and reorganization of healthcare provision dictated by the RHS reform has also led to dramatic reductions of inputs, especially beds, physicians and other personnel.The reduction of physicians can be attributed to the implementation of the region's financial recovery plan, which established a freeze on hiring for public hospitals, with specialized personnel being transferred to HTs and personnel of LPHs remaining on duty until exhaustion.On the contrary, the number of physicians in accredited private hospitals remained unchanged, as an effect of the choice of the Sicilian Health Department to use the workforce as a parameter of performance evaluation to assign tariff bands and the annual budget.It is shown that the high personnel density is a consequence of the high proportion of small hospitals.In Italy, the proportion of doctors is the Evaluating the Reform of Sicilian Health System same in all hospitals, irrespective of their size, while in most other OECD countries it is proportional to the size of the hospital.Moreover, in southern Italy the ratio of medical/nonmedical staff is the highest in the country [2].Therefore, reducing the number of physicians and modifying the composition of the health-care workforce is a key factor in reducing personnel costs per hospital bed and improving efficiency.
Results suggest that the increase of resource usage and the reduction of appropriate care are associated with the increase of output-congestion.It is not surprising because one of the aims of the RHS Reform was to rationalize the use of beds and personnel, to reduce hospital delivery in favour of territorial outpatient healthcare, to reduce waiting lists and passive patients' mobility.However, hospitals must go a long way to counteract inappropriateness, especially with regards to some DRGs at risk of inappropriateness, by transferring patients with low complexity illness to territorial outpatient settings and by keeping only acute and more complex patients.Furthermore, the surplus of personnel and beds rendered available by the reduction of inappropriate admissions should be used to produce appropriate services.Some caution is needed when considering results of this study.First of all, results do not apply to long-term rehabilitation, teaching-and-research hospitals.Therefore, the findings of this study cannot be generalized to these kinds of hospitals and another study should be appropriately designed to this end.Secondly, regarding the choice of inputs and outputs used in this analysis, as the same inputs are likely to be shared among different activities and there is no quantitative information about additional, non care-related activities (e.g. research and development and first aid), it is possible that pure efficiency was under-estimated [26] with consequences on the scale and congestion components as well.Finally, when interpreting results of this study it should be considered that some measures planned by the RHS Reform had not yet been fulfilled in 2010, while others take longer to show their effect, like the conversion of beds from acute-care to long-term care.
DHs.With regard to inappropriate ODs, the indicators of the Annual Report of Hospitalization of the Italian Health Department were chosen.They are discharges with medical DRG from surgical wards; discharges with DRG belonging to the list of 43 DRGs at risk of being inappropriate, excluding cases that have exceptions, as indicated in the LEA Decree; the ordinary admissions with medical DRG and duration of 0-1 days; the admissions with medical DRG of patients aged ≥65 years and length of stay beyond the threshold.With regards to inappropriate DHs, the choice fell on six DRGs indicated as a priority from the Decree n. 875, dated 11th May 2009, containing guidelines on implementation of the RHS Reform.They are DRGs number 06=decompression carpal tunnel; 039=interventions on the lens with and without vitrectomy; 266=skin grafts; 270= other interventions on the skin; 410= chemotherapy; 503= interventions on the knee without principal diagnosis of infection.
FIGURE 1. Percentage of inappropriate ordinary discharges by year and indicator of organizational inappropriateness

TABLE 1 .
Redefinition of the acute-care hospital network following the 2009 Reform, by variation of the organizational structure for either merger or change from HT to LPH § .

TABLE 2 .
Inputs and outputs of healthcare delivery for 118 Sicilian, acute-care, non teaching-and-research hospitals by organizational transformation § .Years 2008-2010 §Variation of the organizational structure occurred if the hospital merged with other hospitals or changed its structure from HT to LPH. § § Wilcoxon matched-pair rank test between hospitals' efficiency scores in 2008 and 2010.

TABLE 3 .
Output-based efficiency scores and output enlargement needed because of inefficiency by organizational transformation §

TABLE 4 .
Efficiency scores and performance indicators: comparison between congested and uncongested hospitals, by organizational transformation §Variation of the organizational structure occurred if the hospital merged with other hospitals or changed its structure from HT to LPH.
1 Kruskall-Wallis test for all variables and z-test for proportions for N(%).

TABLE 5 .
Input and Output Slacks obtained with DEA analysis adjusted for congestion compared to unadjusted DEA, by organizational transformation § Variation of the organizational structure occurred if the hospital merged with other hospitals or changed its structure from HT to LPH.