Evaluating the effect of organization and context on technical efficiency : a second-stage DEA analysis of Italian hospitals

Objective: the purpose of this study was to compare the technical efficiency of italian hospitals at a regional level and to examine if differences could be explained by organisational and contextual factors. technical efficiency was defined as the ability of the operating units evaluated to use optimal resource levels for their level of output. MethOds: the effect of external factors was explored through a second stage data envelopment Analysis (deA). efficiency scores were calculated for each hospital using the deA method (stage i). through tobit regression analysis, the estimated efficiency scores were regressed against a set of organisational and contextual characteristics beyond managerial control, which reflected differences in the population demographics and regional health expenditure (stage ii). stage i and stage ii efficiency scores were compared in order to indirectly assess managerial contribution in relation to hospital efficiency. Results: the highest efficiency (M±sd) was observed in hospitals in the North-West (75.7±15.1), followed by those in the North-east (75.5±15.1), central italy (73.9±16.4) and then southern italy (70.6±17.9). hospital trusts (hts) were shown to be more technically efficient than local Public hospitals (lPhs). Organisational and contextual indicators were statistically significantly different at tobit regression analysis for hts and lPhs. emilia Romagna and lombardia were the regions whose management contributed to increased efficiency. cONclusiONs: in our study, the distribution of regions according to technical efficiency only partly reflected the North-south gradient shown by other studies regarding the gap of expenditure. the important role of organisation and environment in establishing efficiency differences among hospitals was demonstrated.


SECONd-STAGE dEA ANALySIS IN ITALy iNtROductiON
Performance evaluation plays a strategic role in healthcare organizations, in order to address the best use of resources and rationing of demand [1,2].The evaluation of technical efficiency of existing hospitals is necessary to improve hospital performance, so as to employ medical resources effectively and make the healthcare system more efficient and sustainable.
Technical efficiency describes the ability of operating units to transform their inputs into outputs, such that when an operating unit is technically efficient, it works on its production frontier [3].Using this definition, a hospital is considered to be efficient if it produces the maximum amount of output from a given amount of inputs (output-oriented efficiency) or alternatively, if it produces a given output with a minimum consumption of resources (input-oriented efficiency).Most studies in healthcare use an input-oriented model because it is assumed that hospitals cannot reject their patients' demands, while they can reduce resource allocation and consumption levels [4].Alternatively, if hospitals are required to improve the appropriateness of healthcare procedures, whilst maintaining the amount of resources consumed, an output-oriented model can instead be proposed [5].
DEA is a non-parametric technique used in the estimation of production functions and has been used extensively to estimate measures of technical efficiency [24,25].In the standard input-based technical efficiency DEA, hospitals are considered as homogeneous decision making units (DMUs) which are responsible for converting inputs into outputs and whose performances can be evaluated.DEA derives a best practice frontier by solving linear programming problems, which identify the best performing hospitals, minimizing inputs for given outputs.Technical efficiency is measured as a distance to the frontier, so that DEA scores are relative measures of efficiency.The ratio of a weighted sum of outputs to a weighted sum of inputs is obtained, with the condition that the ratio of outputs to inputs should not exceed unity for every DMU.A score of 100% indicates that a hospital is operating on the best practice frontier (i.e., that it is efficient).A score <100% indicates inefficiency with the difference between the actual score and 100 measuring the amount of all inputs which could be decreased whilst maintaining outputs constant.
The purpose of this paper was to compare the technical efficiency of Italian hospitals at a regional level and to examine if differences in technical efficiency could be explained by organization and context, through a second stage DEA analysis.

MethOds data
The data used in this study have been extracted from the 2007 Health System Database of the Italian Health Department, available at http://www.salute.gov.it/servizio/datisis.jsp.Of the 648 facilities included in the Database, only 481 hospitals were considered in the analysis.82 were Hospital Trusts (HTs), bearing full responsibility for their own budgets, management and technical functioning, and 399 were Local Public Hospitals (LPHs), managed by Local Health Authorities (LHAs).LHAs are legal public bodies that have organizational, administrative, fiscal, financial, managerial and technical independence and organize and provide healthcare services within their territorial areas through LPHs and accredited private structures [40] As in the case of any model, the selection of inputs and outputs may affect the final results and/or ranking of hospitals in terms of quality.Being mindful of this concern, we have followed DEA literature in the choice of inputs and outputs [41,42].Outputs included Case Mix Index (CMI) adjusted discharges (MCMI_discharges) calculated as the number of hospital discharges multiplied by the hospital's average CMI and total surgery discharges.Our inputs included inpatient beds and all personnel, categorised into medical staff (physicians, surgeons and dentists), nursing staff, administrative staff and other personnel.Input data about global resources, such as drugs, diagnostic exams and instruments, were not available for hospitals.

statistical Methods
In the first stage of analysis, standard technical-efficiency DEA was conducted and the outcome was referred to as Stage I DEA scores.An input-oriented model seemed more appropriate, because it was assumed that hospitals have control over utilization of resources rather than over demand.Moreover, two scale assumptions can underpin DEA model: constant returns to scale (CRS), and variable returns to scale (VRS).As it was assumed that not all units of analysis were operating at an optimal scale, in this analysis the VRS model was considered in order to assess the extent to which the scale of operations affected productivity [25].The t-test for independent samples was used to compare efficiency scores by hospital type, while the ANOVA test was calculated to assess statistical significance of differences by region.
In the second stage of the analysis, the estimated Stage I DEA scores were regressed against a set of organisational and contextual characteristics beyond managerial control, which reflected differences in the population demographics and regional health expenditure.Predicted efficiency scores (Stage II DEA scores) reflected the amount of efficiency that was predicted by organisational and contextual characteristics.Finally, the two-limit random effects Tobit regression, with hospitals hierarchically nested within regions was used.As explanatory variables, the average length of stay (ALS) and Case-mix Entropy (CME) were considered as organisational variables at the hospital level, which are correlates of inefficiency related to patients and payer mix [42]; the male youth unemployment rate (MYU), the elderly dependency rate (EDR) and the average annual per capita health expenditure in the period 1997-2007 (HE) were included [43], respectively, as indicators of social, demographical and economical dimensions of the regional context [44,45].The log-likelihood criterion has been used to assess the goodness-of-fit of models.A p value <0.05 has been considered statistically significant.Efficiency scores have been calculated for each hospital using the DEA frontier Analyst software [46] while two-level Tobit regression analysis was performed by using Stata/MP 11.2.

Results
Mean outputs and inputs for the 481 hospitals are shown in Table 1.During the year 2007, HTs treated more patients (23 646±12 995) and reported higher complexity (1.06 ±0.18) compared to HPLs (7 878±7 076 and 0.88±0.17,respectively).With regards to territorial distribution of outputs, Northern facilities showed more discharges and surgical discharges and higher complexity than Central and Southern ones.Regarding inputs, all hospitals employed more nursing and administrative staff than medical staff, both for territorial distribution and hospital type.
The average levels of contextual and organisational factors are shown in Table 2. MYU was on average lower in Northern Italy and this is in agreement with the well known Italian North-South economical gradient [45].In Southern regions CME and ALS were at minimum, indicating a minor complexity of treated cases, while EDR was at maximum, indicating an elderly population structure living in the South.), the highest score was observed in the Northwest (75.7±15.1),followed by the Northeast (75.5±15.1),Central Italy (73.9±16.4) and finally, in the South (70.6±17.9).Direct comparison among hospitals at regional level reveals that Emilia Romagna was the most efficient (84.2±12.3)with three hospitals falling on the efficiency frontier.It was followed by Lombardia, whose technical efficiency was 2.0% lower on average and had three fully efficient hospitals.Notably, the highest number of 100% technically efficient hospitals was observed in Campania (11 out of 47) and Lazio (12 out of 45).All regions operated under inefficient scale of operations, as all SE scores were less than unity.
Organisational and contextual factors were found to be statistically significantly different at Tobit regression according to the type of hospital.
For HTs, all included variables were shown to be statistically significant except for the EDR.In particular, DEA scores were increasing on average by decreasing ALS (Coef.=-4.05;p<0.001) and by increasing CME (Coef.=14.89;p=0.038) for organisational factors, and by decreasing MYU (Coef.=-0.65;p=0.031) and by increasing HE (Coef.=0.06; p=0.003) for contextual indicators.For LPHs, none of the ) the intra-class correlation at regional level (Table 4).
Stage II DEA mean scores (Table 5) were calculated for each region and drawn up together with Stage I DEA mean scores (Figure 1).They respectively represented the actual efficiency and the predicted efficiency as a function of organizational factors and context, beyond managerial control.Therefore, any difference between actual and predicted DEA scores could be attributable to the decisionmaker.For regions falling on the dashed bisector line (Toscana, Piemonte, Umbria and Friuli), stage I and stage II DEA mean scores were equal; for regions falling on the right of the bisector line, management contributed to increasing efficiency compared with the standard predicted by organisational and contextual factors alone.This virtuous management effect was much more consistent as we moved away from the bisector, top-down in the graph (in the following order: Campania, Marche, Lombardia, Emilia Romagna) and regions on the extreme right in the graph had the highest stage I mean DEA scores.Regions on the left of the bisector presented some criticism of management because of the lower observed efficiency compared to the standard predicted exclusively by organisational and contextual factors.

discussiON
At the micro level, the proposed study aimed to estimate technical efficiency of Italian hospitals, while at the macro level the aim was to investigate the effect of contextual and organisational factors of regions in the considered time period, beyond managerial control.In order to identify true managerial inefficiency, which is important for designing policies of resource allocation improvement, the effects of these factors must be accounted for when comparing organisations [39].This is especially true in Italy because there are several territorial differences with regards to demographic, economic, social and environmental aspects, and profound differences between Italian regions in various dimensions of health [45,47].
The first result of this study was that the distribution of regions according to technical efficiency only partly reflected the North-South  [48].In our study, the top Stage I DEA mean scores were observed for the northern regions of Emilia Romagna, Lombardia, Bolzano and Marche (Central) whilst the Southern regions of Campania and Sicily were respectively in the fifth and seventh positions, with Lazio (Central) in the sixth position.As shown in Bripi's study [49], the North-Central-South efficiency gradient concealed a variegated territorial pattern, with some Southern regions performing better than other Northern regions, in terms of outcome and efficiency.
Looking at input and output variables for the 100% technically efficient hospitals detected in Campania and Lazio, the highest ratio of surgery discharges by medical staff (Campania) and MCMI_discharges by beds (Lazio), compared to the overall average (data not shown), could be reported.However, in Pammolli's study [48], these regions were included among the regions with the widest gap of expenditure and efficacy from the benchmark.To shed light on this difference, the present study proposed a microeconomic analysis on hospitals, which were defined efficient if they used optimal amounts of resources given the level of output.In this model, HE was considered as an explanatory variable at the regional level, beyond the control of hospital management.Conversely, Pammolli [48] dealt with a macroeconomic analysis of regions, which were defined efficient if they attained high quality standards in terms of containment of HE.In that model, HE was considered as an outcome variable, under the control of the regional management.
In this analysis, HTs were more technically efficient than LPHs.This study did not consider day-hospital discharges as an output, while LPHs tend toward de-hospitalisation of some diseases with increasing recurrence to dayhospital admissions and a reduction in the length of stay.Moreover, LPHs include small hospitals with high average costs per admission.As they fill the gaps of homecare in Italy, they are characterized by fixed costs typical of hospitals and a high incidence of medical staff within the whole personnel team [49].With regards to the relation between technical efficiency and contextual and organisational factors, for a long time [50], it was stated that health expenditure is not the only variable strategic to improving health and can have little impact compared to other social determinants such as housing, education, diet, lifestyle and environment.
Stage II DEA analysis showed that technical efficiency was positively related to CME and HE.Also stage II DEA analysis by Schiavone [20] regarding technical efficiency of Italian hospitals in the period 2000-2004 showed that the variety of services determines an increase of the average levels of efficiency due to the presence of scope economies, especially in hospitals with high complexity.
There was an upward trend in increasing the efficiency by increasing HE because the latter is associated with increased investments, higher quality of services and higher complexity supply.Moreover, the increased complexity of admissions was associated with an increased DRG value.This inflation effect in the HE, which is induced by the DRG based funding system, could also indirectly affect efficiency through complexity.Finally, it should be noted that HE was calculated as an average over the decade 1997-2007 in order to balance the fluctuations of expenditure of the period.
Stage II DEA analysis showed that technical efficiency was inversely related to MYU.There was a downward trend in decreasing the efficiency for hospitals located within Lastly, Stage II DEA analysis confirmed the inverse relation between efficiency and ALS.This result was expected because a significant reduction in patient days and length of stay could be observed in almost all hospitals.The reason is the significant decrease in hospitalisation due to the transition from a reimbursement system based on patient days to another based on the number and complexity of treated cases [49].
The role of management in improving efficiency can be seen through the graphical comparison of Stage I and Stage II efficiency scores.The graphical tool can be considered the easiest and most helpful contribution of the present analysis to the existing knowledge for differentiating the role of management from organisational and contextual factors.Our results are confirmed by Pammolli's study [48], where Umbria and Friuli are proposed as a benchmark for other regions for both expenditure and quality, and Toscana and Piemonte are included within those requiring minor interventions to achieve the benchmark.Emilia Romagna and Lombardia, at the extreme right in the graph, had the highest stage I mean DEA scores and all of them had CMI more than unity.This result confirms the direct correlation between efficiency and complexity [51].The negative role of management shown in our study for regions on the left of the bisector line, Abruzzo, Sardegna and Calabria, is confirmed by Pammolli's study [48], where these regions are included within those requiring major interventions to achieve the benchmark.
Trento and Bolzano are respectively on the left and on the right of the bisector line.This result is not directly comparable with Pammolli's [48], in which these autonomous provinces are evaluated together as a unique region (Trentino Alto Adige), rated as good performance for both expenditure and quality.Some caution is needed when interpreting these results because outputs included inappropriate admissions, and inputs included the staff also used for the provision of services which were excluded from the analysis and did not consider global resources.
Hospital managers need to understand their environment and comply with organisational factors to garner the resources needed to deliver Epidemiology Biostatistics and Public Health -2014, Volume 11, Number 1 SECONd-STAGE dEA ANALySIS IN ITALy

figurE 1 STAgE
figurE 1

Table 3 shows
Stage I DEA results.The efficiency score was 72.8±17.0 on average, with HTs more efficient than LPHs (83.7±12.3 vs. 71.6±17.0,p<0.001).If we consider the distribution of efficiency scores by territorial areas (data not shown in Table

TABLE 1 SummAry
STATiSTicS of ouTpuTS And inpuTS of 481 hoSpiTALS By ArEA And By hoSpiTAL TypE

TABLE 2 SummAry
STATiSTicS of conTExT And orgAnizATionAL indicATorS By TErriToriAL diSTriBuTion MYU= Male youth unemployment; EDR= Elderly dependency rate; CME=Case-mix Entropy; ALS=Average length of stay; HE= average annual per capita health expenditure e 8 7 8 5 -5 O RIGINAL ARTICLES Epidemiology Biostatistics and Public Health -2014, Volume 11, Number 1 SECONd-STAGE dEA ANALySIS IN ITALy gradient shown by other studies with regards to gap of expenditure

TABLE 3
STAgE i EfficiEncy ScorES ovErALL, By rEgionS And By TypE

TABLE 4 ToBiT
Biostatistics and Public Health -2014, Volume 11, Number 1 SECONd-STAGE dEA ANALySIS IN ITALy care efficiently.Socio-economic context plays an important role in establishing differences in efficiency among hospitals.Further analysis on updated data should be done periodically in order to corroborate what comes out from this first analysis, that is to say, hospital efficiency can be increased by improving the socioeconomic context of a region.

TABLE 5
STAgE ii EfficiEncy ScorES AdjuSTEd for conTExT And orgAnizATionAL fAcTorSData for Valle d'Aosta was omitted because it referred to only one hospital; 2 Autonomous provinces; 3 Decreasing order; 4 HTs: Hospital Trusts; LPHs: Local Public Hospitals e 8 7 8 5 -9