Cross-Country External Validation of a Multisource Comorbidity Score

Abstract

Background

The increasing impact of multimorbidity is escalating clinical and economic demands on healthcare systems, underscoring the necessity for effective tools to assess clinical complexity and enhance management strategies [1-3]. The Multisource Comorbidity Score (MCS) is a population-based index based on regional healthcare Utilization databases - hospitalizations and drug prescriptions - of beneficiaries with age equal or greater than 50 years. The MCS was developed in the framework of the Monitoring and Assessing Care Pathways working group of the Italian Ministry of Health, and validated in four Italian regions, showing good performance in predicting mortality, hospitalizations, and healthcare costs [4].

Objective

This study aimed to externally validate and adapt the MCS within the Catalan healthcare system, assessing its predictive capability outside the original Italian setting. This validation seeks to determine whether the MCS can serve as transferable tool in other healthcare systems with different data availability.

Methods

An observational longitudinal study was performed on subjects aged 50 years or older, residing in the health district of Barcelona-Esquerra (ES) continuously during 2014-2015 and followed between January 1^st, 2016, and December 31^st, 2019.

Data were obtained from the Catalan Health Surveillance System [5] which integrates demographic, clinical and healthcare utilization information from several healthcare databases. For this study, we used Catalan healthcare system beneficiary’s, primary care, hospitalization, and pharmacy dispensation databases.

First, the MCS with Italian weights (MCS-1) was applied in the Catalonia setting using the same data sources (hospitalization and pharmacy dispensation databases) as in the original Italian version. Second, new MCS weights (MCS-2) were estimated in predicting one-year mortality (primary outcome) in the Catalonia setting using the methodology and data sources as described in [4]. Finally, a third MCS version (MCS-3) was developed estimating specific weights based on hospitalization, pharmacy dispensation and primary care databases to predict the primary outcome.

Secondary outcomes considered were four-year mortality, one- and four-year hospitalizations (≥1) and one- and four-year hyperfrequency primary care utilization (≥10 visits). To assess the performance of the three MCS versions, generalized linear models (GLMs) with a binomial distribution were used for each outcome. ROC curves and Area Under the Curve (AUC) with 95% Confidence Intervals (95% CI) were estimated to assess the discrimination ability of the three MCS versions. De Long's method was used to compare the AUCs [6].

Net Reclassification Improvement (NRI) [7] was also calculated to assess improvements in risk classification by comparing new MCS versions with the MCS-1. The predicted GLM values, for each outcomes, of the three MCS versions were estimated and a threshold of 0.5 was used to distinguish between high and low risk individuals. The NRI estimates the percentage of individuals who were correctly reclassified into a higher risk category (according to the threshold) if they experienced the outcome, or into a lower risk category if they did not, minus those who were incorrectly reclassified when comparing the two models.

Results

As of January 1^st, 2016, a total of 440,790 individuals had resided in the health district of ES for at least two years. Among them, 198,753 (45%) were aged 50 or older and formed the study cohort.  They were mostly women (57%), with a median age of 66 years (IQR: 57–76).

Table 1 shows the MCS versions performances according to one- and four-year outcomes. All MCS versions demonstrated good discriminatory performance in primary and secondary outcomes.

For one-year mortality, the MCS-1 achieved an AUC of 0.742 (95% CI: 0.734-0.750) similarly to MCS-2 (AUC=0.756, 95% CI: 0.744-0.768), while MCS-3 version showed a significant improved AUC respect to MCS-1 (AUC=0.771, 95% CI: 0.760-0.783, p<0.001). Both new MCS versions performed better in predicting four-year mortality compared to MCS-1 (p= 0.012; p<0.001, respectively).

On the contrary, the MCS-1 showed better performance in predicting all secondary outcomes except one-year hospitalizations with respect to MCS-3.

In addition, significant improvements in risk reclassification for both one-year and four-year mortality were observed with the MCS-2 and MCS-3 compared to the MCS-1. For one-year mortality, the NRI increased by 0.63% (95% CI: 0.14–1.17) in MCS-2 and by 2.17% (95% CI: 1.39–2.97) in MCS-3. Similarly, for four-year mortality, the NRI increased by 1.8% (95% CI: 1.33–2.25) and 2.9% (95% CI: 2.35–3.45), respectively.

An increment in risk reclassification was found in four-year hospitalizations and hyper-frequency when comparing MCS-3 to MCS-1; the reclassification worsened in all secondary outcomes comparing MCS-2 to MCS-1.

Conclusions

The study supports the external validity of the Multisource Comorbidity Score in other healthcare systems with different data availability, such as Catalonia. The local adaptation slightly improved the ability of the score in predicting mortality, however this advantage was not maintained in the secondary outcomes, highlighting the importance of contextual adaptation of such tools.

These findings provide a basis for expanding the use of the score and refining it in different health systems and population segments.