Validity Of Socioeconomic Inequality Indices over Time in Public Health Research: A Case Study on Ivsm and Maternal Data

Abstract

Introduction

In recent years, studies on pregnancy outcomes and reproductive health indicators have increasingly considered the impact of socioeconomic deprivation. Area-based indices such as the Caranci Deprivation Index [1] or Synthetic Index of Socioeconomic Disadvantage (IVSM) from ISTAT [2] are commonly used as proxies to evaluate variations in fertility rates, voluntary and spontaneous abortion, and other maternal-child health outcomes. Moreover, these indices are widely used to examine how socioeconomic disadvantage influences adverse maternal and reproductive health outcomes, as well as accessibility of healthcare services [3-5]. However, both the Caranci Index and IVSM are constructed using data from the 2011 national census. Applying these outdated indicators to more recent healthcare databases risks to introduce bias and misrepresenting the current socioeconomic reality of municipalities.

Objectives

This study aims to determine a municipality-level socioeconomic score that aligns as closely as possible with the IVSM but based on individual-level health data from birth certificates (CEDAP) collected in the years 2010–2012 that are near the year of IVSM determination. The primary objectives are to estimate a predictive model of the IVSM based on aggregated CEDAP data (2010–2012), to validate the model internally within the same time period, and to test the temporal stability of the derived score by applying it to subsequent years and assessing its consistency with the IVSM.

Methods

Municipality-level frequencies were calculated for selected sociodemographic variables present in the CEDAP records, including maternal and paternal citizenship, marital status, educational attainment, occupational status, maternal age class, and parity. The IVSM value, available for each municipality from ISTAT (2011), was used as the dependent variable in a General Linear Model (GLM), where independent variables were the proportions of each sociodemographic category in the reference period 2010–2012. Coefficients estimated from the GLM were then used to compute a composite score (Socieconomic Maternal Score, SMScore) for each municipality by applying the same formula to the frequencies in subsequent time periods (e.g., 2013–2015, 2016–2018).

To evaluate model performance the data of 2010–2012 were splitted into a training and validation sets. In later time periods, the SMScore was compared to the original IVSM using Pearson’s correlation coefficient and Mean Squared Error (MSE). Furthermore, both the IVSM and SMScore were categorized into quartiles to simulate typical use in epidemiological studies. Linearly weighted Cohen’s Kappa was calculated to assess agreement between the quartile-based classifications. This analysis rests on the assumption that, although a municipality’s continuous deprivation score may fluctuate, shifts between quartiles over time are less frequent and may therefore offer a more robust indicator of socioeconomic positioning in longitudinal analyses.

Results

The initial validation of the SMScore against the 2011 IVSM showed a positive correlation (Pearson’s r = 0.64, p < 0.0001) and a relatively low mean squared error (MSE = 3.27), confirming the validity of the constructed scoring model.

When applied retrospectively across five consecutive three-year periods (T1: 2010–2012 through T5: 2018–2020), the correlation between the calculated score and the original IVSM index progressively decreased. In the earlier periods, the relationship remained statistically significant (T1: r = 0.64, MSE = 1.87; T2: r = 0.60, MSE = 1.93). However, the strength of association weakened in later periods, with T3 showing a low correlation (r = 0.24, MSE = 4.19), and T4 a slightly stronger but still moderate low correlation (r = 0.34, MSE = 3.19). By T5, the correlation dropped to non-significant levels (r = 0.10, p = 0.0841) with a marked increase in prediction error (MSE = 6.44).

The heatmap visualization (Fig.1) comparing the quartile classification of municipalities based on IVSM 2011 and the recalculated SMScore across the five periods provides further insight. In the earliest triennia (T1 and T2), the majority of municipalities remained within the same quartile or shifted only marginally, indicating good agreement between classifications. This concordance deteriorated progressively in T3 and T4, with more municipalities diverging from their original IVSM quartile. In T5, the misclassification pattern became more evident, with substantial deviation between the IVSM and SMScore classifications. This trend is quantitatively supported by the linearly weighted Cohen’s Kappa, which declined from 0.83 [0.80–0.87] in T1 to 0.67 [0.62–0.72] in T5, confirming the decreasing agreement over time. Despite changes in the continuous score, quartile stability appeared more robust in the early triennia.

Conclusions

The findings suggest that the use of area-level socioeconomic indices such as the IVSM should be temporally bounded, as their capacity to reflect current population-level vulnerability deteriorates over time. While the index performed well in the early years following its development, its predictive and classificatory coherence weakened in later periods. This divergence may stem from genuine socioeconomic transformations within municipalities or from changes in the demographic and social profiles of the women giving birth, such as age at delivery or parity.

The observed mismatch highlights the importance of regularly updating deprivation indices or developing dynamic proxies that can adapt to shifting population characteristics. Integrating information from different health service databases, which contain additional relevant variables, may refine the scoring approach and enable more robust comparisons with existing socioeconomic indices. This integration can improve area-level deprivation measures and support their effective use in public health monitoring and planning.