Parametric and nonparametric two-sample tests for feature screening in class comparison: a simulation study

Abstract

Background. The identification of a location-, scale- and shape-sensitive test to detect differentially expressed features between two comparison groups represents a key point in high dimensional studies. The most commonly used tests refer to differences in location, but general distributional discrepancies might be important to reveal differential biological processes.                                                        

Methods. A simulation study was conducted to compare the performance of a set of two-sample tests, i.e. Student's t, Welch's t, Wilcoxon-Mann-Whitney, Podgor-Gastwirth PG2, Cucconi, Kolmogorov-Smirnov (KS), Cramer-von Mises (CvM), Anderson-Darling (AD) and Zhang tests (Z_K, Z_C and Z_A) which were investigated under different distributional patterns. We applied the same tests to a real data example.                  

Results. AD, CvM, Z_A and Z_C tests proved to be the most sensitive tests in mixture distribution patterns, while still maintaining a high power in normal distribution patterns. At best, the AD test showed a loss in power of ~ 2% in the comparison of two normal distributions, but a gain of ~ 32% with mixture distributions respect to the parametric tests. Accordingly, the AD test detected the greatest number of differentially expressed features in the real data application.  

Conclusion. The tests for the general two-sample problem introduce a more general concept of 'differential expression', thus overcoming the limitations of the other tests restricted to specific moments of the feature distributions. In particular, the AD test should be considered as a powerful alternative to the parametric tests for feature screening in order to keep as many discriminative features as possible for the class prediction analysis.