A New Approach to the Nonparametric Behrens-Fisher Problem With Compatible Confidence Intervals.

We propose a new method to address the nonparametric Behrens-Fisher problem, allowing for unequal distribution functions across the two samples. The procedure tests the null hypothesis H 0 : θ = 1 / 2 $\mathcal {H}_0: \theta = \nicefrac {1}{2}$ , where θ = P ( X < Y ) + 1 / 2 P ( X = Y ) $\theta = \text{P}(X<Y) + \nicefrac {1}{2}\text{P}(X=Y)$ denotes the Mann-Whitney effect. Apart from the trivial case of one-point distributions, no restrictions are imposed on the underlying data distribution. The test is derived by evaluating the ratio of the true variance σ N 2 $\sigma _N^2$ of the Mann-Whitney effect estimator θ ̂ N $\widehat{\theta }_N$ to its theoretical maximum, as derived from the Birnbaum-Klose inequality. Through simulations, we demonstrate that the proposed test effectively controls the type-I error rate under various conditions, including small and unbalanced sample sizes, and different data-generating mechanisms. Notably, it provides better control of the type-I error rate than the widely used Brunner-Munzel test, particularly at small significance levels such as α = 0.005 $\alpha = 0.005$ . We further construct range-preserving compatible confidence intervals and show that they exhibit improved coverage compared to the confidence intervals compatible to the Brunner-Munzel test. Finally, we illustrate the application of the method in a clinical trial example.