This paper combines extreme value theory for the smallest and largest k observations for some given k>1 with a normal approximation for the average of the remaining observations to construct a more robust alternative to the usual t-test. The new test is found to control size much more successfully in small samples compared to existing methods. This holds for the canonical inference for the mean problem based on an i.i.d. sample, but also when comparing two population means and when conducting inference about linear regression coefficients with clustered standard errors.