Multiple hypothesis testing is widely used to evaluate scientific studies
involving statistical tests. However, for many of these tests, p-values are not
available and are thus often approximated using Monte Carlo tests such as
permutation tests or bootstrap tests. This article presents a simple algorithm
based on Thompson Sampling to test multiple hypotheses. It works with arbitrary
multiple testing procedures, in particular with step-up and step-down
procedures. Its main feature is to sequentially allocate Monte Carlo effort,
generating more Monte Carlo samples for tests whose decisions are so far less
certain. A simulation study demonstrates that for a low computational effort,
the new approach yields a higher power and a higher degree of reproducibility
of its results than previously suggested methods.