The best way to find the problem was by running the test hundreds of times to see the problem.
We used the code below to easily do this:
@RunWith(MyRunner.class)
public class UnusualAndRareProblemTest {
...
..
.
}
public static class MyRunner extends JUnit4ClassRunner {
public MyRunner(Class klass) throws InitializationError {
super(klass);
}
@Override
public void run(final RunNotifier notifier) {
for(int i=0; i<1000; i++) {
super.run(notifier);
}
}
}
It turns out the class under test was spawning threads and we were getting race/dead-locking conditions due to the way we were trying to retrieve the outcomes from different threads.
PS. We fixed this by re-writing the class so that instead of spawning threads directly it made calls to a 'action factory' that we could mock and avoid multi-threading in the test entirely.
