The best way to find the problem was by running the test hundreds of times to see the problem.
We used the code below to easily do this:
@RunWith(MyRunner.class)
public class UnusualAndRareProblemTest {
...
..
.
}
public static class MyRunner extends JUnit4ClassRunner {
public MyRunner(Class klass) throws InitializationError {
super(klass);
}
@Override
public void run(final RunNotifier notifier) {
for(int i=0; i<1000; i++) {
super.run(notifier);
}
}
}
It turns out the class under test was spawning threads and we were getting race/dead-locking conditions due to the way we were trying to retrieve the outcomes from different threads.
PS. We fixed this by re-writing the class so that instead of spawning threads directly it made calls to a 'action factory' that we could mock and avoid multi-threading in the test entirely.
2 comments:
Have you ever checked out testng?
Simple annotations would make running a test multiple times simpler still...
in groovy you could have a test method like:
@Test (invocationCount=3)
void verifyLoudDogBarker(){
assert false : 'dogbarker not loud'
}
to have instances of the test run in parallel just change the annotation to:
@Test (invocationCount=3, threadPoolSize=3)
This is TestNG annotations, not JUnit. JUnit doesn't support such repeated execution.
Post a Comment