Have you ever had a flaky test in your codebase? I did. Let me tell you, it isn't enjoyable. The main problem is (who would have thought it?) the flakyness. It fails randomly on the CI, but if you run it locally, everything seems fine. A simple "Works on my machine" sounds tempting, doesn't it? But that does not solve the problem. So how do you make a test fail, that only fails on every 500th run?

How about running it more than 500 times?

Run your test a thousand times (or more) with a few clicks

It's really simple. Just right-click your test in the "Test Navigator" in Xcode and select Run "testMyAwesomeTest()" repeatedly..., choose how many repetitions you want and hit "Run". That's it.

Sure, you can also play with the other options, but that's basically it.

Doing the same with the command line

Of course, you can do the same using the command line. That might be handy if you want to run the test repeatedly on your build agent via ssh.

For my test project the command would be

xcodebuild test -scheme "running-tests" -destination 'platform=iOS Simulator,name=iPhone 14,OS=16.1' -only-testing "running-testsTests/FizzBuzzServiceTests/testFizzBuzzMagic" -test-iterations 1000

The interesting bits here are:

• xcodebuild test -> let's run some tests

• -only-testing "..." -> I just want to run this single (flaky) test. Not the entire test suite

• -test-iterations 1000 -> let's run the test 1000 times

And again: That's it. 🚀

One more thing

Of course, flaky tests can be much trickier to reproduce than just "it fails randomly. Just run it really often". If you want to dive deeper into "flaky tests", Antoine van der Lee has a great blog post on that matter.

Happy coding 🧑‍💻