Sadly that sort of thing got so common where I work that I'll run the tests three times before considering looking into the error message to see if it is something I broke.

From time to time we take some days just to fix tests with inconsistent results, but there's always more popping up.

[–] OsrsNeedsF2P@lemmy.ml 2 points 1 year ago

Serious answer: You can't write tests for untestable code. Your code needs to be pure if you want reliable tests: https://en.m.wikipedia.org/wiki/Pure_function

For integration tests, they should handle retries themselves

[–] Darorad@lemmy.world 1 points 1 year ago

Yeah, we have a team whose job is to make sure all our tests run well and fixing them if they don't