8

At my company there has been a move away from our large e2e testing suite for various reasons:

  • flaky tests
  • long build times
  • significant burden for multiple teams to contribute

The use of pact is seemingly only covering some small portion of each integration point between applications as large, comprehensive pacts that cover the full API and values used are considered too costly or coupled.

I am conflicted that we are losing test coverage of business use cases because there aren't necessarily any tests that are automatically running a use case end to end.

How do you deal with this in your workplace and what is your position on e2e vs pact testing.

top 8 comments
sorted by: hot top controversial new old
[-] EthicalAI@beehaw.org 7 points 1 year ago* (last edited 1 year ago)

You should end to end test. If it’s slow, do it less often. If it’s costly, again do it less often. But this is your last line of defense against bugs.

Now, every service should also unit test, with mocks of all the other services that they call. I assume this is what you mean by pact testing. These should be locally runnable, offline, as the mocks are cheap offline things. If you need data from a database, I’d recommend mocking the data too.

[-] ZeroEcks@lemmy.blahaj.zone 4 points 1 year ago

I think pact testing means contract testing, basically a test that asserts an agreement between two things (microservice, backend, frontend, etc)

[-] show_me_the_clamps@vlemmy.net 1 points 1 year ago

Yep I could have been more clear this. Pact being the library I am using to fulfil the contract tests.

[-] EthicalAI@beehaw.org 0 points 1 year ago
[-] ZeroEcks@lemmy.blahaj.zone 1 points 1 year ago

Sorry, the one downside of quick updates is I can respond very quickly before the edit is done / sent through hah.

[-] liori@lemm.ee 6 points 1 year ago

We found that flakiness of e2e tests is usually caused by using libraries, frameworks and devops tools that were not designed for being integrated in e2e tests. So we try to get rid of them, or at least wrap them with devops magic. This requires a skilled devops team and buy-in from management.

At some point we were also solving the issue by having dedicated human reviews of e2e failures, it's easy to train a junior QA engineer to have most false positives quickly retried.

But we would never give up on e2e.

[-] ZeroEcks@lemmy.blahaj.zone 4 points 1 year ago

I have never actually managed to get it integrated, but I have always pushed for contract testing that's maintained by both sides of the contract. This has the benefit of being able to create compatibility matrixes between services at the very least

[-] smoke_bird@lemmy.blahaj.zone 3 points 1 year ago

Contract testing and end to end testing should both be different factors that comprise a successful testing strategy. A

load more comments
view more: next ›
this post was submitted on 18 Jun 2023
8 points (100.0% liked)

Programming

13226 readers
1 users here now

All things programming and coding related. Subcommunity of Technology.


This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.

founded 1 year ago
MODERATORS