Btw: 0.19.1, leading zero. It's not yet at 1.0 for a reason.
Fediverse
A community to talk about the Fediverse and all it's related services using ActivityPub (Mastodon, Lemmy, KBin, etc).
If you wanted to get help with moderating your own community then head over to !moderators@lemmy.world!
Rules
- Posts must be on topic.
- Be respectful of others.
- Cite the sources used for graphs and other statistics.
- Follow the general Lemmy.world rules.
Learn more at these websites: Join The Fediverse Wiki, Fediverse.info, Wikipedia Page, The Federation Info (Stats), FediDB (Stats), Sub Rehab (Reddit Migration), Search Lemmy
Sounds like something a cardassian would say.
It took me a second to realize what cardassians you were talking about. I constantly forget the other ones are spelled with a K
Lemmy has been feeling pretty empty the past couple of days. That just really goes to show how important federation really is.
I noticed that as well, thought maybe things were just slowing down, but it's good to know it was the software and not a sudden drop in community involvement. Since admins became aware of the bug and began restarting their servers as an ad-hoc fix, I've been seeing way more engagement on my instance.
Man, the one thing not to break. They broke it.
for the most part federation works just fine. there are occasional issues, yes, but it's not like it can't federate at all.
Does my comment reach anyone?
I’m seeing it on lemmy.ml.
I suspect the bug isn’t absolute, but inconsistent instead.
World here.
Hi
Hello World
Feddit.uk present
What comment?
Sh.itjust.works here
Yep.
I see it on kbin.social.
feddit.it present!
🦈
No.
Hello from. Ttrpg.network!
I would comment on this but since federation is broken....this won't be seen until the instance admin restarts Lemmy and federation works once again.
Federation seems to be working fine from where I stand?
Edit:
Edit 2:
Even the edit federated out to your instance fine...
I have mh suspicions that this is some kind of deadlock issue. After fucking up my config during am upgrade (too many database connections, too many workers) the new multithreaded federation thingy seemed to become quite unstable. From what I could tell, the threads that are supposed to deal with federation all broke/shut down, but didn't get automatically restarted again, leaving some messages and instances in limbo until the entire server got restarted.
After fixing my database connection issue and reducing the amount of workers, I don't think I've encountered many issues since. I occasionally click the little icon above a post to check if my response had reached the community server, and usually it does, pretty quickly.
This issue sounds awful to have to debug, best of luck to the Lemmy devs in finding a fix!
I think it works for a while after restart of the server, and then stops working. Yesterday I was making comments that didn't federate until 12 hours later when I assume the instance admin restarted the server.
My instance has been uo for over 3 days since 0.19.1 update with no issues - very little load though
A while ago, programming.dev had a problem where nothing from the instance federated for a week. Lemmy felt so empty
Am I little naive in wondering how this isn't caught in unit, integration and E2E tests?
Edit: looks more involved than that, but it forced me to dig into how some of the components are tested. Educational.
The main Lemmy developers very often don't have the biggest clue of what they're doing, and they've proven that time and time again. A lot of times they forget normal web practices and rules, and a lot of side things like Jerboa are also just broken.
They also don't know how to interact with the community or how to be normal people. Let's not forget they have been working on Lemmy full-time and this is the state we've come to, ignoring all the database and security and everything else issues they already had.
Dude, lemmy is awesome, give them a break.
Lemmy IS awesome, that doesn't mean we can't want it to be better or criticize it. No one said Lemmy isn't great.
The main Lemmy developers very often don't have the biggest clue of what they're doing
this doesn't sound constructive criticism to me.
that commenter has something personal against the devs. Lemmy is far from perfect but it's functional and improving everyday.
Absolutely, but there are constructive ways to do it and what that guy did.
😂😂 I got downvoted for that. I guess people dislike writing tests.
I love writing tests, It's all the shit that comes after that that sucks.
Those first few pushes that all come up green feel like magic. That first red that points out something you missed, you go back and make a quick change and it's now green and it's the best thing you've ever seen.
It's sooner or later, you throw a couple big red bois on a production build that don't make any sense. You start digging through the code of some guy that only writes comments in haiku and has the impression he gets paid by the number of layers deep he can nest a ternary.
Sooner or later you figure out it's just an edge case there's nothing actually wrong. You'll need to refactor one of the systems but you still have production to push to fix a critical bug, so you hotwire the test and write it off as P1 tech debt.
Eventually, you end up with unit tests that aren't P1 and they fail. If you're understaffed, or overscoped, sooner or later you just have a bunch of half-assed zombie test sitting around. Unless you can convince production to let you go back and clear up your tech debt it's just a unit test graveyard. It still has the big bumpers in place so something serious can't fail. But you never seem to be able to get back to make everything bright new and shiny again.
I wish more developers would do QA. After working with QA my code improved so much because I was proactively thinking about how things might break or potential issues that I never would have thought of.
Yep. When I was still doing QA, I saw some pretty terrible practices and tested code that barely built. Now as a software engineer, I have no QA and rely heavily on my own testing practices, namely, unit testing first, integration testing and system/e2e testing. I can't guarantee the code is bug free and there's parts I know that could be refactored (tech debt), but I know each piece is tested and does what I expect it to. As corny as it sounds, I'm a big fan of TDD. Unit/IT/E2E don't replace QA in my opinion, but better set QA up to focus on the bugs that matter and not basic stuff.
Sheesh.
oof
I'm going to set up a cronjob to restart the Lemmy docker container every 6 hours. That seems to be around when stuff stops working usually.
Yup, absolutely nothing I posted from lemm.ee yesterday made it out into the wider world. In fact I'll probably have to manually federate this comment too.
Hopefully they can get to the bottom of it. Maybe next year we need a rule about releasing anything in December when everyone's too busy to bugfix 😅
Curious: what do you mean by "manually federating" a comment?
My other account is on .world so I can use search to forcibly pull things onto at least that server (search for the original URL of the unfederated post or comment and then wait a few seconds). Sometimes doing that will also encourage whatever it was to federate more widely to more servers, but that doesn't seem to be happening as reliably either since the upgrade 🤷♀️
A restart of Lemmy seems to fix it temporarily according to admin of Lemmy.today.
But it's inconsistent and stops working after a while it seems.