mrmanager

We had to remove the scheduled restarts since they didn't work reliably and brought the server down. So they are not in effect right now. We do manual restarts a few times per day only.

Still hoping for a fix from Lemmy devs soon...

Hi,

It's because of the very annoying outgoing federation issues we have now on Lemmy 0.19.1.

All incoming data works very well but outgoing doesn't get sent until a restart of the Lemmy software, and then only once after restart. We will do a quick restart a couple of times per day and then your comments, posts etc will federate but of course it's really annoying with that long time in between your activity and others seeing it.

Hopefully there is a fix from Lemmy developers after new years. It has a bad idea to upgrade Lemmy before Xmas, and lesson has been very learned, believe me. :)

Same to you Lewdiculous :)

Thanks for sharing!

Its a very nice idea to share positive things. I started listening to the music and its quite nice. I do some programming now and then and this music is very nice for getting into the flow and staying there. :)

Im using FreeTube on the desktop to avoid youtube ads, its a seriously good app: https://freetubeapp.io/

Yeah its really annoying and makes it hard to have any discussions at all.

I will try now to restart only the lemmy docker container and see how it seems to work out. If the restart seems to work reliably many times in a row, I can schedule it to happen automatically.

Not giving up on making this work somehow. :)

Edit: Tried a restart of only the lemmy container and it does give like 20 seconds of downtime so not ideal. But maybe worth doing a couple of times per day at least.

Hehe indeed. :)

Yeah I could try to restart only the Lemmy container instead of the entire stack. Worth trying, thank you. :)

I tried it and it does give 20 seconds of downtime but could be worth doing a couple of times per day anyway. Its so frustrating with no outgoing messages...

Yes, it seems the restart is not as reliable as I was hoping for.

Its a docker compose restart of the entire stack but that has proven to work only sometimes, which is not good enough. This afternoon it failed once (as you noticed), then worked 4 times, then failed once more.

We will figure out a more reliable way to restart, but for now, the restart has been removed again.

Really sorry about all this. Just trying to find a way to make federation work again since its very frustrating when it doesnt.

Anyway, merry Christmas and thanks for being understanding during all this.

Yes.

I think this update has been very problematic in many areas, and in hindsight, we should have waited a few more weeks to update.

Previous updates have not had major problems, but we knew this was a very big update with a big rewrite of how several things work (like federation), so it would have been smarter to wait longer than a week to update.

Unfortunantly there is no way to go back to previous version now, since the 0.18.5 to 0.19.0 update upgraded the database schema as well.

The developers of Lemmy are working on fixes. I know they reopened github tickets where federation was supposed to be fixed. And there are many other issues as well: https://github.com/LemmyNet/lemmy/issues

You can search there and see if anyone else has reported simular issues, and otherwise create a new issue and report the problems.

Next time we will wait longer to install major updates from 19 to 20 and so on.

Edit: Im also getting "Incorrect logon" messages while using the web interface, but it doesnt seem to have any effect. I dont get logged out. But another bug somewhere.

Yeah that's what we are having here too, huge cpu usage and issues with outgoing federation. Sounds spot on.

Yeah it's been a rocky ride since the update, and we had to restart the server a number of times to diagnose issues with outgoing federation that came with Lemmy 0.19.1.

Very sorry about that, and hopefully there is a fix for this coming soon from the developers. Thanks for being so nice about it. :)

What we are seeing is that outgoing federation of posts and comments works after a restart, but new posts and comments made on the local instance are not federating to other instances until the next restart. This is with version 0.19.1.

Also after a restart, the server eats a lot of cpu for several minutes. Im guessing this is normal with the federation queue being processed but cant be sure.