Preface

I've run DubVee for about a year-and-a-half as of this writing. Until now, I've not really had much of a mission statement beyond being an on-ramp to the Fediverse and trying to keep things civil and non-toxic from this instance's point of view.

Recent and semi-recent events have revealed an ugly truth about many of the users in the Fediverse (and social media in general), so I've put together an actual mission statement in hopes of addressing that. It's also my last-ditch effort to not shut down this project out of disgust.

I want DubVee to be an example of what the Fediverse can be when it's not overrun with political idealogies masquerading as people, extremists, lynch mobs, and people too toxic to be anywhere else. I know for a fact that there are plenty of users that don't fit that criteria, and I dare say they are the majority. However, the above-mentioned groups are definitely drowning them out and giving Lemmy (and the Fediverse) a bad image in the process.

To that, I say: Not here. Not on DubVee.

Why DubVee and Why We're Different

DubVee is not your typical Lemmy instance. In order to achieve our goals, we have to do things a little differently compared to other instances. We want the "default" experience to be positive, welcoming, and non-toxic without requiring new users to discover and deal with the toxic parts on their own. We want everyone to be able to go to the homepage without the default experience making them say "what the fuck?!" and turn away. (That's literally what happened when I tried inviting a few Reddit colleages to Lemmy).

Some people may disagree with that; that's fine and is part of the beauty of the Fediverse. There are plenty of other instances that let you rawdog everything (and everyone) the Fediverse has to offer which you can then curate on your own.

Dubvee is not such an instance, and that is a feature we're proud of.

As an instance, we're pretty heavily moderated in order to maintain a welcoming atmosphere that fosters civil discussion. We don't "tone police" everything, but we do expect users, local and remote, to behave themselves and act in a rational, civilized manner. Those who cannot, local or federated, are quickly shown the door.

We also expect people to be here because they want to be here (the Fediverse) rather than being here because they're too toxic and/or have been banned everywhere else.

Note: None of this is throwing shade on other instances. I'm simply highlighting that DubVee recognizes a problem exists, that we are actively addressing it here, and extending an invitation to anyone who wants to be a part of it.

Inspired by Beehaw

Beehaw existed for a year or two before DubVee came online and is seen by many as the "heart" of the Reddit-like corner of the Fediverse due to their mission, philosophy and guiding principles. I've looked to them as a role-model since day 1, and that has not changed. I've always tried to make DubVee as "Beehaw-lite" as I could. In fact, many times, I've described DubVee as "Beehaw with downvotes enabled".

Join Us?

If that's the kind of environment you're looking to be a part of, we welcome you. Even if you want to create an alt here as a "vacation spot" to get away from that on other instances for a while, you're also welcome.

While we're not quite ready to throw open the doors (we're still configured as a regional instance), we're planning to open up soon to a wider audience who wants to be a part of a better, nicer Lemmyverse.

If other instances are taking similar stances against extremism and lynch-mob mentality, then please let me know so I can shake their admins' hands.

---

DubVee's Goals

1. Address the Elephants in the Room

There is one, big elephant in the room that I'm going to point out: there is a not-insignificant userbase in the Fediverse that:

• Is here only because they're too toxic for regular social media
• Has been banned everywhere else and can keep spinning up alts on new instances
• Jump straight to violent solutions to every perceived problem
• Operate in complete bad faith but aren't technically breaking any rules on the more general-purpose instances
• Just want to stir up shit (that's not unique to the Fediverse, just worth mentioning)
• Think it's some kind of absolute free speech zone where they can air their most toxic thoughts
• ...Worse (we'll just leave it at that)

DubVee has always been quick to deal with those accounts, and recent events have reinforced and escalated the need to do so more firmly and more permanently.

2. Provide a Non-Toxic, Chill, Civil, and Safe Environment

Building on the first goal, we take several measures to maintain a positive atmosphere here:

• DubVee is open-federation by default, but we do not federate with any of the "big 3" extreme instances: ml, grad, or hexbear nor any instance known for spreading hate or trolling

• Extremism of any form is not tolerated. Accounts espousing or excusing extremist rhetoric are quickly banned. This includes, but is certainly not limited to:

  • Advocating, excusing, praising, suggesting, or condoning violence against any person or group (including political violence)
  • Hypocritical takes that amount to "Fascism / authoritarianism / whatever-ism is bad, but a little bit is okay when it's my side doing it"
  • "Burn it all down" mentality
  • Stirring the pot by jumping into otherwise civil discussions and injecting extreme rhetoric
  • Pretty much any pattern of behavior that devolves into "lynch mob" mentality.
  • "Wink and a nudge" versions of any of the above

• Hate speech toward any group is prohibited.

• Trolls, bad-faith actors, and people who cannot interact civilly are quickly shown the door.

3. Focus on Quality Over Quantity

We prioritize quality over quantity, and we're not obsessed with growth nor having an endless stream of low-quality content just to fill up space.

We allow very few bots here, AI-generated slop is prohibited, and we do not federate with instances that primarily act as "bridges" reposting from other platforms. Having seen the endless stream of low/no-activity bot posts on other instances, I am very happy with how this policy has been working. This is a platform for humans to interact with one another, share cool stuff, and discuss things. Bot posts have shown to be primarily noise rather than signal.

4. Prevent Misinformation

We have a very strict misinformation policy here, and it is enforced for both local content as well as content arriving via federation. Some of the measures we take to achieve this goal include:

• No low-credibility news sources are allowed
• News articles must always be to the original source and without editorialized titles (exceptions being made for non-news/politics communities).
• Any suspect/wild claims without references are subject to a "mod first, fact check later" moderation policy.
• Any federated news-like communities that are not in compliance with these rules will be administratively removed

More details can be found in our Misinformation Policy page.

5. Ask Users to Leave Their Political Idealogy at the Door

This is a big ask on the Fediverse it seems, but it's not without a good reason. Having run this instance for a year and a half, I've found that 99% of users who make their political idealogy their entire identity tend to be 100% insufferable and cause needless drama.

Just be a person and talk to other people about cool stuff. It's really not hard to not inject political idealogy into every interaction.

This is not to say you can't express your political views - far from it. Hell, there's plenty of politics communities subscribed on this instance. Just don't make everything about it, don't inject it into every conversation, and just be a person and not an arm of your political party.

While this is more of a request to users of this instance, we do ban accounts from federated instances that do nothing but inject political ideaology into every interaction. I will say that since we defederated from .ml that has been less of an issue, but it is still necessary on occasion.

6. Respect the Right to Free Expression - Within the Bounds of Civility

We absolutely believe in the right to free expression of ideas, but it must be done in good faith, in a civilized manner, and without adding needless drama or toxicity. Trolling, hate speech, bigotry, misogyny. harassment, extremist rhetoric/calls to violence, and/or otherwise saying the worst things imaginable under the guise of free expression (aka "freeze peach") is not tolerated.

7. Not Be a Propaganda Platform

We disallow users that are clearly here with an agenda. DubVee is not a propaganda platform; it's for sharing cool stuff, staying informed, and having civil discussions. If someone is here to push an agenda, regardless of what that agenda may be, then they're outside what we consider "good faith" interactions.

Share stuff you're passionate about, sure, but don't be here just to spam your agenda.

8. No Porn

We do not federate with any of the primarily NSFW instances nor do we allow pornographic NSFW communities from general-purpose instances. No judgement if that's your thing, but porn is not our primary focus here, and neither I nor the backup admin want to moderate it or scroll through it when browsing /all.

The main frontend for DubVee, Tesseract, supports multiple profiles. As a compromise, I've unlocked the main UI to allow adding accounts from other instances. With this, you can add an account on another instance that does federate with the NSFW instances, or you can create an account on the NSFW instance(s) and switch to that profile. Many other Lemmy clients also offer the ability to add multiple accounts, so there are plenty of options. We feel this is an acceptable compromise.

News flash: Bad things can happen to people you think are bad without frothing at the mouth and turning into a lynch mob.

DubVee will not tolerate any form of extremism, vigilantism, mob justice, celebration of violence, etc. Please be sure to report any instances if you encounter them.

-- Pat

Two of the three new UPSs arrived, and I'm going to go ahead and swap them in versus waiting for the third one. I'll just have to deal with a little extra planned downtime later to install the 3rd one since there's too much potential for unplanned downtime with the two faulty ones in place now.

Shooting for 6 pm or very close to that. I'm going to cable-manage a bit, so planning for between 10 and 30 minutes of downtime.

Got some wicked winds roaring through here. Power has been flashing like crazy, and surprise, both UPS's are crapping out when I need them (even the one I replaced a few months ago 😠). They're both tripping off after 1-2 seconds.

I ordered three new UPS units Monday, but they're not scheduled to arrive until tomorrow evening. I will probably just go ahead and install them as soon as they are delivered without much warning since the current ones are in such poor shape.

I guess Tripp Lite is no longer a decent UPS brand in 2024?

Fiber's shitting the bed, apparently a cut. We're on backup WAN. ETA from ISP is 5am tomorrow

Upgraded the backup connection last week, so will leave pict-rs enabled for now and see how things go.

---

Update: Fiber came back online around 4:30 this morning and the auto fallback seems to have worked. Yay.

This is the second time today I've had to cut over to the backup WAN (which is much slower). Fiber provider is royally screwing the pooch today, and I currently have no ETA. The only silver lining is it's a widespread issue, so hopefully they're inclined to address the problem in a timely manner.

To relieve congestion on the backup connection, I've temporarily disabled external access to pictrs. All requests (GET or POST) to /pictrs will return a 404 response until the main connection is back online.

Sorry for the inconvenience. Please direct all rage at the shit fiber provider I foolishly "upgraded" to.

Status as of 5:17PM:

Service	Status	Notes
Lemmy ActivityPub	Up (Degraded)	Running on backup WAN
Lemmy API	Up (Degraded)	Backup WAN
Pict-rs	Down	Intentionally scuttled to save bandwidth
Tesseract Media Proxy	Down	Intentionally scuttled
Matrix Server	Up (Degraded)	Backup WAN
Matrix Web Client	Up (Degraded)	Backup WAN
DNS	Up (Degraded)	Operationally up, but no backup servers available
SMTP	Up (Degraded)	Backup WAN

---

Update: 4:30 PM: Tried to call and re-activate my account on my old provider since I own the equipment, and it's all still hooked up. I can even still access my old account on the web portal. Should be easy, right? Fucking wrong! They said they couldn't activate it until they sent me equipment, I returned it, and then called back to say I want to use my own. What the actual fuck?? I'm literally waving money at you, you're still charging me a $100 "install" fee (where you have to do nothing), and all you have to do is take it (and my equipment serial number). Fuck my life (and fuck those idiots, too).

Update 5:17 PM: We remain on backup WAN. ETA for fiber restoration is within 24 hours, but I'm not holding my breath. Like, how do you even fuck up this badly? (the whole goddamned state is affected including business customers)

Update 7:39 PM: Primary connection is back online (for now?). Services have successfully failed back over.

Service	Status	Notes
Lemmy ActivityPub	Up	-
Lemmy API	Up	-
Pict-rs	Up	-
Tesseract Media Proxy	Up	-
Matrix Server	Up	-
Matrix Web Client	Up	-
DNS	Up	-
SMTP	Up	-

Appalachian Power is the absolute worst about giving notifications for scheduled outages. The. Fucking. Worst.

I got an email at at 8:05 PM saying there would be a planned, scheduled outage at 8:00 PM and then 45 seconds later was plunged into darkness. Didn't bother to switch on the generator :sigh: and just decided to sit on the patio and get drunk in the waning daylight.

This is not the first time they've done that, and it probably won't be the last. Guess my next purchase is an automatic transfer switch.

Did I say "fuck Appalachian Power"? Because Fuck. Appalachian. Power.

Sorry about that. lol

Let me start off by saying, in my official admin capacity, that I really don't know if I want to be here anymore (see the problem outlined below). That has implications beyond me just deleting my account and going outside since I would not allow the instance to run unattended. I'm also unsure if my backup admin would have any interest in taking over full time.

If it does come to that, there will be a subsequent announcement and either a transition plan or a sunset plan.

That is still up in the air, but for now....

The Problem

I just wanted to take a moment here to address Lemmy's extremism problem / lynch-mob mentality and how DubVee is responding to that.

Last month, a new site rule was added expressly forbidding any form of extremism with regards to violence. Every rule is there for a reason, and this was added as a direct response to the increasingly violent rhetoric I've been seeing from users.

In full:

I don't care which end of the spectrum you're on. Any post/comment calling for or glorifying violence, especially political violence, will be removed. Offenders will be dealt with on a case-by-case basis, but bans will be the norm and swift for violating this rule.

After running reports against the database, there have been a very large number of bans issued in the last few days for both explicit and implicit violations of that rule:

Explicit Extremism

• Explicitly praising or justifying acts of violence
• Explicitly justifying / advocating for violence / "revolution" / whatever
• Suggesting, justifying, or advocating political violence, vigilantism, and/or breaking with the foundational principles of this country in order to "solve" a problem.
• Hypocritical posts/comments that amount to "Fascism / authoritarianism / whatever -ism is bad, but a little bit is okay when it's my side doing it"
• Stirring the pot by jumping into otherwise civil discussions and suggesting acts of violence (I'll admit, those were mostly new accounts and likely provocateur trolls seizing on current events, but established accounts are also guilty)

Implicit Extremism

• Patterns of upvoting or boosting posts/comments that explicitly praise, endorse, or advocate for violence (or any of the 'explicit' items above)
• Patterns of downvoting posts/comments that denounce violence or those that advocate for lawful/peaceful solutions.

Note: Before you cry "thought crime!" about issuing bans based on vote patterns, it is a well-known tactic for trolls to operate bot or sockpuppet accounts to manipulate votes to serve an agenda. Owing to the federated nature of Lemmy, this becomes easier to do and harder to detect since those can be distributed among different instances that have varying moderation, signup, monitoring, and administrative policies and procedures. So, no, I have no compunction about banning accounts that express support for extremism by way of voting; there's simply no straightforward way to tell bot from person from my point of view.

Even accounting for gallows humor, I must reiterate that extremism and violence are absolutely forbidden, and there will be zero tolerance shown.

While I'm aware there's a certain "mob mentality" inherent to social media in general, "lynch-mob mentality" will not be tolerated; we're better than that, and DubVee is not here to spread hate, violence, or fan the flames.

In Conclusion

If it turns out that we end up blocking 90% of the Fediverse because of this, then I'll just shut this whole instance down. DubVee, by way of federation, will not devolve to 4chan (or 8chan or wherever the shitty little edgelords post their green text nowadays), and I will (figuratively) burn it to the ground before it gets to that point.

What Can You Do as a User?

First and foremost, make sure you report any content that violates this rule and refrain from violating it yourself.

If you're seeing less comments than you were a few days ago or are no longer seeing content from certain users, this is most definitely why. You can check the modlog to see if someone you're expecting to see has been banned.

If you (local user) wish to challenge a ban and advocate for someone who has been blocked, you may send me a DM with your request, and I will review the account further (accounts were reviewed prior to banning, but it never hurts to take a second look).

If you have a problem with this rule and our enforcement of it, there's plenty of other instances you can choose from.

This Weekend

There will be some periodic downtime this weekend as I do some physical cleanup / rearrangement to my equipment. I'll try to keep these to a minimum, but at some point, everything is going to need to go offline for a bit in order to re-cable the UPS and power connections.

Plan is Saturday afternoon between 1 and 4 pm. Hopefully any interruptions will be brief.

Next Week

I'm switching to a new fiber provider next week (FTTH vs current FTTN), so there will be a brief disruption as I cut over to that. That downtime should only be a matter of seconds since both connections will be active, but just in case it turns into a whole thing, I'm mentioning it now. The new fiber is supposed to be installed Tuesday afternoon, and I plan to do the cutover around 7PM local time.

Next Few Months

I'll be expanding the storage server in the coming weeks, but that should not have any immediate impact on DubVee's normal operation.

With these, I hope to provide a better, faster experience. I'm also looking into setting up an Invidious instance to use with DubVee as well as exploring setting up and integrating a Peertube server.

I've been working on the latest release of Tesseract for the last few weeks, and I'm happy to announce the first beta release of the upcoming 1.4.0 series.

I daily drive my dev version throughout the development process, so all the bugs I've encountered/introduced should be fixed. The beta test on the main instance is more of a formality and final shakedown. That said, please report any bugs you encounter either through Github or by describing them in a comment here.

1.4.0 is a significant update as it drops all backwards compatibility with the 0.18.x API and introduces several new features. It's also had some additional polish since 1.3.0 and added things that have been requested for quite some time (pasting images into post/comments, etc).

As always: thanks in advance for being my beta testers. Appreciate it!

Change log for 1.4.0 if you want to see what's new: https://github.com/asimons04/tesseract/blob/1.4.0/ChangeLog.md

If you've noticed content from Lemmy World lagging by a few hours the past few days, it's not just you.

Long story short, it's a problem with how Lemmy sends activities, and it's heavily impacted by latency between sending and receiving server and creates a fixed upper limit on the number of activities per second that can be sent. Lemmy World is hosted in Finland, and DubVee on the US east coast. There's only so much I can do to work around the inherent latency of a trans-Atlantic link.

We're not alone in this. Some instances, particularly in Australia, New Zealand, and US west coast, are impacted more dramatically.

While there's always some federation delay/backlog, it only escalates to noticeable levels every so often. Not sure why, but I've gone over our infrastructure top to bottom several times and cannot find any reason, on this end, for these events. The graph in the post shows the number of activities Dubvee is lagging behind Lemmy World over the last 30 days.

Normally, it's 300-500 activities which usually corresponds to a minute or less of lag between LW sending an event and DubVee processing it. Occasionally, when LW gets busy, we see spikes into the 3000-5000 range (~5-7 minutes of lag). Every so often, though, there will be huge backlog events (the spikes from 15K to 35K) which often take 8-12 hours to catch up. A month or two ago, I think the largest spike was around 180,000 (though that was a separate issue).

I believe this is being addressed in Lemmy itself, but it'll be a while before it's ready (and I'm certainly going to let other instances kick the tires before upgrading).

In the mean time, I've talked with some other admins and have deployed a Federation proxy. I won't go into the nitty-gritty details, but it does seem to be alleviating a lot, but not all, of the congestion. We went from averaging around 10,000 backlogged events to about 2500. So, definitely an improvement.

Update: Buffer has cleared, and things seem to be coming in pretty close to real time. Occasionally the buffer kicks in, but overall, it appears to be helping. Will continue to monitor. Usually 11-12 AM eastern time is when we start to see lag increasing in activities coming from Lemmy World.

Hopefully there's some more optimization I can do in the mean time, and hopefully Lemmy addresses this limitation, but for now, this should make things less bad.

Finally got around to upgrading to 0.19.3.

Upgrade did not go smoothly due to multiple DB migration step failures and less than helpful logging from Lemmy during the DB migrations. (surprised Pikachu).

After digging through to figure out what it was complaining about, the upgrade was finally able to be completed.

You will likely need to log in again (at least I did).

I'm in the process of migrating the pict-rs database to Postgres. Pict-rs has to be offline during this process, so images here will appear to be broken for a bit. You also won't be able to upload any images for the duration of this migration.

Estimated time is about 20-30 minutes.

Currently, we host 4 UIs:

• Tesseract (default UI at https://dubvee.org/)
• Mlmym (old Reddit looking UI at https://old.dubvee.org/)
• Alexandrite (desktop UI at https://alex.dubvee.org/)
• Lemmy-UI (crappy "default" UI at https://lemmy.dubvee.org/).

Due to all the bot crawler traffic that slips through, I'm having a hard time distinguishing actual usage from bot traffic. That said, it does seem like Mlmym and Alexandrite are rarely used by actual people (there's a lot of bot traffic to filter out that spoofs its user agent, so I could easily be wrong here).

Update: Mlmym does all the Lemmy API calls server-side, so all of the actual usage traffic was actually originating from my server (which I was filtering out).

I'm looking to decommission ~~Mlmym (the old Reddit style) and~~ Alexandrite.

I just updated both of those to the latest, so apologies for the brief disruption if you were using them, but ultimately I'd like to trim down the selection to just the ones being actively used.

So, poll time: Are you using old.dubvee.org or alex.dubvee.org? If so, please make your voice heard and I'll keep them around. If not, or if I don't hear back, I'll likely decommission them this weekend.

Update: Will keep Mlmym (old Reddit-style). Alexandrite is still up for decommission unless someone is using it as well.

I'm not quite done with the 1.3.0 release of Tesseract, but the last few bits are going to take longer than I expected. So I think I'm going to hold off and add those in a 1.3.x release a bit later.

I've gutted, re-implemented, and just flat-out re-wrote large portions of the application over the last month or two. I've also been daily driving the dev version, and I guess a couple other people were too. That said, it is stable enough for daily use and ready to get some more eyes on it before an official release.

So, please bear with me if you hit any bugs (I've fixed all the ones I've found so far). If you would be willing and kind enough, either submit a Github issue if you find a bug or just throw a reply to this post with as many details as you can provide.

Hope to run a few betas here before officially releasing 1.3.0.

Apologies for the downtime.

We had some severe weather come through on Tuesday that caused extensive damage to power infrastructure throughout the area. I thought my location had been spared the worst of it, but a tree decided to fall on the transmission line servicing this area Wednesday morning (according to the power company, anyway). By that point, there were about 55,000 other outages ahead of my area in line awaiting repairs (no hard feelings there).

Since power outages are rare here, at least ones lasting longer than an hour or two, I only keep about 2 gallons of fuel on hand for the generator. At typical 3/4 load, that usually last about 4-5 hours. Again, long outages are extremely rare here, so that's usually more than enough runtime. Sadly, this outage lasted much longer (even though I was able to stretch the generator runtime by slightly hacking my UPS)

Federated content should now be coming back in, but it'll take a while to catch up.

Lessons learned:

• Mother nature is a badass bitch. I'm going to start keeping at least 2 days of fuel on hand. Worst case, I'll throw it in the car before it goes stale.
• My 48v e-bike battery and stepdown converter can run my UPS and bare-minimum servers for about 6 1/2 hours (probably more; it was only about half charged and had sat untouched on a shelf for over a year). That came in handy and was why we were able to stay online as long as we did.
• My primary network provider is pretty solid. That stayed up longer than I thought it would.
• Failover to the backup WAN works but failing back to primary does not. That's a "me" problem to fix in my watchdog script.

"Why don't you just throw it in the cloud like a sane person?" you may be asking. Well, it is and it isn't already. It's a hybrid setup. The UI and front end caches/proxies are all cloud based but the DB and API are located on prem where I can throw as much resources at them as I want for free.

I've been tempted to move those up, but it would cost more money than I want to devote to Lemmy at the moment (at least if I want to maintain the same level of performance).

Most of my VPS's are at capacity, but I am going to work on setting up a standby VPS that can scale up and keep the most recent backup dump there.

Mostly, I just don't want to have to rely on donations to keep DubVee online. Right now, all of its components are secondary payloads on my existing VPS hosts or are running on-prem on my own hardware (for free, for all intents and purposes).

I'd rather deal with an outage from time to time than have to constantly wonder if I'm going to be able to pay the cloud hosting bills. It's one of the reasons I've envisioned DubVee remaining relatively small.

Weather / power related. May go offline for a bit until power is restored. Currently on UPS.

Update 1: Yep, expect downtime. UPS ran dry and moved over to aux generator. ETA from power company is tomorrow at 11 PM EST.

Update 2: Still no change in ETA. Have had to shed some load from the generator because my UPS's refuse to charge from it and every time the fridge kicks on, a random piece of my equipment would reboot :sigh:.

Hopefully my reduced power budget will stabilize that.

On the bright side, the primary network connection has remained up and haven't had to switch over to the (much slower) backup.

Update #3: (17:06) Generator running on fumes now and will probably be going offline soon to ensure a safe shutdown and save some fuel to cool the fridge tomorrow if power is still out. Apologizes in advance. I usually have very reliable power here and typically never need the generator for more than a few hours.

Update #4 (04/04/2024 14:48): Power restored 12 hours ahead of ETA (good work AEP crew!). We are now back up and running, but there is about an 18 hour backlog of federation activity that needs to be received. I've confirmed that those are starting to resume, but it will take a while. https://dubvee.org/post/dubvee.org/977774

Lemmy World

There was an issue today with content from Lemmy World not federating to us. After pulling my hair out and testing the DubVee stack top to bottom, I got in touch with the awesome admins over at LW to try to figure out the problem. After they gave their federation service workers a good, stern talking to, we're now receiving content again.

Unfortunately, due to the amount of time (about 4 hours) the federation messages were getting stuck, there is a huge backlog that needs to catch up: about 90,000 messages. As of this writing, there are still 55,024 queued ActivityPub messages in flight (that includes backlog and current messages).

So just a heads up if you're not seeing posts/votes/comments from there or if they don't show up for a while. They will eventually arrive, though they'll be in the feed based on when they were published rather than when they arrive. On the bright side, they are receiving posts/votes/comments from us; you just might not get a reply right away.

Hopefully the queue works through overnight and things are back to normal tomorrow 🤞

Update: Backlog queue down to 37,150 now.

Update #2: 6,531 and falling. Looking like we'll be back in sync shortly.

Update #3: Backlog has finished syncing as of 23:55 EST.

mander.xyz

This one was my fault. lol. About 4 months ago, I was shoring up the firewall against some malicious traffic patterns, and mander.xyz 's server IP got caught in the CIDR block crossfire. Since I see content from their users frequently, I wasn't aware there was a problem. Turns out those were just relayed to us by the home instances of the communities. After I made a firewall allowance for mander's server, I had to reach out to their admin to reset the "last seen" date for DubVee in order to "bring us back to life" as their server had marked ours as dead.

So, all of the communities on mander.xyz should now look alive again. 🎉

My sincerest thanks to @rooki@lemmy.world, @ruud@lemmy.world, and @sal@mander.xyz for their assistance today.

If you've noticed a large number of bulk post removals performed by me in the modlog recently and are wondering if I've gone full authoritarian: no, I haven't gone mad with power. Those were posts that were already removed by other community mods or posts that were self-deleted by their creators. Some were legitimate spam or otherwise violated server rules, but the bulk of them were just cleanup.

"If they're already removed/self-deleted, why remove them again?", you might ask.

Well, Lemmy treats Pictrs (the media subsystem, basically) like a black hole - images go in but it never removes them. When posts are modded or self-deleted, any media attached to them lives forever in pictrs with no clean way to remove them later*. That wastes a huge amount of disk space on my hosting stack for media that will never see the light of day again.

I'm not okay with that for so many reasons. Yeah, object storage is cheap, but why be wasteful?

The Lemmy + Pictrs integration...well, let's just say it leaves much to be desired. "Suggest a feature enhancement" or "ping the Lemmy devs about it" you may be thinking. Haha, right.

I'm building a new API to interact with Lemmy, and the admin/mod components were the first parts that I developed. I've deployed parts of the prototype API to periodically clean up removed/deleted posts along with the media that was attached to them. I could do this silently on the backend and you'd never know, but in the interest of transparency (and also testing that the API works as expected), I've let it log its activities in the modlog as it would when it moves to production.

So, in closing, no, I have not gone mad with power. I'm just trying to keep my disk usage sane and not clutter up storage with abandoned media.

*They can be removed later, but it's a clunky external process that doesn't offer any guarantees.

Apologies for the brief ~20 minute outage. Had a loss of power and one of the two UPS's failed without warning. Had to manually move equipment over to the remaining UPS until power is restored. The cold boot takes frigging forever and was responsible for the bulk of those 20 minutes, but everything should now be back up and running.

We also lost the primary internet connection and are running on the failover, so things may be a bit sluggish.

Utility company said a tree was at fault, so will likely have both electric and network service restored in an hour or two. Unless UPS-A decides to die on me too.

If you've noticed that you're landing on tesseract.dubvee.org by default now, it's because I've done some restructuring to Dubvee's infrastructure to take advantage of the new features I've written into Tesseract. It also means you will need to login to your account again. Sorry about that, but it's a small price to pay for the benefits we'll see.

If you have Dubvee installed as a PWA, you'll probably also need to "re-install" it from the tesseract. domain. It'll redirect gracefully, but unless you resinstall, it will have a title bar and not look app-like. Again, my apologies.

Why?

I've added image proxying and caching to the Tesseract server process which allows it to act as a pseudo-CDN. This takes a significant burden off of my Lemmy + Pict-rs server while also making images and other media load faster for users. It also reduces the load on other Lemmy instances by not repeatedly fetching the same media. Win/win/win!

Additional Perks

I've left this Tesseract server instance unlocked so you can login to multiple Lemmy instances at the same time and seamlessly switch between accounts.

Beta Notice

I still have this version of Tesseract (1.2.8) in beta right now, so please let me know if you encounter any bugs.

Update

Other third-party apps and frontends are unaffected by this. They'll still function as normal but won't benefit from the caching. When configuring other apps (Jerboa, Sync, etc), you'll still put dubvee.org as your instance.

They've got a massive spam problem, and their mod actions do not federate out. So, while they may be cleaning up spam on their end, none of that federates to Lemmy and we get stuck cleaning up the mess in its entirety.

Have had good experiences with users from kbin.social, so it's not them that's the problem. Looking at the magazines on the Kbin side, they are cleaning up spam, so the mods/admins are on the ball. But without those cleanup actions federating out, they're still contributing to fediverse-wide spam that everyone else has to deal with.

Also, Kbin doesn't seem to have the concept of a registration application, only a captcha, which seems to be a very low barrier to entry for the spam bots that keep popping up there.

https://codeberg.org/Kbin/kbin-core/issues/570

Once that issue with Kbin core is addressed and mod actions from Kbin start federating, we will resume federation with kbin.social.

Update: We've re-federated but have removed all communities that are hosted on kbin.social

@Antik@lemmy.world suggested removing the Kbin communities to alleviate the spam vector. That will still allow Kbin users to interact with us (they were never the problem) while preventing spam coming from their communities where their mod actions don't propagate out. If any spam from Kbin comes through to a Lemmy community, mod actions can be taken that will be federated.

Thanks, Antik!

Figured it was time for another status update. Will try to keep it brief.

System/Infrastructure Status

We are within allowances for all metered resources (bandwidth, disk, etc) and will continue to be so for the foreseeable future. Translated: it still isn't costing me any extra money to host DubVee.

Image Uploads Have Been Re-Enabled (With Limits)

Image uploads have been re-enabled but with a limit of 150kb. This is mostly to support setting your profile avatars, but I don't have enough granular control to limit it to just that purpose. As such, you can add images to posts/comments as long as the file you're uploading is less than 150KB.

Consolidating Lemmy Frontends

For quite some time, I've offered just about every alternative Lemmy front-end available. Logs have shown some to never be used, including by me. To simplify administration, I've removed Voyager and Photon from the offerings.

The remaining options are:

• Alexandrite
• Mlmym/"Old" Reddit
• Lemmy-UI

If there are any promising newcomers, I'm open to considering them. Nothing against Voyager app or its dev team. It's just that no one was using it, and I'm in housecleaning mode.

New Default UI

If you've noticed the main DubVee interface is much more sleek and elegant, it's because we've demoted Lemmy-UI from the apex domain and promoted Tesseract to be the new default. Lemmy-UI is still available at a subdomain (lemmy.dubvee.org) if you prefer to use that.

Tesseract used to be my custom build of Photon until I decided to make it an official fork so I could do larger, more comprehensive changes (backporting my customizations to each upstream release was something of a bitch).

For that reason, there's little need to continue hosting Photon which is why it was also removed from the offerings. You can learn more about Tesseract on its GitHub page or ask questions in the Tesseract Support Channel community.

Thoughts, opinions, gripes, bugs? Leave 'em below.

Now that this instance has been running for a little over two months, I thought it was a good time to zoom out a bit and do a little reflection.

Two Month Reflections

First, I cannot believe it's only been two months; it seems way longer than that (in both a good way and an "I'm so tired, you guys" way). While I do run other public-facing services, they're much more hands-off as far as day-to-day work is concerned. Lemmy, on the other hand, is very much like a living organism, almost a pet, that requires near constant supervision, interaction, and attention. The codebase is being rapidly developed, updated, and patched, new vulnerabilities and rough edges being discovered and mitigated, and bad actors constantly looking to exploit, DoS, spam, troll, or just shit all over the place.

It's not all bad, though. In the same two months, many new and exciting projects have sprung up to give Lemmy a fresh face with new UIs. DubVee is happy to offer several of those as official options (Photon, Mlmym, Alexandrite, Voyager). Watching Lemmy and these associated projects evolve, making contributions to the projects, and learning from/sharing tips & tricks with other instance admins have been exciting side-quests in my adventure of running this instance.

That said, there's hardly a dull day when it comes to running/administering the instance, but it is definitely an enjoyable experience overall.

What makes Lemmy unique, compared to the other more static services I run, is the sysadmin tasks are only half the work. The other half comes in the form of interacting with the communities I've created/joined, posting content, and moderating. While it's been rewarding and fun, it is also a lot of work sometimes. Lemmy is very much still small and growing, so there is also the pressure to "be the change you want to see in the world" which further motivates me to contribute content.

If I was just made aware of Lemmy today, would I still stand up my own instance?

Yeah, probably (ask me in again in 6 months though 😆).

Federated and decentralized ActivityPub applications are something I find very cool and hope to see gain traction as we move into the next phase of the web, so I'm happy to be a part of the early days of that.

I hail from the tail-end of the old days of the internet where people ran BBS's, forums, and sites as a hobby, when ads were rare or non-existent, and when everything, everywhere wasn't trying to monetize/inflame every human interaction you were trying to have. Lemmy seems like a modern take on the hobbyist-run forums of days past, and it's been like a breath of fresh air. Posting/commenting on Lemmy feels more like having an actual conversation/debate than the "shouting dick jokes into the void" feeling of larger platforms.

Operational Status, Funding, and Donations

Operational Status and Hosting Costs

This instance is 100% a hobby for me, so I have been prepared since day one to cover all associated costs as long as they remain reasonable. The Lemmy services are hosted from my project servers in my basement, so it costs me nothing, upfront anyway, to run them. DubVee's biggest expense to date, aside from my time/sanity, was the domain name. That was a vanity choice, though, as my first/test Lemmy instance, like many others out there, ran under my personal domain at no extra cost. At $20/yr for dubvee.org, I don't have a problem with the additional expense (I used to spend almost that much per day on fast food -- don't judge).

I use two cloud servers for the public-facing frontends and for caching/DDoS protection plus one more for status monitoring, but those are shared with my other services and have incurred no additional cost with the addition of Lemmy. Between the two servers and factoring in all of my other services, projections show my traffic egress limits and other metered resources to be sufficient for the foreseeable future and likely beyond.

With the core services being hosted on-prem, DubVee is susceptible to power and internet outages. I am fortunate that both have been largely reliable to date with minimal outages. I have approximately 40 minutes of UPS runtime which, to date, has covered 95% or more of my power interruptions. Internet uptime and stability is comparable.

I do have a backup/auxiliary internet source, but it is not of sufficient capacity to keep DubVee online for the public. However, I do have things configured to route federation traffic over the backup connection, when needed, as that traffic is more lightweight and less sensitive to latency. I do that so posts/comments from other instances will still come through.

Our application and infrastructure status page can be found here: https://status.dubvee.org/

Funding/Donations

At this time, I have no plans to solicit or even accept donations. Barring a need to move fully to a cloud provider, I also do not see that changing in the future since it would, in my mind at least, turn a hobby into an obligation.

Thoughts for the Future

As stated earlier, I'm happy to be part of what I hope becomes the new model for social interactions online. I think it's in everyone's benefit to get away from engagement algorithms, constant ads, and centralized content, so I'm planning to stick with the Fediverse for the long haul.

I made an intentional choice to try to keep this instance small. The decision was made for both technical requirement reasons as well as administrative/moderation overhead. Currently, I am the sole admin of this instance, and I don't have the time/energy to manage a large number of users (at least with the current state of Lemmy's moderation toolset). Should signups/usage here increase significantly, I may need to recruit another admin as well as explore my contingency plan of moving the Lemmy backend services to a cloud provider.

If moving the instance fully to a cloud host becomes necessary, and depending on costs associated with that, I may re-evaluate soliciting/accepting donations. At this time, these are contingency plans only.

What happens if I ever decide to shut down?

While I currently have no plans to abandon DubVee or the Fediverse, things can and do change. As DubVee is still a small instance, current shutdown plans are to simply reach out to active users and let them know when I plan to sunset the instance and help them move to another. I'll also be happy to provide any data I can export for them to assist in their migration.

If by chance DubVee grows sufficiently (or is still small but someone else wants to take over operations in my stead), I would be open to transferring the domain and server-side resources to another party, assuming acceptable arrangements and guarantees can be made. I'd also be willing to co-admin or otherwise remain involved in such a scenario.

None of that should be taken as a hint I plan to shut down. My goal with the above statement is to simply codify the steps I would take if/when that scenario should arise.

What are your thoughts?

If there's anything you'd like to see implemented, spelled out in policy, or something I can do to improve the experience here, please feel free to suggest in the comments.

Did you know there's more than one way to Lemmy? Since Lemmy usage has taken off, so have the projects to develop alternate frontends. When one seems promising, we will usually adopt it as an option.

Below are the official and 3rd party Lemmy user interfaces supported by DubVee:

Lemmy UI

The default Lemmy experience through the official frontend.

• URL: https://dubvee.org/
• Github: https://github.com/LemmyNet/lemmy-ui

Photon

A sleek, mobile and desktop friendly interface that can be installed as a PWA (progressive web app). This is currently my "editor's choice" for Lemmy mobile access.

• URL: https://m.dubvee.org/
• Github: https://github.com/Xyphyn/photon

Voyager

A progressive web app (PWA) designed for mobile. Has both an Android and iOS (default) skin.

• URL: https://voyager.dubvee.org/
• Github: https://github.com/aeharding/voyager/blob/main/Dockerfile

Mlmym

An old-Reddit style UI for Lemmy. Definitely nostalgic and scratches the "Reddit" itch, but feels a little claustrophobic sometimes. Still, it's a solid UI, and I'm sure it will improve with time.

• URL: https://old.dubvee.org/
• Github: https://github.com/rystaf/mlmym

Alexandrite

A sleek Lemmy UI designed primarily for desktops. The default option is annoyingly purple, and you can only change the theme once you're logged in. The theme switcher is a slider which is weird/annoying. This UI is starting to grow on me, though.

• URL: https://alex.dubvee.org/
• Github: https://github.com/sheodox/alexandrite