41
submitted 2 weeks ago* (last edited 2 weeks ago) by TankieTanuki@hexbear.net to c/technology@hexbear.net

Your answer could potentially help me un-fuck my website!

Details here.


Edit: The total capacity is currently 160 GB, so no amount of pruning will free up 1 TB. I need to mount an external share or something.


Edit2: I'm trying this but the command isn't working despite zero exit status.

top 34 comments
sorted by: hot top controversial new old
[-] CyberSyndicalist@hexbear.net 10 points 2 weeks ago

Do you have your own physical server or are renting one?

[-] TankieTanuki@hexbear.net 9 points 2 weeks ago

Renting a VPS in the Netherlands.

[-] fox@hexbear.net 7 points 2 weeks ago

You may be able to just buy more capacity from the vendor by writing to their support. It's all containers

[-] TankieTanuki@hexbear.net 6 points 2 weeks ago* (last edited 2 weeks ago)

Storage isn't one of the things the sell on their business page but yeah, maybe their team can rig up something for a fee. It's worth a shot.

[-] Awoo@hexbear.net 6 points 2 weeks ago* (last edited 2 weeks ago)

Some of these companies can be really good, others can be like "no". InmotionHosting support were the primary reason I used them despite being slightly more expensive because their support would literally help with any issue even ones no other company would, and they would do stuff like give free storage for a problem like this. At one point I was using like 21TB of space on their service and really testing the limitations of "unlimited storage" they offered on their plan and they were totally cool with it even though I was clearly taking the piss a little. The only thing they held me to was that all things being stored on it had to be frontend accessible.

[-] TankieTanuki@hexbear.net 3 points 2 weeks ago

frontend accessible

What does that mean, not encrypted at rest?

[-] Awoo@hexbear.net 2 points 2 weeks ago* (last edited 2 weeks ago)

No like physically used as part of the website. Not just sitting on the server that the website was on without actually being in use on the website itself.

At the time what we were doing was archiving all pictures and video from a very very large and active discord. These were then searchable, you could see who posted what pictures and video, in what order historically, even sort by total number of reacts.

I don't like the fact that discords buries all media for guilds and communities behind a god awful chat. So many screenshots and videos and all kinds of media are just buried in discord, extracting JUST the media and making it viewable and searchable by user etc preserves genuinely emotionally sentimental content. Things like that time a group of friends defeated their first dungeon together and stuff like that deserves to not get forgotten about never to be found ever again somewhere in the depths of a discord server.

Also I am a hoarder.

[-] PaX@hexbear.net 4 points 2 weeks ago

Idk if you pay for bandwidth or anything but if you do that might be cheaper than exchanging 1 TB of traffic or more over their network agony-deep

[-] TankieTanuki@hexbear.net 3 points 2 weeks ago* (last edited 2 weeks ago)

Bandwidth in and out of my VPS is free. Object storage ingress and egress is what will cost me.

[-] Tabitha@hexbear.net 3 points 2 weeks ago

IDK I'm drunk right now but maybe they meant "rent a 2TB VM in the same datacenter for a few days"

[-] TankieTanuki@hexbear.net 3 points 2 weeks ago* (last edited 2 weeks ago)

Unfortunately the storage on their largest VM is only 320 GB.

[-] What_Religion_R_They@hexbear.net 5 points 2 weeks ago

regarding your cgnat idea you can try to work with an ssh tunnel between your PC and the VPS to bridge them and do what you were planning to by mounting

[-] PaX@hexbear.net 5 points 2 weeks ago* (last edited 2 weeks ago)

I have been in similar situations before and there is no easy answer agony-wholesome

I have no idea how Peertube works but if you're on Linux you might be able to turn on filesystem compression.... but if the data you're storing is already compressed (like video) that probably won't help very much

You could try sshfs like you mentioned in your post but it might be unreliable :( , maybe you could try NFS or even mount remote storage over Plan 9 protocol

[-] TankieTanuki@hexbear.net 5 points 2 weeks ago* (last edited 2 weeks ago)

To mount an sshfs share I think I need to shell into my PC from the webserver (not the other way around) and I can't do that.

[-] PaX@hexbear.net 5 points 2 weeks ago

Unlimited destruction upon NAT qin-shi-huangdi-fireball

Do both hosts have internet connection via IPv6? You could try this over IPv6, usually no NAT, but your ISP might do some kind of firewalling

You could also probably put up a Wireguard (VPN) connection between them since the webserver is not behind NAT

Or also "forward a port" if you have access to the NATing router between your PC and webserver

[-] TankieTanuki@hexbear.net 5 points 2 weeks ago

I might be able to do IPv6 but I would need to drop my VPN (IPv4 only).

I know very little about Wireguard.

[-] PaX@hexbear.net 5 points 2 weeks ago

Sorry, had to log off for a bit

It seems you need some kind of NAT traversal mechanism. Wireguard is pretty easy to set up, it's built-into the Linux kernel these days. I've heard good things about Tailscale as well.

Sorry I wish I could help more :(

[-] JoeByeThen@hexbear.net 4 points 2 weeks ago

I just setup tailscale on my home Ubuntu jellyfin server the other day for the first time and it was a totally painless experience. Been working great.🤞

[-] Tabitha@hexbear.net 3 points 2 weeks ago

I could totally see tailscale working in this scenario lol

[-] What_Religion_R_They@hexbear.net 3 points 2 weeks ago

I don't know what peertube can do but can you move stuff to /dev/null as a storage device?

[-] comrade_pibb@hexbear.net 3 points 2 weeks ago

There's a cloud option if you don't want to self host: https://devnull-as-a-service.com/

[-] TankieTanuki@hexbear.net 4 points 2 weeks ago

lol reminds me of the odd/even API.

[-] comrade_pibb@hexbear.net 3 points 2 weeks ago

npm and its consequences etc etc

[-] TankieTanuki@hexbear.net 1 points 2 weeks ago

That's a neat idea, but when peertube goes to put the files back into remote storage it would fail to find them locally.

[-] Findom_DeLuise@hexbear.net 3 points 2 weeks ago* (last edited 2 weeks ago)

You probably have some archival shit under /var/log that you can prune. I will usually go level-by-level running du -sh {PARENT_PATH}/* to narrow things down one level at a time until I find the offender. Check under /opt and user home dirs, too. Otherwise, you're probably stuck adding another disk or pushing data out to an S3 bucket.

[-] TankieTanuki@hexbear.net 3 points 2 weeks ago

Can I make a BackBlaze bucket and mount that on the filesystem?

[-] Findom_DeLuise@hexbear.net 3 points 2 weeks ago

Depends on if you can free up enough space on / to install any packages you may need for the filesystem drivers. I've never had to do it either way; if you have a local rig that you can test with, that might be a start.

[-] farting_weedman@hexbear.net 2 points 2 weeks ago

Alternate solution: use atop sorting by fs access and kill the process doing it.

[-] farting_weedman@hexbear.net 2 points 2 weeks ago

Stop the service that’s doing the thing causing you a problem.

Fix the thing you screwed up

Restart the fixed service that was causing the problem

What could possibly go wrong?

In this case, and I’m making wild assumptions because I don’t have a peertube instance to play with, stop peertube, connect your local system up to it with a vpn to traverse your nat, mount some volume in the local system as the remote systems target for the file system move, then start peertube back up and see if it starts filling your local drive with some data.

[-] TankieTanuki@hexbear.net 1 points 2 weeks ago* (last edited 2 weeks ago)

When I made the wrong command, the application registered hundreds of move jobs in its database. The developer said there is no way to cancel the jobs. If I restart peertube.service then it will notice the incomplete jobs and immediately resume them. I think I'm going to use btrfs next time so I can take a system snapshot before executing risky server commands.

connect your local system up to it with a vpn to traverse your nat

I need a little more detail on this. Here's my takeaway:

  1. Host an (e.g.) OpenVPN server on the webserver, after creating the proper certificate infrastructure.
  2. Connect as client from home PC.
  3. VPN Success. Now I can access each other on the local subnet that starts with a 10. or something.
  4. Mount home PC's NFS share on the webserver.
[-] farting_weedman@hexbear.net 3 points 2 weeks ago

Yeah that’s what I was thinking. Standing up openvpn especially is pretty well documented.

[-] Sickos@hexbear.net 1 points 2 weeks ago

So if I'm reading this right, the issue is going to be that when you revive peertube, it's going to resume trying to move stuff? Is it running in a container or directly on that machine?

[-] Sickos@hexbear.net 1 points 2 weeks ago* (last edited 2 weeks ago)

If I'm understanding this correctly, and keep in mind I know nothing of peertube, I'm just an old coder tearing through their source, you could break your config. Configure it not to use object storage and restart. If can't find the storage it's trying to move stuff from, it can't move that stuff.

OBJECT_STORAGE: { ENABLED: config.get<boolean>('object_storage.enabled'),

[-] Sickos@hexbear.net 1 points 2 weeks ago

Or make the target folder/file read only

this post was submitted on 15 Jun 2024
41 points (100.0% liked)

technology

22966 readers
244 users here now

On the road to fully automated luxury gay space communism.

Spreading Linux propaganda since 2020

Rules:

founded 3 years ago
MODERATORS