Selfhosted

39239 readers

298 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago

MODERATORS

HybridSarcasm@lemmy.world

HybridSarcasm@lemmy.hybridsarcasm.xyz

Questions on backing up to S3 Glacier Deep Archive. (lemmy.world)

submitted 10 months ago by MigratingtoLemmy@lemmy.world to c/selfhosted@lemmy.world

48 comments fedilink hide all child comments

Hi everyone. I was considering backup options to Glacier Deep Archive, and wanted to know:

Which software do you use to encrypt client-side, obfuscate, compress and deduplicate the data before you send it to S3?
What is the difference between Restore Requests (bulk) and Outbound data transfer and which one will I be using when I want to pull my data from AWS?

I'll be storing approximately 8TB or so of data, which is why I was looking at inexpensive ways to back it up other than buying an HDD outright.

Thanks!

you are viewing a single comment's thread
view the rest of the comments

[–] MigratingtoLemmy@lemmy.world 2 points 10 months ago (1 children)

Thanks! Which cloud provider do you use?

[–] Rootiest@lemmy.world 2 points 10 months ago (1 children)

I've been using S3 but I'm considering Cloudflare R2 as it might be a bit cheaper

[–] MigratingtoLemmy@lemmy.world 1 points 10 months ago (1 children)

Really? Their pricing is even more expensive than AWS' S3 Glacier Archive! I'd much rather use BackBlaze B2 than pay that much!

[–] Rootiest@lemmy.world 2 points 10 months ago* (last edited 10 months ago) (1 children)

Tbh I don't really bother with Glacier. It is a lot more expensive than it seems especially when you want to restore anything.

I generally just use intelligent tiering and it kind of balances out.

You might think "oh well I'm probably never going to restore from here anyway"

I am here to tell you that's a very foolish attitude.

If you aren't testing your backups you might as well not have them.

My honest advice if you must insist on using Glacier is to start off in a normal tier, and keep it there long enough to have tested the backups before transferring it as-is into Glacier.

It's not perfect as there's really no guarantee that data remains safe but at least it mitigates the possibility and reduces the cost to initially use standard tiers before retiring it to Glacier.

[–] MigratingtoLemmy@lemmy.world 1 points 10 months ago (1 children)

You're right about testing backups. I will have 2 different backups, one for my config and the second for the irreplaceable media. Indeed, restoration from Glacier is too expensive for the data that I plan to back-up.

I was looking at Scaleway's Glacier offering, B2 and iDrive. How do you propose I test my backups? I could certainly pull in my config and test it on a VM, but how do I check that I have backed up my media? I plan to encrypt, compress, deduplicate and then ship it off.

[–] Rootiest@lemmy.world 1 points 10 months ago (1 children)

I guess it depends on how you do it.

I use Kopia so I can easily mount a snapshot like a removable disk or restore a snapshot so I typically test my backups by simply restoring them

[–] MigratingtoLemmy@lemmy.world 1 points 10 months ago

Thanks