datahoarder
Who are we?
We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.
We are one. We are legion. And we're trying really hard not to forget.
-- 5-4-3-2-1-bang from this thread
view the rest of the comments
Im curious on why you are using zfs snapshots? (i never used them myself as I handle my backups with duplicity)
Snapshots on the file system are instant and can be rolled back or opened in read only mode if you need an older version of the file.
Sending a snapshot through replication is also much faster as the system just sends the snapshot itself which updates the files system in one pass by sending all of the changes made rather than having to compare file by file.
For comparison, using rsync to update my backup with 250,000 files might take 20-30 minutes because it has to check every file on the source and destination to see if it has been modified. Sending a zfs snapshot is 15 seconds because it just sends the differences in one pass.
do you encrypt them?
Yes they’re all encrypted with LUKS. Using unRAID as my base OS which handles most aspects of the process.