datahoarder

6699 readers

1 users here now

Who are we?

We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.

We are one. We are legion. And we're trying really hard not to forget.

-- 5-4-3-2-1-bang from this thread

founded 4 years ago

MODERATORS

archivist@lemmy.ml

Archiveteam Veoh grab (tracker.archiveteam.org)

submitted 3 days ago* (last edited 2 days ago) by kabi@lemm.ee to c/datahoarder@lemmy.ml

7 comments fedilink hide all child comments

Just looking at the numbers, it doesn't seem to me like archival will complete before the shutdown date (nov. 11). There are 2million+ elements left, likely 100TB+ of videos.

If you care to help them out, see instructions at the top of the page. Be sure you have a "clean connection", though.

edit: They're saying that the current rate seems to be plenty enough to finish by the deadline. Workers are often left idling at the moment.

you are viewing a single comment's thread
view the rest of the comments

[–] clb92@feddit.dk 3 points 2 days ago* (last edited 2 days ago) (2 children)

I've started a few warriors, but it's not helping because they've activated rate limiting. I just get:

Tracker rate limiting is active. We don't want to overload the site we're archiving, so we've limited the number of downloads per minute. Retrying after 240 seconds...

So the bottleneck is that they don't want to overload Veoh.

[–] kabi@lemm.ee 1 points 2 days ago

They are about to double the rate lmit, so it should be a little better...

[–] kabi@lemm.ee 1 points 2 days ago

Yeah, that's what I'm getting now, too. Hopefully they take the deadline into account for rate limiting... Still good to have warriors ready to retrieve, I guess.