this post was submitted on 08 Jun 2023
22 points (92.3% liked)

General Programming Discussion

7792 readers
42 users here now

A general programming discussion community.

Rules:

  1. Be civil.
  2. Please start discussions that spark conversation

Other communities

Systems

Functional Programming

Also related

founded 5 years ago
MODERATORS
 

Idea: Scrape all the posts from a subreddit as they're being made, and "archive" them on a lemmy instance, making it very clear it's being rehosted, and linking back to the original. It would probably have to be a "closed" lemmy instance specifically for this purpose. The tool would run for multiple subreddits, allowing Lemmy users to still be updated about and discuss any potential content that gets left behind.

Thoughts? It's probably iffy copyright-wise, but I think I can square my conscience with it.

top 7 comments
sorted by: hot top controversial new old
[–] dessalines@lemmy.ml 20 points 1 year ago (2 children)

This instance, like most instances, would prefer most content come organically, rather than just being a mirror of whatever reddit has.

[–] usernotfound@lemmy.ml 7 points 1 year ago* (last edited 1 year ago)

I fully understand and respect that, and would never run it on an instance that wasn't specifically set up for this purpose.

The intention is for it to be something people can OPT IN to, not OPT OUT of.

[–] Dubois_arache@lemmy.blahaj.zone 2 points 1 year ago* (last edited 1 year ago)

Anyway some of the content will be repeated because of the waves of information in the internet everyday, so... would be better to gain something of that accumulate of info that reddit has.

[–] BackOnMyBS@kbin.social 6 points 1 year ago

I'm not against it as long as I can avoid reddit taking over my feed. I'm happy to get away from reddit, not just because of their business practices, but the culture has changed to something I rather avoid. I'm liking this cozy federated feel so far and would prefer to keep reddit's culture from taking over my feed again.

[–] fruitywelsh@lemmy.ml 4 points 1 year ago* (last edited 1 year ago)

Well there is a tool that uses the API for this purpose https://github.com/rileynull/RedditLemmyImporter Maybe one of the little niches carved out for free API use, like for bots or something might work for that, but definitely should work now.

Another interesting idea is for getting historical data, pulling it from the archive.org project from the project r/datahoarders are doing now.

[–] stanleytweedle@lemmy.ml 4 points 1 year ago

I think that would be great. There's a wealth of posts and comments that users have made that deserve to be preserved and shared. It would help Lemmy grow and just be a good policy to make sure Reddit doesn't control access to the content those users generated.

As long as you link or referce back to the post and user I don't see how that would be legally or morally problematic. It's all public anyway, but IANAL so this this not legal advice, just my thoughts.

[–] xurxia@lemmy.ml 4 points 1 year ago

If you are talking about your posts there is no problem, but if you are talking about all the posts and replies of a subreddit, I think it is a bad idea as you are replicating comments and opinions of other people who couldn't delete them in a future if they wanted.

load more comments
view more: next ›