this post was submitted on 25 Aug 2023
44 points (100.0% liked)
cybersecurity
3177 readers
1 users here now
An umbrella community for all things cybersecurity / infosec. News, research, questions, are all welcome!
Community Rules
- Be kind
- Limit promotional activities
- Non-cybersecurity posts should be redirected to other communities within infosec.pub.
Enjoy!
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I can't speak specifically to the infosec bots, but I suspect it has something to do with all of the Lemmy instances mirroring every post. It could add a lot of weight to SEO for a various websites. So if they can get a post that doesn't get deleted, that's SEO fodder
Seems like Lemmy should add a
rel=canonical
link when browsing federated communities - this would “solve“ this issue (and would be the correct thing to do anyway).I believe Lemmy instances disallow crawling by default, so SEO is probably not why. Would be nice to find Lemmy results in Google if they can sort out the canonical URL problem. Reddit was a great resource for random questions, and if people move here it should still be easy to find.
What makes you say that? robot.txt just disallows things like /create_community and there's no robots, googlebot, etc meta tags in the source that I can see, and no nofollow apart from on a few things like feeds.
Also, I'm sure I've seen Lemmy appearing in search results already.