this post was submitted on 31 Jul 2024
28 points (100.0% liked)
Chat
7499 readers
6 users here now
Relaxed section for discussion and debate that doesn't fit anywhere else. Whether it's advice, how your week is going, a link that's at the back of your mind, or something like that, it can likely go here.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Do you find that your searches are higher quality than something like, say, DuckDuckGo?
Yeah, I'd say they're pretty comparable. And you can add filters, so like for example, I have all of Reddit blocked out from searches.
That reddit filter will have less than an effect than it used to. Reddit blocked all crawlers except google.
Fair point.
Brave search works for Reddit
reddit recently updated their robots.txt to disallow all crawlers Google paid a bunch of money to have access to crawl reddit
You'll still see old stuff, but crawlers that care about robots.txt will get no new information.
The best part of that robots.txt is:
Sure Jan.
Reddit believes in ~~an open~~ pay to access internet, but not the ~~mis~~use of ~~public content~~ our content we didn't make.
https://search.brave.com/search?q=site%3Areddit.com+trump&source=web&tf=pd Past day is not that old
A few possibilities,
No matter the reason, well behaving crawlers will no longer crawl reddit, Everything is disallowed in the robots.txt
Oh 100%. You can try kagi out on a free trial, which is what I'm on. It's nice because you get different search results that aren't constrained to the Google algorithm.
What I mean by this, I search for recipes a lot, on Google all the recipes are identical. When I use Kagi, they differ with the main search team I want.
I plan on using Kagi and subbing when I have the money.
It's better to pay for a product than be the product imho.
edit: for the recipes I mean on Google it's the same promoted main steam food network bs, etc every. Single. Time. With Kagi I feel like it's smaller people with recipes with more relevancy to the terms I type in.