It's by design, thanks capitalism + AI shit
this post was submitted on 26 May 2025
115 points (97.5% liked)
No Stupid Questions
2788 readers
54 users here now
There is no such thing as a Stupid Question!
Don't be embarrassed of your curiosity; everyone has questions that they may feel uncomfortable asking certain people, so this place gives you a nice area not to be judged about asking it. Everyone here is willing to help.
- ex. How do I change oil
- ex. How to tie shoes
- ex. Can you cry underwater?
Reminder that the rules for lemmy.ca still apply!
Thanks for reading all of this, even if you didn't read all of this, and your eye started somewhere else, have a watermelon slice 🍉.
founded 3 years ago
MODERATORS
To reiterate: My problem is not with SEO, or spam, or AI-generated websites, or irrelevant results. My problem is getting no results much more frequently than in the past.
Ed Zitron has been reporting on this in his newsletter and on his podcast. Yes, Google deliberately made search worse to drive engagement, hence ad dollars. You can find Ed's work pretty easily, but here's a specific article on the topic:
The Man Who Killed Google Search
It's available in podcast form, if you enjoy podcasts. The show is called Better Offline. I listen every week.
It’s all search engines and it’s a near-impossible task. SEO spam is drowning and strangling the internet. There’s far more fake sites out there than there are legit sites. They steal content from legit sites (with mass bot-crawling), they generate fake content with AI, they link to other sites to form vast networks of “fake engagement.”
The harder you try to filter them out the harder they try to trick you! It’s a never ending arms race.
Can you ELI5 what SEO spam is? And what you mean by fake sites steal content?
SEO spam is search engine optimization spam. These are fake sites full of ads that make money when people visit them and click on the ads. The way they get people to visit is via SEO: tricking Google’s search engine to rank the site higher than other sites undeservedly. The way they pull off this trick is manifold (since there’s an arms race of techniques to detect SEO and new SEO that evades detection). Classic techniques include:
- filling the page with irrelevant keywords
- creating networks of sites that link to each other (link farms; this targets the PageRank algorithm which uses sites linking to each other as an honest signal of credibility and relevance)
- showing different content to the search engine’s crawler bots than you do to real users (cloaking)
- stealing content from popular sites (scraping) and using it to bulk up your fake sites (classic targets include Wikipedia, Stackoverflow, and tons of recipe blogs)
- tricking legitimate websites into linking to you (link farms tend to be detected and down-ranked en masse; legitimate sites linking into them help avoid this detection by reputation laundering)
- hacking people’s social media accounts to promote your fake websites
- hosting shady/illegal content such as porn, gambling, or TV and sports streams
Mate that's so helpful! Thanks for taking the time to write that out it's really easy to follow. I wondered why I sometimes ended up on bizarre sites like that...
I don't have official information either, but for a potential explanation which doesn't sound like a conspiracy theory:
In my experience, search engines dropped off in quality rapidly in the first few months after LLMs (ChatGPT and such) became publicly available. LLMs made it trivial to craft plausible-looking webpages and there's a financial incentive to do so, because you can put ads on the webpage and get paid for any unfortunate visitor. The result is tons of spam webpages
Search engines had to deal with those spam webpages, i.e. filter them out, otherwise the results would've been even more useless. But that's where the problem comes in: Because LLMs are specifically built to imitate human language, it's extremely difficult to filter for them. This means that lots of non-LLM-generated webpages would get caught up in these filters, too.
In particular, I'm guessing, they're aggressively filtering webpages which aren't widely known, which could be either a new spam webpage or it could be that niche blog with the answer you're looking for. They can't really discern between the two, so both get filtered out.
There's a host of factors that lead into the degredation of (at least google) search over the last decade or so. Many have already been brought up, but one that hasn't is the result of the SEO arms race. When google first got popular, people realized they could game the algorithm by embedding keywords in their page to make the page appear in more searches. Google found a way to correct that, but it's been whack-a-mole with the so-called "Search Engine Optimization" industry ever since. These days, google says 'eh, fuck it' and basically runs a pay-to-play service to rank up people who are willing to pay to have their page listed higher. Most people don't even click past the first page, so not getting on the first page is effectively a death sentence for that page/site. It's a real incestuous/autocannibalizing cesspool anymore. Google search is now quite literally an ad listing rather than a search index.
I don't know if they're getting worse. I think the net is just getting worse.
It feels like I CANNOT find answers to questions anymore. Anything software or video game related is a living nightmare of blatantly wrong answers, irrelevant forum posts from 2011, or reddit posts with no comments/answers...
It's to push you into their AI stuff.
Gimp the search engine enough that you need to turn to AI to get a worse version than what you had before. Also a form of enshittification.
Is it just me, or are search engines getting worse?
It's both.
Social media control.
The faster you find what you're looking for, the earlier you leave the site which is a problem for corporations that pursue profit over their mission (which is most big public and private equity owned ones now).
It's a problem not just for search engines but social media, dating apps, grocery stores and other things.
This is true, but it’s also not helped by the fact that the web itself has gotten worse. A lot of stuff that previously would have been on a crawlable page is now on an amorphous infinite scroll feed behind a login screen, or worse still a YouTube video.
And the crawlable pages have been poisoned with SEO so there's often lots of keywords but almost no information.
It fascinates me that a model of non-profit app ownership hasn't come along to create utilitarian apps that actually achieve their goals. Like an open-source dating app or friendship finding app, maybe with mental health pros on the board.
Someone suggested the Swedish search engine Marginalia in another discussion recently. It seems uniquely suited to restoring the exploratoration/discovery aspect that online searches once had. (I don't know how it will be for the particular application you need or for local services in general, though.)
The exploration/discovery aspect, as in, exploring unknown, barren territory and discovering that you're lost.
This engine is a massive downgrade, whenever I enter an even slightly weird query, or put too large a phrase in quotes, I get no results at all.
Which, ironically, is the problem with standard search engines that induced me to make this post.
How do you mean weird phrases? How long a phrase in quotes? Sounds like you're playing that game people used to play, trying to stump Google back when it was new and gaining popularity, except people at that time were aiming to get no results on purpose.
It's been going downhill for about a decade in my experience. These companies put profits above their founding ideals. Google, bing, yandex and baidu are the only major search indexes out there, and they still use each other's data.
But for none of these companies search is a priority. It's just a means to an end. Google wants to sell you ads, bing wants to sell you ads, yandex and baidu are under the thumb of Russia and China.
It's unfortunate their goals and motivations aren't what they used to be. I explored alternatives. And although not perfect, I've got something decent now
What about Kagi? I know almost nothing about it but I heard good things about it on Lemmy. Is it just a paid, slightly better google?
A heck of a lot better than google. It's what I use
The more you stay, more ads you get. It’s in their interest to keep you longer, so they can justify to sell more ads. If they give you the result you want right away, you leave, which causes them a loss of potential ad revenue.
I haven't noticed that myself. I've found that pretty much anything pulls up at least some results.
I note that you don't give an example, and just thinking...if what you're searching for is NSFW, most search engines that I've glanced at these days default to having some form of adult content filter on by default.
But other than that, I can't think of a new filter that's shown up. And I think that search engines are only indexing more content these days.
If your search is something that you don't mind sharing an example of, would you mind doing so?