No Stupid Questions

To reiterate: My problem is not with SEO, or spam, or AI-generated websites, or irrelevant results. My problem is getting no results much more frequently than in the past.

It’s all search engines and it’s a near-impossible task. SEO spam is drowning and strangling the internet. There’s far more fake sites out there than there are legit sites. They steal content from legit sites (with mass bot-crawling), they generate fake content with AI, they link to other sites to form vast networks of “fake engagement.”

The harder you try to filter them out the harder they try to trick you! It’s a never ending arms race.

It's by design, thanks capitalism + AI shit

I don't know if they're getting worse. I think the net is just getting worse.

There's a host of factors that lead into the degredation of (at least google) search over the last decade or so. Many have already been brought up, but one that hasn't is the result of the SEO arms race. When google first got popular, people realized they could game the algorithm by embedding keywords in their page to make the page appear in more searches. Google found a way to correct that, but it's been whack-a-mole with the so-called "Search Engine Optimization" industry ever since. These days, google says 'eh, fuck it' and basically runs a pay-to-play service to rank up people who are willing to pay to have their page listed higher. Most people don't even click past the first page, so not getting on the first page is effectively a death sentence for that page/site. It's a real incestuous/autocannibalizing cesspool anymore. Google search is now quite literally an ad listing rather than a search index.

I don't have official information either, but for a potential explanation which doesn't sound like a conspiracy theory:

In my experience, search engines dropped off in quality rapidly in the first few months after LLMs (ChatGPT and such) became publicly available. LLMs made it trivial to craft plausible-looking webpages and there's a financial incentive to do so, because you can put ads on the webpage and get paid for any unfortunate visitor. The result is tons of spam webpages

Search engines had to deal with those spam webpages, i.e. filter them out, otherwise the results would've been even more useless. But that's where the problem comes in: Because LLMs are specifically built to imitate human language, it's extremely difficult to filter for them. This means that lots of non-LLM-generated webpages would get caught up in these filters, too.

In particular, I'm guessing, they're aggressively filtering webpages which aren't widely known, which could be either a new spam webpage or it could be that niche blog with the answer you're looking for. They can't really discern between the two, so both get filtered out.

It feels like I CANNOT find answers to questions anymore. Anything software or video game related is a living nightmare of blatantly wrong answers, irrelevant forum posts from 2011, or reddit posts with no comments/answers...

I think what it is is, they want to serve you generic results so the further you go into specific territory you might get one or two results but not a lot. But search for the switch 2 or something popular that they can advertise over, and they'l have all the results in the world.

Is it just me, or are search engines getting worse?

It's both.

Someone suggested the Swedish search engine Marginalia in another discussion recently. It seems uniquely suited to restoring the exploratoration/discovery aspect that online searches once had. (I don't know how it will be for the particular application you need or for local services in general, though.)

It's to push you into their AI stuff.
Gimp the search engine enough that you need to turn to AI to get a worse version than what you had before. Also a form of enshittification.

Social media control.

It's been going downhill for about a decade in my experience. These companies put profits above their founding ideals. Google, bing, yandex and baidu are the only major search indexes out there, and they still use each other's data.

But for none of these companies search is a priority. It's just a means to an end. Google wants to sell you ads, bing wants to sell you ads, yandex and baidu are under the thumb of Russia and China.

It's unfortunate their goals and motivations aren't what they used to be. I explored alternatives. And although not perfect, I've got something decent now

The faster you find what you're looking for, the earlier you leave the site which is a problem for corporations that pursue profit over their mission (which is most big public and private equity owned ones now).

It's a problem not just for search engines but social media, dating apps, grocery stores and other things.

The more you stay, more ads you get. It’s in their interest to keep you longer, so they can justify to sell more ads. If they give you the result you want right away, you leave, which causes them a loss of potential ad revenue.

I haven't noticed that myself. I've found that pretty much anything pulls up at least some results.

I note that you don't give an example, and just thinking...if what you're searching for is NSFW, most search engines that I've glanced at these days default to having some form of adult content filter on by default.

But other than that, I can't think of a new filter that's shown up. And I think that search engines are only indexing more content these days.

If your search is something that you don't mind sharing an example of, would you mind doing so?