This is a most excellent place for technology news and articles.
[Warning: "ideas guy" tier babble]
It's somewhat clear that search engines are too prone to go to shit, either due to malice or something worse (like stupidity).
Based on that, I wonder if a user-run, free-as-speech and open source decentralised search system wouldn't work. Roughly in the spirit of torrents - where anyone can use the system but if you're using it you're expected to contribute with it too.
There was (is?) the yacy project which used a distributed index, and the individual nodes would contribute to the index.
A hybrid of original Yahoo! and Google is probably the best option. Sites submit themselves, they get reviewed, and an algorithm catalogs the contents. So curation and automatic indexing together.
That exists, it's called Searx, and in my experience it isn't great
Searx is a meta-engine, as bdonvr mentioned.
One part of this (which isn't really covered in the article) is that Google historically had a give-and-take relationship with people gaming search engine results. SEO has been a thing for a long time, and it's impossible to make it go away. However, Google used to punish sites that took it too far. It wasn't necessarily ideal, but it worked well enough to keep egregious spam out of the top level results, and companies could still direct users to their site when they had something they were actually looking for. SEO consulting companies sprang up who knew Google's rules well, and that arguably meant a bunch of grifters being overpaid, but at least the results stayed relevant.
Google seems to have given up on enforcing many of those rules.
There's also some minor discussion in the MeFi thread about "federated search" as well.
Self-hosted search also seems like a strong possibility.
The problem that I see with self-hosting is that it isn't a practical reality for most people, due to different tech expertises and machine capabilities. Instead I think that a better system would allow you to simply install some software, and contribute as much as you can while you use it.
I'm not informed on MetaFilter. From your other comment it seems that it's also an indexing site (besides being a community - from their "About" page). Is this correct?
From your other comment it seems that it’s also an indexing site (besides being a community - from their “About” page). Is this correct?
Yes, it's got a really old-school layout, because it's been around since 1999. To me, the fact that they've managed to avoid advertising for 25 years while having the main indexing site as well as things like Ask MetaFilter, IRL meetups, and even a jobs board, it means they've been pretty darn good at managing their finances and figuring out how to support the site long-term without ads. They're also in the process of becoming an actual non-profit organization. They pay their moderators a living wage, because it's a job. That's... pretty amazing.
The comment section takes a bit to get used to, because it's just chronological order of comments, no sorted threads. Very, very old school web ethos. However, if you can get used to it, some really amazing discussion can happen in there.
One of the benefits of the ways MeFi posts work is often you have users doing massive amounts of research and providing literally mountains of links and analysis, you can get pretty lost in the weeds on some posts.
It's been the source of high quality discussions for a long time and there's some really interesting professionals on there who have been staples of the community for a long time. Think hackernews and how many people it has from the industry, but instead of it all being tech people (MeFi has it own share of techies) it is thoughtful and sometimes expert opinion from a large variety of disciplines, as well as first person accounts from people of all walks of life.
It's also where I first found this link (The Man Who Killed Google Search) and decided to post it here.
