Newtra

Wow, I just scrolled through the front page and it was 100% depressing/anxiety-inducing news.

I don't think I want this in my life.

I had no idea Omeleto existed. Looks like I've got a few weekends of watching their vids ahead of me!

Good to see them learning LaTeX young. It's one of those life skills that no one should need, but everybody does need at some point

Why do I find "match-3" most offensive part of that thought?

Note: For this guide, we’ll focus on functions that operate on the scalar preactivations at each neuron individually.

Very frustrating to see this, as large models have shown that scalar activation functions make only a tiny impact when your model is wide enough.

https://arxiv.org/abs/2002.05202v1 shows GLU-based activation functions (2 inputs->1 output) almost universally beat their equivalent scalar functions. IMO there needs to be more work around these kinds of multi-input constructions, as there are much bigger potential gains.

E.g. even for cases where the network only needs static routing (tabular data), transformers sometimes perform magically better than MLPs. This suggests there's something special about self-attention as an "activation function". If that magic can be extracted and made sub-quadratic, it could be a paradigm shift in NN design.

You're right. Everything is suspiciously wordy, substance is sparse, and every headline is clickbaity. It's like they tuned the content specifically for google, not human readers...

EDIT: Because my comment was also lacking substance: e.g. the Steam Deck review in "30 Best Retro Handhelds Of 2024 [All Reviewed]" says "Yes it’s big, and the battery life… pretty terrible", then gives no further information about size or battery life, which seems extremely relevant to potential buyers. They wrote 8 paragraphs and shared only 3 shallow facts.

Google also is responsible for the SEO industry. They made ads hugely profitable, then started directing traffic to sites that serve more of their ads, regardless of quality.

"zero commercial prospects"? That sounds exactly like the sort of movie I'd pay money for!

I'd say it's more like they're failing upwards. It's certainly good for AMD, but it seems like it happened in spite of their involvement, not because of it:

For reasons unknown to me, AMD decided this year to discontinue funding the effort and not release it as any software product. But the good news was that there was a clause in case of this eventuality: Janik could open-source the work if/when the contract ended.

AMD didn't want this advertised or released, and even canned this project despite it reaching better performance than the OpenCL alternative. I really don't get their thought process. It's surreal. Do they not want to support AI? Do they not like selling GPUs?

With jobs, maybe. With careers, especially in STEM, you get lots of exceptions like extremely rewarding but low paying positions in academia, and tech companies that think they can just spend money instead of effort to fix their culture and broken hiring process.

Saying goodbye to their life in France, where they were paying around £2,574 (€3,000) in taxes every year,

So these people who were rich enough to own a second home wanted to spend more than 50% of their time in France, but were paying the vast majority of their taxes back to the UK?

No wonder the laws got tightened.

To accept it, you just move on with your life. Find the next thing you should do and do it. The more you dwell, the harder it will be to stop dwelling, so just break the cycle and go do anything else.

You will encounter plenty more people who are insistently wrong. Each one will affect you less than the last.