OH NO, I have TERRIBLE NEWS my new FAVOURITE SHOW got CANCELLED (probably by the WOKE MOB)
Hey mate what do you think learning is. Like genuinely, if you were to describe the process of learning a subject to me.
Ok now I'm wasted af and this show rules, this is the best fucking thing ever, gather all your friends and watch this shit, they hired the Mooch and someone whose credential is that they're a YouTuber to not give money to people whose ideas are "what if X but crypto" like one of them is literally "what if water but there's an NFT on it"
The first Crypto Name they introduce is fucking Anthony "Unit of Time Measurement" Scaramucci and I got severe whiplash, I am not mentally ready to watch this, I need to refill my drug drawer
I wouldn’t argue with someone who said reasoning models are a substantial advance
Oh, I would.
I've seen people say stuff like "you can't disagree the models have rapidly advanced" and I'm just like yes I can, here: no they didn't. If you're claiming they advanced in any way please show me a metric by which you're judging it. Are they cheaper? Are they more efficient? Are they able to actually do anything? I want data, I want a chart, I want a proper experiment where the model didn't have access to the test data when it was being trained and I want that published in a reputable venue. If the advances are so substantial you should be able to give me like five papers that contain this stuff. Absent that I cannot help but think that the claim here is "it vibes better".
If they're an AGI believer then the bar is even higher, since in their dictionary an advancement would mean the models getting closer to AGI, at which point I'd be fucked to see the metric by which they describe the distance of their current favourite model to AGI. They can't even properly define the latter in computer-scientific terms, only vibes.
I advocate for a strict approach, like physicist dismissing any claim containing "quantum" but no maths, I will immediately dismiss any AI claims if you can't describe the metric you used to evaluate the model and isolate the changes between the old and new version to evaluate their efficacy. You know, the bog-standard shit you always put in any CS systems Experimental section.
A company that forces you to write a "Connect" every half-year where you reflect on your performance and Impact™ : (click here for the definition of Impact™ in Microsoft® Sharepoint™)
There is only one True Word
Some of them are pretty spot on.
Quantum computing reality vs quantum computing in popculture and marketing follows precisely the same line as quantum physics reality vs popular quantum physics.
I think the end is way too generous. I don't think we deserve an end.
I've been thinking about this post for a full day now. It's truly bizzare, in a "I'd like to talk to this person and study their brain" kind of way.
Put aside the technical impossibility of LLMs acting as the agents he describes. That's small potatoes. The only thing that stays in my mind is this:
take 2 minutes to think of precisely the information I need
I can't even put into words the full nonsense of this statement. How do you think this would work? This is not how learning works. This is not how research works. This is not how anything works.
I can't understand this. Like yes, of course, some times there's this moment where you think "god I remember there was this particular chart I saw" or "how many people lived in Tokio again?" or "I read exactly the solution to this problem on StackOverflow once". In the days of yore you'd write one Google query and you'd get it. Nowadays maybe you can find it on Wikipedia. Sure. But that doesn't actually take two minutes either, it's like an instant one-second thought of "oh I know I saw exactly this factoid somewhere". You don't read books for that though. Does this person think books are just sequences of facts you're supposed to memorise?
How on earth do you think of "precisely the information you need". What does that mean? How many problems are there in your life where you precisely know how the solution would look like, you just need an elaborate query through an encyclopedia to get it? Maybe this is useful if your entire goal is creating a survey of existing research into a topic, but that's a really small fraction of applications for reading a fucking book. How often do you precisely know what you don't know? Like genuinely. How can your curiosity be distilled into a precise, well-structured query? Don't you ever read something and go "oh, I never even thought about this", "I didn't know this was a problem", "I wouldn't have thought of this myself". If not then what the fuck are you reading??
I am also presuming this is about purely non-fiction technical books, because otherwise this gets more nonsensical. Like what do you ask your agents for, "did they indeed take the hobbits to Isengard? Prepare a comprehensive review of conflicting points of view."
This single point presumes that none of the reasons for you absorbing knowledge from other people is to use it in a creative way, get inspired by something, or just find out about something you didn't know you didn't know. It's something so alien to me, so detached from what I consider the human experience, I simply don't comprehend this. Is this a real person? How does the day-to-day life of this person look like? What goes on in their head when they read a book? What are we moving towards as a species?
Turns out software engineering cannot be easily solved with a ~~small shell script~~ large language model.
The author of the article appears to be a genuine ML engineer, although some of his takes aged like fine milk. He seems to be shilling Google a bit too much for my taste. However, the sneer content is good nonetheless.
First off, the "Devin solves a task on Upwork" demo is 1. cherry picked, 2. not even correctly solved.
Second, and this is the absolutely fantastic golden nugget here, to show off its "bug solving capability" it creates its own nonsensical bugs and then reverses them. It's the ideal corporate worker, able to appear busy by creating useless work for itself out of thin air.
It also takes over 6 hours to perform this task, which would be reasonable for an experienced software engineer, but an experienced software engineer's workflow doesn't include burning a small nuclear explosion worth of energy while coding and then not actually solving the task. We don't drink that much coffee.
The next demo is a bait-and-switch again. In this case I think the author of the article fails to sneer quite as much as it's worthy -- the task the AI solves is writing test cases for finding the Least Common Multiple modulo a number. Come on, that task is fucking trivial, all those tests are oneliners! It's famously much easier to verify modulo arithmetic than it is to actually compute it. And it takes the AI an hour to do it!
It is a bit refreshing though that it didn't turn out DEVIN is just Dinesh, Eesha, Vikram, Ishani, and Niranjan working for $2/h from a slum in India.
They were still early