this post was submitted on 21 May 2025
95 points (94.4% liked)
Programming
20294 readers
296 users here now
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
- Follow the programming.dev instance rules
- Keep content related to programming in some way
- If you're posting long videos try to add in some form of tldr for those who don't want to watch videos
Wormhole
Follow the wormhole through a path of communities !webdev@programming.dev
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
There's avante.nvim for LLM integration, it supports most if not all LLM vendors at the moment.
I tried it, however, and got to the same conclusion as you. Not worth it.
I found the rough areas did different by model
e.g. Claude could not correct issues it introduced, it would sort of spiral through half refactors for an hour if unchecked.
Gemini Pro asked for way too much permission. "Should I do this?" Yes, go, do it! "Okay, should I now add my edits" Yes. "Okay I added edits, now we can run
make test
", okay run make Gemini requests to run make ugh it's worse than a bad intern.Gemini would also frequently hallucinate APIs because I used a non-standard api for my hash table (allocate/dereference instead of get/set/update, which I find is more natural for managing ownership). And Gemini would rewrite my code style and order of operations for no reason (eg move a counter increment before updating another field).
At no point could I just point the model at a small problem unsupervised. Even "update the test suite for 100% coverage of this module, make sure the tests are as small in scope as possible" had highly mixed results.
And all models I tried would update my cmakelists and break it, and I hate dealing with cmake.
I've been told the new Gemini is good at SQL and programming, but I'm underwhelmed on both. Gemini frequently doesn't even know all the BigQuery functions, which being integrated into BigQuery Studio it should.
They're decent at code review, but a language server is still better at catching bugs.