UraniumBlazer

You're ignoring the whole Job of a judge, where they put the actions and laws into a procedural, historical and social context (something which LLMs can't emulate) to reach a verdict.

LLMs would have no problem doing any of this. There's a discernible pattern in any judge's verdict. LLMs can easily pick this pattern up.

You know what's the quality of the code LLMs shit out?

LLMs in their current form are "spitting out" code in a very literal way. Actual programmers never do that. No one is smart enough to code by intuition. We write code, take a look at it, run it, see warnings/errors if any, fix them and repeat. No programmer writes code and gets it correct in the first try itself.

LLMs till now have had their hands tied behind their backs. They haven't been able to run the code by themselves at all. They haven't been able to do recursive reasoning. TILL NOW.

The new O1 model (I think) is able to do that. It'll just get better from here. Look at the sudden increase in the quality of code output. There's a very strong reason as to why I believe this as well.

I heavily use LLMs for my code. They seem to write shit code in the first pass. I give it the output, the issues with the code, semantic errors if any and so on. By the third or fourth time I get back to it, the code it writes is perfect. I have stopped needing to manually type out comments and so on. LLMs do that for me now (of course, I supervise what it writes n don't blindly trust it). Using LLMs has sped up my coding at least by 4 times (and I'm not even using a fine tuned model).

I also don't agree with your assessment. If an LLM passes a perfect law exam (a thing that doesn't really exist) and afterwards only invents laws and precedent cases, it's still useless.

There's no reason as to why it would do that. The underlying function behind verdicts/legal arguments has been the same, and will remain the same, because it's based on logic and human morals. Tackling morals is easy because LLMs have been trained on human data. Their morals are a reflection of ours. If we want to specify our morals explicitly, then we could make them law (and we already have for the ones that matter most), which makes stuff even easier.

Law = rules. Action A is legal. Action B isn't. Doing X + Y + Z constitutes action A and so on. Legal arguments have to abide by all rules of logic. Law is one of the most logic heavy fields out there.

As for LLMs not being able to reason, it's very debatable. Whether they reason or not depends upon your definition of "reasoning". Debating definitions here is useless however, as the end result speaks for itself.

If LLMs can pass certification exams for lawyers, then it means either one of two things:

1. the exams are flawed
2. LLMs are capable of practicing law.

And I believe them 100%.

The legal profession revolves around logic. Legal arguments are formed based on the rules of logic. A fine tuned model would absolutely demolish any human opponent here.

Take chess for example. There's a predefined set of rules. Chess was one of the first games where ML models beat humans.

Sure, the magnitude of complexity of rules in law is more than that in chess. The ground principle is still the same.