A child may hallucinate, lie, misunderstand, etc, but we wouldn't say the foundations of a complete adult are not there, and we wouldn't assess the child as not conscious. I'm not saying that LLMs are conscious because they say so (they can be made to say anything), but rather that it's difficult to be confident that humans possess some special spice of consciousness that LLMs do not, because we can also be convinced to say anything.
LLMs can reason (somewhat unreliably) with a fraction of a human brains compute power while running on hardware that was made for graphics processing. Maybe they are conscious, but only in some pathetically small way, which will only become evident when they scale up, like a child.
Really? I mean, it's melodramatic, but if you went throughout time and asked writers and intellectuals if a machine could write poetry, solve mathmatical equations, and radicalize people effectively t enough to cause a minor mental health crisis, I think they'd be pretty surprised.
LLMs do expose something about intelligence, which is that much of what we recognize as intelligence and reason can be distilled from sufficiently large quantities of natural language. Not perfectly, but isn't it just the slightest bit revealing?