The real problem is that our concept of Infinite growth fails to describe the growth of use value. We only measure the growth of exchange value.

Strictly speaking, both are possible. But only one is good.

You could do the SMBC web comic joke about two computers that just trade bits of data as a form of economic activity and achieve growth capped only by the maximum processor speed of the transactions. Viola! Infinite growth! But its all exchange growth, no utility.

You could also describe the Cambrian Explosion as an "Infinite Growth" strategy. Rapid proliferation of species, new ecological niches that are filled by species specifically focused on occupying those niches, revolutionary new means of biologically observing and processing information, etc etc etc. Now we've got enormous utilitarian growth, but there's no real exchange process (unless you consider organisms eating one another a market mechanic).

From a biological perspective, the real upper limit on growth is efficient use of solar energy to process planetary materials. And we're still nowhere near that limit. Plants and planktons and insects and crustaceans have been lapping us on that front even into the modern industrial age.

Our model of economic growth just fixates on the exchange side so heavily that we end up with the Two-Computers-On-An-Island-Trading-Bits proxy for economic success. We've abandoned our conceptualization of utility growth in our quest to fully financialize. We don't recognize volunteer labor as a form of economic growth. We don't recognize procreation as a form of economic growth or mortality as a form of de-growth. We don't recognize ecological destruction as a form of economic de-growth or extinction as a massive loss in economic value.

Until we adjust our measures, the infinite growth we're aiming for is purely a growth in accounting statistics. We're sacrificing real prosperity for a financial fairy tale.