For those wondering, it also works with a Linux VM:
- Host: AMD Ryzen 9 3900X + Proxmox
- PCI passthrough for an Nvidia RTX 3060 12GB
- A Debian VM with 16GB and as many cores as the host have (if you set less cores, you will have to tune cpu affinity/pinning)
- An HDMI dummy
- I stream the VM to my office using Sunshine and Moonlight
It's not easy to set up, but it works. I'm able to run some games like Borderlands 3 running at ~50FPS with a resolution of 1920x1080 with visual effects set to the max (important: disable vsync in the games !).
Only problem is disk access. It tends to add some latency. So with games coded with their ass (ex: Raft), the framerate drops a lot (Raft goes down to 20FPS sometimes).