A few weeks ago I wrote about my experience migrating a HA installation from a Raspberry Pi to a NUC running proxmox. Since I can't help but to tinker, here's my experience transferring the installation to bare metal.
Reasons I had for making the switch:
- There are HAOS add-ons for pretty much every extra service I'm interested in running right now
- According to its documentation, Frigate works better when not run inside a VM (passing through the iGPU can be problematic)
- Proxmox nags me about a subscription every time I log in to the admin console
- Random crashes (that I misattributed to proxmox)
- Potentially lower idle power consumption (made no difference, it turned out)
I started the migration by making a full backup again. I installed Puppy Linux on a USB stick, booted the NUC with it, downloaded the HA image and wiped the boot drive:
dd if=hass.img of=/dev/nvme0n0 status=progress
(Fun sidenote: writing an image from RAM to a NVME drive was so ridiculously fast, the USB stick felt like a floppy disk in comparison.) After rebooting, I had a fresh HA install once again. This time, I monitored the restoration progress by periodically checking the supervisor logs on the console (ha supervisor logs command.) Running supervisor stats showed CPU usage at around 50% (one core at 100%). The restore took roughly 16 minutes to complete.
I had some connection trouble after restoring but at a quick glance, everything seemed to work after a reboot. After a closer look, I noticed most of my addons were missing. I ran a partial restore of everything except HA core, which appeared to fail due to not being able to fetch add-on images.
Before, I had proxmox seemingly crash on me a couple of times. Actually, it was losing its network connection and needed a reboot to recover. I had thought this might be a problem with proxmox but turns out it was even worse when running baremetal HAOS! Every time the network cut out, I saw this message on the console:
rtl_rxtx_empty_cond == 0 (loop 42 delay 100)
There appears to be a bug in the RTL network chip or its driver. Intel doesn't list which chipset it uses in this NUC model, probably because they're ashamed of it. In proxmox the bug was triggered maybe once a week but in HAOS it was more like once every few minutes. Going back to proxmox wouldn't be an acceptable fix because I still couldn't trust the server to remain online.
I worked around the problem by running to a local computer shop and getting a USB ethernet adapter. Hopefully, a future kernel update will fix the issue and I no longer need to use it, but for now the USB adapter (with an AX88179 chip) has been working perfectly. After fixing the network problem, partial restore worked and all addons were reinstalled.
Finally, I wanted to add a second interface for my IoT VLAN. This was easy in proxmox, as I could simply add a second virtual adapter, but it can be done in plain HAOS just as easily. This feature doesn't seem to be mentioned in the documentation anywhere, but the ha command line tool can configure VLANs for you:
ha network vlan enp0s20f0u1 200 --ipv4-method auto --ipv6-method auto
This adds a new virtual interface to the physical interface enp0s20f0u1 for VLAN tag 200. (This can also be done using NetworkManager.)
Having HA on two subnets simultaneously has worked well so far. Traffic to my IoT devices no longer needs to go through the router and, in the future, setting up Matter devices on the IoT subnet ought to be possible as (to my understanding) they utilize link local IPv6 addresses.
Lastly, I got a PoE camera and added Frigate. Configuring it was bit of a chore and the documentation feels a bit fragmented, but I did get it working in a couple of hours. Some relevant notes:
- OpenVINO detector seems to work well enough on the NUC. I currently have just one camera and feel no need to get a Coral accelerator
- VAAPI acceleration for ffmpeg requires protected mode to be disabled ("full access" version of the add-on needed)
- I used go2rtc to restream the detect stream, since that stream is also good for live view. It can be viewed from Home Assistant's UI, even through nabu casa.
- Frigate-card supports casting locally! (figured out how this works:
media_player.play_mediaaction, content id ismedia-source://camera/CAMERA_ENTITY_ID, and content type isapplication/vnd.apple.mpegurl) - Having frigate continuously running doesn't have any measurable effect on the NUC's power consumption. Maybe there's something wrong with my power settings and it wasn't idling as much as it should have?
- Using Frigate's person detection as an occupancy sensor works really well. This might actually replace a PIR sensor once I move the camera to its final location.
In the end, was it worth it moving from proxmox to bare metal? Maybe. One less moving part to worry about at least. It did not solve the random (network) crash issue, but I did figure out the root cause. There was no change in power consumption, the NUC still draws 10 watts (with or without Frigate running.) If I come up with something that needs to run in a VM I might go back, but I'm also planning on building a NAS in the near future which I could also use for running VMs and containers.
One problem still remains: the power button does nothing! I think hassos is missing acpid or its configuration. This is not a showstopper, but it would be nice to be able to reboot the system gracefully when/if it loses networking.
The problem, I believe, is that stable diffusion presently only supports Python 3.10, but Arch ships 3.12, and some of the dependencies aren't compatible with the newer version. Here's what I did to get it working on Arch + AMD 7800XT GPU.
python3.10 -m venv venv(in stable diffusion root directory)This should be enough for the dependencies to install correctly. To get GPU acceleration to work, I also had to add this environment variable:
HSA_OVERRIDE_GFX_VERSION=11.0.0(Not sure if this is needed or if the value is same for 7900 XTX)