Selfhosted

So, this question is very difficult to answer. I don't want you to be discouraged though.

I can't answer you because I don't know your goals. Since we're in /c/selfhosted, I assume you're experimenting with some self-hosted setups, which is awesome! But what exactly are you hoping to do with OpenWRT? And what’s the plan for the switch? Are you aiming for better network control, VLANs, firewall rules, or are you just looking to have network area storage?

If you can share more about what you’re trying to accomplish, folks here will be much better equipped to help you figure out your next steps.

Building on the advice others gave:

1. Make a list of the precise goals you want to achieve. Even if you don't know precisely what you're trying to do, if you can describe the intent well, someone who does know can point you in the right direction.
2. Networking is not super hard, but it is not super easy, either. You should take note of every configuration change from stock, and you should optimally have an understanding of what a majority of those do. Ticking boxes at random will have results varying from "nothing happens" to "nothing happened... yet" to "the network is suddenly down" to "my switch is on but I can't even ping it anymore."
3. My advice is that routers, switches, and WiFi APs should remain as just routers, switches, and APs. I would not put services like networked storage on them, as that will significantly increase the complexity involved when you inevitably have to replace or maintenance them down the road.

Going off your response to foggy:

achieve better security through segmentation by isolating cloud-connected devices, guest devices from trusted devices.

You're describing VLANs. VLANs are something that the OWRT documentation (last I used it) was simply very shit at. I'll make the assumption you understand or are capable of learning about how VLANs work. (TLDR is that devices on different VLANs can not talk to one another without going through a router or a layer-3 switch, which I don't think OWRT handles anyway. Once you know what tagged/untagged means, then you're good to proceed.)

The way you access VLANs in modern OWRT is: Network > Interfaces > Devices (tab). From here, you may see different things depending on your hardware. In my case (I use consumer routers), I have several "network devices" which map to a physical port, and a single bridge device. From there, I can click on "configure" for the bridge device and select the "Bridge VLAN Filtering" tab to configure the vlans on the various ports.

Note that VLANs if incorrectly configured can easily make it impossible for you to access your device, requiring you reset it.

Being able to “pin” a Mac address to an IP, and being able to use internal network name resolution to reach those devices.

To my knowledge, OWRT lacks the ability to pin MACs to specific ports, at least in the web UI. It may be possible to do this manually in the configuration files, but I have never attempted to do so myself.

a blocklist for known ad-domains / malicious domains.

You generally do this on your (core) router, not the switch. (Unless your switch is doing some really funky behavior, in which case you're not here asking questions.) Most devices OWRT runs on, however, have very little flash and not much RAM. While you can probably get Pi-Hole or Adguard Home to run on them, I do it differently.

I run Adguard Home on a device separate from my router, and on the router, I have set the AGH device as the first DNS sever (OWRT: Network > DHCP and DNS > Forwards (tab)), then I enable Strict Order ("Resolv & Hosts Files" tab).****___

a high level monitoring capability to seen what devices are communicating with what domains / IPs

I would do this on the router level, not switch level. That said you can actually just follow this tutorial here https://grafana.com/blog/2021/02/09/how-i-monitor-my-openwrt-router-with-grafana-cloud-and-prometheus/

An IDS capability of some sort to be able to detect anomalies in my LAN.

This is not something I've ever attempted or done, so I'm interested in hearing what you come up with when/if you ever get there.

What should my first configurations and preparations

Write on paper your goals. Write on paper a list of your systems and what needs to speak with what.

Then pick the most important or simplest device and get it connected the way you want.

Maybe run a bandwidth speed test, and enable/set qos to 95% of that value... I found that's an easy way to kill the buffer bloat (way better latency).

should I do

Read. For example the tutorials in the openwrt wiki. I found them quite helpful.

before flashing Openwrt

Nothing... except the exact prerequisites for the flashing itself.

Your network config comes afterwards.

Be prepared for your router to not work until you figure out how to set stuff up. So hopefully this isn't your only router. I converted a router to OpenWRT and my knowledge of networking was super basic. It was very confusing but I did get it working in a few days. However, I couldn't get some things working like a VPN or updating without wiping all my settings. OpenWRT is router firmware, not NAS software. You'll need to run something like CasaOS, OpenMediaVault, TrueNAS, etc for that.

This website is a great starting point for router configurations: https://routersecurity.org/#StartHere