Ethernet splitter?

What you will need is a network that provides the capability to use VLANs and what VikingOSX is eluding to. Network clients can be separated on each VLAN as required for isolation between VLANs. Using an Ethernet switch, by itself, may not solve this, regardless of where you place it.

You would need a router (and potentially managed Ethernet switches) that support VLANs to accomplish what your networking goals are. So, to start with you will need to determine if your Linksys Velop system supports VLANs. I know a number of Linksys product do, but not sure about their Velop mesh system.

Reroute the ethernet wiring so that the modem, iMac, and Linksys all connect to the ethernet switch instead of having the modem and Linksys connect directly to each other.

Read your Linksys documentation about setting up a sub-net strictly for your iMac. That will isolate your iMac on the network from all the other network devices (except the router and the switch) on the default network. With that subnet, your router will be the .1 node, and your iMac can either receive IP addresses via DHCP, or can actually have a static address known to the router greater than that .1 node address.

I am not using a mesh network here, nor a Linksys edge device. However, I have two distinct subnets in the house that are isolated from one another, which functionally restricts access to computers, NAS servers, and printer on the main subnet. The same Ethernet cabling exists as prior to the subnetting.

The key is not to have the Ethernet switch between the modem and all of your other devices. Your "main" Velop router should be directly connected to the modem, and then, you can elect to use one or more switches downstream of the router as your networking goals require. The modem only expects a single device to be connected to it to work properly.

To separate your iMac from the Velop system, the Velop router must provide VLAN functionality. You would then create a minimum of two VLANs: one for the Velop wireless network to support your alarm system, and one for your iMac. Again, I do know that certain Linksys hardware does support VLANs. I just don't know for sure if their Velop system does.