Access Mac Mini Remotely

How are you trying to connect from the remote computer? Your private local network ip address is not routable from an external network.

I have been able to do this successfully, using Back to My Mac to sign into my home Mac from a remote location on my MacBook Pro.

You say you try connecting to the Shared section in the Finder sidebar, which means you are successfully seeing the name of your Mac when you're away from home. But what happens after you select the name of the Mac? Does nothing come up?

You might want to review the Apple tech notes on this, especially the second one, and make sure you've done everything it says:

Set up and use Back to My Mac - Apple Support

Get help using Back to My Mac - Apple Support

The main point is that both Macs have to be logged into the same iCloud account, that is how it can get past the router firewalls. But some router configurations can still prevent connection or slow it way down, like not having UPnP or NAT-PMP enabled. I don't know if your type of modem also might have its own firewall blocking ports, my modem is a different model.

Which exact model of Arris Surfboard do you have? Is it only a modem or is it also acting as a router?