if i purchase the subscription will this be accessible across all devices logged in with the same icloud account?
In general, most (but not all) streaming video services on iPhone will have a matching app on the Apple TV box, for which the same subscription can be used. You don’t have to uninstall them on iPhone, but you may do so.
… with the same icloud account
Apple makes a distinction between your Apple ID for Store services (including in-app subscriptions) and your Apple ID for iCloud services (e.g. iCloud Photos access, HomeKit). This is unrelated to whether your Apple ID uses the ‘@icloud.com’ domain for a name. For most users, the Apple ID for both will be the same. But (mostly historical) those could use different accounts. And one might be signed in for one, but not for the other at any given time: separate sign-ins, and separate sign-outs.
The access to a third party video streaming service may be through the content provider directly (with separate login credentials), or started as in-app subscription through your Apple ID account (for Store services). For the latter, the app on a secondary device may require to use the “Restore Purchases” option, instead of logging in with an Apple ID account.
… accessible across all devices
Many content providers do not limit the number of devices per se, but do limit the number of simultaneous streams on one account/subscription. How much, that differs per content provider. E.g. Disney+ allows 10 devices with a signed-in app, and 4 simultaneous streams (e.g. 4 family members on the same household watching something different from Disney+). For Max, it depends on your plan: 2 or 4 simultaneous streams. Netflix: 1, 2, or 4 simultaneous streams, depending on plan. Apple TV+: 6 simultaneous streams per subscription (intended to accommodate Family Sharing groups of up to 6 users).