App not available in my Apple TV App Store

The TV and the iPhone are different devices and require that the developer create different apps for each device. iPhone apps don't work for the TV and vice versa. Beyond that, you would need to ask the church's app developer(s) why there are differences if both apps are indeed for that church.

Which app? Why do you believe that you should be able to use the app with your TV?

M1 Mac mini, macOS 14.1.2;  Watch, series 7 GPS/cellular, watchOS 10.1.1; iPad Pro 2nd Gen, iPadOS 17.1.2; iPhone 12 Pro, iOS 17.1.2; TV 4th Gen, tvOS 17; HomePods, audioOS 17.1.1 and Airport Express w/ Airplay 2