You have more than one issue at hand here. As far as the Firesticks, those are not Apple products, so those would have to be handled separately, however I suspect something similar is causing the issue. As far as what your Apple TV provides, since you are seeing YouTube, it would appear that you are connected to the internet.
What happens when you attempt to use other options of the Apple TV? Have you verified your setup? Set up your Apple TV - Apple Support
Do you have any other internet problems with other devices such as computers? Have you checked with Amazon support for questions regarding the Firesticks? Considering you have connection issues with more than one type of device, it sounds like Comcast should have more information for you.