Please reset the TC to factory before you start. No files are deleted on the hard disk if you previously used it for backups.
You hold in reset button, then power on TC without releasing the reset for about 10sec. The front LED will flash rapidly.
Wait until it is finished rebooting back to defaults. Plug the WAN port on TC into the Comcast router LAN port. (Wan port is marked with sunburst symbol) Then open the airport utility in your MBA, click on the Other WiFi Devices (1) click on the newly discovered Time Capsule and run through the setup. From there it is pretty much automatic.

Let me recommend you change the recommended Apple names to short names <8 characters, no spaces, pure alphanumerics. Passwords same rules but 8-20 characters.
Decide if you prefer the wireless network of your comcast or the TC. It will depend on a number of factors but either can be used.
Once completed you should be able to access the TC from Finder as well as from Time Machine.
If you run into trouble give me a screenshot of exactly where you have issues.
NOTE.. Catalina is not great with Time Capsule. Give it a try but if you run into problems it is par for the course.
We generally recommend using an alternative backup software like Carbon Copy Cloner.
Also note the TC can disappear after a day or two. Again this is standard issue.. and there are a few changes you can make to help it work more reliably but see how you go getting to that point.