I just got a Apple Series 4 the Cellular+ GPS one

You do not need completely separate plan for your  Watch. The Watch needs to be added to your existing iPhone cellular plan. There is normally an additional charge applied to your iPhone plan when you do this. If you are having problems setting this up with your iPhone you should contact your iPhone cellular provider directly for assistance.

AT&T is telling you that your current plan will not allow you to add the Watch. If you wish to add the Watch, you need to have a different all around plan. Then you can add the Watch. The current plan you have will not work to add the Watch, you have to change your plan first, then you can add the Watch. You need to work with AT&T.

You need to contact your telephone carrier and add a cellular plan. My plan with AT&T is $10/month plus tax.

Yes, you need an additional plan for the watch. The screen you show is self-explanatory. You cannot add the watch to the current plan that you have. You will need to contact AT&T customer support to make changes. The final image you provided explains that to you quite clearly.

OR Change your cellular carrier (in the U.S. T-Mobile supports the same iPhone).

I will acknowledge that T-Mobile is good in metro areas, but if you are in someplace like Maine, or other less densely populated areas, the coverage may not be as good as AT&T. I do have T-Mobile, but I also live in a metro area (OK, the suburbs, but T-Mobile works for, except when on vacation in Maine 😀 Then it is kind-of-OK during vacation, but not something I would want if I was living in Maine.

You set it up using the Watch app on your phone, not from the AT&T Web site.