Ok, this modem should work fine with your AirPort Extreme base station. It's similar to the one Cox provided me when I was using their service a while back. However, depending on the exact model of your Extreme, I would suspect (like you already did) that it may need to be replaced. However, before doing so, if your Cox-provided modem is 2-3 + years old, I would suggest that you ask Cox to replace it regardless.
Cox has been rolling out IPv6 in many of their service areas, but maybe not yours just yet. Typically, an ISP would request that your configure your router for IPv6 Link-local only is so that the router does not interfere with their IPv6 service.