You can.
The App Store Updates page only informs you of updates available for the OS you are running.
If you want to or are able to install Big Sur, open Safari and click on the link below.
How to upgrade to macOS Big Sur - Apple Support
If you are considering upgrading to Catalina or Big Sur be warned they do not support 32bit apps.
Your Mac is probably running 32bit apps and these will need to be updated to 64bit apps,
some updates may be free, some may be paid. Use this app to discover which apps you have
are 32bit, https://www.stclairsoft.com/Go64/ then contact the developers of the software to see
if they are offering 64bit updates.