Always helpful if you include some information about the mac you have, what model and what year it is will decide which OS's you can upgrade to. You haven't included which app it is you are trying to install, which version or what OS it is asking for.
Here is a general answer.
The App Store Updates page only informs you of updates available for the OS you are currently running.
As you are running an older system then you are probably up to date on that system.
If you want to install a newer OS that is called an Upgrade.
The model and year of your Mac will determine which macOS you can upgrade to,
read Section 1 in each of the How To's.... below.
Please use Safari to access the following links do not use Firefox
How to upgrade to macOS Mojave - Apple Support
How to upgrade to macOS High Sierra - Apple Support
How to upgrade to OS X El Capitan - Apple Support
Before upgrading make a full back up of your Mac.
Check that third party apps you use have updates available for the new OS.
https://roaringapps.com
Check that your printers/ scanners etc have updated drivers and software available for the new OS.
Connect your mac to your router via cable rather than rely on WiFi.
Do not use your mac while it is downloading
and do not use any other devices that share the same internet connection.
If the App Store says ‘Open’ then it is detecting a previous download of the install app
go to the Applications folder locate Install macOS ******.app.
Move it to the Trash, empty the Trash.
Restart, click on the link again, the App Store should say ‘Get' or 'Download'.
If you are considering upgrading to Catalina be warned that it does not support 32bit apps.
Your Mac is probably running 32bit apps and these will need to be updated to 64bit apps,
some updates may be free, some may be paid. Use this app to discover which apps you have
are 32bit, https://www.stclairsoft.com/Go64/. Once the list has been created you can then
contact the developers of the apps to see if they provide 64bit updates.
How to upgrade to macOS Catalina – Apple Support