You have downloaded and tried to install updates to macOS High Sierra probably from the Apple Support Downloads
site these updates can only be installed on a mac that is already running a previous version of macOS High Sierra.
Installing a newer OS is called an upgrade.
Which OS you can upgrade to will be governed by which model and year your mac is.
Click on the top left of your screen click on About This Mac to find out the year of your mac.
For macOS High Sierra,
Open Safari and click on this link,
How to get old versions of macOS – Apple Support
Go to the Download OS section and click on macOS High Sierra
Before upgrading make a full back up of your Mac.
Check that third party apps you use have updates available for the new OS.
https://roaringapps.com
Check that your printers/ scanners etc have updated drivers and software available for the new OS.
Connect your mac to your router via cable rather than rely on WiFi.
Do not use your mac while it is downloading
and do not use any other devices that share the same internet connection.
If the App Store says ‘Open’ then it is detecting a previous download of the install app
go to the Applications folder locate Install macOS macOS High Sierra.app.
Move it to the Trash, empty the Trash.
Restart, click on the link again, the App Store should say ‘Get' or 'Download'.