If removing the Notification from Safari in Safari > Settings > Websites > Notifications, does not completely resolve the issue, then I would suggest downloading MalwareBytes. I recommend just the free one time scan of the system and it will remove the Malware. I do not recommend any software that monitors the system for Malware/Anti-Virus and runs constantly in the background and that includes MalwareBytes.
The "ask you" Malware was most likely installed by a fake "Flash Player Update" message the looked legitimate. Be wary of any update or install messages that come up while browsing the web.