I'd consider following MS's instructions to remove everything & then reinstall Office, but make sure that you deselect the option to install Outlook.
All of the MS installers I have used on the Mac has an advanced or 'custom install' option that allowed for each app to be (de)selected, launch the Office installer to see if that is possible, it will be a few screens in, if you want to be sure before you begin.
Maybe grab a screenshot of one of these dialogs & post it here (cmd+shift+4, then click & drag the crosshair to capture that area). It's possible calendar reminders are coming from Apple's own Calendar app?
Another option is to reinstall Outlook & try disabling the reminders…
http://office.microsoft.com/en-us/mac-outlook-help/turn-off-office-reminders-HA1 02928340.aspx
but this suggests it's 'buggy' on Mac?
http://answers.microsoft.com/en-us/mac/forum/macoffice2011-macoutlook/turn-off-o ffice-reminders-doesnt-appear-to-work/df2bf139-360a-4879-8090-6d1a8f26d6a8
If Outlook is gone from disk (empty the trash) there must be another process that is doing this, did you check the login items?