What you describe seems abnormal, lets see if these instructions change anything…
Start by hooking the device up to your iTunes on your Mac/PC.
Sync it so you have the current state backed up. Note any errors. (cmd+shift+4, drag the crosshairs & you will get a screenshot of that part of the screen, attach it here. The screenshot will be on the Desktop).
With it still connected select the device in the iTunes sidebar (or in the 'tabs' across the main window).
Select the 'Apps' view for that device.
Deselect any apps in the list that you want to remove, or use the 'Home screens' list to delete them.
Apply the changes with the button at the bottom.
Check the apps are gone when the device finishes it's sync - please report any error messages you get here.
Now you can go to the Apps that are saved in iTunes, delete any you do not want on the Mac. They should only come back if the device still has copies installed.
Re-sync the device to see if any come back.
NOTE: You may want to disable automatic downloads of purchases in the iTunes prefs, it is also a setting on iOS, check the 'Settings > iTunes & App Store'.
To hide applications from past purchases …
In iTunes on your computer, select the iTunes Store & sign in, purchased on the right – it's under 'quick links' (you may need to authenticate again). You can then click the ✖ to 'hide these items. They will no longer show up in iTunes when you browse your purchases, however you can still 'show all' to see them, or disable the 'hide option' at a later date.
iTunes Store: Hiding and unhiding purchases