Are you talking about a folder on the "Dock"? If so, then that is just an Alias to a real directory. So removing the folder alias displaying as a stack from the dock just deletes the alias.
If you wish to remove a single item from the stack, then you remove that item from the actual folder. The stack should have an "Open in Finder" button, which should take you to the real folder.
You could also create a new folder, then populate that folder with Aliases to all the apps or files you wish to have in a stack, then drag the folder to the dock, then option-click on the folder in the stack and make it a stack.
DO NOT move the actual applications, folders, or files to a collection folder, just create aliases for the apps, folders, files you want in that collection stack. Command-Option-Drag an app, folder, file to the collection folder will create an alias, and not move the item itself.