Can I delete the "Public" folder?

You shouldn't delete it, it's there so files can be transfered between user accounts on the same machine.

If you look inside it, you'll see Drop Box, which is where you drop files from one user account to another and it changes permissions to be used by the other account when logged into it.

This is useful if your having problems in one account, you can transfer just your files into the other account and delete/recreate the problematic account.

The Public folder isn't publicly viewed online unless it's published or made part of a network share, which is disabled by default.

Public folder is used by File Sharing to share files with other computers connected to the same network as your MacBook Pro.

If you delete it and you want to share files with other computers in the future, you will have to choose a new shared folder on System Preferences > Sharing > File Sharing. There isn't any reason to delete it, but it shouldn't damage Mac OS X