If you click on an icon in the Finder, and then press ⌘-Delete, the item will fly into the Trash. It’s still retrievable, same as anything else you put into the Trash. It just gets there a really easy way.
Clicking on an icon and then pressing the Delete key by itself does nothing. You have to add the Command key.
If you look in the Finder’s File menu you’ll see the shortcut.
UPDATE: I wrote about this in April 2015. Oopsy. I’ll make up for it.