Command + Option (Mac) / Control + Alt (Win) + Shift + Delete (or Backspace) will delete a photo from a folder in the Library Module and put it in the trash while bypassing the warning dialog box asking if you want to delete the image or simply remove it from the Lightroom catalog.
Be careful, you can not use Command (Mac) / Control (Win) + Z to undo this action. If you accidently removed a photo you wanted, you will have to find the photo in the trash/recycle bin (through the operating system), and pull it out manually and place it where it belongs.

Thanks as always for these tips, Julieanne.
Note that shortcut above should be:
Command + Option (Mac) / Control + Alt (Win) + Shift + Delete (or Backspace)
Tell us how to delete without the shortcut.
You can select Photo > Delete Photo but you will always be asked if you want to remove or delete. The shortcut can be used to bypass that dialog.
Thank you for the correction Trevor. I apologize for the error!