just click on the pdf file, hold down the shift key and press the delete key, that will bypass the recycle bin and delete the file.

However there are programs available that can get the files back if they aren't overwritten.  Deleting removes the reference of the file from the file system table.

If you really want all trace of the file removed

http://www.heidi.ie/node/6   a free program to wipe the file from the hard drive