Although the Recycle Bin appears on all disks, the actual contents are always stored on the same disk where the file was originally deleted from. If you plug in an external HDD and delete a file from your desktop, then remove the external HDD, it will still be there. If you delete a file from the external HDD, you won't have access after removing it. The Recycle Bin properties is confusing in that it appears to have a choice where to place it, but that's actually to tab between settings for each disk's Recycle Bin, not where all recyclables are placed.
It's not the Recycle Bin that prevents removal of external HDDs. It is the Microsoft Windows Search Indexer, which prevents external storage from being safely removed (feedback at...