If you are in Windows Explorer view when initiating the upload you simply need to change to Small or Large Icons and a thumbnail of the image should appear. If no thumbnails are appearing at all, make sure that you have thumbnails enabled under Start -> Search -> Visual Effects