I recommend at least 8GB of space for the install in virtualbox.
The Ubuntu system itself needs at least 4GB of space to handle all of its parts such as the GUI and other things (Unlike windows the GUI is not tied into the OS so keep that in mind)
If you have enough hard disk space I would not bother with installing a virtual drive onto a USB thumb drive, especially if you only have a 4 gig stick.
How much room do you have on your HDD?
Really a virtual hard disk drive does not eat up that much room unless you have one of those small capacity sold state drives or a very old HDD.
It will not take up any more room then you want it to for the virtual HDD, and it can be easily removed if you honestly do feel the space is too tight.
If that...