I would check for memory leaks. use performance monitor. and check the reliability monitor,
Action Center / Maintenance / View reliability history
But, with that said, I have 4G's on 7 x64. Have been using it alot, and am using 1.3G of memory sitting at the desktop now with very little open. This is normal.
Vista/7 isn't XP,,,, please please please quit comparing the 2. They are apples and oranges.
On the front of x32 or x64....... it depends on the software you use. Yes, there are still software out there that is very good, and better than most new stuff that just will not run properly in x64. So, you need to install the RC and test all your apps extensivly before you make the leap to keep it.
If I had my way, I would have not made 7 an x32 system. Everything would be forced to x64 period. We should have been there a couple years ago already. Next OS, will be x64 only, as it should be.