Where, on your system, is it generating the extra copies? I would suspect the problem may be there, or where you keep (as in your post) you keep your emails?
Perhaps you should start clean again. If it is important, make some hard copies of the email.
I have, on an external, a folder called "Mailbox". When I make a fresh install of an OS, in the "Mail" - "Advanced" tab - Maintenance, I point to this folder as the Store folder, default.
This picks up all my earlier mail and account setting automatically.
This is your choice but, with your problem, first uninstall totally Windows Live Mail. Then download a fresh installation, preferebly from the source (
Download Windows Live Mail 2011 from Official Microsoft Download Center).
I am sure, that ,if you then, as I have suggested, create a new folder somewhere on your system, and point to it, your problem will be gone. If so, you will probably have to examine previously received mail to see where the culprit is.