If I read your post as written. You are selecting a picture with Windows explorer and then 1. It is requesting a program to open it with , or 2. You are right clcicking and selecting Live mail to open it with?
What is happening is that everytime you do this, you are changing the default opening programs (imo). If this is the case, you have to put them back to defaults. Control Panel - Default programs - Set your default programs. Select "Windows Live Mail" "Set as Default"
Now, before going further as I have no idea what program you have installed, do this. Right click the picture of your choice and then "Open with" browse to your photo or graphic program, if you cannot see one, otherwise select "Paint" if visible on the list. and make sure you have the small box, bottom left, Always open..etc. ticked.
Now you can open Windows Live Mail, and hopefully the message has gone. Select you new mail (Email message - top left) and "Attach File.". Browse to your picture and select it - done.