You'll find the best stability by physically removing the original 2GB of memory. This is because it is mismatched with your new RAM and the settings for the new RAM do not apply for that.
It would probably not be impossible to have both memory types running at the same time well, but that would call for lots of manual bios tweaking, testing and adjusting.
So with that, it is my recommendation to run only the new memory modules and forget the old. You're not going to push the memory requirements of the pc to where there would be any noticeable difference anyhow. 4GB is plenty.