Hi. I have had presided problem after installing, even win7. Then I hit win10 and it was remain after new installation of windows.
What was. I have 5.1 speakers. They was connected in 5.1 mode. Every time while rebooting or turning off it was that ugly peachy sound.
Solution was after long two years(not drivers issue) I have found. I reconnected 5.1 to 7.1 and that was it. My sound system is 10 years old electronics. Maybe cos of that.
You could try tracking it down by uninstalling all your third party applications such as media players or pdf readers and just leave windows as well as it's drivers installed. A process of elimination.
If the sound does stop then slowly start adding apps back one at a time to see if the sound returns. If your sure things are ok then install the next thing and so on..