True, it sounds like maybe the PSU is even overheating due to a faulty or broken fan. Even if that isn't exactly what's happening, it has the symptoms because PSUs are designed so that if they're too hot, they will not turn on until they cool. You can easily tell if this is happening by looking at the fan spinning well or not, and by touching the case near the PSU area. If it was so hot that it couldn't turn on, you couldn't leave your hand on it for more than a few seconds without becoming uncomfortable.
If you have to leave the power on for an extended period of time to get it to start booting, then this signifies blown capacitors most likely in the PSU but it could also be the motherboard caps. Inspect the motherboard with a flashlight to see if there is any brown substance leaking from them. If the motherboard looks good with no signs of component failure, then sure, replace the PSU and hope for the best.