The PSU does not directly supply power, in terms of voltage regulation, to the CPU, but it does supply power to other modular components connected to the motherboard such as the GPU rails and SATA drives. A component can not reasonably be damaged from being undervolted. Several things in an under-voltage scenario would happen here:
1. If the PSU supplies a lack of power to the motherboard, which regulates the voltage for the CPU (and in many cases the CPU is also self-regulating its own voltage), the system will blue screen, fail to start, go into endless reboot cycles, and show other signs of problems.
2. If the PSU supplies a lack of power to the video graphics card or the SATA drives these devices will fail to operate properly, but under no circumstances would they overheat.
A lack of proper voltage does not create an overheating scenario. If there is not enough power it will just malfunction and turn off. The only scenario where you can create physical damage this way is from going over the voltage or the PSU/CPU/motherboard/and other components losing their capability to regulate the voltage.
If a CPU is undervolted it will hang it will not damage the processor or any other components. Constantly restarting the system over and over again in a boot loop could damage components, but this would have to go on for days. If anything the PSU can overheat from undervolting the other components in extreme circumstances.
My suggestion is that the CPU is overheating not being under too low of a voltage.