Windows 10 Mic input comes out of speakers

I just got a microphone for my computer, and the input it receives sometimes comes out of the speakers I have. Both are free standing, i.e. they're not part of a headset or anything.

I have checked for the "listen to this device" feature, and that's turned off.

This happens when I open Recording Devices, and when I open the build in Voice Recorder app.

Any suggestions on how to stop this?