disable noise

Recently I got myself a new set of headphones and they sound great but a bit too great because whenever I hold down any key I hear a rapid clicking sound. The sound isn't very loud but loud enough that it has been annoying me since I got these headphones. Is there any way to disable this...