You could do it in powershell. Suppose you wanted to delete all items in a folder on your desktop named test_dir that do not have a ! in them.
You could run the following.
Foreach ($file in (Get-ChildItem -Path $env:Userprofile\Desktop\test_dir -Recurse -File -Exclude *!*)) { Remove-Item $file.FullName }