Server 2016 Unresponsive - Windows Installer

[DuncIT]

I have a Server 2016 (Standard) that I use to run Remote Desktop Services for 7 users. The box has two Xeon E5-2690 (2.9 Ghz) cpus, 256 Gb, and two 2 TB SSD drives. The box had been working flawlessly for ages. Within the last two weeks I've had to resort to restarting it every day. Throughout the day if I watch the Task Manager I see the CPU usage creep up until it is 100%, and then it becomes totally unresponsive. None of the users are running any additional tasks - they have / use Edge, Thunderbird, Calc, Adobe Reader, and Open Office (infrequently). Looking at the processes, though, I see more and more "Windows Installer" threads opening all day long. I have no idea what is spawning msiexec.exe, or how to curtail it. If anyone has any idea what's going on I'm desperate for help.

Thanks for the help,

Dunc

 

[Neemobeer]

Look at the event log and see what is spawning those installers.

 

[ussnorway]

Is Adobe creative cloud installed on the workstations... That would be making copies of the individual library files and sending to the C drive

Sent from my SP9 using Windows Forums mobile app

 

[DuncIT]

Look at the event log and see what is spawning those installers.

I have searched the event logs, and I can't find the source. I restarted the box last night at 2:00 am and it was running at 10% CPU utilization with 1009 processes running. It is currently (10 hours later, with no additional users logged on) running at 100% CPU with 8627 processes running.

 

[DuncIT]

Is Adobe creative cloud installed on the workstations... That would be making copies of the individual library files and sending to the C drive

Sent from my SP9 using Windows Forums mobile app

No, we don't use the Adobe creative cloud. The only Adobe product installed is Reader, and that wasn't running this morning between 2:00 and 11:00 am

 

[Neemobeer]

The windows installer service logs quite a bit of data to the Application log. I would image there is record of these installers. I would also look at the schedule tasks for something reoccurring. You can also use procexp to look at the details of the processes which would give you clues as to what they are.

 

[DuncIT]

The windows installer service logs quite a bit of data to the Application log. I would image there is record of these installers. I would also look at the schedule tasks for something reoccurring. You can also use procexp to look at the details of the processes which would give you clues as to what they are.

There are, what appear to be, 1000's of msiexec log files, each of them is exactly 2bytes and empty.

 

[DuncIT]

As I said, previously, there are 1000's of empty log files - it appears that MSIEXEC is creating a log entry anywhere from 7 to 16 times / minute. Occasionally, it creates a different log entry similar to this:

MSI (c) (E4:E0) [11:55:48:481]: Failed to connect to server. Error: 0x80070008
1: 2774 2: 0x80070008

DNS for the box is set to 8.8.8.8, and 8.8.4.4, both of which are pingable. I've cleared the DNS cache and restarted the service. I've also restarted the BITS and Windows Installer services. I've also done multiple Windows Defender scans (quick and full), just incase - they have not turned up anything.

Does anyone have any suggestions?

 

[Neemobeer]

How is software deployed in the environment? GPO, some packaging system like SCCM or Altaris?


Also you didn't run through my other suggestion from my last post. Get procexp and look at the running windows installer processes, specifically the commandline column they may give you an indicator to what the process is.

 

[DuncIT]

Also you didn't run through my other suggestion from my last post. Get procexp and look at the running windows installer processes, specifically the commandline column they may give you an indicator to what the process is.

I ran procexp64, as you suggested, and it identified the problem! The processes were being spawned by the Synology Active Backup client that was running on the box. I have no idea why that app has suddenly changed / failed, but we will obviously be examining new backup solutions. Thank you so much for your help / insight. Procexp now has a permanent spot in my tool chest!

 

[Neemobeer]

Yeah it's handy as are many of the sysinternals tools.

I'd also give this a read worth the $20
Link Removed