Windows 7 Same BSOD on multiple identical laptops

[Francis V]

Greetings BSOD Gurus!

I have one that's stumping me, and even though I tried I haven't been able to figure it out on my own.

Quick backstory:

These laptops are identical to each other, right now I'm looking at 5 or 6 of them doing the same thing, seemingly at random throwing a BSOD 0x00000124. Since they do seem to be doing the same thing, I can't imagine it's a hardware failure, probably just a driver, I just don't have enough experience to figure out which (or if I'm wrong in that assumption, what else it would be).

Although you can probably tell all this from the attachments, the basic laptop info is:

Specs
HP ProBook G3
i7-6500U
A single stick of 8 GB RAM (the OEM RAM that came with the laptop)
500 GB Samsung 850 EVO (used this to replace the stock WD Blue)

1. When I got the laptops, replaced the WD Blue with a 500 GB Samsung 850 EVO
2. Loaded Win 7 Pro on it from HP OS Install CD
3. Installed all drivers for that model from HPs website
4. Ran Windows Update to current
5. Installed a number of programs (I can provide a list of all if necessary)
6. Joined to a domain

From there the BSODs seem pretty random, I've seen them when typing in notepad, while browsing the web, watching a video on YouTube, while its sitting idle with the screen saver on, overnight when no one's around, etc.

I'm just starting to get familiar with WinDBG, so that hasn't told me much yet, so all I've done so far is try to find solutions online as well as used BlueScreenView, and at least in that all the crashes look identical to me across all the laptops.

I hope that's good info to get started, please let me know if I can provide anything else. The attachments are from one of the laptops that I picked as the guinea pig to work on, it's been powered off for a few weeks as it was deemed unusable due to the crashes and is now sitting on my desk as of today.

I'll keep playing with it, but any pointers would be super appreciated! This is driving me insane!

Thanks!!!

 

[kemical]

*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck 124, {4, fffffa80066fe8d8, 0, 0}

Probably caused by : GenuineIntel

Followup:     MachineOwner

Hi,
all your dump files are like the above Bugcheck 124. This usually caused by a hardware error typically heat or overheating. However it can also be tripped by other issues such as software.
Have you checked your temperatures? This application is excellent:
HWiNFO - Hardware Information, Analysis and Monitoring Tools
Now each Bugcheck usually has a sub-type which can give more information on the actual issue. If you notice in the brackets above the number 4, this is the sub-type. Sub-type 4 means an un-correctable PCI Express error occurred.

Try running the diagnostic application supplied by HP for your machine:
Link Removed

If that doesn't come up with much then test your CPU using Intel processor diagnostic:
Link Removed

When you installed your Windows 7 Pro chipset drivers, did you install both versions? You'll see that they are separated by a USB 3 driver but basically you need both. (if you haven't already got them of course.)

Your Bios also appears to be old but if your unsure about the process of updating it then leave well alone.
BiosVendor = HP
BiosVersion = N78 Ver. 01.10
BiosReleaseDate = 04/10/2016
SystemManufacturer = HP
SystemProductName = HP ProBook 450 G3

Current version stands at 1.14
Link Removed

If after making the above changes the bsod still occurs try running the driver verifier. This will stress the drivers on boot up to see which if any blue screens. If a bsod occurs then the culprit is written in the dump file. This guide will show you how to go on:
Link Removed

If the verifier should cause a bsod boot loop then boot into safe mode and turn it off.

Post any new dump files.

 

[Francis V]

Awesome, thanks for the tips! I won't have time to go through all the steps above till a bit later in the day today, but I'll go through all and then I'll report back!

This is happening on at least 5 identical laptops that are brand new, with only the SSDs having been added, so if it's problem hardware that's causing the issue, that would be unfortunate.

 

[kemical]

so if it's problem hardware that's causing the issue, that would be unfortunate.

Indeed.. Good luck with the tests..

 

[Francis V]

Update so far.

Checked temps, they looked fine. I'll upload a screenshot of them once the HP Diagnostics are done running, I left them on the laptop's desktop. But CPU was idling at around 39/40C, and went up to 58 to 62C during a stress test.

The Intel CPU Diagnostics passed.

I'm now doing the Extensive Systems Test from HP which has 5 hours 34 minutes left. There's also a Component Diagnostics process I'll run once that's done. Yay, probably be going on till tomorrow.

No BSOD's so far since it's been on my desk, so I definitely haven't found anything that triggers it.

I'll check back in when the diagnostics are done, if nothing turns up that'll just leave me to run the verifier.

 

[kemical]

Thanks for the update Francis. Hope all goes well.

 

[BIGBEARJEDI]

Hi Francis,
Interesting thread you have here. In your post #5 you mention that you did a Stress Test. What software did you use to conduct the Stress Test with? There are Stress Tests and there are Stress Tests. We use several different ones; but when you are trying to determine whether a computer has been thermally damaged (Cooked!) and it overheats intermittently as in your case, you can't reproduce the BSOD again on one of the laptops you randomly picked out of the lot.

We suggest that you use the Passmark BurnInTest software that is free from here: PassMark BurnInTest software - PC Reliability and Load Testing.
The key to using this software however, is to really stress it, you'll have to heat up the ambient room temperature where you are conducting the test on the laptop. If this is at your workplace you may not have access to Thermostat control for your workspace room, so you should consider taking it home if you can or can get permission to do so. You'll need to run the Passmark BurnInTest for at least 24 hours continuous time on. That means that you'll have to go into the laptops Power Settings and disable all auto-sleep, auto-hibernate, all battery-saving settings. Make certain, that all those options are set to happen with the "never" setting, otherwise windows will disable the laptops hard drive and other internal components, and the stress test won't work as well. When I test PCs or laptops here in my Home office, I turn my thermostat up to 87 degrees F which is hot as it goes. This is no problem in the winter when we have snow on the ground, but in the summer when I do these and it's 90 degrees plus outside it's very uncomfortable. Especially for 24 hours as I mention. It's even better if you can leave it going for 48 hours. I don't have pets but if you do, this may not be possible as it may injure them so you'll have to find another way. You can use another free program to monitor your CPU core temps during the 24 hour Stress Test period such as REALTEMP from TechPowerUp we also like you can get for free from here: TechPowerUp.

This will allow you to watch your laptop CPU core temps heat up. What you are looking for is if REALTEMP reports temps in excess of 72 deg C or 160 deg F. If you can heat up that test laptop to about 73-75 deg C you are above the rated temp of most CPU processor chips, and if you don't generate a BSOD during that period, the CPU chip and other Motherboard components probably are not thermally damaged and the BSODs are coming from corrupted or out-of-date drivers, or apps that got installed incorrectly, or a virus infection etc. I've worked for a number of computer and server manufacturers and we have special Industrial Temp Controlled Ovens the size of a Hotel room to do this in. You'll need to be careful and be physically there when the laptop gets close to that max temp of 72 deg C. and you need to shut down the test and take the laptop outside of your house or apartment (assuming the outside temp is less than your inside temp!). If it's 108 where you live, you need to get that laptop cooled down in a hurry--I will often put laptops into my fridge for 15 min. or so to get them back under the Max so I don't overcook them! This works great. Just make sure not to leave it in there if your roommate or family member throws a pizza box on top of the laptop and you leave it in there overnight by mistake! I did this once, and interestingly enough when I took it out of the fridge the next morning and turned it on--it still worked! I think it was a Dell.

Running a true Stress Test program such as the BurnInTest along with having the heat turned wayyyy up in your house or apartment will definitely get that laptop to BSOD if there is anything hinky in the components from being thermally damaged. Of course, if it turns out to be a driver or registry issue that's correctlable you wouldn't need to do this to the other laptops. Chances are however, if the BSODs come back with a vengeance on during the 24 hour Stress Test period, and do not come back after you bring the laptop back down to normal temps, you could have an overtemp situation with all the laptops, and you'd have to take the rest of them all home with you and repeat this test with all of them to see.

Hope this proves useful to you. kemical will continue helping you with software repair attempts.
Best of luck,
<<<BIGBEARJEDI>>>

 

[kemical]

Hope this proves useful to you. kemical will continue helping you with software repair attempts.

Well to be honest the apps you describe aren't really the best and I wouldn't advise using them.

I'm also unsure who 'we' is?

 

[Francis V]

Hello again,

BIGBEARJEDI - I used nothing that fancy, I transcoded video for 20 minutes which held the CPU at 95 to 100%. I definitely can temp stress the CPU further in the future, but the idea was just to give it a short workout and see how high the temps go under short heavy usage, since nothing the laptops are currently used for actually ever drive up the CPU usage this high. I'll keep your heavy duty stress test in mind for if all else fails.

I let the HP Diagnostics run on "Loop until crash" over the weekend, and it went through 14 full passes of all components with no problems.

I'm going to save the BIOS Update for last, since if that fixes the issue and I have to do that on all the laptops then I want to make sure that was it for sure and nothing else worked first.

Driver Verifier is running now and I'll let that run for a day or two, let's see what happens.

I'm also loading Win 10 onto one of the laptops out of curiosity, if it's drivers then that one will (should?) have no issues (unless HPs Win 10 driver pack also has a problem ).

 

[BIGBEARJEDI]

Sounds good. Thanks for your response back. Glad to try and help. kemical doesn't approve of the apps I'm using which I discovered from other forums over the years and used to help me track down thermal problems on PCs and laptops as described. Therefore, "we" can't recommend them, but "I" do, so that's up to you to use them. For me, the last thing I wanted to do when I was in IT was to deploy computers to employees that will randomly crash under temperature or operational stress as that sort of negligence in the business venue can get you fired.

Using these procedures may and has killed marginal computers that belong to my Clients. In some cases involving laptops, my Clients chose not to replace them but let me buy them laptop cooler pads to keep the temps down below the 72 deg Max to give them a few more months of service before the crashes got too bad and they could put off buying a replacement laptop. But, it you have several laptops and they were all bought at the same time from the same reseller you could be experiencing what we call a "lot failure". You should gather up all the serial numbers of the identical laptop models you have and see if the numbers are all sequential; for example 12340093, 12340094, 1230095, 1230096, etc. If they are, best practice would dictate that you use a more comprehensive stress test and you have to run it on each and every laptop, not just the one that seems to be failing occasionally. If you do that and document your results, you can go back to the reseller and use the "lemon law" (if you live in California or a state that has a similar law) and get those machines replaced. Make sure your documentation is copious and accurate. I had to do this back in 1999 when I got a batch of bad Toshiba laptops that were bad due to manufacturing defects; and Toshiba replaced them all for me.

These are your laptops, or your company laptops (and your Rep on the line), so it's up to you how to handle it. After 30 years or so in IT shops, that's what I would do.

Best of luck to you,
<<<BBJ>>>

 

[kemical]

Glad to try and help. kemical doesn't approve of the apps I'm using which I discovered from other forums over the years and used to help me track down thermal problems on PCs and laptops as described. Therefore, "we" can't recommend them, but "I" do

Why you had to write a huge diatribe about stressing the machine is beyond me especially if one reads the thread through properly..

 

[BIGBEARJEDI]

Mmmm. Guess I was trying to make sure the OP covered all his bases, as it sounded like a business deployment situation, possibly IT though he never told us what his end-user situation was. For a home user situation, I agree this was an overkill post. But, on the off chance it was a business or IT deployment situation I felt he wasn't stressing the laptop hard enough to force BSOD to occur within the short time frame he tested. Also, he was doing "marginal testing". Companies that did this often went bankrupt since they didn't know how to properly stress-test computer-electronics. Examples are AST Research and Compaq. I worked at companies where we experienced DOA rates on their computers and laptops up to 40%!!! That's because whatever they were doing in their QC or Manufacturing was not adequate to catch these unacceptable high rates of failures. Since you thought it was inappropriate to warn the OP that his testing methods were not adequate in my opinion, can we pull this post from the thread? I know the OP read it, but it would remove the whole thing from other folks who happen upon reading it. I'm Ok if you wish to do this. --BBJ

 

[Francis V]

Alright, back again!

So the verifier's been running for a few days now, but didn't cause any blue screens. In addition it hasn't BSOD'd once since sitting on my desk, but I haven't done anything I'm aware of that would have corrected any issues, so I'm not really sure where to go from here.

The laptop I loaded Win 10 on has been happily working away with no problems, with the same usage that was causing 7 to BSOD, so I'm still relatively sure it's just a Win 7 driver issue, just don't know how to narrow that down further.

Any other ideas? Upgrading them all to Win 10 is really not a popular option at the moment.

 

[kemical]

Hi Francis,
please disable the verifier as anything it found now would most likely be a false positive. If nothing was found straight away then test for 24hrs, which you have more than covered.

Bugcheck 124 is quite a fickle bsod, I've seen all sorts cause it, even an out of date browser although it was probably more to do with flash player. Anyhoo the chances are that whatever it was has been updated already?
Best thing to do is simply monitor the situation and as long as they (the lappies) are running error free, leave it be. No point fixing things which ain't broken...

Of course if a blue screen occurs please post the dump.