Debugging Windows 11 crashes

Once to twice a week, I have been finding my system asleep when it should have been folding. After looking at "Event Viewer" after each of these I noticed a trend of a Kernel-Power event proceeded (but not immediately) by a volmgr one.

To view the events open up "Event Viewer", expand "Windows Logs", and click on "System"

Simplified Example:

Level	Date and Time	Source	Event ID	Task Category
Critical	5/7/2024 2:13:36 AM	Kernel-Power	41	(63)
	The system has rebooted without cleanly shutting down first. This error could be caused if the system stopped responding, crashed, or lost power unexpectedly.
Error	5/7/2024 2:13:36 AM	volmgr	162	None
	Dump file generation succeded.
Warning	5/7/2024 2:11:00 AM	Display	4101	None
	Display driver amduw23g stopped responding and has successfully recovered.
Warning	5/7/2024 2:06:07 AM	Display	4101	None
	Display driver amduw23g stopped responding and has successfully recovered.

Updating Drivers

As a first step to try and remedy this, I always update drivers. I updated the following:

• chipset: 
• 4.07.13.2243 -> 5.11.02.217 (from Asus)
  
• 6.02.07.2300 (from AMD after above didn't fix it)
  
• video: a version installed in 2024 -> 24.5.1
  

Updating the BIOS

I found some posts online that said it could be due to the motherboard, so I decided to try updating the BIOS to see if that would help. Unfortunately, it did not.

Before updating the BIOS/UEFI make sure to write down your changes as they will be reset.

BIOS/UEFI Settings

• Expo -> Enabled
• Fan Curves
• CPU
• 20C -> 20%
• 60C -> 40%
• 85C -> 70%
• 90C -> 100%
• System
• 20C -> 30%
• 50C -> 50%
• 85C -> 80%
• 90C -> 100%
• Advanced Mode (F7)
• Tool -> ASUS Armoury Crate
•  Download & Install -> Disabled
• AI Tweaker -> Precision Boost Override
• Curve Optimizer
• All Cores
• Negative
• 30
• Precision Boost Override -> AMD Eco Mode
• AMD Eco Mode -> cTDP 65W
  
• AI Tweaker -> SOC Offset -> negative -> 0.03 (however this caused the pc to hang on warm-boots so I use the below settings instead)
• AI Tweaker -> CPU SOC Voltage -> Manual
• VDD SOC Voltage Override -> 1.2

Update the BIOS

• https://www.asus.com/motherboards-components/motherboards/tuf-gaming/tuf-gaming-x670e-plus-wifi/helpdesk_bios/?model2Name=TUF-GAMING-X670E-PLUS-WIFI
• 1618 (2023-05-25) -> 2613 (2024-04-17)
  

After updating the BIOS, you will need to reapply the settings.

Old video drivers

Since all the updates didn't work, I tried installing old video drivers that I knew worked from 24.5.1 to 23.10.2. Luckily this seems to have done the trick and my system is stable again with >27 days of uptime.

Slow folding

Unfortunately, it looks like my solution may be short lived as to get expected folding performance, I need to update to 24.6.1: https://foldingforum.org/viewtopic.php?t=41637&sid=d9b1c6f33f52801aaca8c31fc3fe52d1 So fingers crossed that this release is also stable.

Update 2024-07-29: I had a freeze after roughly 11 days of uptime

Update 2024-07-31: My PC crashed overnight after only 1.5 days of uptime

24.7.1

Update 2024-07-31: I updated the graphics drivers to 24.7.1. Here is the process that I used:

• Downloaded the AMD driver cleanup utility (from here)
  
• Downloaded the full updated AMD graphics driver (for 5600 XT)
• Disabled Ethernet/Wi-Fi
• I do this so that Windows doesn't try to "helpfully" download and install other graphic driver versions
  
• Ran the AMD driver cleanup utility
• Had it reboot me into Safe-Mode
• It removed the drivers and prompted me to reboot, which I did
• Installed the new drivers (had to approve the big red scary box because of no internet)
• Rebooted
• Enabled Ethernet/Wi-Fi

Update 2024-08-02: My computer has crashed twice in the past 2 days so I decided to try DDU

• Downloaded Display Driver Uninstaller (aka DDU)
• I downloaded from Guru3D
• Disabled Ethernet/Wi-Fi
• Rebooted into Safe Mode
• https://support.microsoft.com/en-us/windows/start-your-pc-in-safe-mode-in-windows-92c27cff-db89-8644-1ce4-b3e5e56fe234
• Settings -> System -> Recovery
• next to "Advanced startup", select "Restart now"
• System will reboot
• Troubleshoot -> Advanced options -> Startup Settings -> Restart
• System will reboot again
• Type "4" for Safe Mode
• Extracted the EXE from the ZIP file
• Ran the EXE, it will extract even more files
• Ran "Display Driver Uninstaller.exe"
• Unchecked the final option (Disable Windows Update Driver Downloads)
• Since we already disabled internet access this is unnecessary and DDU recommends reenabling afterwards anyway so this saves a step
• Close the options
• Device Type -> GPU
• Device -> AMD
• Clicked "Clean and restart"
• Waited for it to do its thing and reboot
• Installed the 24.7.1 drivers again
• Rebooted
• Enabled Ethernet/Wi-Fi

 

Appendix

Sources

• https://forums.tomshardware.com/threads/idle-bsod-kernel-power-41-with-vol-mgr-162-161.3777171/
• https://www.reddit.com/r/WindowsHelp/comments/vkkwmf/windows_11_volmgr_162_kernelpower_41_bsod/

FAHControl on Chromebook

As I wanted to be able to control my folding from my Chromebook, but the official FAH Control still has a Python2 dependency, I had to use a workaround to get it to work.

1. Download:
• The deb file from: https://github.com/cdberkstresser/fah-control/releases
• found it in this thread: https://foldingforum.org/viewtopic.php?t=34757&sid=e7eec792658e20ad2da3a89e905685c6&start=15
2. Install some requirements:
• In a linux terminal run:
• sudo apt install python3-gi gir1.2-gtk-3.0
3. Right click the deb file and select install with Linux

Now I wanted to add an icon to the launcher for being able to quickly launch it

1. Get a suitable icon. Here is what I chose
• https://icons.iconarchive.com/icons/papirus-team/papirus-apps/512/FAHControl-icon.png
2. Move the icon file into the "Linux files" directory
3. Create the applications directory:
• mkdir -p .local/share/applications
4. Create the shortcut
• touch .local/share/applications/FAHControl.desktop
5. Put the following in it

[Desktop Entry]

Name=FAHControl

Comment=Folding at Home Control interface

GenericName=FAH

X-GNOME-FullName=Folding at Home Conrol

Exec=/usr/bin/FAHControl

Terminal=false

Type=Application

Icon=/home/XXX/FAHControl-icon.png

Categories=Settings

Sources:

• https://support.google.com/pixelbook/thread/708306/how-do-i-add-linux-apps-to-the-home-screen?hl=en

Adventures in a Graphics Card Upgrade

Decision Time

As my HIS IceQ Radeon HD 7850 2GB was getting on in age, I decided it was time for an upgrade. I was trying to stay around the 130W TDP of the current card so that my folding did not increase my electric bill. The main options that I was considering was:

1. Used GTX 1060 ~$135-150
2. GTX 1650 ~$159
3. GTX 1650 Super ~$200
4. GTX 1660 ~$215
5. GTX 1660 Super ~$230
6. RTX 2060 KO ~$300
7. Used RX 470 4GB (avoid mining only) ~$75
8. Used RX 480 4GB ~$90
9. RX 570 4GB ~$120
10. RX 580 ~$155
11. RX 5500 XT 4GB ~$159.99

My first elimination was anything over $200 as that was just too expensive to pair with my old Core2 Quad Q6600. I then eliminated the RX 570 and GTX 1060 as it was just overpriced compared to the RX 480 and RX 5500 XT. The RX 580 and GTX 1650 were also overpriced compared to RX 5500 XT. This narrowed my choices to:

1. Used RX 470
2. Used RX 480 4GB
3. RX 5500 XT

The Purchase

I then scoured ebay for the used graphics cards. I placed the following price guidelines (including shipping) RX 470 for $70, RX 480 4GB for $80, RX 480 8GB for $90. It seemed that the 8GB models were pulling a large premium and the 470 and 480 4GB models seemed to be about the same price. I also wanted a backplate on the GPU to try to avoid GPU sag.

I was able to score a MSI RX 480 Gaming X 4GB model for ~$81 with shipping. When it arrived, it was in great physical shape and looked clean. I gave it a blast of compressed air to ensure maximum airflow. I installed the GPU after having to remove a fan mount in my old Antec 900 to increase the room for it. It then booted up without any issue. When I went to run benchmarks is when the bad news came. Under any kind of load the GPU spiked to 90C under 100% fan.

The Fix

To try to resolve this I tried:

1. Reducing the Core Voltage
2. Checking for a firmware upgrade
3. Tightening the 4 screws that maintained mounting pressure between the cooler on the card a quarter turn (be careful as over tightening can damage the chip)

However, none of those helped. It was finally time to try the last thing that I could think of, replacing the thermal paste. Here are the steps that I followed:

1. Removed the back plate taking care to keep track of which screws came out of where
2. Removed the two screws on the pcie bracket
3. Removed the 4 spring loaded screws that maintain the mounting pressure
4. Unplug one 4 pin connector between the PCB and cooler which allowed me to open it up like a book
5. This gave me a good look at the largely dried and cracking gray thermal paste
6. Carefully cleaned this off the GPU and the heatsink
7. Applied some Silicone thermal paste that I had leftover from my Q6600 build
8. Reattached the 4 pin connector
9. Put the 4 spring loaded screws back in, in a X pattern slowly tightening each one to maintain even pressure
10. Screwed the 2 screws on the pcie bracket back in
11. Screwed the back plate back on

Then the for the test. I reinstalled the graphic card and booted up the system. Thankfully, the card now worked like new!

Undervolting

The card's defaults are 1303MHz at 1150mV. I turned this down to 1250MHz at 1065mV using MSI Afterburner. This cost me about 4% theoretical performance for a savings of ~15% power (because Voltage is squared), which put the max GPU core/memory power usage at about 105W. This should keep me in the ~130W total board power range that I was looking for. The card is now doing about 65C with 40% fan.

I am still verifying the stability in folding, but at 1060mV it was just barely coming across an occasional error. I am hoping that bumping to 1065mV will eliminate those errors.

Update 2020-07-01: The additional 5mV have resolved the occasional error. I have seen some work units spike to 110W and hit 67C with 50% fan.

Update 2020-07-12: I still saw an infrequent error. I pushed to 1070mV in state 7 and 1055mV in state 6.

Update 2020-09-11:

Here are the default clock speeds and voltages:

1303MHz @ 1150mV

1235MHz @ 1143mV

1191MHz @ 1087mV

1145MHz @ 1037mV

1077MHz @ 973mV

910MHz @ 850mV

Default Memory: 1750MHz @ 975mV

Here is what my final stable underclock is:

1250MHz @ 1077mV

1235MHz @ 1075mV

1191MHz @ 1075mV

1145MHz @ 1037mV

1077MHz @ 973mV

910MHz @ 850mV

Memory Clocks/Voltages unchanged

GPU power usage (not total board power) drops from ~120Watts to ~100Watts

 

Update 2023-05-23:

I repasted using Arctic MX-4 thermal paste and saw a  temperature reduction. I also found this post and used it as a base for a new underclock: https://forums.tomshardware.com/threads/frequency-and-fps-drop-stuttering-on-rx-480.3624457/

1191MHz @ 990mV

1145MHz @ 980mV

1077MHz @ 975mV

910MHz @ 850mV

 

I have it capping out at 1191MHz to keep it nice and quiet at under 55C at 30% fan speed under load.

Folding on a HD 7850

I have been folding for years, but I just recently found out that adding a couple configuration options have doubled my PPD on my HD7850.


Enable Core17 beta units. Add the two following gpu extra slot options:

1.  First
•  Name: client-type
•  Value: beta
2. Second
• Name: extra-core-args
• Value: -gpu-vendor=ati

I found out about the options on this forum post: http://www.overclock.net/t/1367557/core-17-beta-wu


This took my HD7850 from 6-7k PPD to 12-13k PPD!

This is using the 12.8 drivers. I am letting it stabilize before trying 13.3 Beta drivers, because I had problems folding on 13.1 drivers. I will update this post when I try the new drivers.


4/20/2013 Update: I installed the 13.3 Beta 3 drivers (including the new APP SDK) and my cpu usage went from ~25% to ~2%. I am also seeing about a 40 second reduction in TPF resulting in about 3k more PPD. I have also seen some Bad work units that lower my PPD.

5/3/2013 Update: Folding has put the beta units back into a closed beta and I am only seeing Core16 units. So to maximize my PPD, I have gone back to 12.8 drivers and AMD APP SDK 2.7.

 Process:

• Disable Folding and make sure it does not restart automatically on reboot
• Download 64bit DRIVERS or 32bit DRIVERS 
• Download APP SDK 2.7
• Uninstall all AMD software/drivers using control panel
• Reboot
• Use Driver Fusion to remove all remnants of the AMD drivers
• Reboot
• Install drivers using custom install and uncheck AMD APP SDK runtime
• Install AMD APP SDK
• Reboot
• Restart Folding

5/7/2013 Update: They have released the Core17 work units again and now I am getting 18-20K PPD with them now.

6/14/2013 Update: With the release of the 8900 work units, I have moved to the 13.4 drivers and am now getting 28-30K PPD now.