Intro
I recently acquired a Dell Poweredge T440, and after installing a GPU the fans skyrocketed. This turned the server into a noise machine. To solve this, I created a script that disables third-party PCI mode and based on a threshold, set the fan speed to 20% or to be controlled by Dell’s fan profile. These are the steps I took to make my T440 silent again.
Downgrading iDRAC 9 firmware to 3.30.30.30
For this whole setup to work, iDRAC needs to be on firmware 3.30.30.30, as later firmware revisions remove the ability to control the fan speed. The firmware can be downloaded from Dell’s website. To downgrade, navigate to your iDRAC’s web admin page, and under Maintenance->System Update you can apply the downgrade. After the downgrade, make sure IPMI over LAN is enabled under iDRAC Settings->Connectivity->Network->IPMI Settings.
Script to control the fan speed
Once iDRAC is on firmware 3.30.30.30 and you can use IPMI over LAN, you can start using my script to control the fan speed. My script first checks that a VM has been up for 60 seconds. I added this check as pfSense is virtualized in my lab and as such needs to wait for it to be up to issue LAN commands. You can remove this check and the script will work as intended. Before running, make sure to fill in the blanks.
#!/bin/bash
check_start_time() {
local start_time="$1"
local current_time=$(date +%s)
local time_diff=$((current_time - start_time))
if [ $time_diff -ge 60 ]; then
echo 0
else
echo 1
fi
}
check_temperature() {
temp_lines=$(ipmitool -I lanplus -H "$HOST" -U "$USER" -P "$PASS" sdr type temperature 2>/dev/null | grep -e degrees | awk '{print $(NF-2)}')
temp_arr=""
readarray -t temp_arr <<< "$temp_lines"
sorted_temps=($(printf "%s\n" "${temp_arr[@]}" | sort -nr))
max_temp=${sorted_temps[0]}
if [ "$max_temp" -le 60 ] && [ "$CURRENT_PROFILE" != "MANUAL" ]; then
# enable manual fan control
ipmitool -I lanplus -H "$HOST" -U "$USER" -P "$PASS" raw 0x30 0x30 0x01 0x00 &> /dev/null
# set initial fan to 20%
ipmitool -I lanplus -H "$HOST" -U "$USER" -P "$PASS" raw 0x30 0x30 0x02 0xff 0x14 &> /dev/null
LOW_TO_HIGH=false
CURRENT_PROFILE="MANUAL"
elif [ "$max_temp" -le 65 ] && [ "$CURRENT_PROFILE" != "MANUAL2" ]; then
# enable manual fan control
ipmitool -I lanplus -H "$HOST" -U "$USER" -P "$PASS" raw 0x30 0x30 0x01 0x00 &> /dev/null
# set initial fan to 30%
ipmitool -I lanplus -H "$HOST" -U "$USER" -P "$PASS" raw 0x30 0x30 0x02 0xff 0x1e &> /dev/null
# set low_to_high used to determine wait time
if [[ "$CURRENT_PROFILE" == "MANUAL"]]; then
LOW_TO_HIGH=true
else
LOW_TO_HIGH=false
fi
CURRENT_PROFILE="MANUAL2"
elif [ "$max_temp" -gt 65 ] && [ "$CURRENT_PROFILE" != "AUTO" ]; then
# enable Dell fan control
ipmitool -I lanplus -H "$HOST" -U "$USER" -P "$PASS" raw 0x30 0x30 0x01 0x01 &> /dev/null
# set low_to_high used to determine wait time
if [[ "$CURRENT_PROFILE" == "MANUAL2"]]; then
LOW_TO_HIGH=true
else
LOW_TO_HIGH=false
fi
CURRENT_PROFILE="AUTO"
fi
}
disable_third_party_mode (){
# disable 3rd party pcie fan mode
output=$(sshpass -p "$PASS" ssh -o StrictHostKeyChecking=no "$USER@$HOST" "racadm get system.pcieslotlfm" 2>/dev/null)
pcie_ports_count=$(echo "$output" | grep -c 'System.pcieslotlfm')
# Iterate through each PCIe port and disable third party mode
for ((i=1; i<=pcie_ports_count; i++)); do
sshpass -p "$PASS" ssh -o StrictHostKeyChecking=no "$USER@$HOST" "racadm set system.pcieslotlfm.$i.lfmmode disabled" &> /dev/null
done
}
HOST=""
USER=""
PASS=""
VMID=""
CURRENT_PROFILE=""
LOW_TO_HIGH=false
start_time=$(ps -o lstart= -p $(cat /var/run/qemu-server/$VMID.pid))
start_time=$(date -d "$start_time" +%s)
while [[ $(check_start_time "$start_time") -eq 1 ]]
do
sleep 5
done
disable_third_party_mode
while true; do
check_temperature
if [[ "$CURRENT_PROFILE" == "AUTO" || "$LOW_TO_HIGH" == true]]; then
sleep 180
else
sleep 60
fi
done
Automating the launching of the script
Instead of manually running this every time my server reboots, I created a service. To do so, I used the following service file:
[Unit]
Description=User defined temperature fan control for iDrac 9
[Service]
Type=simple
ExecStart=/usr/local/sbin/fan_control.sh
Restart=always
[Install]
WantedBy=multi-user.target
On my Proxmox install, the bash script is placed at /usr/local/sbin/fan_control.sh with the appropriate permissions and the service file is at /etc/systemd/system. After these files are created, it takes a “systemctl daemon-reload” and then a “systemctl enable fan_control” to start the service and make sure it starts every reboot.
Done!
Once everything is set up, your Dell server with iDRAC 9 should be extremely quiet.