Linux Percpu memory leak

High Percpu allocator memory usage:

Symptom:

/proc/meminfo shows too low “MemFree”

$ free -g
              total        used        free      shared  buff/cache   available
Mem:             94          81           1           0          10          14
Swap:             0           0           0

$ ps -aux |  awk '{sum+=$6} END {print sum/1024/1024 " GB"}'
16.2599 GB

$ grep Mem /proc/meminfo  | awk '{print $1,$2/1024/1024,"GB"}'
MemTotal: 94.2432 GB
MemFree: 2.08708 GB
MemAvailable: 14.7526 GB

Percpu:

$ grep Percpu /proc/meminfo 
Percpu:         57867840 kB

Percpu keep claiming and you also have hotplug CPU enabled:

$ grep hotplug /var/log/dmesg.0
[    0.021454] kernel: ACPI: SRAT: Node 0 PXM 0 [mem 0x1040000000-0x443fffffff] hotplug
[    0.202068] kernel: smpboot: Allowing 240 CPUs, 220 hotplug CPU

This can make memory leak.

Solution, just disable hotplug CPU and specific number of CPU (nr_cpus) to boot cmdline e.g. nr_cpus=8 and reboot.

$ cat /proc/cmdline 
BOOT_IMAGE=/boot/vmlinuz-5.4.0-135-generic root=UUID=bbd3ea56-da3b-4e1a-b14e-159e41299ea3 ro net.ifnames=0 nr_cpus=8

Hope this help.