I only have 8 cores CPU but Linux kernel only 4 cores when I run top/htop and GUI tools

linux-rockz · November 25, 2019, 5:38pm

Hi!

I can’t bring cpu core 1,3,5,7 online, when I’m on root via mate-terminal, write permission errors.

Here is the full story for my problem.

I updated my machine, rebooted and then I only have 4 cores showing in “mate-system-monitor”, same with “htop”.

I booted Ubuntu-Mate 18.10 Live DVD and then “mate-system-monitor” shows me 8 cores.

I tried out a work around.

Added to my rc.local:

Next try, I’ll did:

added to: /etc/rc.local

/bin/touch /sys/devices/system/cpu/cpu1/online
/bin/touch /sys/devices/system/cpu/cpu3/online
/bin/touch /sys/devices/system/cpu/cpu5/online
/bin/touch /sys/devices/system/cpu/cpu7/online
/bin/sleep 2

/bin/echo 1 > /sys/devices/system/cpu/cpu1/online
/bin/echo 1 > /sys/devices/system/cpu/cpu3/online
/bin/echo 1 > /sys/devices/system/cpu/cpu5/online
/bin/echo 1 > /sys/devices/system/cpu/cpu7/online
/bin/sleep 2

/bin/echo 0,1,2,3,4,5,6,7 > /sys/devices/system/cpu/online
/bin/sleep 1

Rebooted… The file “online” are now present in /sys/devices/system/cpu/cpu1/ (3+5+7)

but with value 0, not 1.

cat /sys/devices/system/cpu/cpu1/online
= 0


/bin/echo 1 > /sys/devices/system/cpu/cpu1/online
/bin/echo: write error: Operation not permitted

Please help to fix that, thanks.

linux-rockz · November 25, 2019, 5:59pm

lscpu | grep CPU.s

Output:

CPU(s): 8
On-line CPU(s) list: 0,2,4,6
Off-line CPU(s) list: 1,3,5,7
NUMA node0 CPU(s): 0,2,4,6

lscpu | egrep ‘NUMA|ore’

Output:

NUMA node(s): 1
Model name: AMD FX™-8350 Eight-Core Processor
NUMA node0 CPU(s): 0,2,4,6

nixcraft · November 25, 2019, 6:47pm

Did you disable HT in BIOS? Go to bios set default settings. Make sure HT (hyper threading) is enabled. If you still getting error, paste output of the following commands:

cat /proc/cmdline 
uname -mrs
getconf LONG_BIT

In another thread you talked about microcode update. Remove that package and reboot the system. Does that helps?

linux-rockz · November 26, 2019, 8:27am

cat /proc/cmdline

Output:

BOOT_IMAGE=path-to… ro quiet nosplash biosdevname=0 net.ifnames=0 ipv6.disable=1 maxcpus=8 nr_cpus=8 possible_cpus=8 numa=off amd_iommu=on mem_encrypt=on kvm_amd.sev=1 apparmor=1 security=apparmor ramdisk_size=6291456K nopersistence noprompt timezone=Etc/UTC block.events_dfl_poll_msecs=1000 noautologin mce=0 vsyscall=none page_poison=1 aslr pti=on kpti=on slab_nomerge slub_debug=FPZ nosmt spectre_v2=on nobp=1 spec_store_bypass_disable=on video=vesafb:off,efifb:off nouveau.modeset=0 modprobe.blacklist=nouveau rd.driver.blacklist=nouveau pcie_aspm.policy=performance elevator=deadline selinux=0 ht=on acpi=ht acpi_osi=Linux acpi_backlight=vendor

uname -mrs + getconf LONG_BIT

Output:

Linux 5.0.0-36-generic x86_64

64

HT is on.
Core 1,3,5,7 are offline.

I didn’t take core 1,3,5,7 offline.

When I boot with an old version of Ubuntu-Mate v18.10 I have 8 cores available.
Look here:

I removed amd64-microcode. Rebooted. No effect.

A few days ago I had 8 cores available.

Could it be the kernel update?

linux-rockz · November 26, 2019, 8:32am

lscpu

Output:

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 48 bits physical, 48 bits virtual
CPU(s): 8
On-line CPU(s) list: 0,2,4,6
Off-line CPU(s) list: 1,3,5,7
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: AuthenticAMD
CPU family: 21
Model: 2
Model name: AMD FX™-8350 Eight-Core Processor
Stepping: 0
CPU MHz: 4012.980
BogoMIPS: 8027.04
Virtualization: AMD-V
L1d cache: 16K
L1i cache: 64K
L2 cache: 2048K
L3 cache: 8192K
NUMA node0 CPU(s): 0,2,4,6
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb cpb hw_pstate pti ssbd ibpb vmmcall bmi1 arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold

I didn’t set a NUMA node.

Is it possible to remove the NUMA-node?

ls /sys/devices/system/cpu

Output:

cpu0 cpu2 cpu4 cpu6 cpufreq hotplug kernel_max modalias online power smt vulnerabilities
cpu1 cpu3 cpu5 cpu7 cpuidle isolated microcode offline possible present uevent

lscpu | grep CPU.s

Output:

CPU(s): 8
On-line CPU(s) list: 0,2,4,6
Off-line CPU(s) list: 1,3,5,7
NUMA node0 CPU(s): 0,2,4,6

I didn’t set a NUMA node0.

My idea…
If you can’t remove the NUMA-Node0 maybe it is
it is possible to add another NUMA node using the
CPU cores 1,3,5,7 which are currently offline.

linux-rockz · November 26, 2019, 9:28am

I did a test and tried kernel “5.0.0-13-lowlatency”.

The problem that only 4 CPU cores are displayed still exists.

nixcraft · November 26, 2019, 1:44pm

Delete these options from boot loader and reboot it.

maxcpus=8 nr_cpus=8 possible_cpus=8

linux-rockz · November 26, 2019, 4:03pm

‘done’ …no effect, look:

lscpu

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 48 bits physical, 48 bits virtual

CPU(s): 8

On-line CPU(s) list: 0,2,4,6
Off-line CPU(s) list: 1,3,5,7

Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: AuthenticAMD
CPU family: 21
Model: 2
Model name: AMD FX™-8350 Eight-Core Processor
Stepping: 0
CPU MHz: 4010.831
BogoMIPS: 8027.43
Virtualization: AMD-V
L1d cache: 16K
L1i cache: 64K
L2 cache: 2048K
L3 cache: 8192K
NUMA node0 CPU(s): 0,2,4,6

I added to grub.cfg = numa=off numa_balancing=disable

But that didn’t help.

NUMA node0 CPU(s): 0,2,4,6
Off-line CPU(s) list: 1,3,5,7

linux-rockz · November 26, 2019, 6:20pm

“numactl” …from package “numactl”…

usage: numactl [--all | -a] [--interleave= | -i ] [--preferred= | -p ]
[--physcpubind= | -C ] [--cpunodebind= | -N ]
[--membind= | -m ] [--localalloc | -l] command args ...
numactl [--show | -s]
numactl [--hardware | -H]
numactl [--length | -l ] [--offset | -o ] [--shmmode | -M ]
[--strict | -t]
[--shmid | -I ] --shm | -S
[--shmid | -I ] --file | -f
[--huge | -u] [--touch | -T]
memory policy | --dump | -d | --dump-nodes | -D

memory policy is --interleave | -i, --preferred | -p, --membind | -m, --localalloc | -l
is a comma delimited list of node numbers or A-B ranges or all.
Instead of a number a node can also be:
netdev:DEV the node connected to network device DEV
file:PATH the node the block device of path is connected to
ip:HOST the node of the network device host routes through
block:PATH the node of block device path
pci:[seg:]bus:dev[:func] The node of a PCI device
is a comma delimited list of cpu numbers or A-B ranges or all
all ranges can be inverted with !
all numbers and ranges can be made cpuset-relative with +
the old --cpubind argument is deprecated.
use --cpunodebind or --physcpubind instead
can have g (GB), m (MB) or k (KB) suffixes

If I knew what the command would have to be called
deactivate NUMA node0 or
to add another node for CPU core 1,3,5,7.

Tried:

numactl -a --physcpubind=0 --cpunodebind=0 --membind=0

commad didn’t work

Does anyone have an idea how the right command is to turn off NUMA?

nixcraft · November 26, 2019, 11:58pm

The numactl run a process with a specified scheduling or memory placement policy. Numactl can also set a persistent policy for shared memory segments or files, and set the processor affinity and memory affinity of a process. You don’t need it. After reading all this you have a few options:

Install latest kernel from source
Switch back to older kernel where you go 8 core option
Reinstall the OS

linux-rockz · November 27, 2019, 7:33am

Solution:

I added to rc.local in /etc/ directory:

echo on > /sys/devices/system/cpu/smt/control

Rebooted and now my 8 cores are back.

Raj · November 27, 2019, 9:12am

You also have nosmt (see cat /proc/cmdline) set in kernel variable. Delete that line too. That is the main root cause of problem.

linux-rockz · November 27, 2019, 9:45am

thanks for the info @ Raj, done.

nixcraft · November 27, 2019, 2:03pm

Thank you for posting solution as well.

linux-rockz · December 2, 2019, 8:53am

@ Raj, I removed the “nosmt” from cmdline in grub.cfg.

Also, removed the line "echo on > /sys/devices/system/cpu/smt/control” from my rc.local.
Thanks!

Raj · December 6, 2019, 10:35am

Hey @linux-rockz

I think you added cpu attack mitigation (spectra and others) that is why those core were hidden. if you remove them, a patched linux kernel will protect it. so if you need those core remove nosmt and other such options.