After upgrading a webserver running Apache to Debian Jessie (from Wheezy), I noticed that the system CPU usage was higher. Running an strace on one of the Apache processes was giving me very little info:
strace -c -p 10112
Process 10112 attached - interrupt to quit
^CProcess 10112 detached
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
95.71 28.549784 3172198 9 4 futex
I had previously tried to spread interrupts across cores and also limited network activity – as they are something one can do to work out why System CPU is high – but they didn’t bring down the CPU usage.
One culprit remained: Futex/Mutex locks. I changed the default Mutex [1] to file and magically the System CPU usage went down.