Skip to content

Commit

Permalink
OCPBUGS-13153: Limit the value of GOMAXPROCS on node-exporter to 4.
Browse files Browse the repository at this point in the history
On nodes with multiple CPU cores, this should help avoid lock contentions
without having any side effects, see the ticket for more details.

node-exporter versions that have "--runtime.gomaxprocs" can still
override this: the flag has precedence over this automatic setting.

Make the entrypoint set the env var before runnning the process.
Get the CPU count from /proc/cpuinfo without taking CPU affinity (via cpuset)
into account as the container's CPU requests is not an integer, thus no
affinity is applied.

Signed-off-by: Ayoub Mrini <amrini@redhat.com>
  • Loading branch information
machine424 committed Jun 20, 2023
1 parent bcda99d commit 58997b8
Show file tree
Hide file tree
Showing 2 changed files with 28 additions and 0 deletions.
13 changes: 13 additions & 0 deletions assets/node-exporter/daemonset.yaml
Expand Up @@ -41,6 +41,19 @@ spec:
- --collector.cpu.info
- --collector.textfile.directory=/var/node_exporter/textfile
- --no-collector.btrfs
command:
- /bin/sh
- -c
- |
export GOMAXPROCS=4
# We don't take CPU affinity into account as the container doesn't have integer CPU requests.
# In case of error, fallback to the default value.
NUM_CPUS=$(grep -c '^processor' "/proc/cpuinfo" 2>/dev/null || echo "0")
if [ "$NUM_CPUS" -lt "$GOMAXPROCS" ]; then
export GOMAXPROCS="$NUM_CPUS"
fi
echo "ts=$(date --iso-8601=seconds) num_cpus=$NUM_CPUS gomaxprocs=$GOMAXPROCS"
exec /bin/node_exporter "$0" "$@"
image: quay.io/prometheus/node-exporter:v1.6.0
name: node-exporter
resources:
Expand Down
15 changes: 15 additions & 0 deletions jsonnet/components/node-exporter.libsonnet
Expand Up @@ -250,6 +250,21 @@ function(params)
'--collector.textfile.directory=' + textfileDir,
'--no-collector.btrfs',
],
command: [
'/bin/sh',
'-c',
|||
export GOMAXPROCS=4
# We don't take CPU affinity into account as the container doesn't have integer CPU requests.
# In case of error, fallback to the default value.
NUM_CPUS=$(grep -c '^processor' "/proc/cpuinfo" 2>/dev/null || echo "0")
if [ "$NUM_CPUS" -lt "$GOMAXPROCS" ]; then
export GOMAXPROCS="$NUM_CPUS"
fi
echo "ts=$(date --iso-8601=seconds) num_cpus=$NUM_CPUS gomaxprocs=$GOMAXPROCS"
exec /bin/node_exporter "$0" "$@"
|||,
],
terminationMessagePolicy: 'FallbackToLogsOnError',
volumeMounts+: [{
mountPath: textfileDir,
Expand Down

0 comments on commit 58997b8

Please sign in to comment.