etcd StatefulSet livenessProbe

httpGet /health livenessProbe make my etcd unstable in error state “pkg/osutil: received terminated signal, shutting down”, still don’t know why…

livenessProbe:
failureThreshold: 3
httpGet:
  host: 127.0.0.1
  path: /health
  port: 2381
  scheme: HTTP
initialDelaySeconds: 60
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 3

It must better after changed to command livenessProbe

livenessProbe:
failureThreshold: 3
  exec:
    command:
    - sh
    - -c
    - |-
      ETCDCTL_API=3  etcdctl  --cacert=/ca.crt  --cert=/kubernetes.pem --key=/kubernetes-key.pem --endpoints=127.0.0.1:2379 endpoint health
  initialDelaySeconds: 60
  periodSeconds: 10
  successThreshold: 1
  timeoutSeconds: 3

etcdctl endpoint health have more accurate than /health and no need to open metrics port 2371.

in normal state, this command response time < 50ms.

# ETCDCTL_API=3  etcdctl endpoint health  --cacert=/ca.crt  --cert=/kubernetes.pem --key=/kubernetes-key.pem --endpoints=127.0.0.1:2379
127.0.0.1:2379 is healthy: successfully committed proposal: took = 48.516245ms

NAME     READY   STATUS    RESTARTS   AGE   IP             
etcd-0   1/1     Running   0          1h   10.169.132.230
etcd-1   1/1     Running   0          1h   10.169.125.179
etcd-2   1/1     Running   0          1h   10.169.176.100

Hope that help.