k8s磁盘压力DiskPressure解决方案

本实践将在 Ubuntu 22.04.3LTS 系统上进行安装测试
docker版本 24.0.6
kubeadm版本 v1.28.2 版本

操作步骤

常用命令

# node.kubernetes.io/disk-pressure:NoSchedule
# docker system prune
# kubectl get pods -A -owide
# kubectl get events -owide

# kubectl cordon k8s-node1
# kubectl uncordon k8s-node1

# kubectl delete event --all
vim /var/lib/kubelet/config.yaml

#https://kubernetes.io/docs/concepts/scheduling-eviction/node-pressure-eviction/#eviction-thresholds
#https://kubernetes.io/zh-cn/docs/reference/config-api/kubelet-config.v1beta1/

主要修改配置文件

/var/lib/kubelet/config.yaml

evictionMinimumReclaim:
  memory.available: "0Mi"
  nodefs.available: "1%"
  imagefs.available: "1%"
evictionHard:
  memory.available: "100Mi"
  nodefs.available: "2%"
  nodefs.inodesFree: "2%"
  imagefs.available: "2%"
  imagefs.inodesFree: "2%"
evictionSoft:
  memory.available: "100Mi"
  nodefs.available: "2%"
  nodefs.inodesFree: "2%"
  imagefs.available: "2%"
  imagefs.inodesFree: "2%"
# 等待驱逐时间
evictionSoftGracePeriod:
  memory.available: "2m"
  nodefs.available: "2m"
  nodefs.inodesFree: "2m"
  imagefs.available: "2m"
  imagefs.inodesFree: "2m"
evictionPressureTransitionPeriod: 5m0s
evictionMaxPodGracePeriod: 360

查看系统日志

systemctl restart kubelet
journalctl -u kubelet --since "1min ago"

增加更加宽容的磁盘参数

imageGCHighThresholdPercent: 100
evictionHard:
  memory.available:  "100Mi"
  nodefs.available:  "1%"
  nodefs.inodesFree: "1%"
  imagefs.available: "100Mi"