pod(九)：污點taint 與容忍度tolerations

2022 年 11 月 7 日
筆記
Docker容器, Kubernetes(k8s)管理

一.系統環境
二.前言
三.污點taint
- 3.1 污點taint概覽
- 3.2 給節點添加污點taint
四.容忍度tolerations
- 4.1 容忍度tolerations概覽
- 4.2 設置容忍度tolerations

一.系統環境

伺服器版本	docker軟體版本	Kubernetes(k8s)集群版本	CPU架構
CentOS Linux release 7.4.1708 (Core)	Docker version 20.10.12	v1.21.9	x86_64

Kubernetes集群架構：k8scloude1作為master節點，k8scloude2，k8scloude3作為worker節點

伺服器	作業系統版本	CPU架構	進程	功能描述
k8scloude1/192.168.110.130	CentOS Linux release 7.4.1708 (Core)	x86_64	docker，kube-apiserver，etcd，kube-scheduler，kube-controller-manager，kubelet，kube-proxy，coredns，calico	k8s master節點
k8scloude2/192.168.110.129	CentOS Linux release 7.4.1708 (Core)	x86_64	docker，kubelet，kube-proxy，calico	k8s worker節點
k8scloude3/192.168.110.128	CentOS Linux release 7.4.1708 (Core)	x86_64	docker，kubelet，kube-proxy，calico	k8s worker節點

二.前言

本文介紹污點taint 與容忍度tolerations，可以影響pod的調度。

使用污點taint 與容忍度tolerations的前提是已經有一套可以正常運行的Kubernetes集群，關於Kubernetes(k8s)集群的安裝部署，可以查看部落格《Centos7 安裝部署Kubernetes(k8s)集群》//www.cnblogs.com/renshengdezheli/p/16686769.html

三.污點taint

3.1 污點taint概覽

節點親和性是 Pod 的一種屬性，它使 Pod 被吸引到一類特定的節點（這可能出於一種偏好，也可能是硬性要求）。污點（Taint）則相反——它使節點能夠排斥一類特定的 Pod。

3.2 給節點添加污點taint

給節點增加一個污點的語法如下：給節點 node1 增加一個污點，它的鍵名是 key1，鍵值是 value1，效果是 NoSchedule。這表示只有擁有和這個污點相匹配的容忍度的 Pod 才能夠被分配到 node1 這個節點。

#污點的格式：鍵=值:NoSchedule
kubectl taint nodes node1 key1=value1:NoSchedule

#只有鍵沒有值的話，格式為：鍵:NoSchedule
kubectl taint nodes node1 key1:NoSchedule

移除污點語法如下：

kubectl taint nodes node1 key1=value1:NoSchedule-

節點的描述資訊里有一個Taints欄位，Taints欄位表示節點有沒有污點

[root@k8scloude1 deploy]# kubectl get nodes -o wide
NAME         STATUS   ROLES                  AGE   VERSION   INTERNAL-IP       EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION          CONTAINER-RUNTIME
k8scloude1   Ready    control-plane,master   8d    v1.21.0   192.168.110.130   <none>        CentOS Linux 7 (Core)   3.10.0-693.el7.x86_64   docker://20.10.12
k8scloude2   Ready    <none>                 8d    v1.21.0   192.168.110.129   <none>        CentOS Linux 7 (Core)   3.10.0-693.el7.x86_64   docker://20.10.12
k8scloude3   Ready    <none>                 8d    v1.21.0   192.168.110.128   <none>        CentOS Linux 7 (Core)   3.10.0-693.el7.x86_64   docker://20.10.12

[root@k8scloude1 deploy]# kubectl describe nodes k8scloude1
Name:               k8scloude1
Roles:              control-plane,master
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    kubernetes.io/arch=amd64
                    kubernetes.io/hostname=k8scloude1
                    kubernetes.io/os=linux
                    node-role.kubernetes.io/control-plane=
                    node-role.kubernetes.io/master=
                    node.kubernetes.io/exclude-from-external-load-balancers=
Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
                    node.alpha.kubernetes.io/ttl: 0
                    projectcalico.org/IPv4Address: 192.168.110.130/24
                    projectcalico.org/IPv4IPIPTunnelAddr: 10.244.158.64
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Sun, 09 Jan 2022 16:19:06 +0800
Taints:             node-role.kubernetes.io/master:NoSchedule
Unschedulable:      false
......

查看節點是否有污點，Taints: node-role.kubernetes.io/master:NoSchedule表示k8s集群的master節點有污點，這是默認就存在的污點，這也是master節點為什麼不能運行應用pod的原因。

[root@k8scloude1 deploy]# kubectl describe nodes k8scloude2 | grep -i Taints
Taints:             <none>

[root@k8scloude1 deploy]# kubectl describe nodes k8scloude1 | grep -i Taints
Taints:             node-role.kubernetes.io/master:NoSchedule

[root@k8scloude1 deploy]# kubectl describe nodes k8scloude3 | grep -i Taints
Taints:             <none>

創建pod，nodeSelector:kubernetes.io/hostname: k8scloude1表示pod運行在標籤為kubernetes.io/hostname=k8scloude1的節點上。

關於pod的調度詳細內容，請查看部落格《pod(八)：pod的調度——將 Pod 指派給節點》//www.cnblogs.com/renshengdezheli/p/16863405.html

[root@k8scloude1 pod]# vim schedulepod4.yaml 

[root@k8scloude1 pod]# cat schedulepod4.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: pod1
  name: pod1
  namespace: pod
spec:
  nodeSelector:
    kubernetes.io/hostname: k8scloude1
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: pod1
    resources: {}
    ports:
    - name: http
      containerPort: 80
      protocol: TCP
      hostPort: 80
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

標籤為kubernetes.io/hostname=k8scloude1的節點為k8scloude1節點

[root@k8scloude1 pod]# kubectl get nodes -l kubernetes.io/hostname=k8scloude1
NAME         STATUS   ROLES                  AGE   VERSION
k8scloude1   Ready    control-plane,master   8d    v1.21.0

創建pod，因為k8scloude1上有污點，pod1不能運行在k8scloude1上，所以pod1狀態為Pending

[root@k8scloude1 pod]# kubectl apply -f schedulepod4.yaml 
pod/pod1 created

 #因為k8scloude1上有污點，pod1不能運行在k8scloude1上，所以pod1狀態為Pending
[root@k8scloude1 pod]# kubectl get pod -o wide
NAME   READY   STATUS    RESTARTS   AGE   IP       NODE     NOMINATED NODE   READINESS GATES
pod1   0/1     Pending   0          9s    <none>   <none>   <none>           <none>

[root@k8scloude1 pod]# kubectl delete pod pod1 --force
warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "pod1" force deleted

[root@k8scloude1 pod]# kubectl get pod -o wide
No resources found in pod namespace.

四.容忍度tolerations

4.1 容忍度tolerations概覽

容忍度（Toleration）是應用於 Pod 上的。容忍度允許調度器調度帶有對應污點的 Pod。容忍度允許調度但並不保證調度：作為其功能的一部分，調度器也會評估其他參數。

污點和容忍度（Toleration）相互配合，可以用來避免 Pod 被分配到不合適的節點上。每個節點上都可以應用一個或多個污點，這表示對於那些不能容忍這些污點的 Pod，是不會被該節點接受的。

4.2 設置容忍度tolerations

只有擁有和這個污點相匹配的容忍度的 Pod 才能夠被分配到 node節點。

查看k8scloude1節點的污點

[root@k8scloude1 pod]# kubectl describe nodes k8scloude1 | grep -i taint
Taints:             node-role.kubernetes.io/master:NoSchedule

你可以在 Pod 規約中為 Pod 設置容忍度，創建pod，tolerations參數表示可以容忍污點：node-role.kubernetes.io/master:NoSchedule ，nodeSelector:kubernetes.io/hostname: k8scloude1表示pod運行在標籤為kubernetes.io/hostname=k8scloude1的節點上。

[root@k8scloude1 pod]# vim schedulepod4.yaml 

[root@k8scloude1 pod]# cat schedulepod4.yaml 
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: pod1
  name: pod1
  namespace: pod
spec:
  tolerations:
  - key: "node-role.kubernetes.io/master"
    operator: "Equal"
    value: ""
    effect: "NoSchedule"
  nodeSelector:
    kubernetes.io/hostname: k8scloude1
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: pod1
    resources: {}
    ports:
    - name: http
      containerPort: 80
      protocol: TCP
      hostPort: 80
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

[root@k8scloude1 pod]# kubectl get pods -o wide
No resources found in pod namespace.

[root@k8scloude1 pod]# kubectl apply -f schedulepod4.yaml 
pod/pod1 created

查看pod，即使k8scloude1節點有污點，pod還是正常運行。

taint污點和cordon，drain的區別：某個節點上有污點，可以設置tolerations容忍度，讓pod運行在該節點，某個節點被cordon，drain，則該節點不能被分配出去運行pod。

關於cordon，drain的詳細資訊，請查看部落格《cordon節點，drain驅逐節點，delete 節點》//www.cnblogs.com/renshengdezheli/p/16860674.html

[root@k8scloude1 pod]# kubectl get pods -o wide
NAME   READY   STATUS    RESTARTS   AGE   IP              NODE         NOMINATED NODE   READINESS GATES
pod1   1/1     Running   0          4s    10.244.158.84   k8scloude1   <none>           <none>

[root@k8scloude1 pod]# kubectl delete pod pod1 --force
warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "pod1" force deleted

[root@k8scloude1 pod]# kubectl get pods -o wide
No resources found in pod namespace.

注意，tolerations容忍度有兩種寫法，任選一種即可:

tolerations:
- key: "key1"
  operator: "Equal"
  value: "value1"
  effect: "NoSchedule"
  
tolerations:
- key: "key1"
  operator: "Exists"
  effect: "NoSchedule"

給k8scloude2節點打標籤

[root@k8scloude1 pod]# kubectl label nodes k8scloude2 taint=T
node/k8scloude2 labeled

[root@k8scloude1 pod]# kubectl get node --show-labels
NAME         STATUS   ROLES                  AGE   VERSION   LABELS
k8scloude1   Ready    control-plane,master   8d    v1.21.0   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8scloude1,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=,node-role.kubernetes.io/master=,node.kubernetes.io/exclude-from-external-load-balancers=
k8scloude2   Ready    <none>                 8d    v1.21.0   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8scloude2,kubernetes.io/os=linux,taint=T
k8scloude3   Ready    <none>                 8d    v1.21.0   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8scloude3,kubernetes.io/os=linux

對k8scloude2設置污點

#污點taint的格式：鍵=值:NoSchedule
[root@k8scloude1 pod]# kubectl taint node k8scloude2 wudian=true:NoSchedule
node/k8scloude2 tainted

[root@k8scloude1 pod]# kubectl describe nodes k8scloude2 | grep -i Taints
Taints:             wudian=true:NoSchedule

創建pod，tolerations參數表示容忍污點wudian=true:NoSchedule，nodeSelector:taint: T參數表示pod運行在標籤為nodeSelector=taint: T的節點。

[root@k8scloude1 pod]# vim schedulepod4.yaml 

[root@k8scloude1 pod]# cat schedulepod4.yaml 
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: pod1
  name: pod1
  namespace: pod
spec:
  tolerations:
  - key: "wudian"
    operator: "Equal"
    value: "true"
    effect: "NoSchedule"
  nodeSelector:
    taint: T
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: pod1
    resources: {}
    ports:
    - name: http
      containerPort: 80
      protocol: TCP
      hostPort: 80
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

[root@k8scloude1 pod]# kubectl get pod -o wide
No resources found in pod namespace.

[root@k8scloude1 pod]# kubectl apply -f schedulepod4.yaml 
pod/pod1 created

查看pod，k8scloude2節點就算有污點也能運行pod

[root@k8scloude1 pod]# kubectl get pods -o wide
NAME   READY   STATUS    RESTARTS   AGE   IP               NODE         NOMINATED NODE   READINESS GATES
pod1   1/1     Running   0          8s    10.244.112.177   k8scloude2   <none>           <none>

[root@k8scloude1 pod]# kubectl delete pod pod1 --force
warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "pod1" force deleted

[root@k8scloude1 pod]# kubectl get pods -o wide
No resources found in pod namespace.

污點容忍的另一種寫法：operator: “Exists”，沒有value值。

[root@k8scloude1 pod]# vim schedulepod4.yaml 

[root@k8scloude1 pod]# cat schedulepod4.yaml 
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: pod1
  name: pod1
  namespace: pod
spec:
  tolerations:
  - key: "wudian"
    operator: "Exists"
    effect: "NoSchedule"
  nodeSelector:
    taint: T
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: pod1
    resources: {}
    ports:
    - name: http
      containerPort: 80
      protocol: TCP
      hostPort: 80
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

[root@k8scloude1 pod]# kubectl apply -f schedulepod4.yaml 
pod/pod1 created

查看pod，k8scloude2節點就算有污點也能運行pod

[root@k8scloude1 pod]# kubectl get pods -o wide
NAME   READY   STATUS    RESTARTS   AGE   IP               NODE         NOMINATED NODE   READINESS GATES
pod1   1/1     Running   0          10s   10.244.112.178   k8scloude2   <none>           <none>

[root@k8scloude1 pod]# kubectl delete pod pod1 --force
warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "pod1" force deleted

[root@k8scloude1 pod]# kubectl get pods -o wide
No resources found in pod namespace.

給k8scloude2節點再添加一個污點

[root@k8scloude1 pod]# kubectl describe nodes k8scloude2 | grep Taints
Taints:             wudian=true:NoSchedule

[root@k8scloude1 pod]# kubectl taint node k8scloude2 zang=shide:NoSchedule
node/k8scloude2 tainted

[root@k8scloude1 pod]# kubectl describe nodes k8scloude2 | grep Taints
Taints:             wudian=true:NoSchedule

[root@k8scloude1 pod]# kubectl describe nodes k8scloude2 | grep -A2 Taints
Taints:             wudian=true:NoSchedule
                    zang=shide:NoSchedule
Unschedulable:      false

[root@k8scloude1 pod]# kubectl describe nodes k8scloude2 | grep -A1 Taints
Taints:             wudian=true:NoSchedule
                    zang=shide:NoSchedule

創建pod，tolerations參數表示容忍2個污點：wudian=true:NoSchedule和zang=shide:NoSchedule，nodeSelector:taint: T參數表示pod運行在標籤為nodeSelector=taint: T的節點。

[root@k8scloude1 pod]# vim schedulepod4.yaml 

[root@k8scloude1 pod]# cat schedulepod4.yaml 
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: pod1
  name: pod1
  namespace: pod
spec:
  tolerations:
  - key: "wudian"
    operator: "Equal"
    value: "true"
    effect: "NoSchedule"
  - key: "zang"
    operator: "Equal"
    value: "shide"
    effect: "NoSchedule"
  nodeSelector:
    taint: T
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: pod1
    resources: {}
    ports:
    - name: http
      containerPort: 80
      protocol: TCP
      hostPort: 80
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

[root@k8scloude1 pod]# kubectl apply -f schedulepod4.yaml 
pod/pod1 created

查看pod，k8scloude2節點就算有2個污點也能運行pod

[root@k8scloude1 pod]# kubectl get pods -o wide
NAME   READY   STATUS    RESTARTS   AGE   IP               NODE         NOMINATED NODE   READINESS GATES
pod1   1/1     Running   0          6s    10.244.112.179   k8scloude2   <none>           <none>

[root@k8scloude1 pod]# kubectl delete pod pod1 --force
warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "pod1" force deleted

創建pod，tolerations參數表示容忍污點：wudian=true:NoSchedule，nodeSelector:taint: T參數表示pod運行在標籤為nodeSelector=taint: T的節點。

[root@k8scloude1 pod]# vim schedulepod4.yaml 

[root@k8scloude1 pod]# cat schedulepod4.yaml 
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: pod1
  name: pod1
  namespace: pod
spec:
  tolerations:
  - key: "wudian"
    operator: "Equal"
    value: "true"
    effect: "NoSchedule"
  nodeSelector:
    taint: T
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: pod1
    resources: {}
    ports:
    - name: http
      containerPort: 80
      protocol: TCP
      hostPort: 80
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

[root@k8scloude1 pod]# kubectl apply -f schedulepod4.yaml 
pod/pod1 created

查看pod，一個節點有兩個污點值，但是yaml文件只容忍一個，所以pod創建不成功。

[root@k8scloude1 pod]# kubectl get pods -o wide
NAME   READY   STATUS    RESTARTS   AGE   IP       NODE     NOMINATED NODE   READINESS GATES
pod1   0/1     Pending   0          8s    <none>   <none>   <none>           <none>

[root@k8scloude1 pod]# kubectl delete pod pod1 --force
warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "pod1" force deleted

[root@k8scloude1 pod]# kubectl get pods -o wide
No resources found in pod namespace.

取消k8scloude2污點

[root@k8scloude1 pod]# kubectl describe nodes k8scloude2 | grep -A2 Taints
Taints:             wudian=true:NoSchedule
                    zang=shide:NoSchedule
Unschedulable:      false

#取消污點
[root@k8scloude1 pod]# kubectl taint node k8scloude2 zang-
node/k8scloude2 untainted

[root@k8scloude1 pod]# kubectl taint node k8scloude2 wudian-
node/k8scloude2 untainted

[root@k8scloude1 pod]# kubectl describe nodes k8scloude1 | grep -A2 Taints
Taints:             node-role.kubernetes.io/master:NoSchedule
Unschedulable:      false
Lease:

[root@k8scloude1 pod]# kubectl describe nodes k8scloude2 | grep -A2 Taints
Taints:             <none>
Unschedulable:      false
Lease:

[root@k8scloude1 pod]# kubectl describe nodes k8scloude3 | grep -A2 Taints
Taints:             <none>
Unschedulable:      false
Lease:

Tips:如果自身機器有限，只能有一台機器，則可以把master節點的污點taint取消，就可以在master上運行pod了。

Tags: Docker容器 Kubernetes(k8s)管理

pod(九)：污點taint 與容忍度tolerations

一.系統環境

二.前言

三.污點taint

3.1 污點taint概覽

3.2 給節點添加污點taint

四.容忍度tolerations

4.1 容忍度tolerations概覽

4.2 設置容忍度tolerations

VirMach 便宜 VPS

QNews

pod(九)：污點taint 與容忍度tolerations

一.系統環境

二.前言

三.污點taint

3.1 污點taint概覽

3.2 給節點添加污點taint

四.容忍度tolerations

4.1 容忍度tolerations概覽

4.2 設置容忍度tolerations

分享此文：

Related Posts

對話九陽股份資訊總監劉大勇：企業數智化轉型關鍵要素

Python-異常處理與調試

介面自動化測試用例如何設計

第2-1-3章 docker-compose安裝FastDFS,實現文件存儲服務

VirMach 便宜 VPS

QNews

熱門搜尋