前文我们了解了k8s上的kube-scheduler的工作方式,以及pod调度策略的定义;回顾请参考:https://www.cnblogs.com/qiuhom-1874/p/14243312.html;今天我们来聊一下k8s上的节点污点和pod容忍度相关话题;
节点污点是什么呢?
节点污点有点类似节点上的标签或注解信息,它们都是用来描述对应节点的元数据信息;污点定义的格式和标签、注解的定义方式很类似,都是用一个kv数据来表示,不同于节点标签,污点的键值数据中包含对应污点的effect,污点的effect是用于描述对应节点上的污点有什么作用;在k8s上污点有三个效用(effect),第一个效用是NoSchedule,表示拒绝pod调度到对应节点上运行;第二个效用是PreferSchedule,表示尽量不把pod调度到此节点上运行;第三个效用是NoExecute,表示拒绝将pod调度到此节点上运行;该效用相比NoSchedule要严苛一点;从上面的描述来看,对应污点就是来描述拒绝pod运行在对应节点的节点属性;
pod对节点污点的容忍度
从字面意思就能够理解,pod要想运行在对应有污点的节点上,对应pod就要容忍对应节点上的污点;我们把这种容忍节点污点的定义叫做pod对节点污点的容忍度;pod对节点污点的容忍度就是在对应pod中定义怎么去匹配节点污点;通常匹配节点污点的方式有两种,一种是等值匹配,一种是存在性匹配;所谓等值匹配表示对应pod的污点容忍度,必须和节点上的污点属性相等,所谓污点属性是指污点的key、value以及effect;即容忍度必须满足和对应污点的key,value和effect相同,这样表示等值匹配关系,其操作符为Equal;存在性匹配是指对应容忍度只需要匹配污点的key和effect即可,value不纳入匹配标准,即容忍度只要满足和对应污点的key和effect相同就表示能够容忍对应污点,其操作符为Exists;
节点污点和pod容忍度的关系
提示:如上图所示,只有能够容忍对应节点污点的pod才能够被调度到对应节点运行,不能容忍节点污点的pod是一定不能调度到对应节点上运行(除节点污点为PreferNoSchedule);
节点污点管理
给节点添加污点命令使用语法格式
Usage: kubectl taint NODE NAME KEY_1=VAL_1:TAINT_EFFECT_1 ... KEY_N=VAL_N:TAINT_EFFECT_N [options]
提示:给节点增加污点我们可以用kubectl taint node命令来增加节点污点,只需要指定对应节点名称和污点即可,污点可以指定多个,用空格隔开;
示例:给node01添加一个test=test:NoSchedule的污点
[root@master01 ~]# kubectl taint node node01.k8s.org test=test:NoSchedule node/node01.k8s.org tainted [root@master01 ~]#
查看节点污点
[root@master01 ~]# kubectl describe node node01.k8s.org |grep Taint Taints: test=test:NoSchedule [root@master01 ~]#
删除污点
[root@master01 ~]# kubectl describe node node01.k8s.org |grep Taint Taints: test=test:NoSchedule [root@master01 ~]# kubectl taint node node01.k8s.org test:NoSchedule- node/node01.k8s.org untainted [root@master01 ~]# kubectl describe node node01.k8s.org |grep Taint Taints: <none> [root@master01 ~]#
提示:删除污点可以指定对应节点上的污点的key和对应污点的effect,也可以直接在对应污点的key后面加“-”,表示删除对应名为对应key的所有污点;
pod容忍度定义
示例:创建一个pod,其容忍度为对应节点有 node-role.kubernetes.io/master:NoSchedule的污点
[root@master01 ~]# cat pod-demo-taints.yaml apiVersion: v1 kind: Pod Metadata: name: redis-demo labels: app: db spec: containers: - name: redis image: redis:4-alpine ports: - name: redis containerPort: 6379 tolerations: - key: node-role.kubernetes.io/master operator: Exists effect: NoSchedule [root@master01 ~]#
提示:定义pod对节点污点的容忍度需要用tolerations字段定义,该字段为一个列表对象;其中key是用来指定对应污点的key,这个key必须和对应节点污点上的key相等;operator字段用于指定对应的操作符,即描述容忍度怎么匹配污点,这个操作符只有两个,Equal和Exists;effect字段用于描述对应的效用,该字段的值通常有三个,NoSchedule、PreferNoSchedule、NoExecute;这个字段的值必须和对应的污点相同;上述清单表示,redis-demo这个pod能够容忍节点上有node-role.kubernetes.io/master:NoSchedule的污点;
应用清单
[root@master01 ~]# kubectl apply -f pod-demo-taints.yaml pod/redis-demo created [root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo 1/1 Running 0 7s 10.244.4.35 node04.k8s.org <none> <none> [root@master01 ~]#
提示:可以看到对应pod运行在node04上;这里需要注意,定义pod容忍度只是表示对应pod可以运行在对应有污点的节点上,并非它一定运行在对应节点上;它也可以运行在那些没有污点的节点上;
验证:删除pod,给node01,node02,03,04都打上test:NoSchedule的污点,再次应用清单,看看对应pod是否能够正常运行?
[root@master01 ~]# kubectl delete -f pod-demo-taints.yaml pod "redis-demo" deleted [root@master01 ~]# kubectl taint node node01.k8s.org test:NoSchedule node/node01.k8s.org tainted [root@master01 ~]# kubectl taint node node02.k8s.org test:NoSchedule node/node02.k8s.org tainted [root@master01 ~]# kubectl taint node node03.k8s.org test:NoSchedule node/node03.k8s.org tainted [root@master01 ~]# kubectl taint node node04.k8s.org test:NoSchedule node/node04.k8s.org tainted [root@master01 ~]# kubectl describe node node01.k8s.org |grep Taints Taints: test:NoSchedule [root@master01 ~]# kubectl describe node node02.k8s.org |grep Taints Taints: test:NoSchedule [root@master01 ~]# kubectl describe node node03.k8s.org |grep Taints Taints: test:NoSchedule [root@master01 ~]# kubectl describe node node04.k8s.org |grep Taints Taints: test:NoSchedule [root@master01 ~]# kubectl apply -f pod-demo-taints.yaml pod/redis-demo created [root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo 1/1 Running 0 18s 10.244.0.14 master01.k8s.org <none> <none> [root@master01 ~]#
提示:可以看到对应pod,被调度到master节点上运行了;其原因是对应pod能够容忍master节点上的污点;对应其他node节点上的污点,它并不能容忍,所以只能运行在master节点;
删除对应pod中容忍度的定义,再次应用pod清单,看看对应pod是否会正常运行?
[root@master01 ~]# kubectl delete pod redis-demo pod "redis-demo" deleted [root@master01 ~]# cat pod-demo-taints.yaml apiVersion: v1 kind: Pod Metadata: name: redis-demo labels: app: db spec: containers: - name: redis image: redis:4-alpine ports: - name: redis containerPort: 6379 [root@master01 ~]# kubectl apply -f pod-demo-taints.yaml pod/redis-demo created [root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo 0/1 Pending 0 6s <none> <none> <none> <none> [root@master01 ~]#
提示:可以看到对应pod处于pending状态;其原因是对应pod没法容忍对应节点污点;即所有节点都排斥对应pod运行在对应节点上;
示例:定义等值匹配关系污点容忍度
[root@master01 ~]# cat pod-demo-taints.yaml apiVersion: v1 kind: Pod Metadata: name: redis-demo labels: app: db spec: containers: - name: redis image: redis:4-alpine ports: - name: redis containerPort: 6379 tolerations: - key: test operator: Equal value: test effect: NoSchedule [root@master01 ~]#
提示:定义等值匹配关系的容忍度,需要指定对应污点中的value属性;
删除原有pod,应用清单
[root@master01 ~]# kubectl delete pod redis-demo pod "redis-demo" deleted [root@master01 ~]# kubectl apply -f pod-demo-taints.yaml pod/redis-demo created [root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo 0/1 Pending 0 4s <none> <none> <none> <none> [root@master01 ~]#
提示:可以看到应用对应清单以后,pod处于pending状态,其原因是没有满足对应pod容忍度的节点,所以对应pod无法正常调度到节点上运行;
验证:修改node01节点的污点为test=test:NoSchedule
[root@master01 ~]# kubectl describe node node01.k8s.org |grep Taints Taints: test:NoSchedule [root@master01 ~]# kubectl taint node node01.k8s.org test=test:NoSchedule --overwrite node/node01.k8s.org modified [root@master01 ~]# kubectl describe node node01.k8s.org |grep Taints Taints: test=test:NoSchedule [root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo 1/1 Running 0 4m46s 10.244.1.44 node01.k8s.org <none> <none> [root@master01 ~]#
提示:可以看到把node01的污点修改为test=test:NoSchedule以后,对应pod就被调度到node01上运行;
验证:修改node01节点上的污点为test:NoSchedule,看看对应pod是否被驱离呢?
[root@master01 ~]# kubectl taint node node01.k8s.org test:NoSchedule --overwrite node/node01.k8s.org modified [root@master01 ~]# kubectl describe node node01.k8s.org |grep Taints Taints: test:NoSchedule [root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo 1/1 Running 0 7m27s 10.244.1.44 node01.k8s.org <none> <none> [root@master01 ~]#
提示:可以看到对应节点污点修改为test:NoSchedule以后,对应pod也不会被驱离,说明效用为NoSchedule的污点只是在pod调度时起作用,对于调度完成的pod不起作用;
示例:定义pod容忍度为test:PreferNoSchedule
[root@master01 ~]# cat pod-demo-taints.yaml apiVersion: v1 kind: Pod Metadata: name: redis-demo1 labels: app: db spec: containers: - name: redis image: redis:4-alpine ports: - name: redis containerPort: 6379 tolerations: - key: test operator: Exists effect: PreferNoSchedule [root@master01 ~]#
应用清单
[root@master01 ~]# kubectl apply -f pod-demo-taints.yaml pod/redis-demo1 created [root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo 1/1 Running 0 11m 10.244.1.44 node01.k8s.org <none> <none> redis-demo1 0/1 Pending 0 6s <none> <none> <none> <none> [root@master01 ~]#
提示:可以看到对应pod处于pending状态,其原因是没有节点污点是test:PerferNoSchedule,所以对应pod不能被调度运行;
给node02节点添加test:PreferNoSchedule污点
[root@master01 ~]# kubectl describe node node02.k8s.org |grep Taints Taints: test:NoSchedule [root@master01 ~]# kubectl taint node node02.k8s.org test:PreferNoSchedule node/node02.k8s.org tainted [root@master01 ~]# kubectl describe node node02.k8s.org |grep -A 1 Taints Taints: test:NoSchedule test:PreferNoSchedule [root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo 1/1 Running 0 18m 10.244.1.44 node01.k8s.org <none> <none> redis-demo1 0/1 Pending 0 6m21s <none> <none> <none> <none> [root@master01 ~]#
提示:可以看到对应node02上有两个污点,对应pod也没有正常运行起来,其原因是node02上有一个test:NoSchedule污点,对应pod容忍度不能容忍此类污点;
验证:修改node01,node03,node04上的节点污点为test:PreferNoSchedule,修改pod的容忍度为test:NoSchedule,再次应用清单,看看对应pod怎么调度
[root@master01 ~]# kubectl taint node node01.k8s.org test:NoSchedule- node/node01.k8s.org untainted [root@master01 ~]# kubectl taint node node03.k8s.org test:NoSchedule- node/node03.k8s.org untainted [root@master01 ~]# kubectl taint node node04.k8s.org test:NoSchedule- node/node04.k8s.org untainted [root@master01 ~]# kubectl taint node node01.k8s.org test:PreferNoSchedule node/node01.k8s.org tainted [root@master01 ~]# kubectl taint node node03.k8s.org test:PreferNoSchedule node/node03.k8s.org tainted [root@master01 ~]# kubectl taint node node04.k8s.org test:PreferNoSchedule node/node04.k8s.org tainted [root@master01 ~]# kubectl describe node node01.k8s.org |grep -A 1 Taints Taints: test:PreferNoSchedule Unschedulable: false [root@master01 ~]# kubectl describe node node02.k8s.org |grep -A 1 Taints Taints: test:NoSchedule test:PreferNoSchedule [root@master01 ~]# kubectl describe node node03.k8s.org |grep -A 1 Taints Taints: test:PreferNoSchedule Unschedulable: false [root@master01 ~]# kubectl describe node node04.k8s.org |grep -A 1 Taints Taints: test:PreferNoSchedule Unschedulable: false [root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo 1/1 Running 0 31m 10.244.1.44 node01.k8s.org <none> <none> redis-demo1 1/1 Running 0 19m 10.244.1.45 node01.k8s.org <none> <none> [root@master01 ~]# kubectl delete pod --all pod "redis-demo" deleted pod "redis-demo1" deleted [root@master01 ~]# cat pod-demo-taints.yaml apiVersion: v1 kind: Pod Metadata: name: redis-demo1 labels: app: db spec: containers: - name: redis image: redis:4-alpine ports: - name: redis containerPort: 6379 tolerations: - key: test operator: Exists effect: NoSchedule [root@master01 ~]# kubectl apply -f pod-demo-taints.yaml pod/redis-demo1 created [root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo1 1/1 Running 0 5s 10.244.4.36 node04.k8s.org <none> <none> [root@master01 ~]#
提示:从上面的验证过程来看,当我们把node01,node03,node04节点上的污点删除以后,刚才创建的redis-demo1pod被调度到node01上运行了;其原因是node01上的污点第一个被删除;但我们把pod的容忍对修改成test:NoSchedule以后,再次应用清单,对应pod被调度到node04上运行;这意味着NoSchedule效用污点容忍度是可以正常容忍PreferNoSchedule污点;
示例:定义pod容忍度为test:NoExecute
[root@master01 ~]# cat pod-demo-taints.yaml apiVersion: v1 kind: Pod Metadata: name: redis-demo2 labels: app: db spec: containers: - name: redis image: redis:4-alpine ports: - name: redis containerPort: 6379 tolerations: - key: test operator: Exists effect: NoExecute [root@master01 ~]#
应用清单
[root@master01 ~]# kubectl apply -f pod-demo-taints.yaml pod/redis-demo2 created [root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo1 1/1 Running 0 35m 10.244.4.36 node04.k8s.org <none> <none> redis-demo2 1/1 Running 0 5s 10.244.4.38 node04.k8s.org <none> <none> [root@master01 ~]#
提示:可以看到对应pod被调度到node04上运行,说明容忍效用为NoExecute能够容忍污点效用为PreferNoSchedule的节点;
验证:更改所有node节点污点为test:NoSchedule,删除原有pod,再次应用清单,看看对应pod是否还会正常运行?
[root@master01 ~]# kubectl taint node node01.k8s.org test- node/node01.k8s.org untainted [root@master01 ~]# kubectl taint node node02.k8s.org test- node/node02.k8s.org untainted [root@master01 ~]# kubectl taint node node03.k8s.org test- node/node03.k8s.org untainted [root@master01 ~]# kubectl taint node node04.k8s.org test- node/node04.k8s.org untainted [root@master01 ~]# kubectl taint node node01.k8s.org test:NoSchedule node/node01.k8s.org tainted [root@master01 ~]# kubectl taint node node02.k8s.org test:NoSchedule node/node02.k8s.org tainted [root@master01 ~]# kubectl taint node node03.k8s.org test:NoSchedule node/node03.k8s.org tainted [root@master01 ~]# kubectl taint node node04.k8s.org test:NoSchedule node/node04.k8s.org tainted [root@master01 ~]# kubectl describe node node01.k8s.org |grep -A 1 Taints Taints: test:NoSchedule Unschedulable: false [root@master01 ~]# kubectl describe node node02.k8s.org |grep -A 1 Taints Taints: test:NoSchedule Unschedulable: false [root@master01 ~]# kubectl describe node node03.k8s.org |grep -A 1 Taints Taints: test:NoSchedule Unschedulable: false [root@master01 ~]# kubectl describe node node04.k8s.org |grep -A 1 Taints Taints: test:NoSchedule Unschedulable: false [root@master01 ~]# kubectl delete pod --all pod "redis-demo1" deleted pod "redis-demo2" deleted [root@master01 ~]# kubectl apply -f pod-demo-taints.yaml pod/redis-demo2 created [root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo2 0/1 Pending 0 6s <none> <none> <none> <none> [root@master01 ~]#
提示:可以看到对应pod处于pending状态,说明pod容忍效用为NoExecute,并不能容忍污点效用为NoSchedule;
删除pod,修改所有节点污点为test:NoExecute,把pod容忍度修改为NoScheudle,然后应用清单,看看对应pod怎么调度
[root@master01 ~]# kubectl delete pod --all pod "redis-demo2" deleted [root@master01 ~]# kubectl taint node node01.k8s.org test- node/node01.k8s.org untainted [root@master01 ~]# kubectl taint node node02.k8s.org test- node/node02.k8s.org untainted [root@master01 ~]# kubectl taint node node03.k8s.org test- node/node03.k8s.org untainted [root@master01 ~]# kubectl taint node node04.k8s.org test- node/node04.k8s.org untainted [root@master01 ~]# kubectl taint node node01.k8s.org test:NoExecute node/node01.k8s.org tainted [root@master01 ~]# kubectl taint node node02.k8s.org test:NoExecute node/node02.k8s.org tainted [root@master01 ~]# kubectl taint node node03.k8s.org test:NoExecute node/node03.k8s.org tainted [root@master01 ~]# kubectl taint node node04.k8s.org test:NoExecute node/node04.k8s.org tainted [root@master01 ~]# kubectl describe node node01.k8s.org |grep -A 1 Taints Taints: test:NoExecute Unschedulable: false [root@master01 ~]# kubectl describe node node02.k8s.org |grep -A 1 Taints Taints: test:NoExecute Unschedulable: false [root@master01 ~]# kubectl describe node node03.k8s.org |grep -A 1 Taints Taints: test:NoExecute Unschedulable: false [root@master01 ~]# kubectl describe node node04.k8s.org |grep -A 1 Taints Taints: test:NoExecute Unschedulable: false [root@master01 ~]# cat pod-demo-taints.yaml apiVersion: v1 kind: Pod Metadata: name: redis-demo2 labels: app: db spec: containers: - name: redis image: redis:4-alpine ports: - name: redis containerPort: 6379 tolerations: - key: test operator: Exists effect: NoSchedule [root@master01 ~]# kubectl apply -f pod-demo-taints.yaml pod/redis-demo2 created [root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo2 0/1 Pending 0 8s <none> <none> <none> <none> [root@master01 ~]#
提示:从上面的演示来看,pod容忍度效用为NoSchedule也不能容忍污点效用为NoExecute;
删除pod,修改对应pod的容忍度为test:NoExecute
[root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo2 0/1 Pending 0 5m5s <none> <none> <none> <none> [root@master01 ~]# kubectl delete pod --all pod "redis-demo2" deleted [root@master01 ~]# cat pod-demo-taints.yaml apiVersion: v1 kind: Pod Metadata: name: redis-demo2 labels: app: db spec: containers: - name: redis image: redis:4-alpine ports: - name: redis containerPort: 6379 tolerations: - key: test operator: Exists effect: NoExecute [root@master01 ~]# kubectl apply -f pod-demo-taints.yaml pod/redis-demo2 created [root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo2 1/1 Running 0 6s 10.244.4.43 node04.k8s.org <none> <none> [root@master01 ~]#
修改node04节点污点为test:NoSchedule,看看对应pod是否可以正常运行?
[root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo2 1/1 Running 0 4m38s 10.244.4.43 node04.k8s.org <none> <none> [root@master01 ~]# kubectl taint node node04.k8s.org test- node/node04.k8s.org untainted [root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo2 1/1 Running 0 8m2s 10.244.4.43 node04.k8s.org <none> <none> [root@master01 ~]# kubectl taint node node04.k8s.org test:NoSchedule node/node04.k8s.org tainted [root@master01 ~]# kubectl describe node node04.k8s.org |grep -A 1 Taints Taints: test:NoSchedule Unschedulable: false [root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo2 1/1 Running 0 8m25s 10.244.4.43 node04.k8s.org <none> <none> [root@master01 ~]#
提示:从NoExecute更改为NoSchedule,对原有pod不会进行驱离;
修改pod的容忍度为test:NoSchedule,再次应用清单
[root@master01 ~]# cat pod-demo-taints.yaml apiVersion: v1 kind: Pod Metadata: name: redis-demo3 labels: app: db spec: containers: - name: redis image: redis:4-alpine ports: - name: redis containerPort: 6379 tolerations: - key: test operator: Exists effect: NoSchedule --- apiVersion: v1 kind: Pod Metadata: name: redis-demo4 labels: app: db spec: containers: - name: redis image: redis:4-alpine ports: - name: redis containerPort: 6379 tolerations: - key: test operator: Exists effect: NoSchedule [root@master01 ~]# kubectl apply -f pod-demo-taints.yaml pod/redis-demo3 created pod/redis-demo4 created [root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo2 1/1 Running 0 14m 10.244.4.43 node04.k8s.org <none> <none> redis-demo3 1/1 Running 0 4s 10.244.4.45 node04.k8s.org <none> <none> redis-demo4 1/1 Running 0 4s 10.244.4.46 node04.k8s.org <none> <none> [root@master01 ~]#
提示:可以看到后面两个pod都被调度node04上运行;其原因是对应pod的容忍度test:NoSchedule只能容忍node04上的污点test:NoSchedule;
修改node04的污点为NoExecute,看看对应pod是否会被驱离?
[root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo2 1/1 Running 0 17m 10.244.4.43 node04.k8s.org <none> <none> redis-demo3 1/1 Running 0 2m32s 10.244.4.45 node04.k8s.org <none> <none> redis-demo4 1/1 Running 0 2m32s 10.244.4.46 node04.k8s.org <none> <none> [root@master01 ~]# kubectl describe node node04.k8s.org |grep -A 1 Taints Taints: test:NoSchedule Unschedulable: false [root@master01 ~]# kubectl taint node node04.k8s.org test- node/node04.k8s.org untainted [root@master01 ~]# kubectl taint node node04.k8s.org test:NoExecute node/node04.k8s.org tainted [root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo2 1/1 Running 0 18m 10.244.4.43 node04.k8s.org <none> <none> redis-demo3 0/1 Terminating 0 3m43s 10.244.4.45 node04.k8s.org <none> <none> redis-demo4 0/1 Terminating 0 3m43s 10.244.4.46 node04.k8s.org <none> <none> [root@master01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-demo2 1/1 Running 0 18m 10.244.4.43 node04.k8s.org <none> <none> [root@master01 ~]#
提示:可以看到修改node04的污点为test:NoExecute以后,对应pod容忍污点效用为不是NoExecute的pod被驱离了;说明污点效用为NoExecute,它会驱离不能容忍该污点效用的所有pod;
创建一个deploy,其指定容器的容忍度为test:NoExecute,并指定其驱离延迟施加为10秒
[root@master01 ~]# cat deploy-demo-taint.yaml apiVersion: apps/v1 kind: Deployment Metadata: name: deploy-demo spec: replicas: 3 selector: matchLabels: app: redis template: Metadata: labels: app: redis spec: containers: - name: redis image: redis:4-alpine ports: - name: redis containerPort: 6379 tolerations: - key: test operator: Exists effect: NoExecute tolerationSeconds: 10 [root@master01 ~]#
提示:tolerationSeconds字段用于指定其驱离宽限其时长;该字段只能用在其容忍污点效用为NoExecute的容忍度中使用;其他污点效用不能使用该字段来指定其容忍宽限时长;
应用配置清单
[root@master01 ~]# kubectl apply -f deploy-demo-taint.yaml deployment.apps/deploy-demo created [root@master01 ~]# kubectl get pods -o wide -w NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES deploy-demo-79b89f9847-9zk8j 1/1 Running 0 7s 10.244.2.71 node02.k8s.org <none> <none> deploy-demo-79b89f9847-h8zlc 1/1 Running 0 7s 10.244.3.61 node03.k8s.org <none> <none> deploy-demo-79b89f9847-shscr 1/1 Running 0 7s 10.244.1.62 node01.k8s.org <none> <none> redis-demo2 1/1 Running 0 54m 10.244.4.43 node04.k8s.org <none> <none> deploy-demo-79b89f9847-h8zlc 1/1 Terminating 0 10s 10.244.3.61 node03.k8s.org <none> <none> deploy-demo-79b89f9847-shscr 1/1 Terminating 0 10s 10.244.1.62 node01.k8s.org <none> <none> deploy-demo-79b89f9847-2x8w6 0/1 Pending 0 0s <none> <none> <none> <none> deploy-demo-79b89f9847-2x8w6 0/1 Pending 0 0s <none> node03.k8s.org <none> <none> deploy-demo-79b89f9847-lhltv 0/1 Pending 0 0s <none> <none> <none> <none> deploy-demo-79b89f9847-9zk8j 1/1 Terminating 0 10s 10.244.2.71 node02.k8s.org <none> <none> deploy-demo-79b89f9847-2x8w6 0/1 ContainerCreating 0 0s <none> node03.k8s.org <none> <none> deploy-demo-79b89f9847-lhltv 0/1 Pending 0 0s <none> node02.k8s.org <none> <none> deploy-demo-79b89f9847-lhltv 0/1 ContainerCreating 0 0s <none> node02.k8s.org <none> <none> deploy-demo-79b89f9847-w8xjw 0/1 Pending 0 0s <none> <none> <none> <none> deploy-demo-79b89f9847-w8xjw 0/1 Pending 0 0s <none> node01.k8s.org <none> <none> deploy-demo-79b89f9847-w8xjw 0/1 ContainerCreating 0 0s <none> node01.k8s.org <none> <none> deploy-demo-79b89f9847-shscr 1/1 Terminating 0 10s 10.244.1.62 node01.k8s.org <none> <none> deploy-demo-79b89f9847-h8zlc 1/1 Terminating 0 10s 10.244.3.61 node03.k8s.org <none> <none> deploy-demo-79b89f9847-9zk8j 1/1 Terminating 0 10s 10.244.2.71 node02.k8s.org <none> <none> deploy-demo-79b89f9847-shscr 0/1 Terminating 0 11s 10.244.1.62 node01.k8s.org <none> <none> deploy-demo-79b89f9847-2x8w6 0/1 ContainerCreating 0 1s <none> node03.k8s.org <none> <none> deploy-demo-79b89f9847-lhltv 0/1 ContainerCreating 0 1s <none> node02.k8s.org <none> <none> deploy-demo-79b89f9847-w8xjw 0/1 ContainerCreating 0 1s <none> node01.k8s.org <none> <none> deploy-demo-79b89f9847-h8zlc 0/1 Terminating 0 11s 10.244.3.61 node03.k8s.org <none> <none> deploy-demo-79b89f9847-2x8w6 1/1 Running 0 1s 10.244.3.62 node03.k8s.org <none> <none> deploy-demo-79b89f9847-9zk8j 0/1 Terminating 0 11s 10.244.2.71 node02.k8s.org <none> <none> deploy-demo-79b89f9847-lhltv 1/1 Running 0 1s 10.244.2.72 node02.k8s.org <none> <none> deploy-demo-79b89f9847-w8xjw 1/1 Running 0 2s 10.244.1.63 node01.k8s.org <none> <none> deploy-demo-79b89f9847-h8zlc 0/1 Terminating 0 15s 10.244.3.61 node03.k8s.org <none> <none> deploy-demo-79b89f9847-h8zlc 0/1 Terminating 0 15s 10.244.3.61 node03.k8s.org <none> <none> ^C[root@master01 ~]#
提示:可以看到对应pod只能在对应节点上运行10秒,随后就被驱离,因为我们创建的是一个deploy,对应pod被驱离以后,对应deploy又会重建;
总结:对于污点效用为NoSchedule来说,它只会拒绝新建的pod,不会对原有pod进行驱离;如果对应pod能够容忍该污点,则对应pod就有可能运行在对应节点上;如果不能容忍,则对应pod一定不会调度到对应节点运行;对于污点效用为PreferNoSchedule来说,它也不会驱离已存在pod,它只有在所有节点都不满足对应pod容忍度时,对应pod可以勉强运行在此类污点效用的节点上;对于污点效用为NoExecute来说,默认不指定其容忍宽限时长,表示能够一直容忍,如果指定了其宽限时长,则到了宽限时长对应pod将会被驱离;对应之前被调度到该节点上的pod,在节点污点效用变为NoExecute后,该节点会立即驱离所有不能容忍污点效用为NoExecute的pod;