Kubernetes 学习总结(3) M
- 2020 年 1 月 6 日
- 筆記
APIserver符合RESTful风格，支持GET/PUT/DELETE/POST等各种操作。所以也支持kubectl通过一系列命令对各处资源进行管理控制。 常用的资源 1)、workLoad(工作负载型资源，运行APP，对外提供服务): Pod/ReplicaSet/Deployment/ StatefulSet/ DaemonSet/ Job/ Cronjob / 2)、service discovery and Load Balance(服务发现及均衡型资源)：Service/ Ingress 3)、configuration and storage(配置与存储类型资源) ：Volume,CSI(容器存储接口，扩展第三方的存储) ConfigMap，Secret(特殊的配置类型资源) Downward API(配置类型资源) 4)、集群级资源(配置在名称空间级别)： namespace, node, role, clusterRole, roleBinding, clusterRoleBinding 5)、元数据类型资源：HPA、PodTemplate、limitRange（读取权限)
一、配置清单(configuration mainfest) kubectl 通常通过ymal格式的配置清单对资源进行管理控制。APIserver仅处理Json格式定义的资源定义。ymal格式的配置清单由APIserver自动将其转为json格式，而后再提交处理。 [root@docker79 ~]# kubectl get deployment NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE nginx-deploy 2 2 2 2 1d [root@docker79 ~]# kubectl get deployment nginx-deploy -o yaml (查看nginx-deploy的配置清单) 配置清单主格式如下 apiVersion: group/version (如果group省略表示core组) kind: Pod (资源类型) metadata: 元数据 spec: 期望状态 disired state status: 当前状态， current state ，本字段由kubernetes 集群维护;
[root@docker79 ~]# kubectl api-versions (查看默认支持的group与version)
admissionregistration.k8s.io/v1beta1 apiextensions.k8s.io/v1beta1 apiregistration.k8s.io/v1 apiregistration.k8s.io/v1beta1 apps/v1 apps/v1beta1 apps/v1beta2 authentication.k8s.io/v1 authentication.k8s.io/v1beta1 authorization.k8s.io/v1 authorization.k8s.io/v1beta1 autoscaling/v1 autoscaling/v2beta1 batch/v1 batch/v1beta1 certificates.k8s.io/v1beta1 events.k8s.io/v1beta1 extensions/v1beta1 networking.k8s.io/v1 policy/v1beta1 rbac.authorization.k8s.io/v1 rbac.authorization.k8s.io/v1beta1 scheduling.k8s.io/v1beta1 storage.k8s.io/v1 storage.k8s.io/v1beta1 v1
[root@docker79 ~]#
kubectl explain kubectl explain Resource_Kind[.Subitem…] 查看相关resource 在定义配置清单时的帮助信息。在帮助信息中常见格式如下： apiVersion <string> 表示字符串类型 metadata <Object> 表示需要嵌套多层字段 labels <map[string]string> 表示由k:v组成的映射 finalizers <[]string> 表示字串列表 ownerReferences <[]Object> 表示对象列表
资源清单 例1： [root@docker79 ~]# mkdir manifests [root@docker79 ~]# cd manifests/ [root@docker79 manifests]# vim pod-demo.yaml [root@docker79 manifests]# cat pod-demo.yaml
apiVersion: v1 kind: Pod metadata: name: pod-demo namespace: default labels: app: nginx tier: frontend spec: containers: - name: nginx image: nginx:1-alpine - name: busybox image: busybox:latest command: - "/bin/sh" - "-c" - "sleep 3600"
[root@docker79 manifests]# kubectl create -f pod-demo.yaml pod/pod-demo created [root@docker79 manifests]# kubectl get pods NAME READY STATUS RESTARTS AGE pod-demo 2/2 Running 0 8s [root@docker79 manifests]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE pod-demo 2/2 Running 0 17s 10.244.1.9 docker78 <none> [root@docker79 manifests]# [root@docker79 manifests]# kubectl logs pod-demo busybox [root@docker79 manifests]# kubectl logs pod-demo nginx [root@docker79 manifests]# kubectl exec -it pod-demo -c busybox — /bin/sh / # netstat -tnlp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN – / # exit [root@docker79 manifests]# kubectl delete -f pod-demo.yaml pod "pod-demo" deleted [root@docker79 manifests]#
再次修改资源清单，例2： [root@docker79 manifests]# vim pod-demo.yaml [root@docker79 manifests]# cat pod-demo.yaml
apiVersion: v1 kind: Pod metadata: name: pod-demo namespace: default labels: app: nginx tier: frontend spec: containers: - name: nginx image: nginx:1-alpine imagePullPolicy: IfNotPresent ports: - containerPort: 80 name: http - name: https containerPort: 443
[root@docker79 manifests]# kubectl apply -f pod-demo.yaml pod/pod-demo created [root@docker79 manifests]# kubectl get pods NAME READY STATUS RESTARTS AGE pod-demo 1/1 Running 0 6s [root@docker79 manifests]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE pod-demo 1/1 Running 0 14s 10.244.1.10 docker78 <none> [root@docker79 manifests]# 说明：imagePullPolicy: <string> 有三个值，分别是 Always ，Never，IfNotPresent （如果标签是latest，默认策略为Always) ，一项创建本项不能更新。 ports 选项可参考：kubectl explain pod.spec.containers.ports
二、Labels、Selector及 Annotation Lables 是 key=value 格式的键值对儿。key最大长度 63字符，只能以字母或数字开头，由字母、数字、-、_、. 组成。value 最大长度 63字符，可以为空。标签支持前缀，但不能超出253字符(一般都是name或FQDN) [root@docker79 manifests]# kubectl get pods –show-labels NAME READY STATUS RESTARTS AGE LABELS pod-demo 1/1 Running 0 14m app=nginx,tier=frontend [root@docker79 manifests]# kubectl get pods -L app NAME READY STATUS RESTARTS AGE APP pod-demo 1/1 Running 0 15m nginx [root@docker79 manifests]# kubectl get pods -l app –show-labels NAME READY STATUS RESTARTS AGE LABELS pod-demo 1/1 Running 0 15m app=nginx,tier=frontend [root@docker79 manifests]# kubectl label pods pod-demo release=canary 手工打标签 pod/pod-demo labeled [root@docker79 manifests]# kubectl get pods –show-labels NAME READY STATUS RESTARTS AGE LABELS pod-demo 1/1 Running 0 24m app=nginx,release=canary,tier=frontend [root@docker79 manifests]# kubectl label pods pod-demo release=canary2 –overwrite pod/pod-demo labeled [root@docker79 manifests]# kubectl get pods –show-labels NAME READY STATUS RESTARTS AGE LABELS pod-demo 1/1 Running 0 25m app=nginx,release=canary2,tier=frontend [root@docker79 manifests]#
Selector标签选择器 支持两类：1、等值关系的标签选择器 ； 2、集合关系的标签选择器。 等值关系： = , == , != 集合关系： in , notin [root@docker79 manifests]# kubectl get pods -l "release in (canary,beta,alpha)" NAME READY STATUS RESTARTS AGE pod-demo 1/1 Running 0 28m [root@docker79 manifests]#
标签选择器关联其它资源（如pod控制器和service) 。使用以下两个字段进行关联： matchLabels: 直接给定键值 matchExpressions: 基于给定的表达式来定义使用标签选择器 ,定义格式如 { key: "KEY", operator: "OPERATER", value: [ VAL1, VAL2, …]} 操作符： In , NotIn (values字段的值必须为非空列表） Exists , NotExists（values字段的值必须为空列表)
节点标签选择器 nodeSelector <map[string]string> 例：
[root@docker79 manifests]# kubectl get nodes --show-labels NAME STATUS ROLES AGE VERSION LABELS docker77 Ready <none> 2d v1.11.2 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=docker77 docker78 Ready <none> 2d v1.11.2 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=docker78 docker79 Ready master 2d v1.11.2 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=docker79,node-role.kubernetes.io/master= [root@docker79 manifests]# [root@docker79 manifests]# kubectl label node docker77 diskType=ssd node/docker77 labeled [root@docker79 manifests]# kubectl get nodes --show-labels NAME STATUS ROLES AGE VERSION LABELS docker77 Ready <none> 2d v1.11.2 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,diskType=ssd,kubernetes.io/hostname=docker77 docker78 Ready <none> 2d v1.11.2 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=docker78 docker79 Ready master 2d v1.11.2 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/hostname=docker79,node-role.kubernetes.io/master= [root@docker79 manifests]# [root@docker79 manifests]# kubectl delete -f pod-demo.yaml pod "pod-demo" deleted [root@docker79 manifests]# vim pod-demo.yaml [root@docker79 manifests]# cat pod-demo.yaml apiVersion: v1 kind: Pod metadata: name: pod-demo namespace: default labels: app: nginx tier: frontend spec: containers: - name: nginx image: nginx:1-alpine imagePullPolicy: IfNotPresent ports: - containerPort: 80 name: http - name: https containerPort: 443 nodeSelector: diskType: ssd [root@docker79 manifests]# [root@docker79 manifests]# kubectl apply -f pod-demo.yaml pod/pod-demo created [root@docker79 manifests]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE pod-demo 0/1 ContainerCreating 0 17s <none> docker77 <none> [root@docker79 manifests]#
annotations (资源注解) 与label不同之处在于： 它不能用于挑选资源对象，仅用于为对象提供“元数据”。 例：
[root@docker79 manifests]# kubectl delete -f pod-demo.yaml pod "pod-demo" deleted [root@docker79 manifests]# vim pod-demo.yaml [root@docker79 manifests]# cat pod-demo.yaml apiVersion: v1 kind: Pod metadata: name: pod-demo namespace: default labels: app: nginx tier: frontend annotations: inspiry.cn/author: "cluster admin" spec: containers: - name: nginx image: nginx:1-alpine imagePullPolicy: IfNotPresent ports: - containerPort: 80 name: http - name: https containerPort: 443 nodeSelector: diskType: ssd [root@docker79 manifests]# kubectl apply -f pod-demo.yaml pod/pod-demo created [root@docker79 manifests]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE pod-demo 1/1 Running 0 13s 10.244.2.9 docker77 <none> [root@docker79 manifests]# kubectl describe pod pod-demo Name: pod-demo Namespace: default Priority: 0 PriorityClassName: <none> Node: docker77/192.168.20.77 Start Time: Wed, 26 Sep 2018 16:25:44 +0800 Labels: app=nginx tier=frontend Annotations: inspiry.cn/author=cluster admin ......
三、pod生命周期 状态： Pending ， Running ，Failed， Successded ， Unknown Pod生命周期中的重要行为： 1) 初始化container 2) container探测：liveness probe（存活探测) 和 readiness probe (就绪探测) 以上两种探测使用的探针类型有：ExeAction、TCPSocketAction、HTTPGetAction 探针探测时的常用选项如下： failureThreshold 探测多少次宣告失败，默认3次 periodSeconds 每次间隔时长，默认10s timeoutSeconds 每次的超时时长，默认1s initialDelaySeconds 初始延时的时长，默认立即探测 探测结果的重启策略restartPolicy 有以下三种值： Always: 一旦故障，立即重启 Never: 不重启 OnFailure: 只有出现 failure状态时才重启 3) 启动后勾子和中止前勾子 lifecycle kubectl explain pods.spec.containers.lifecycle.postStart kubectl explain pods.spec.containers.lifecycle.preStop
例1-liveness：
[root@docker79 manifests]# vim liveness-exec.yaml [root@docker79 manifests]# cat liveness-exec.yaml apiVersion: v1 kind: Pod metadata: name: liveness-exec-pod namespace: default spec: containers: - name: liveness-exec-container image: busybox:latest imagePullPolicy: IfNotPresent command: ["/bin/sh", "-c", "touch /tmp/healthy; sleep 30; rm -f /tmp/healthy; sleep 3600" ] livenessProbe: exec: command: ["test", "-e" , "/tmp/health"] initialDelaySeconds: 1 periodSeconds: 3 restartPolicy: Never [root@docker79 manifests]# kubectl apply -f liveness-exec.yaml pod/liveness-exec-pod created [root@docker79 manifests]# kubectl get pods NAME READY STATUS RESTARTS AGE liveness-exec-pod 1/1 Running 0 6s pod-demo 1/1 Running 0 18m [root@docker79 manifests]# kubectl get pods NAME READY STATUS RESTARTS AGE liveness-exec-pod 0/1 Error 0 51s pod-demo 1/1 Running 0 19m [root@docker79 manifests]#
说明：container启动之后在30s之内 成功探测/tmp/health 文件存在，所以container status处于running状态；但30s之后/tmp/health文件被rm，导致探测/tmp/health文件不存在，所以container status处于Eroor 。
例2-liveness：
[root@docker79 manifests]# vim liveness-httpsocket.yaml [root@docker79 manifests]# cat liveness-httpsocket.yaml apiVersion: v1 kind: Pod metadata: name: liveness-httpget-pod namespace: default spec: containers: - name: liveness-httpget-container image: nginx:1-alpine imagePullPolicy: IfNotPresent ports: - name: http containerPort: 80 livenessProbe: httpGet: port: http path: /index.html initialDelaySeconds: 1 periodSeconds: 3 [root@docker79 manifests]# kubectl apply -f liveness-httpsocket.yaml pod/liveness-httpget-pod created [root@docker79 manifests]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE liveness-httpget-pod 1/1 Running 0 8s 10.244.1.13 docker78 <none> pod-demo 1/1 Running 0 2h 10.244.2.9 docker77 <none> [root@docker79 manifests]# [root@docker79 manifests]# kubectl exec -it liveness-httpget-pod -- /bin/sh / # ls /usr/share/nginx/html/ 50x.html index.html / # rm /usr/share/nginx/html/index.html -f / # exit [root@docker79 manifests]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE liveness-httpget-pod 1/1 Running 1 2m 10.244.1.13 docker78 <none> pod-demo 1/1 Running 0 2h 10.244.2.9 docker77 <none> [root@docker79 manifests]#
说明：当httpGetAction探针探测到无法访问 /index.html 时，立即执行restartPolicy重启策略，该策略默认值always，所以可以看到 pod 的 restarts 为1.
例3-readiness：
[root@docker79 manifests]# vim readiness-httpget.yaml [root@docker79 manifests]# cat readiness-httpget.yaml apiVersion: v1 kind: Pod metadata: name: readiness-httpget-pod namespace: default spec: containers: - name: readiness-httpget-container image: nginx:1-alpine imagePullPolicy: IfNotPresent ports: - name: http containerPort: 80 readinessProbe: httpGet: port: http path: /index.html initialDelaySeconds: 1 periodSeconds: 3 [root@docker79 manifests]# kubectl apply -f readiness-httpget.yaml pod/readiness-httpget-pod created [root@docker79 manifests]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE pod-demo 1/1 Running 0 3h 10.244.2.9 docker77 <none> readiness-httpget-pod 1/1 Running 0 9s 10.244.1.14 docker78 <none> [root@docker79 manifests]# kubectl exec -it readiness-httpget-pod -- /bin/sh / # rm -f /usr/share/nginx/html/index.html / # [root@docker79 manifests]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE pod-demo 1/1 Running 0 3h 10.244.2.9 docker77 <none> readiness-httpget-pod 0/1 Running 0 1m 10.244.1.14 docker78 <none> [root@docker79 manifests]# kubectl exec -it readiness-httpget-pod -- /bin/sh / # echo readiness > /usr/share/nginx/html/index.html / # [root@docker79 manifests]# [root@docker79 manifests]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE pod-demo 1/1 Running 0 3h 10.244.2.9 docker77 <none> readiness-httpget-pod 1/1 Running 0 2m 10.244.1.14 docker78 <none> [root@docker79 manifests]#
说明：就绪探测与service 调度有紧密关联，只有就绪的pod才能被调度。例3中当删除/usr/share/nginx/html/index.html之后，探针无法获取/index.html，所以pod的READY状态处于0/1状态；当再次创建/usr/share/nginx/html/index.html之后，探针有效探测到/index.html之后，pod的READY状态处于1/1状态。
例4-lifecycle：
[root@docker79 manifests]# vim lifecycle-poststart.yaml [root@docker79 manifests]# cat lifecycle-poststart.yaml apiVersion: v1 kind: Pod metadata: name: lifecycle-poststart-pod namespace: default spec: containers: - name: lifecycle-poststart-container image: busybox:latest imagePullPolicy: IfNotPresent lifecycle: postStart: exec: command: ["mkdir","-p","/var/www/inspiry"] command: ["/bin/sh","-c","sleep 60; /bin/httpd -f -h /var/www/inspiry"] [root@docker79 manifests]# kubectl apply -f lifecycle-poststart.yaml pod/lifecycle-poststart-pod created [root@docker79 manifests]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE lifecycle-poststart-pod 1/1 Running 0 9s 10.244.1.15 docker78 <none> [root@docker79 manifests]# kubectl exec -it lifecycle-poststart-pod -- /bin/sh / # ls /var/www/ inspiry / # netstat -tnlp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 :::80 :::* LISTEN 1/httpd / # exit [root@docker79 manifests]#
说明：例4中使用lifecycle.postStart.exec 表示pod中container启动后执行的命令.
