Centos下使用containerd管理容器：5分鐘從docker轉型到containerd
- 2022 年 9 月 11 日
- 筆記
- containerd容器, Docker容器
目錄
一.系統環境
| 服務器版本 | containerd軟件版本 | CPU架構 |
|---|---|---|
| CentOS Linux release 7.4.1708 (Core) | 1.6.8 | x86_64 |
二.前言
在 Kubernetes 的早期，我們只支持一個容器運行時。那個運行時是 Docker 引擎。那時，並沒有太多其他的選擇，而 Docker 是處理容器的主要工具，所以這不是一個有爭議的選擇。最終，我們開始添加更多的容器運行時，比如 rkt 和 hypernetes，很明顯 Kubernetes 用戶希望選擇最適合他們的運行時。因此，Kubernetes 需要一種方法來允許集群操作員靈活地使用他們選擇的任何運行時。
發佈了容器運行時接口(CRI) 以實現這種靈活性。CRI 的引入對項目和用戶來說都很棒，但它確實引入了一個問題：Docker Engine 作為容器運行時的使用早於 CRI，並且 Docker Engine 不兼容 CRI。為了解決這個問題，引入了一個小型軟件 shim (dockershim) 作為 kubelet 組件的一部分，專門用於填補 Docker Engine 和 CRI 之間的空白，允許集群操作員繼續使用 Docker Engine 作為他們的容器運行時基本上不間斷。
然而，這個小軟件墊片從來沒有打算成為一個永久的解決方案。多年來，它的存在給 kubelet 本身帶來了許多不必要的複雜性。由於這個 shim，Docker 的一些集成實現不一致，導致維護人員的負擔增加，並且維護供應商特定的代碼不符合我們的開源理念。為了減少這種維護負擔並朝着支持開放標準的更具協作性的社區邁進，引入了 KEP-2221，建議刪除 dockershim。隨着 Kubernetes v1.20 的發佈，正式棄用。
自 Kubernetes v1.24 起，Dockershim 已被刪除，這對項目來說是一個積極的舉措。
自 Kubernetes v1.24 起，K8S默認使用containerd作為容器運行時！可以理解為containerd是docker的替代品。
好消息是containerd用法和docker類似，轉型相當簡單！
三.containerd
containerd 是一個行業標準的容器運行時，強調簡單性、健壯性和可移植性。它可作為 Linux 和 Windows 的守護進程使用，可以管理其主機系統的完整容器生命周期：圖像傳輸和存儲、容器執行和監督、低級存儲和網絡附件等。containerd 旨在嵌入到更大的系統中，而不是由開發人員或最終用戶直接使用。
containerd是從Docker中分離出來的一個項目，可以作為一個底層容器運行時，現在它成了Kubernete容器運行時更好的選擇。docker 對容器的管理和操作基本都是通過 containerd 完成的
K8S發佈CRI（Container Runtime Interface），統一了容器運行時接口，凡是支持CRI的容器運行時，皆可作為K8S的底層容器運行時。
如果你使用Docker作為K8S容器運行時的話，kubelet需要先要通過dockershim去調用Docker，再通過Docker去調用containerd。
如果你使用containerd作為K8S容器運行時的話，由於containerd內置了CRI插件，kubelet可以直接調用containerd。使用containerd不僅性能提高了（調用鏈變短了），而且資源佔用也會變小（Docker不是一個純粹的容器運行時，具有大量其他功能）。
四.部署containerd
4.1 安裝containerd
安裝containerd
[root@k8sworker2 ~]# yum -y install containerd.io cri-tools
已加載插件：fastestmirror
base | 3.6 kB 00:00:00
......
驗證中 : containerd.io-1.4.12-3.1.el7.x86_64 4/4
更新完畢:
containerd.io.x86_64 0:1.6.8-3.1.el7 cri-tools.x86_64 0:1.24.2-0
完畢！
設置containerd開機自啟動並啟動containerd
[root@k8sworker2 ~]# systemctl enable containerd --now
Created symlink from /etc/systemd/system/multi-user.target.wants/containerd.service to /usr/lib/systemd/system/containerd.service.
[root@k8sworker2 ~]# systemctl status containerd
● containerd.service - containerd container runtime
Loaded: loaded (/usr/lib/systemd/system/containerd.service; enabled; vendor preset: disabled)
Active: active (running) since 四 2022-01-06 00:23:13 CST; 1min 4s ago
Docs: //containerd.io
Main PID: 88228 (containerd)
CGroup: /system.slice/containerd.service
└─88228 /usr/bin/containerd
4.2 containerd配置文件
containerd的配置文件為/etc/containerd/config.toml
[root@k8sworker2 ~]# ll -h /etc/containerd/config.toml
-rw-r--r-- 1 root root 886 11月 18 04:15 /etc/containerd/config.toml
containerd的默認配置文件/etc/containerd/config.toml 內容如下
[root@k8sworker2 ~]# cat /etc/containerd/config.toml
# Copyright 2018-2020 Docker Inc.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# //www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
disabled_plugins = ["cri"]
#root = "/var/lib/containerd"
#state = "/run/containerd"
#subreaper = true
#oom_score = 0
#[grpc]
# address = "/run/containerd/containerd.sock"
# uid = 0
# gid = 0
#[debug]
# address = "/run/containerd/debug.sock"
# uid = 0
# gid = 0
# level = "info"
可以使用containerd config default > /etc/containerd/config.toml生成默認的配置文件，這個命令生成的配置文件內容很多
[root@k8sworker2 ~]# containerd config default
version = 2
root = "/var/lib/containerd"
state = "/run/containerd"
plugin_dir = ""
disabled_plugins = []
required_plugins = []
oom_score = 0
[grpc]
address = "/run/containerd/containerd.sock"
tcp_address = ""
tcp_tls_cert = ""
tcp_tls_key = ""
uid = 0
gid = 0
max_recv_message_size = 16777216
max_send_message_size = 16777216
[ttrpc]
address = ""
uid = 0
gid = 0
[debug]
address = ""
uid = 0
gid = 0
level = ""
[metrics]
address = ""
grpc_histogram = false
[cgroup]
path = ""
[timeouts]
"io.containerd.timeout.shim.cleanup" = "5s"
"io.containerd.timeout.shim.load" = "5s"
"io.containerd.timeout.shim.shutdown" = "3s"
"io.containerd.timeout.task.state" = "2s"
[plugins]
[plugins."io.containerd.gc.v1.scheduler"]
pause_threshold = 0.02
deletion_threshold = 0
mutation_threshold = 100
schedule_delay = "0s"
startup_delay = "100ms"
[plugins."io.containerd.grpc.v1.cri"]
disable_tcp_service = true
stream_server_address = "127.0.0.1"
stream_server_port = "0"
stream_idle_timeout = "4h0m0s"
enable_selinux = false
selinux_category_range = 1024
sandbox_image = "k8s.gcr.io/pause:3.2"
stats_collect_period = 10
systemd_cgroup = false
enable_tls_streaming = false
max_container_log_line_size = 16384
disable_cgroup = false
disable_apparmor = false
restrict_oom_score_adj = false
max_concurrent_downloads = 3
disable_proc_mount = false
unset_seccomp_profile = ""
tolerate_missing_hugetlb_controller = true
disable_hugetlb_controller = true
ignore_image_defined_volumes = false
[plugins."io.containerd.grpc.v1.cri".containerd]
snapshotter = "overlayfs"
default_runtime_name = "runc"
no_pivot = false
disable_snapshot_annotations = true
discard_unpacked_layers = false
[plugins."io.containerd.grpc.v1.cri".containerd.default_runtime]
runtime_type = ""
runtime_engine = ""
runtime_root = ""
privileged_without_host_devices = false
base_runtime_spec = ""
[plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime]
runtime_type = ""
runtime_engine = ""
runtime_root = ""
privileged_without_host_devices = false
base_runtime_spec = ""
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes]
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
runtime_type = "io.containerd.runc.v2"
runtime_engine = ""
runtime_root = ""
privileged_without_host_devices = false
base_runtime_spec = ""
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
[plugins."io.containerd.grpc.v1.cri".cni]
bin_dir = "/opt/cni/bin"
conf_dir = "/etc/cni/net.d"
max_conf_num = 1
conf_template = ""
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["//registry-1.docker.io"]
[plugins."io.containerd.grpc.v1.cri".image_decryption]
key_model = ""
[plugins."io.containerd.grpc.v1.cri".x509_key_pair_streaming]
tls_cert_file = ""
tls_key_file = ""
[plugins."io.containerd.internal.v1.opt"]
path = "/opt/containerd"
[plugins."io.containerd.internal.v1.restart"]
interval = "10s"
[plugins."io.containerd.metadata.v1.bolt"]
content_sharing_policy = "shared"
[plugins."io.containerd.monitor.v1.cgroups"]
no_prometheus = false
[plugins."io.containerd.runtime.v1.linux"]
shim = "containerd-shim"
runtime = "runc"
runtime_root = ""
no_shim = false
shim_debug = false
[plugins."io.containerd.runtime.v2.task"]
platforms = ["linux/amd64"]
[plugins."io.containerd.service.v1.diff-service"]
default = ["walking"]
[plugins."io.containerd.snapshotter.v1.devmapper"]
root_path = ""
pool_name = ""
base_image_size = ""
async_remove = false
查看默認的配置
[root@k8sworker2 ~]# containerd config dump
version = 2
root = "/var/lib/containerd"
state = "/run/containerd"
plugin_dir = ""
disabled_plugins = ["cri"]
required_plugins = []
oom_score = 0
imports = ["/etc/containerd/config.toml"]
[grpc]
address = "/run/containerd/containerd.sock"
tcp_address = ""
tcp_tls_cert = ""
tcp_tls_key = ""
uid = 0
gid = 0
max_recv_message_size = 16777216
max_send_message_size = 16777216
[ttrpc]
address = ""
uid = 0
gid = 0
[debug]
address = ""
uid = 0
gid = 0
level = ""
[metrics]
address = ""
grpc_histogram = false
[cgroup]
path = ""
[timeouts]
"io.containerd.timeout.shim.cleanup" = "5s"
"io.containerd.timeout.shim.load" = "5s"
"io.containerd.timeout.shim.shutdown" = "3s"
"io.containerd.timeout.task.state" = "2s"
[plugins]
[plugins."io.containerd.gc.v1.scheduler"]
pause_threshold = 0.02
deletion_threshold = 0
mutation_threshold = 100
schedule_delay = "0s"
startup_delay = "100ms"
[plugins."io.containerd.internal.v1.opt"]
path = "/opt/containerd"
[plugins."io.containerd.internal.v1.restart"]
interval = "10s"
[plugins."io.containerd.metadata.v1.bolt"]
content_sharing_policy = "shared"
[plugins."io.containerd.monitor.v1.cgroups"]
no_prometheus = false
[plugins."io.containerd.runtime.v1.linux"]
shim = "containerd-shim"
runtime = "runc"
runtime_root = ""
no_shim = false
shim_debug = false
[plugins."io.containerd.runtime.v2.task"]
platforms = ["linux/amd64"]
[plugins."io.containerd.service.v1.diff-service"]
default = ["walking"]
[plugins."io.containerd.snapshotter.v1.devmapper"]
root_path = ""
pool_name = ""
base_image_size = ""
async_remove = false
4.3 配置containerd阿里雲鏡像加速器
修改配置文件，添加鏡像加速器
[root@k8sworker2 ~]# cat /etc/containerd/config.toml
disabled_plugins = ["restart"]
[plugins]
[plugins.cri.registry.mirrors."docker.io"]
endpoint = ["//frz7i079.mirror.aliyuncs.com"]
重啟containerd
[root@k8sworker2 ~]# systemctl restart containerd
[root@k8sworker2 ~]# systemctl status containerd
● containerd.service - containerd container runtime
Loaded: loaded (/usr/lib/systemd/system/containerd.service; enabled; vendor preset: disabled)
Active: active (running) since 四 2022-01-06 00:34:18 CST; 6s ago
Docs: //containerd.io
Process: 90199 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
Main PID: 90201 (containerd)
Memory: 19.6M
CGroup: /system.slice/containerd.service
└─90201 /usr/bin/containerd
五.管理containerd鏡像/容器
5.1 使用ctr管理容器
ctr是containerd自帶的用於管理容器的命令行工具，雖然該ctr工具與 containerd 捆綁在一起，但應注意該ctr工具僅用於調試 containerd，用戶體驗並不好，不推薦使用。
查看命名空間
[root@k8sworker2 ~]# ctr ns list
NAME LABELS
moby
查看鏡像
[root@k8sworker2 ~]# ctr i list
REF TYPE DIGEST SIZE PLATFORMS LABELS
5.2 使用nerdctl管理容器
5.2.1 安裝nerdctl
因為containerd自帶的命令行管理工具用戶體驗不好，所以Containerd 官方推出了一個新的 CLI 叫 nerdctl。nerdctl 的使用體驗和 docker 一樣順滑，使用最廣，推薦使用。
nerdctl是containerd的cli客戶端工具,與docker cli語法大部分兼容，用法類似。需要兩個安裝包nerdctl-0.15.0-linux-amd64.tar.gz和cni-plugins-linux-amd64-v1.0.1.tgz ，兩個安裝包下載地址如下：
- nerdctl-0.15.0-linux-amd64.tar.gz下載地址：//github.com/containerd/nerdctl/releases
- 網絡插件cni-plugins-linux-amd64-v1.0.1.tgz 下載地址：//github.com/containernetworking/plugins/releases
[root@k8sworker2 ~]# ll -h nerdctl-0.15.0-linux-amd64.tar.gz
-rw-r--r-- 1 root root 9.6M 1月 6 00:45 nerdctl-0.15.0-linux-amd64.tar.gz
[root@k8sworker2 ~]# ll -h cni-plugins-linux-amd64-v1.0.1.tgz
-rw-r--r-- 1 root root 36M 1月 6 00:59 cni-plugins-linux-amd64-v1.0.1.tgz
分別進行解壓
[root@k8sworker2 ~]# tar xf nerdctl-0.15.0-linux-amd64.tar.gz -C /usr/local/bin/
[root@k8sworker2 ~]# ls /usr/local/bin/
containerd-rootless-setuptool.sh containerd-rootless.sh nerdctl
[root@k8sworker2 ~]# mkdir -p /opt/cni/bin
[root@k8sworker2 ~]# tar xf cni-plugins-linux-amd64-v1.0.1.tgz -C /opt/cni/bin/
[root@k8sworker2 ~]# ls /opt/cni/bin/
bandwidth bridge dhcp firewall host-device host-local ipvlan loopback macvlan portmap ptp sbr static tuning vlan vrf
5.2.2 配置nerdctl命令tab鍵自動補全
先安裝必要的依賴包
[root@k8sworker2 ~]# yum install -y epel-release bash-completion
[root@k8sworker2 ~]# source /usr/share/bash-completion/bash_completion
修改配置文件
[root@k8sworker2 ~]# source <(nerdctl completion bash)
#添加source <(nerdctl completion bash)到/etc/profile里
[root@k8sworker2 ~]# cat /etc/profile | head -3
# /etc/profile
source <(nerdctl completion bash)
使/etc/profile文件生效
[root@k8sworker2 buildtest]# source /etc/profile
此時nerdctl r按tab鍵就可以自動補全了
[root@k8sworker2 ~]# nerdctl r
restart (Restart one or more running containers)
rmi (Remove one or more images)
rm (Remove one or more containers)
run (Run a command in a new container. Optionally specify "ipfs://" or "ipns://" scheme to pull image from IPFS.)
5.2.3 nerdctl常用命令
nerdctl的命令和docker命令類似
#查看nerdctl版本
nerdctl version
#nerdctl run運行容器
nerdctl run -d -p 80:80 --name=nginx --restart=always nginx
#執行容器
nerdctl exec -it nginx /bin/sh
#列出正在運行的容器
nerdctl ps
#獲取容器的詳細信息
nerdctl inspect nginx
#獲取容器日誌
nerdctl logs -f nginx
#停止容器
nerdctl stop nginx
#刪除容器
nerdctl rm -f nginx
#鏡像列表
nerdctl images
#登錄鏡像倉庫
nerdctl login --username xxx --password xxx ip/主機名
nerdctl login --username xxx --password xxx harbor.k8s.local
#註銷退出登錄
nerdctl logout
#拉取鏡像
nerdctl pull busybox
#推送鏡像
nerdctl push harbor.k8s.local/course/nginx:alpine
#給鏡像起別名
nerdctl tag nginx:alpine harbor.k8s.local/course/nginx:alpine
#導出鏡像為tar包
nerdctl save -o busybox.tar busybox:latest
nerdctl save hub.c.163.com/library/nginx:latest >nginx.tar
#刪除鏡像
nerdctl rmi busybox
#導入鏡像
nerdctl load -i busybox.tar.gz
#查看數據卷
[root@k8sworker2 ~]# nerdctl volume list
VOLUME NAME DIRECTORY
#查看網絡
[root@k8sworker2 ~]# nerdctl network list
NETWORK ID NAME FILE
0 bridge
k8s-pod-network /etc/cni/net.d/10-calico.conflist
host
none
#查看命名空間
[root@k8sworker2 ~]# nerdctl namespace list
NAME CONTAINERS IMAGES VOLUMES
buildkit 0 0 0
default 2 2 0
moby 0 0 0
5.2.4 使用nerdctl管理容器
查看鏡像
[root@k8sworker2 ~]# nerdctl images
REPOSITORY TAG IMAGE ID CREATED PLATFORM SIZE
拉取nginx鏡像
[root@k8sworker2 ~]# nerdctl pull nginx
docker.io/library/nginx:latest: resolved |++++++++++++++++++++++++++++++++++++++|
docker.io/library/nginx:latest: resolved
......
|++++++++++++++++++++++++++++++++++++++|
layer-sha256:a9edb18cadd1336142d6567ebee31be2a03c0905eeefe26cb150de7b0fbc520b: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:b4df32aa5a72e2a4316aad3414508ccd907d87b4ad177abd7cbd62fa4dab2a2f: done |++++++++++++++++++++++++++++++++++++++|
elapsed: 30.9s total: 54.1 M (1.8 MiB/s)
[root@k8sworker2 ~]# nerdctl images
REPOSITORY TAG IMAGE ID CREATED PLATFORM SIZE
nginx latest 0d17b565c37b 40 seconds ago linux/amd64 146.2 MiB
查看容器
#沒有容器運行
[root@k8sworker2 ~]# nerdctl ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
使用Nginx鏡像創建一個容器，-p 80:80端口映射
[root@k8sworker2 ~]# nerdctl run -d --name=nginx --restart=always -p 80:80 nginx
5d1c3f1ca77f17d77f75895c29b6ce491d1921e61ad4f59f4ace3046436d10d4
[root@k8sworker2 ~]# nerdctl ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5d1c3f1ca77f docker.io/library/nginx:latest "/docker-entrypoint.…" 17 seconds ago Up 0.0.0.0:80->80/tcp nginx
訪問nginx容器：物理機IP:80端口
[root@k8sworker2 ~]# curl //192.168.110.139:80
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="//nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="//nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
查看nerdctl信息
[root@k8sworker2 ~]# nerdctl info
Client:
Namespace: default
Debug Mode: false
Server:
Server Version: 1.4.12
Storage Driver: overlayfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Log: json-file
Storage: native overlayfs
Security Options:
seccomp
Profile: default
Kernel Version: 3.10.0-693.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 2.371GiB
Name: k8sworker2
ID: 160b58af-90bf-420f-99d3-ef2d4b9305ef
六.使用nerdctl構建鏡像
nerdctl構建鏡像和docker build類似，docker構建自定義鏡像可以查看博客《構建自定義鏡像並優化dockerfile文件》//www.cnblogs.com/renshengdezheli/p/16645144.html
注意：使用nerdctl構建鏡像需要提前安裝buildkit
配置Dockerfile文件，此Dockerfile文件的意思是構建一個可以ssh的centos鏡像，Dockerfile文件的具體解釋請查看博客《構建自定義鏡像並優化dockerfile文件》//www.cnblogs.com/renshengdezheli/p/16645144.html
[root@k8sworker2 ~]# cat Dockerfile_ssh2
FROM hub.c.163.com/library/centos:latest
MAINTAINER LZ
RUN yum -y install openssh-server openssh-clients && \
ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key && \
ssh-keygen -t ecdsa -f /etc/ssh/ssh_host_ecdsa_key && \
ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key && \
echo qweasd | passwd --stdin root && \
sed -i '/UseDNS/cUseDNS no' /etc/ssh/sshd_config && \
yum -y install net-tools && \
yum -y install iproute && \
yum clean all
EXPOSE 22
CMD ["/usr/sbin/sshd","-D"]
構建鏡像
#構建鏡像失敗，報錯依賴於buildkit
[root@k8sworker2 ~]# nerdctl build -t centos:ssh . -f Dockerfile_ssh2
FATA[0000] `buildctl` needs to be installed and `buildkitd` needs to be running, see //github.com/moby/buildkit: exec: "buildctl": executable file not found in $PATH
6.1 安裝buildkit
為了能使用nerdctl構建鏡像，需要先安裝buildkit
buildkit-v0.9.3.linux-amd64.tar.gz下載網址：//github.com/moby/buildkit/releases/tag/v0.9.3
準備好buildkit安裝包並解壓
[root@k8sworker2 ~]# ll -h buildkit-v0.9.3.linux-amd64.tar.gz
-rw-r--r-- 1 root root 46M 1月 6 22:11 buildkit-v0.9.3.linux-amd64.tar.gz
#解壓
[root@k8sworker2 ~]# tar xf buildkit-v0.9.3.linux-amd64.tar.gz -C /usr/local/bin/
[root@k8sworker2 ~]# ls /usr/local/bin/
bin containerd-rootless-setuptool.sh containerd-rootless.sh nerdctl
[root@k8sworker2 ~]# ls /usr/local/bin/bin/
buildctl buildkit-qemu-aarch64 buildkit-qemu-i386 buildkit-qemu-mips64el buildkit-qemu-riscv64 buildkit-runc
buildkitd buildkit-qemu-arm buildkit-qemu-mips64 buildkit-qemu-ppc64le buildkit-qemu-s390x
[root@k8sworker2 ~]# mv /usr/local/bin/bin/* /usr/local/bin/
[root@k8sworker2 ~]# ls /usr/local/bin/
bin buildkitd buildkit-qemu-arm buildkit-qemu-mips64 buildkit-qemu-ppc64le buildkit-qemu-s390x containerd-rootless-setuptool.sh nerdctl
buildctl buildkit-qemu-aarch64 buildkit-qemu-i386 buildkit-qemu-mips64el buildkit-qemu-riscv64 buildkit-runc containerd-rootless.sh
配置buildkit使用systemd啟動
[root@k8sworker2 buildtest]# cat /etc/systemd/system/buildkit.service
[Unit]
Description=BuildKit
Documentation=//github.com/moby/buildkit
[Service]
ExecStart=/usr/local/bin/buildkitd --oci-worker=false --containerd-worker=true
[Install]
WantedBy=multi-user.target
確保/usr/local/bin/buildkitd這個可執行文件存在
[root@k8sworker2 ~]# ls /usr/local/bin/buildkitd
/usr/local/bin/buildkitd
重啟加載配置文件
[root@k8sworker2 ~]# systemctl daemon-reload
配置buildkit開機自啟動並現在啟動buildkit
[root@k8sworker2 ~]# systemctl enable buildkit --now
Created symlink from /etc/systemd/system/multi-user.target.wants/buildkit.service to /etc/systemd/system/buildkit.service.
[root@k8sworker2 ~]# systemctl status buildkit
● buildkit.service - BuildKit
Loaded: loaded (/etc/systemd/system/buildkit.service; enabled; vendor preset: disabled)
Active: active (running) since 四 2022-01-06 22:18:45 CST; 9s ago
Docs: //github.com/moby/buildkit
Main PID: 93007 (buildkitd)
Memory: 10.6M
CGroup: /system.slice/buildkit.service
└─93007 /usr/local/bin/buildkitd --oci-worker=false --containerd-worker=true
可以查看buildkit的日誌
[root@k8sworker2 ~]# journalctl -u buildkit
-- Logs begin at 三 2022-01-05 10:28:14 CST, end at 四 2022-01-06 22:19:31 CST. --
1月 06 22:18:45 k8sworker2 systemd[1]: Started BuildKit.
1月 06 22:18:45 k8sworker2 systemd[1]: Starting BuildKit...
1月 06 22:18:45 k8sworker2 buildkitd[93007]: time="2022-01-06T22:18:45+08:00" level=warning msg="using host network as the default"
1月 06 22:18:45 k8sworker2 buildkitd[93007]: time="2022-01-06T22:18:45+08:00" level=warning msg="git source cannot be enabled: failed to find git binary: exec: \"git\
1月 06 22:18:45 k8sworker2 buildkitd[93007]: time="2022-01-06T22:18:45+08:00" level=info msg="found worker \"bb19mi8dhi1sk5r4qn6wxic8e\", labels=map[org.mobyproject.b
1月 06 22:18:45 k8sworker2 buildkitd[93007]: time="2022-01-06T22:18:45+08:00" level=info msg="found 1 workers, default=\"bb19mi8dhi1sk5r4qn6wxic8e\""
1月 06 22:18:45 k8sworker2 buildkitd[93007]: time="2022-01-06T22:18:45+08:00" level=warning msg="currently, only the default worker can be used."
1月 06 22:18:45 k8sworker2 buildkitd[93007]: time="2022-01-06T22:18:45+08:00" level=info msg="running server on /run/buildkit/buildkitd.sock"
6.2 構建鏡像
配置Dockerfile文件，此Dockerfile文件的意思是構建一個可以ssh的centos鏡像
[root@k8sworker2 ~]# cat Dockerfile_ssh2
FROM hub.c.163.com/library/centos:latest
MAINTAINER LZ
RUN yum -y install openssh-server openssh-clients && \
ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key && \
ssh-keygen -t ecdsa -f /etc/ssh/ssh_host_ecdsa_key && \
ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key && \
echo qweasd | passwd --stdin root && \
sed -i '/UseDNS/cUseDNS no' /etc/ssh/sshd_config && \
yum -y install net-tools && \
yum -y install iproute && \
yum clean all
EXPOSE 22
CMD ["/usr/sbin/sshd","-D"]
使用nerdctl根據Dockerfile_ssh2文件構建鏡像
[root@k8sworker2 ~]# nerdctl build -t centos:ssh . -f Dockerfile_ssh2
[+] Building 35.1s (6/6) FINISHED
=> [internal] load build definition from Dockerfile_ssh2 0.0s
=> => transferring dockerfile: 557B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for hub.c.163.com/library/centos:latest 1.0s
=> CACHED [1/2] FROM hub.c.163.com/library/centos:latest@sha256:ab7e9c357fa8e5c822dd22615d3f704090780df1e089ac4ff8c6098f26a71fef 0.0s
=> => resolve hub.c.163.com/library/centos:latest@sha256:ab7e9c357fa8e5c822dd22615d3f704090780df1e089ac4ff8c6098f26a71fef 0.2s
=> [2/2] RUN yum -y install openssh-server openssh-clients && ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key && ssh-keygen -t ecdsa -f /etc/ssh/ssh_host_ecdsa_key && ssh-keygen -t ed25519 27.3s
=> exporting to oci image format 6.6s
=> => exporting layers 2.0s
=> => exporting manifest sha256:d250fe183c437af049cf0c03e616c543574f889d6f7c271c37918104537503ba 0.0s
=> => exporting config sha256:06aa9d95ed27cbb080d669c66131ae9975910efd34de434fc407b08da66e2a59 0.0s
=> => sending tarball 4.6s
unpacking docker.io/library/centos:ssh (sha256:d250fe183c437af049cf0c03e616c543574f889d6f7c271c37918104537503ba)...done
可以看到鏡像已經構建完畢，centos:ssh就是新構建的鏡像
[root@k8sworker2 ~]# nerdctl images
REPOSITORY TAG IMAGE ID CREATED PLATFORM SIZE
centos ssh d250fe183c43 16 seconds ago linux/amd64 232.0 MiB
nginx latest 0d17b565c37b 6 hours ago linux/amd64 146.2 MiB
使用新構建的鏡像centos:ssh創建容器
[root@k8sworker2 ~]# nerdctl run -d --name=centos-ssh --restart=always centos:ssh
baf0b7d8f067a12eb67576dbaf9249d8134f04a46df6d36b7c3d213065cdf44e
查看正在運行的容器
[root@k8sworker2 ~]# nerdctl ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5d1c3f1ca77f docker.io/library/nginx:latest "/docker-entrypoint.…" 6 hours ago Up 0.0.0.0:80->80/tcp nginx
baf0b7d8f067 docker.io/library/centos:ssh "/usr/sbin/sshd -D" 5 seconds ago Up centos-ssh
查看容器centos-ssh的IP地址，IP地址為：10.4.0.6
[root@k8sworker2 ~]# nerdctl exec -it centos-ssh ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0@if9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 66:ff:8f:43:5c:af brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.4.0.6/24 brd 10.4.0.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::64ff:8fff:fe43:5caf/64 scope link tentative dadfailed
valid_lft forever preferred_lft forever
使用ssh連接容器centos-ssh，root密碼為：qweasd
[root@k8sworker2 ~]# ssh root@10.4.0.6
The authenticity of host '10.4.0.6 (10.4.0.6)' can't be established.
ECDSA key fingerprint is SHA256:gaD8Yb7WsdXxXtAx1Iwz2zkvzhYPb4TqluriL39OzxM.
ECDSA key fingerprint is MD5:e8:32:27:d1:75:58:8f:b3:5b:71:43:e8:fd:cc:7d:ee.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.4.0.6' (ECDSA) to the list of known hosts.
root@10.4.0.6's password:
#成功ssh進入容器
[root@baf0b7d8f067 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0@if9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 66:ff:8f:43:5c:af brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.4.0.6/24 brd 10.4.0.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::64ff:8fff:fe43:5caf/64 scope link tentative dadfailed
valid_lft forever preferred_lft forever
[root@baf0b7d8f067 ~]# exit
logout
Connection to 10.4.0.6 closed.
