1. 轻量级容器管理工具 Containerd
2. Containerd的两种安装方式
3. Containerd容器镜像管理
4. Containerd数据持久化和网络管理
1、Containerd镜像管理
1.1 Containerd容器镜像管理命令
- docker使用
docker images命令管理镜像 - 单机containerd使用
ctr images命令管理镜像,containerd本身的CLI - k8s中containerd使用
crictl images命令管理镜像,Kubernetes社区的专用CLI工具
获取命令帮助
# ctr --help
NAME:
ctr -
__
_____/ /______
/ ___/ __/ ___/
/ /__/ /_/ /
\___/\__/_/
......
获取命令帮助
# ctr images
NAME:
ctr images - manage images
USAGE:
ctr images command [command options] [arguments...]
COMMANDS:
check check existing images to ensure all content is available locally
export export images
import import images
list, ls list images known to containerd
mount mount an image to a target path
unmount unmount the image from the target
pull pull an image from a remote
push push an image to a remote
delete, del, remove, rm remove one or more images by reference
tag tag an image
label set and clear labels for an image
convert convert an image
OPTIONS:
--help, -h show help
1.2 查看镜像
# ctr images ls
REF TYPE DIGEST SIZE PLATFORMS LABELS
1.3 下载镜像
containerd支持oci标准的镜像，所以可以直接使用docker官方或dockerfile构建的镜像
# 镜像名不能简写，--all-platforms：所有平台，不加的话下载当前平台架构
# ctr images pull --all-platforms docker.io/library/nginx:alpine
# ctr images pull docker.io/library/nginx:latest
# uname -a
说明：
这里ctr命令pull镜像时，不能直接把镜像名字写成`nginx:alpine`
查看已下载容器镜像
# ctr images ls
| REF | TYPE | DIGEST |
|---|---|---|
| docker.io/library/nginx:alpine | application/vnd.docker.distribution.manifest.list.v2+json | sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3 |
| SIZE | PLATFORMS | LABELS |
|---|---|---|
| 9.7 MiB | linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/ppc64le,linux/s390x | - |
指定平台下载容器镜像
# ctr images pull --platform linux/amd64 docker.io/library/nginx:alpine
没有镜像是无法运行容器的。
1.4 镜像挂载
方便查看镜像中包含的内容。
把已下载的容器镜像挂载至当前文件系统
# ctr images mount docker.io/library/nginx:alpine /mnt
sha256:af2fcce448e2e4451a5f4796a9bf9cb5c9b5f88e0d6d10029cada42fb9d268ac
/mnt
[root@localhost ~]# ls /mnt
bin dev docker-entrypoint.d docker-entrypoint.sh etc home lib media mnt opt proc root run sbin srv sys tmp usr var
卸载
# umount /mnt
1.5 镜像导出
把容器镜像导出
# ctr i export --all-platforms nginx.img docker.io/library/nginx:alpine
说明
--all-platforms,导出所有平台镜像，本版本为1.6版本，1.4版本不需要添加此选项。
查看已导出容器镜像
# ls
nginx.img
# ls -lh
总用量 196M
-rw-r--r-- 1 root root 73M 2月 18 14:48 nginx.img
1.6 镜像删除
删除指定容器镜像
# ctr image rm docker.io/library/nginx:alpine
docker.io/library/nginx:alpine
再次查看容器镜像
[root@192 ~]# ctr images ls
REF TYPE DIGEST SIZE PLATFORMS LABELS
1.7 镜像导入
导入容器镜像
# ctr images import nginx.img
unpacking docker.io/library/nginx:alpine (sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3)...done
1.8 修改镜像tag
# ctr images tag docker.io/library/nginx:alpine nginx:alpine
nginx:alpine
# 建议修改tag的名称为：镜像仓库地址/目录/镜像名
说明：
把docker.io/library/nginx:alpine 修改为 nginx:alpine
查看修改后的容器镜像
# ctr images ls
REF TYPE DIGEST SIZE PLATFORMS LABELS
docker.io/library/nginx:alpine application/vnd.docker.distribution.manifest.list.v2+json sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3 9.7 MiB linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/ppc64le,linux/s390x -
nginx:alpine application/vnd.docker.distribution.manifest.list.v2+json sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3 9.7 MiB linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/ppc64le,linux/s390x -
修改后对容器镜像做检查比对
# ctr images check
REF TYPE DIGEST STATUS SIZE UNPACKED
docker.io/library/nginx:alpine application/vnd.docker.distribution.manifest.list.v2+json sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3 complete (7/7) 9.7 MiB/9.7 MiB true
nginx:alpine application/vnd.docker.distribution.manifest.list.v2+json sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3 complete (7/7) 9.7 MiB/9.7 MiB true
2、Containerd容器管理
2.1 获取命令帮助
2.1.1 获取ctr命令帮助
[root@localhost ~]# ctr --help
NAME:
ctr -
__
_____/ /______
/ ___/ __/ ___/
/ /__/ /_/ /
\___/\__/_/
containerd CLI
......
2.1.2 获取创建静态容器命令帮助
# ctr container --help
NAME:
ctr containers - manage containers
USAGE:
ctr containers command [command options] [arguments...]
COMMANDS:
create create container
delete, del, remove, rm delete one or more existing containers
info get info about a container
list, ls list containers
label set and clear labels for a container
checkpoint checkpoint a container
restore restore a container from checkpoint
OPTIONS:
--help, -h show help
说明：
使用`ctr container create `命令创建容器后，容器并没有处于运行状态，其只是一个静态的容器。这个 container 对象只是包含了运行一个容器所需的资源及配置的数据结构，例如： namespaces、rootfs 和容器的配置都已经初始化成功了，只是用户进程(本案例为nginx)还没有启动。需要使用`ctr tasks`命令才能获取一个动态容器。
2.1.3 获取动态容器命令帮助
# ctr run --help
NAME:
ctr run - run a container
......
说明：
使用`ctr run`命令可以创建一个静态容器并使其运行。一步到位运行容器。
2.2 查看容器
container表示静态容器，可用c缩写代表container
# ctr container ls
CONTAINER IMAGE RUNTIME
或
# ctr c ls
CONTAINER IMAGE RUNTIME
以上命令无法查看出来容器的静态的还是动态的
2.3 查看任务
task表示容器里跑的进程, 可用t缩写代表task
# ctr task ls
TASK PID STATUS
或
# ctr t ls
TASK PID STATUS
2.4 创建静态容器
# ctr c create docker.io/library/nginx:alpine nginx1
# ctr container ls
CONTAINER IMAGE RUNTIME
nginx1 docker.io/library/nginx:alpine io.containerd.runc.v2
查看容器详细信息
# ctr container info nginx1
2.5 静态容器启动为动态容器
复制containerd连接runC垫片工具至系统
# ls usr/local/bin/
containerd containerd-shim containerd-shim-runc-v1 containerd-shim-runc-v2 containerd-stress crictl critest ctd-decoder ctr
[root@localhost ~]# cp usr/local/bin/containerd-shim-runc-v2 /usr/bin/
启动task，即表时在容器中运行了进程，即为动态容器。
# ctr task start -d nginx1
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
说明：
-d表示daemon或者后台的意思，否则会卡住终端
查看容器所在宿主机进程，是以宿主机进程的方式存在的。
# ctr task ls
TASK PID STATUS
nginx1 3395 RUNNING
查看容器的进程(都是物理机的进程)
# ctr task ps nginx1
PID INFO
3395 -
3434 -
物理机查看到相应的进程
# ps -ef | grep 3395
root 3395 3375 0 19:16 ? 00:00:00 nginx: master process nginx -g daemon off;
101 3434 3395 0 19:16 ? 00:00:00 nginx: worker process
2.6 进入容器操作
# ctr task exec --exec-id 1 nginx1 /bin/sh
# ifconfig 查看网卡信息
...
# curl 127.0.0.1 访问本地提供的web服务
...
说明：
为exec进程设定一个id，可以随意输入，只要保证唯一即可，也可使用$RANDOM变量。
2.7 直接运行一个动态容器
# ctr run -d --net-host docker.io/library/nginx:alpine nginx2
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
说明：
* -d 代表dameon，后台运行
* --net-host 代表容器的IP就是宿主机的IP(相当于docker里的host类型网络)
查看已运行容器
# ctr container ls
CONTAINER IMAGE RUNTIME
nginx2 docker.io/library/nginx:alpine io.containerd.runc.v2
查看已运行容器中运行的进程,既tasks
# ctr tasks ls
TASK PID STATUS
nginx2 4061 RUNNING
进入容器
# ctr task exec --exec-id 1 -t nginx2 /bin/sh
/ # ifconfig
ens33 Link encap:Ethernet HWaddr 00:0C:29:B1:B6:1D
inet addr:192.168.10.164 Bcast:192.168.10.255 Mask:255.255.255.0
inet6 addr: fe80::2b33:40ed:9311:8812/64 Scope:Link
inet6 addr: fe80::adf4:a8bc:a1c:a9f7/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:55360 errors:0 dropped:0 overruns:0 frame:0
TX packets:30526 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:53511295 (51.0 MiB) TX bytes:2735050 (2.6 MiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:68 errors:0 dropped:0 overruns:0 frame:0
TX packets:68 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:5916 (5.7 KiB) TX bytes:5916 (5.7 KiB)
virbr0 Link encap:Ethernet HWaddr 52:54:00:E9:51:82
inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
为容器中运行的网站添加网站文件
/ # echo "nginx2" > /usr/share/nginx/html/index.html
/ # exit
在宿主机上访问网站
[root@localhost ~]# curl 192.168.10.164
nginx2
2.8 暂停容器
查看容器状态
# ctr tasks ls
TASK PID STATUS
nginx2 4061 RUNNING
暂停容器
# ctr tasks pause nginx2
再次查看容器状态，看到其状态为PAUSED，表示停止。
# ctr tasks ls
TASK PID STATUS
nginx2 4061 PAUSED
[root@localhost ~]# curl http://192.168.10.164
在宿主机访问，发现不可以访问到网站
2.9 恢复容器
使用resume命令恢复容器
# ctr tasks resume nginx2
查看恢复后状态
# ctr tasks ls
TASK PID STATUS
nginx2 4061 RUNNING
在宿主机上访问容器中提供的服务
# curl http://192.168.10.164
nginx2
2.10 停止容器
# ctr tasks --help
......
使用kill命令停止容器中运行的进程，既为停止容器
# ctr tasks kill nginx2
查看容器停止后状态，STATUS为STOPPED
# ctr tasks ls
TASK PID STATUS
nginx1 3395 RUNNING
nginx2 4061 STOPPED
2.11 删除容器
# ctr tasks delete nginx2
必须先删除task，再删除容器
查看静态容器，确认其还存在于系统中
# ctr container ls
CONTAINER IMAGE RUNTIME
nginx2 docker.io/library/nginx:alpine io.containerd.runc.v2
删除容器
# ctr container delete nginx2
3、Containerd使用私有容器镜像仓库 Harbor
3.1 Harbor准备
3.2 配置Containerd使用Harbor仓库
3.2.1 Harbor主机名解析
在所有安装containerd宿主机上添加此配置信息。
# vim /etc/hosts
# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.10.165 harbor.kubemsb.com
说明
* 192.168.10.165是harbor的IP
* harbor.kubemsb.com建议用FQDN形式，如果用类似harbor这种短名，后面下载镜像会出问题
3.2.2 修改Containerd配置文件
此配置文件已提前替换过，仅修改本地容器镜像仓库地址即可。
# vim /etc/containerd/config.toml
# cat /etc/containerd/config.toml
root = "/var/lib/containerd"
state = "/run/containerd"
oom_score = -999
[grpc]
address = "/run/containerd/containerd.sock"
uid = 0
gid = 0
max_recv_message_size = 16777216
max_send_message_size = 16777216
[debug]
address = ""
uid = 0
gid = 0
level = ""
[metrics]
address = ""
grpc_histogram = false
[cgroup]
path = ""
[plugins]
[plugins.cgroups]
no_prometheus = false
[plugins.cri]
stream_server_address = "127.0.0.1"
stream_server_port = "0"
enable_selinux = false
sandbox_image = "easzlab/pause-amd64:3.2"
stats_collect_period = 10
systemd_cgroup = false
enable_tls_streaming = false
max_container_log_line_size = 16384
[plugins.cri.containerd]
snapshotter = "overlayfs"
no_pivot = false
[plugins.cri.containerd.default_runtime]
runtime_type = "io.containerd.runtime.v1.linux"
runtime_engine = ""
runtime_root = ""
[plugins.cri.containerd.untrusted_workload_runtime]
runtime_type = ""
runtime_engine = ""
runtime_root = ""
[plugins.cri.cni]
bin_dir = "/opt/kube/bin"
conf_dir = "/etc/cni/net.d"
conf_template = "/etc/cni/net.d/10-default.conf"
[plugins.cri.registry]
[plugins.cri.registry.mirrors]
[plugins.cri.registry.mirrors."docker.io"]
endpoint = [
"https://docker.mirrors.ustc.edu.cn",
"http://hub-mirror.c.163.com"
]
[plugins.cri.registry.mirrors."gcr.io"]
endpoint = [
"https://gcr.mirrors.ustc.edu.cn"
]
[plugins.cri.registry.mirrors."k8s.gcr.io"]
endpoint = [
"https://gcr.mirrors.ustc.edu.cn/google-containers/"
]
[plugins.cri.registry.mirrors."quay.io"]
endpoint = [
"https://quay.mirrors.ustc.edu.cn"
]
[plugins.cri.registry.mirrors."harbor.kubemsb.com"] 在此处添加,在镜像加速器下面添加这一段
endpoint = [
"http://harbor.kubemsb.com"
]
[plugins.cri.x509_key_pair_streaming]
tls_cert_file = ""
tls_key_file = ""
[plugins.diff-service]
default = ["walking"]
[plugins.linux]
shim = "containerd-shim"
runtime = "runc"
runtime_root = ""
no_shim = false
shim_debug = false
[plugins.opt]
path = "/opt/containerd"
[plugins.restart]
interval = "10s"
[plugins.scheduler]
pause_threshold = 0.02
deletion_threshold = 0
mutation_threshold = 100
schedule_delay = "0s"
startup_delay = "100ms"
重启containerd，以便于重新加载配置文件。
# systemctl daemon-reload
# systemctl restart containerd
3.2.3 ctr下载镜像
下载容器镜像
# ctr images pull --platform linux/amd64 docker.io/library/nginx:latest
说明:
* --platform linux/amd64 指定系统平台，也可以使用--all-platforms指定所有平台镜像。
查看已下载容器镜像
# ctr images ls
REF TYPE DIGEST SIZE PLATFORMS LABELS
docker.io/library/nginx:latest application/vnd.docker.distribution.manifest.list.v2+json sha256:2834dc507516af02784808c5f48b7cbe38b8ed5d0f4837f16e78d00deb7e7767 54.1 MiB linux/386,linux/amd64,linux/arm/v5,linux/arm/v7,linux/arm64/v8,linux/mips64le,linux/ppc64le,linux/s390x -
3.2.4 ctr上传镜像
上传到Harbor library公有项目
重新生成新的tag
# ctr images tag docker.io/library/nginx:latest harbor.kubemsb.com/library/nginx:latest
harbor.kubemsb.com/library/nginx:latest
查看已生成容器镜像
# ctr images ls
REF TYPE DIGEST SIZE PLATFORMS LABELS
docker.io/library/nginx:latest application/vnd.docker.distribution.manifest.list.v2+json sha256:2834dc507516af02784808c5f48b7cbe38b8ed5d0f4837f16e78d00deb7e7767 54.1 MiB linux/386,linux/amd64,linux/arm/v5,linux/arm/v7,linux/arm64/v8,linux/mips64le,linux/ppc64le,linux/s390x -
harbor.kubemsb.com/library/nginx:latest application/vnd.docker.distribution.manifest.list.v2+json sha256:2834dc507516af02784808c5f48b7cbe38b8ed5d0f4837f16e78d00deb7e7767 54.1 MiB linux/386,linux/amd64,linux/arm/v5,linux/arm/v7,linux/arm64/v8,linux/mips64le,linux/ppc64le,linux/s390x -
推送容器镜像至Harbor
# ctr images push --platform linux/amd64 --plain-http -u admin:密码 harbor.kubemsb.com/library/nginx:latest
说明:
* 先tag再push
* 因为我们harbor是http协议，不是https协议，所以需要加上`--plain-http`
* `--user admin:Harbor12345`指定harbor的用户名与密码
输出：
manifest-sha256:0fd68ec4b64b8dbb2bef1f1a5de9d47b658afd3635dc9c45bf0cbeac46e72101: done |++++++++++++++++++++++++++++++++++++++|
config-sha256:dd025cdfe837e1c6395365870a491cf16bae668218edb07d85c626928a60e478: done |++++++++++++++++++++++++++++++++++++++|
elapsed: 0.5 s total: 9.3 Ki (18.1 KiB/s)
下载已上传容器镜像
# ctr images pull --plain-http harbor.kubemsb.com/library/nginx:latest
