目录
引言
一、搭建Harbor仓库
（一）关闭防护
（二）安装docker
（三）安装docker-compose
（四）安装harbor-offline
1.获取安装包
2.修改配置文件
3.启动服务
4.登录仓库验证
二、设置免密登录
（一）创建新项目
（二）添加仓库地址
（三）创建secret资源
1.查看登录凭据
2.创建登录凭据资源清单
3.删除镜像
4.创建ngiinx
引言
在Kubernetes（k8s）环境中，使用私有镜像仓库如Harbor来存储和管理容器镜像是一种常见做法。Harbor是由VMware公司开源的企业级Docker Registry管理项目，支持丰富的权限控制和完善的架构设计，尤其适合大规模Docker集群部署。然而，每次Pod拉取私有镜像时都需要进行身份验证，这可能会增加系统的复杂性和运维成本。本文将介绍如何在Kubernetes中配置免密使用Harbor私有仓库的方法。
环境准备
| 主机名 | IP地址 | 部署服务 | 服务器类型 |
| master01 | 192.168.83.30 | k8s集群 | 控制节点、ETCD节点 |
| node01 | 192.168.83.40 | k8s集群 | 工作节点 |
| node02 | 192.168.83.50 | k8s集群 | 工作节点 |
| harbor | 192.168.83.60 | harbor-offline-installer-v1.2.2 | harbor私有仓库 |
一、搭建Harbor仓库
在harbor节点上进行操作
（一）关闭防护
[root@harbor ~]# systemctl stop firewalld.service
[root@harbor ~]# systemctl disable firewalld.service
[root@harbor ~]# setenforce 0
（二）安装docker
[root@harbor ~]#yum install -y yum-utils device-mapper-persistent-data lvm2
#安装依赖包
[root@harbor ~]#yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
#获取阿里云镜像源
[root@harbor ~]#yum install -y docker-ce-20.10.18 docker-ce-cli-20.10.18 containerd.io
#安装Docker-CE（社区版）20.10.18版本
[root@harbor ~]#systemctl enable --now docker.service
#设置开机自启并立即启动docker服务
[root@harbor ~]#cat > /etc/docker/daemon.json <<EOF
{
"registry-mirrors": ["https://gix2yhc1.mirror.aliyuncs.com"]
}
EOF
（三）安装docker-compose
下载地址：Releases · docker/compose · GitHub
[root@harbor data]# wget https://github.com/docker/compose/releases/download/1.25.0/docker-compose-Linux-x86_64
[root@harbor data]# ls
docker-compose-Linux-x86_64
[root@harbor data]# mv docker-compose-Linux-x86_64 /usr/bin/docker-compose
[root@harbor data]# chmod +x /usr/bin/docker-compose
[root@harbor data]# docker-compose --version
docker-compose version 1.25.0, build 0a186604（四）安装harbor-offline
1.获取安装包
[root@harbor data]# wget http://harbor.orientsoft.cn/harbor-1.2.2/harbor-offline-installer-v1.2.2.tgz
#下载数据包，或者在官方网站下载完毕之后，上传到服务器当中
[root@harbor data]# ls
harbor-offline-installer-v1.2.2.tgz
[root@harbor data]# tar zxvf harbor-offline-installer-v1.2.2.tgz -C /usr/local/2.修改配置文件
[root@harbor data]# cd /usr/local/harbor/
[root@harbor harbor]# ls
common docker-compose.notary.yml harbor_1_1_0_template harbor.v1.2.2.tar.gz LICENSE prepare
docker-compose.clair.yml docker-compose.yml harbor.cfg install.sh NOTICE upgrade
[root@harbor harbor]# vim harbor.cfg
5 hostname = 192.168.83.60 #修改仓库地址为本地地址
......
59 harbor_admin_password = Harbor12345 #harbor登录密码，可自定义
......
3.启动服务
在配置好了 harbor.cfg 之后，执行 ./prepare 命令，为 harbor 启动的容器生成一些必要的文件（环境），再执行命令 ./install.sh 以 pull 镜像并启动容器
[root@harbor harbor]# ./prepare
Generated and saved secret to file: /data/secretkey
Generated configuration file: ./common/config/nginx/nginx.conf
Generated configuration file: ./common/config/adminserver/env
Generated configuration file: ./common/config/ui/env
Generated configuration file: ./common/config/registry/config.yml
Generated configuration file: ./common/config/db/env
Generated configuration file: ./common/config/jobservice/env
Generated configuration file: ./common/config/jobservice/app.conf
Generated configuration file: ./common/config/ui/app.conf
Generated certificate, key file: ./common/config/ui/private_key.pem, cert file: ./common/config/registry/root.crt
The configuration files are ready, please use docker-compose to start the service.
[root@harbor harbor]# ./install.sh
[Step 0]: checking installation environment ...
Note: docker version: 20.10.18
Note: docker-compose version: 1.25.0
[Step 1]: loading Harbor images ...
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
[root@harbor harbor]# systemctl start docker
[root@harbor harbor]# systemctl start docker
[root@harbor harbor]# ./install.sh
[Step 0]: checking installation environment ...
Note: docker version: 20.10.18
Note: docker-compose version: 1.25.0
[Step 1]: loading Harbor images ...
dd60b611baaa: Loading layer [==================================================>] 133.2MB/133.2MB
abf0579c40fd: Loading layer [==================================================>] 1.536kB/1.536kB
ea1fc7bed9c5: Loading layer [==================================================>] 22.48MB/22.48MB
.......4.登录仓库验证
浏览器访问http://harbo服务器ip/
用户名：admin
密码：Harbor12345
二、设置免密登录
（一）创建新项目
登录harbor仓库web界面
创建一个新项目。点击“+项目”按钮
填写项目名称为“new-project”，点击“确定”按钮，创建新项目
（二）添加仓库地址
在每个工作节点配置连接私有仓库，而后登录仓库
//node01节点
[root@node01 ~]#cat > /etc/docker/daemon.json <<EOF
{
"registry-mirrors": ["https://gix2yhc1.mirror.aliyuncs.com"],
"insecure-registries": ["192.168.83.60"]
}
EOF
[root@node01 ~]#systemctl daemon-reload
[root@node01 ~]#systemctl restart docker
[root@node01 ~]#docker login -u admin -p Harbor12345 http://192.168.83.60
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
//node02节点
[root@node02 ~]#cat > /etc/docker/daemon.json <<EOF
{
"registry-mirrors": ["https://gix2yhc1.mirror.aliyuncs.com"],
"insecure-registries": ["192.168.83.60"]
}
EOF
[root@node02 ~]#systemctl daemon-reload
[root@node02 ~]#systemctl restart docker
[root@node02 ~]#docker login -u admin -p Harbor12345 http://192.168.83.60
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded（三）添加镜像
将本地镜像添加到私有仓库
在其中一个节点下载nginx:1.18.0的镜像，并打上标签之后上传到私有仓库
[root@node02 ~]#docker pull nginx:1.18.0
#下载镜像
1.18.0: Pulling from library/nginx
f7ec5a41d630: Pull complete
0b20d28b5eb3: Pull complete
1576642c9776: Pull complete
c12a848bad84: Pull complete
03f221d9cf00: Pull complete
Digest: sha256:e90ac5331fe095cea01b121a3627174b2e33e06e83720e9a934c7b8ccc9c55a0
Status: Downloaded newer image for nginx:1.18.0
docker.io/library/nginx:1.18.0
[root@node02 ~]#docker tag nginx:1.18.0 192.168.83.60/new-project/nginx:v1
#添加镜像标签
[root@node02 ~]#docker images |grep nginx
nginx latest 605c77e624dd 2 years ago 141MB
192.168.83.60/new-project/nginx v1 c2c45d506085 3 years ago 133MB
nginx 1.18.0 c2c45d506085 3 years ago 133MB
[root@node02 ~]#docker push 192.168.83.60/new-project/nginx:v1
#上传到harbo私有仓库
The push refers to repository [192.168.83.60/new-project/nginx]
4fa6704c8474: Pushed
4fe7d87c8e14: Pushed
6fcbf7acaafd: Pushed
f3fdf88f1cb7: Pushed
7e718b9c0c8c: Pushed
v1: digest: sha256:9b0fc8e09ae1abb0144ce57018fc1e13d23abd108540f135dc83c0ed661081cf size: 1362
在web界面查看是否上传成功
（三）创建secret资源
1.查看登录凭据
//在任意工作节点上查看登录凭据
[root@node02 ~]#cat /root/.docker/config.json | base64 -w 0
ewoJImF1dGhzIjogewoJCSIxOTIuMTY4LjgzLjYwIjogewoJCQkiYXV0aCI6ICJZV1J0YVc0NlNHRnlZbTl5TVRJek5EVT0iCgkJfQoJfQp9
#各节点登录凭据相同
#base64 -w 0：进行 base64 加密并禁止自动换行
2.创建登录凭据资源清单
在master节点创建harbor登录凭据资源清单用于 K8S 访问 Harbor 私服拉取镜像所需要的密钥权限凭证 secret 资源
[root@master01 data]#vim harbor-secret.yaml
[root@master01 data]#cat harbor-secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: harbor-secret
data:
.dockerconfigjson: ewoJImF1dGhzIjogewoJCSIxOTIuMTY4LjgzLjYwIjogewoJCQkiYXV0aCI6ICJZV1J0YVc0NlNHRnlZbTl5TVRJek5EVT0iCgkJfQoJfQp9 #复制粘贴上述查看的登陆凭据
type: kubernetes.io/dockerconfigjson
[root@master01 data]#kubectl apply -f harbor-secret.yaml
secret/harbor-secret created
[root@master01 data]#kubectl get secret harbor-secret
NAME TYPE DATA AGE
harbor-secret kubernetes.io/dockerconfigjson 1 27s3.删除镜像
删除之前在node节点下载的nginx镜像，已经自定义标签的镜像
[root@node02 ~]#docker rmi nginx:1.18.0
Untagged: nginx:1.18.0
Untagged: nginx@sha256:e90ac5331fe095cea01b121a3627174b2e33e06e83720e9a934c7b8ccc9c55a0
[root@node02 ~]#docker rmi 192.168.83.60/new-project/nginx:v1
Untagged: 192.168.83.60/new-project/nginx:v1
Untagged: 192.168.83.60/new-project/nginx@sha256:9b0fc8e09ae1abb0144ce57018fc1e13d23abd108540f135dc83c0ed661081cf
Deleted: sha256:c2c45d506085d300b72a6d4b10e3dce104228080a2cf095fc38333afe237e2be
Deleted: sha256:43d6c481a041dbcc1d8ea9c565b1b692bcb28da3414683c316703c669c012ebc
Deleted: sha256:defebc732c194dd0b5b39e20c4d014896ce120207f5dfdb41ed6696b0e8224d6
Deleted: sha256:4ea0f2550407442f808812429981c0b62d8dd6a531db8a412640293a1faf8f3c
Deleted: sha256:778ca58cf39b8fa0776ade88562750a035a24ec5afb7dc4ab2aa892b2c09769d
Deleted: sha256:7e718b9c0c8c2e6420fe9c4d1d551088e314fe923dce4b2caf75891d82fb227d
4.创建ngiinx
指定使用harbor仓库的镜像资源去创建pod
4.1 定义yaml文件
[root@master01 data]#vim nginx.yaml
[root@master01 data]#cat nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
imagePullSecrets: #添加K8S访问Harbor私服拉取镜像所需要的secret资源选项
- name: harbor-secret #指定 secret 资源名称
containers:
- name: nginx
image: 192.168.83.60/new-project/nginx:v1 #指定harbor中的镜像名
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx
spec:
type: NodePort
ports:
- port: 80
targetPort: 80
nodePort: 31111
selector:
app: nginx4.2 创建资源
[root@master01 data]#kubectl apply -f nginx.yaml
deployment.apps/nginx created
service/nginx created
[root@master01 data]#kubectl get pod,svc
NAME READY STATUS RESTARTS AGE
pod/nginx-5b658db7f8-kldfk 1/1 Running 0 3s
pod/nginx-5b658db7f8-vd967 1/1 Running 0 3s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 73d
service/nginx NodePort 10.96.135.253 <none> 80:31111/TCP 3s
[root@master01 data]#curl 192.168.83.30:31111 -I
HTTP/1.1 200 OK
Server: nginx/1.18.0
Date: Mon, 29 Jul 2024 06:42:32 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Tue, 21 Apr 2020 14:09:01 GMT
Connection: keep-alive
ETag: "5e9efe7d-264"
Accept-Ranges: bytes4.3 查看镜像下载地址
[root@master01 data]#kubectl describe pod nginx-5b658db7f8-kldfk
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Pulling 22s kubelet Pulling image "192.168.83.60/new-project/nginx:v1"
Normal Pulled 13s kubelet Successfully pulled image "192.168.83.60/new-project/nginx:v1" in 9.569090599s
Normal Created 13s kubelet Created container nginx
Normal Started 12s kubelet Started container nginx
#可以发现镜像时从harbor下载的刷新harbor页面，可以看到镜像的下载次数增加了
