Helm(二)

一、Chart模板流程控制if_with_range

1.if

修改values.yaml

cat > values.yaml <<EOF
myname: yeunyi
service:
type: ClusterIP
port: 80
myport: 8080
EOF

修改service.yaml

cat > templates/service.yaml <<EOF
apiVersion: v1
kind: Service
metadata:
name: testserivce
labels:
app: myapp
spec:
type: {{ .Values.service.type }}
ports:
{{- if eq .Values.web "nginx" }}
- port: {{ .Values.service.port }}
{{- else }}
- port: {{ .Values.service.myport }}
{{- end }}
targetPort: http
protocol: TCP
name: http
EOF

说明：在if else end 左边加-，是为了去掉空行。{{- 表示删除左边的所有空格，直到非空格字符，而 -}}表示删除右边的所有空格。注意，换行符也是空格，当然还包括空格，TAB字符

渲染

helm template testrelease . --set web=nginx

[root@aminglinux01 redis-cluster]# helm template testrelease . --set web=nginx
---
# Source: redis-cluster/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: testrelease-configmap
data:
myname: "yeunyi"
myvalue: "Hello World"
---
# Source: redis-cluster/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
name: testserivce
labels:
app: myapp
spec:
type: ClusterIP
ports:
- port: 80
targetPort: http
protocol: TCP
name: http
[root@aminglinux01 redis-cluster]#

如果不定义web变量的值，port为8080

[root@aminglinux01 redis-cluster]# helm template testrelease .
---
# Source: redis-cluster/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: testrelease-configmap
data:
myname: "yeunyi"
myvalue: "Hello World"
---
# Source: redis-cluster/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
name: testserivce
labels:
app: myapp
spec:
type: ClusterIP
ports:
- port: 8080
targetPort: http
protocol: TCP
name: http

2.with 限定作用域

with 的语法类似简单的 if:

{{ with PIPELINE }}
# restricted scope
{{ end }}

没有用 with 的例子：values.yaml：

[root@aminglinux01 redis-cluster]# cat > values.yaml << EOF
> env:
> host: localhost
> user: test
> hello: world
> EOF

deployment.yaml 的引用：

{{- if .Values.env }}
env:
- name: host
value: {{ .Values.env.host }}
- name: user
value: {{ .Values.env.user }}
- name: hello
value: {{ .Values.env.hello }}
{{- end }}

上面的变量引用都需要从.Values开始，有点繁琐。
with 的例子：deployment.yaml 添加 with 以后：

{{- with .Values.env }}
env:
- name: host
value: {{ .host }}
- name: user
value: {{ .user }}
- name: hello
value: {{ .hello }}
{{- end }}

with 语句块里，把当前范围定义在了.Values.env这个变量上了。渲染后结果

env:
- name: host
value: localhost
- name: user
value: test
- name: hello
value: world

3.range 实现循环

在values.yaml 文件中添加上一个变量列表：

cat > values.yaml <<EOF
myname: aming
service:
type: ClusterIP
port: 80
myport: 8080
test:
- 1
- 2
- 3
EOF

循环打印该列表：

cat > templates/configmap.yaml <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Release.Name }}-configmap
data:
myvalue: "Hello World"
myname: {{ quote .Values.myname }}
test: |
{{- range .Values.test }}
- {{ . }} ##遍历循环打印所有元素
{{- end }}
EOF

渲染

[root@aminglinux01 redis-cluster]# helm template testrelease .
---
# Source: redis-cluster/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: testrelease-configmap
data:
myvalue: "Hello World"
myname: "aming"
test: |
- 1 ##遍历循环打印所有元素
- 2 ##遍历循环打印所有元素
- 3 ##遍历循环打印所有元素
---
# Source: redis-cluster/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
name: testserivce
labels:
app: myapp
spec:
type: ClusterIP
ports:
- port: 8080
targetPort: http
protocol: TCP
name: http

二、Chart模板中的变量

变量在模板中，使用变量的场合不多，但个别情况下不得不使用变量。

问题 1：获取数组键值

values.yaml

env:
NAME: "gateway"
JAVA_OPTS: "-Xmx1G"

deployment.yaml

...
env:
{{- range $k, $v := .Values.env }}
- name: {{ $k }}
value: {{ $v | quote }}
{{- end }}

结果如下：

env:
- name: JAVA_OPTS
value: "-Xmx1G"
- name: NAME
value: "gateway"

上面在 range 循环中使用 $k 和 $v 两个变量来接收后面列表循环的键和值。

问题 2：with 中不能使用内置对象

with 语句块内不能带 .Release.Name 对象，否则报错。我们可以将该对象赋值给一个变量可以来解决这个问题：

apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Release.Name }}-deployment
spec:
replicas: {{ .Values.replicas }}
template:
metadata:
labels:
project: {{ .Values.label.project }}
app: {{ quote .Values.label.app }}
{{- with .Values.label }}
project: {{ .project }}
app: {{ .app }}
release: {{ .Release.Name }}
{{- end }}

上面会出错，但可以先定义变量，再调用它

{{- $releaseName := .Release.Name -}}
{{- with .Values.label }}
project: {{ .project }}
app: {{ .app }}
release: {{ $releaseName }}

可以看到在with 语句上面增加了一句 {{-$releaseName:=.Release.Name- }}，其中$releaseName 就是后面的对象的一个引用变量，它的形式就是 $name，赋值操作使用:=，这样with 语句块内部的$releaseName 变量仍然指向的是.Release.Name

三、Chart的命名模板

命名模板有时候也被称为部分或子模板。

相对于 deployment.yaml 这种主模板，命名模板只是定义部分通用内容，然后在各个主模板中调用。
templates目录下有个_helpers.tpl文件。公共的命名模板都放在这个文件里。命名模板使用 define 来定义。如，这里先简单定义一个只包含字符串的模板，用作资源名称。

cat > templates/_helpers.tpl <<EOF
{{/* 定义资源名称 */}}
{{ define "mytest.name" -}}
yeyunyi
{{- end }}
EOF

使用template引用

cat > templates/test.yaml <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ template "mytest.name" }}
labels:
app: {{ .Values.myname }}
EOF

渲染

[root@aminglinux01 redis-cluster]# helm template testrelease .
---
# Source: redis-cluster/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: testrelease-configmap
data:
myvalue: "Hello World"
myname: "aming"
test: |
- 1 ##遍历循环打印所有元素
- 2 ##遍历循环打印所有元素
- 3 ##遍历循环打印所有元素
---
# Source: redis-cluster/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
name: testserivce
labels:
app: myapp
spec:
type: ClusterIP
ports:
- port: 8080
targetPort: http
protocol: TCP
name: http
---
# Source: redis-cluster/templates/test.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: yeyunyi
  labels:
    app: aming
[root@aminglinux01 redis-cluster]#

include用法：

cat > templates/_helpers.tpl <<EOF
{{/* 定义资源名称 */}}
{{ define "mytest.name" -}}
yeyunyi
{{- end }}
{{/* 定义label */}}
{{- define "mytest.label" -}}
app: {{ .Release.Name }}
release: stable
env: qa
{{- end }}
EOF

在template的yaml文件里调用

cat > templates/test.yaml <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
name: test-{{ template "mytest.name" . }}
labels:
{{- include "mytest.label" . | nindent 4 }}
EOF

渲染

[root@aminglinux01 redis-cluster]# helm template testrelease .
---
# Source: redis-cluster/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: testrelease-configmap
data:
myvalue: "Hello World"
myname: "aming"
test: |
- 1 ##遍历循环打印所有元素
- 2 ##遍历循环打印所有元素
- 3 ##遍历循环打印所有元素
---
# Source: redis-cluster/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
name: testserivce
labels:
app: myapp
spec:
type: ClusterIP
ports:
- port: 8080
targetPort: http
protocol: TCP
name: http
---
# Source: redis-cluster/templates/test.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: test-yeyunyi
labels:
app: testrelease
release: stable
env: qa
[root@aminglinux01 redis-cluster]#

四、自定义Chart实战

1.创建chart包

helm create aminglinux

[root@aminglinux01 redis-cluster]# helm create yeyunyilinux
Creating yeyunyilinux
[root@aminglinux01 redis-cluster]# ls
2           charts      README.md  templates.bak  values.yaml.bak
Chart.lock  Chart.yaml  templates  values.yaml    yeyunyilinux
[root@aminglinux01 redis-cluster]#

2.自定义templates模板文件

删除掉默认的模板文件

cd aminglinux
rm -rf templates/*

[root@aminglinux01 redis-cluster]# cd yeyunyilinux/
[root@aminglinux01 yeyunyilinux]# ls
charts  Chart.yaml  templates  values.yaml
[root@aminglinux01 yeyunyilinux]# rm -rf templates/*
[root@aminglinux01 yeyunyilinux]#

生成一个deployment模板

kubectl create deployment yeyunyilinux --image=registry.cn-hangzhou.aliyuncs.com/daliyused/nginx:latest -o yaml --dry-run > templates/deployment.yaml

修改deployment.yaml

编辑svc模板

apiVersion: v1
kind: Service
metadata:
labels:
app: {{ .Values.appname }} #service要管理deployment的pod资源，因此这里的标签要和pod资源的标签对应上，直接调用appname这个变量
name: {{ .Values.appname }}-svc #service资源的名称，也可以直接调用appname这个变量，后面加一个-svc
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: {{ .Values.appname }} #标签选择器还是调用appname这个变量
type: NodePort

编辑configmap模板

vi templates/configmap.yaml ##写入如下内容
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Values.appname }}-cm #引入appname变量加上-cm作为cm资源的名称
data:
test.aminglinux.com.conf: |
server {
listen 80;
server_name test.aminglinux.com;
location / {
root /data/code/aminglinux;
index index.html;
}
}

编辑pv/pvc模板

[root@aminglinux01 yeyunyilinux]# cat > templates/pv-pvc.yaml << EOF
> apiVersion: v1
> kind: PersistentVolume
> metadata:
> name: {{ .Values.appname }}-pv #引入appname变量加上-pv作为pv资源的名称
> labels:
> pv: {{ .Values.appname }}-pv #标签也可以使用和pv名称一样的名字
> spec:
> capacity:
> storage: 2Gi
> accessModes:
> - ReadWriteMany
> persistentVolumeReclaimPolicy: Retain
> nfs:
> path: {{ .Values.nfsPath }} #这里会引入nfsPath变量的值
> server: {{ .Values.nfsServer }} #这里会引入nfsServer变量的值
> ---
> apiVersion: v1
> kind: PersistentVolumeClaim
> metadata:
> name: {{ .Values.appname }}-pvc #引入appname变量加上-pvc作为pvc资源的名称
> spec:
> accessModes:
> - ReadWriteMany
> resources:
> requests:
> storage: 2Gi
> selector:
> matchLabels:
> pv: {{ .Values.appname }}-pv #指定pv的标签
> EOF
[root@aminglinux01 yeyunyilinux]#

定义values.yaml

cat > values.yaml << EOF
appname: yeyunyilinux
replicas: 2
image: yeyunyilinux/helm-custom-chart ##这是一个测试的镜像
imageTag: v0
nfsPath: /data/nfs/yeyunyilinux ##这个目录需要提前创建好
nfsServer: 192.168.100.160
EOF

注意：假定NFS服务器已经搭建好了

3.安装chart

helm install aminglinux-release .

[root@aminglinux01 yeyunyilinux]# helm install yeyunyilinux-release .
NAME: yeyunyilinux-release
LAST DEPLOYED: Fri Jul 26 01:47:51 2024
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
[root@aminglinux01 yeyunyilinux]#

4. 查看svc

[root@aminglinux01 yeyunyilinux]# kubectl get svc
NAME                        TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)              AGE
yeyunyilinux-svc            NodePort    10.15.191.78    <none>        80:30802/TCP         55s
[root@aminglinux01 yeyunyilinux]#

5.到NFS服务器上创建一个测试页

echo "This is a test site." > /data/nfs/yeyunyilinux/index.html

[root@bogon ~]# cat /data/nfs/yeyunyilinux/index.html 
This is a test site.
[root@bogon ~]#

6）浏览器访问
http://192.168.222.131:32745

五、使用Helm安装harbor

注意：如果你的harbor是之前docker-compose安装的，还需要额外做一个动作，让它支持chart

docker-compose stop
./install.sh --with-chartmuseum

1.下载harbor的chart包

Harbor的chartmuseum可以让Helm直接将chart包推送到harbor里，但是注意，harbor从2.8.0开始已经不支持chartmuseum了，而是改为了OCI ，鉴于新版本不太成熟和使用人太少，所以当前，我们安装2.6.2版本

helm search repo harbor -l #查看历史版本
helm pull bitnami/harbor --version 16.1.0 --untar
##16.1.0是chart的版本，而harbor版本为2.6.2

2.修改默认values.yaml

cd harbor
vi values.yaml #更改
storageClass: "nfs-client" ##这个是提前搭建好的nfs的storageclass
将所有"core.harbor.domain"替换为你自己的域名

3.安装

helm install myharbor --version 22.0.5 .

[root@aminglinux01 harbor]# helm install myharbor .
NAME: myharbor
LAST DEPLOYED: Mon Jul 29 22:46:49 2024
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: harbor
CHART VERSION: 22.0.5
APP VERSION: 2.11.0

** Please be patient while the chart is being deployed **

1. Get the Harbor URL:

NOTE: It may take a few minutes for the LoadBalancer IP to be available.
Watch the status with: 'kubectl get svc --namespace default -w myharbor'
export SERVICE_IP=$(kubectl get svc --namespace default myharbor --template "{{ range (index .status.loadBalancer.ingress 0) }}{{ . }}{{ end }}")
echo "Harbor URL: http://$SERVICE_IP/"

2. Login with the following credentials to see your Harbor application

echo Username: "admin"
echo Password: $(kubectl get secret --namespace default myharbor-core-envvars -o jsonpath="{.data.HARBOR_ADMIN_PASSWORD}" | base64 -d)

WARNING: There are "resources" sections in the chart not set. Using "resourcesPreset" is not recommended for production. For production installations, please set the following values according to your workload needs:
- core.resources
- exporter.resources
- jobservice.resources
- nginx.resources
- portal.resources
- registry.controller.resources
- registry.server.resources
- trivy.resources
+info https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/

⚠ SECURITY WARNING: Original containers have been substituted. This Helm chart was designed, tested, and validated on multiple platforms using a specific set of Bitnami and Tanzu Application Catalog containers. Substituting other containers is likely to cause degraded security and performance, broken chart features, and missing environment variables.

Substituted images detected:
- %!s(<nil>)/registry.cn-hangzhou.aliyuncs.com/daliyused/os-shell:12-debian-12
- %!s(<nil>)/registry.cn-hangzhou.aliyuncs.com/daliyused/nginx:1.27.0-debian-12-r3
- %!s(<nil>)/registry.cn-hangzhou.aliyuncs.com/daliyused/harbor-portal:2.11.0-debian-12-r2
- %!s(<nil>)/registry.cn-hangzhou.aliyuncs.com/daliyused/harbor-core:2.11.0-debian-12-r2
- %!s(<nil>)/registry.cn-hangzhou.aliyuncs.com/daliyused/harbor-jobservice:2.11.0-debian-12-r2
- %!s(<nil>)/registry.cn-hangzhou.aliyuncs.com/daliyused/harbor-registry:2.11.0-debian-12-r3
- %!s(<nil>)/registry.cn-hangzhou.aliyuncs.com/daliyused/harbor-registryctl:2.11.0-debian-12-r2
- %!s(<nil>)/registry.cn-hangzhou.aliyuncs.com/daliyused/harbor-adapter-trivy:2.11.0-debian-12-r3
- %!s(<nil>)/registry.cn-hangzhou.aliyuncs.com/daliyused/harbor-exporter:2.11.0-debian-12-r2
- %!s(<nil>)/registry.cn-hangzhou.aliyuncs.com/daliyused/postgresql:13.15.0-debian-12-r12
[root@aminglinux01 harbor]#

[root@aminglinux01 harbor]# kubectl get pod | grep myharbor
myharbor-core-b9d48ccdd-v9jdz 1/1 Running 1 (5m22s ago) 6m33s
myharbor-jobservice-6f5dbfcc4f-q852z 1/1 Running 2 (4m22s ago) 6m33s
myharbor-nginx-65b8c5764d-vz4vn 1/1 Running 0 6m33s
myharbor-portal-ff7fd4949-lj6jw 1/1 Running 0 6m33s
myharbor-postgresql-0 1/1 Running 0 6m33s
myharbor-redis-master-0 1/1 Running 0 6m33s
myharbor-registry-5b59458d9-4j79b 2/2 Running 0 6m33s
myharbor-trivy-0 1/1 Running 0 6m33s
[root@aminglinux01 harbor]#

注意：如果镜像有问题，处理修改value.yaml文件外，也可以看一下Chart文件夹下是否还有应用文件夹，文件夹下如果还有value.yaml文件也要修改镜像地址，否则会存在拉取不到镜像的问题。

4.查看端口

kubectl get svc |grep harbor |grep LoadBalancer

[root@aminglinux01 harbor]# kubectl get svc |grep harbor |grep LoadBalancer
myharbor LoadBalancer 10.15.36.193 <pending> 80:30979/TCP,443:32161/TCP 36m
[root@aminglinux01 harbor]#

[root@aminglinux01 harbor]# kubectl get pod -owide | grep myharbor
myharbor-core-b9d48ccdd-v9jdz 1/1 Running 1 (36m ago) 38m 10.18.206.199 aminglinux02 <none> <none>
myharbor-jobservice-6f5dbfcc4f-q852z 1/1 Running 2 (35m ago) 38m 10.18.68.135 aminglinux03 <none> <none>
myharbor-nginx-65b8c5764d-vz4vn 1/1 Running 0 38m 10.18.68.187 aminglinux03 <none> <none>
myharbor-portal-ff7fd4949-lj6jw 1/1 Running 0 38m 10.18.206.195 aminglinux02 <none> <none>
myharbor-postgresql-0 1/1 Running 0 38m 10.18.68.184 aminglinux03 <none> <none>
myharbor-redis-master-0 1/1 Running 0 38m 10.18.68.189 aminglinux03 <none> <none>
myharbor-registry-5b59458d9-4j79b 2/2 Running 0 38m 10.18.206.213 aminglinux02 <none> <none>
myharbor-trivy-0 1/1 Running 0 38m 10.18.68.134 aminglinux03 <none> <none>

5.查看密码

kubectl get secret --namespace default myharbor-core-envvars -o jsonpath="{.data.HARBOR_ADMIN_PASSWORD}" | base64 -d

[root@aminglinux01 harbor]# kubectl get secret --namespace default myharbor-core-envvars -o jsonpath="{.data.HARBOR_ADMIN_PASSWORD}" | base64 -d
oYKiQMMFYH[root@aminglinux01 harbor]#

6.浏览器登录

https://192.168.100.153:32161

四、将Chart推送到私有仓库harbor

1.安装helm-push插件

helm plugin install https://github.com/chartmuseum/helm-push

[root@aminglinux01 harbor]# helm plugin install https://github.com/chartmuseum/helm-push
Downloading and installing helm-push v0.10.4 ...
https://github.com/chartmuseum/helm-push/releases/download/v0.10.4/helm-push_0.10.4_linux_amd64.tar.gz
Installed plugin: cm-push
[root@aminglinux01 harbor]#

检查plugins列表

[root@aminglinux01 harbor]# helm plugin list
NAME    VERSION   DESCRIPTION
cm-push   0.10.4    Push chart package to ChartMuseum
[root@aminglinux01 harbor]#

2.添加harbor地址

到harbor浏览器后台添加新的项目 chart_repo

helm添加新仓库

helm repo add myharbor https://harbor.yuankeedu.com/chartrepo/chart_repo --username=admin --password=1qaz@WSX

如果出现x509的错误提示，执行

echo -n | openssl s_client -showcerts -connect 域名:443 2>/dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' >> /etc/ssl/certs/ca-bundle.trust.crt