一、接着上一篇的话
docker镜像已经有了,本文我们将接着演示如何使用helm部署应用到k8s。
分为两大部分:
- 制作helm chart,推送到私有仓库nexus
- helm拉取chart,部署到k8s
二、制作helm chart
要求你先安装helm,随便一台linux机器即可,不要求你要有k8s或者docker环境。
xxx@local:~/Downloads$ wget https://get.helm.sh/helm-v3.12.1-linux-amd64.tar.gz
--2023-06-30 10:49:03-- https://get.helm.sh/helm-v3.12.1-linux-amd64.tar.gz
正在解析主机 get.helm.sh (get.helm.sh)... 152.199.39.108, 2606:2800:247:1cb7:261b:1f9c:2074:3c
正在连接 get.helm.sh (get.helm.sh)|152.199.39.108|:443... 已连接。
已发出 HTTP 请求,正在等待回应... 200 OK
长度: 16036346 (15M) [application/x-tar]
正在保存至: “helm-v3.12.1-linux-amd64.tar.gz”
helm-v3.12.1-linux- 100%[===================>] 15.29M 1.19MB/s 用时 13s
2023-06-30 10:49:18 (1.14 MB/s) - 已保存 “helm-v3.12.1-linux-amd64.tar.gz” [16036346/16036346])
xxx@local:~/Downloads$ tar -xf helm-v3.12.1-linux-amd64.tar.gz
xxx@local:~/Downloads$ sudo mv linux-amd64/helm /usr/local/bin/helm
[sudo] xxx 的密码:
xxx@local:~/Downloads$ helm version
version.BuildInfo{Version:"v3.12.1", GitCommit:"f32a527a060157990e2aa86bf45010dfb3cc8b8d", GitTreeState:"clean", GoVersion:"go1.20.4"}
xxx@local:~/Downloads$ helm repo list
NAME URL
bitnami https://charts.bitnami.com/bitnami
1、创建helm模板
helm creta java
这里,我删除了没用到的一些文件,最后保留的见下:
templates下的三个文件就是对应K8S容器的三个yaml,没有什么好说的,helm这里额外增加了一个values.yaml文件,它是用来替换templates下的yaml文件里的变量。
那有人要问了,values.yaml文件里的变量可以替换吗?
在helm install的时候,你仍然可以替换values.yaml中的变量。所以我这设计的所有java应用,不同的环境,都使用同一个helm chart。
不同应用存在环境的差异,这个问题也就因此解决了。
2. deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Values.appName }}
namespace: {{ .Values.namespace }}
labels:
app: {{ .Values.appName }}
spec:
progressDeadlineSeconds: 600
{{- if not .Values.autoscaling.enabled }}
replicas: {{ .Values.replicaCount }}
{{- end }}
revisionHistoryLimit: 10
selector:
matchLabels:
app: {{ .Values.appName }}
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
{{- with .Values.podAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
app: {{ .Values.appName }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
containers:
- name: {{ .Values.appName }}
env:
- name: TZ
value: Asia/Shanghai
- name: APPNAME
value: {{ .Values.appName }}
- name: CONFIG_SERVICE_ADDR
value: {{ .Values.env.configServiceAddr }}
- name: CONFIG_SERVICE_NAMESPACE
value: {{ .Values.env.configServiceNameSpace }}
- name: CONFIG_SERVICE_GROUP
value: {{ .Values.env.configServiceGroup }}
- name: CONFIG_EPHEMERAL
value: 'false'
- name: CONFIG_SERVICE_ENABLED
value: '{{ .Values.env.configServiceEnabled }}'
- name: spring.cloud.nacos.config.accessKey
value: {{ .Values.env.configAccessKey }}
- name: spring.cloud.nacos.config.secretKey
value: {{ .Values.env.configSecretKey }}
- name: spring.cloud.nacos.discovery.accessKey
value: {{ .Values.env.configAccessKey }}
- name: spring.cloud.nacos.discovery.secretKey
value: {{ .Values.env.configSecretKey }}
- name: JAVA_OPTS
value: {{ .Values.env.javaOpts }}
image: "{{ .Values.image.repository }}{{ .Values.appName }}:{{ .Values.image.tag }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
ports:
- name: http
containerPort: {{ .Values.service.port }}
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /mgm/health
port: {{ .Values.service.port }}
scheme: HTTP
initialDelaySeconds: 1
periodSeconds: 5
successThreshold: 1
timeoutSeconds: 3
startupProbe:
failureThreshold: 22
httpGet:
path: /mgm/health
port: {{ .Values.service.port }}
scheme: HTTP
initialDelaySeconds: 25
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
resources:
limits:
cpu: {{ .Values.resource.limitCpu }}
memory: {{ .Values.resource.limitMemory }}
requests:
cpu: {{ .Values.resource.requestCpu }}
memory: {{ .Values.resource.requestMemory }}
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
3、values.yaml
# Default values for java.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
namespace: java-service
appName: devops-service
replicaCount: 1
env:
javaOpts:
# consul
configServiceEnabled: true
configServiceHost: 192.168.10.19
configServicePort: 8500
# nacos
configServiceAddr: 192.168.5.28:8848
configServiceNameSpace: a5fdc665-4267-462e-9b03-d363984d9963
configServiceGroup: DEFAULT_GROUP
configAccessKey:
configSecretKey:
image:
repository: 192.168.5.6:8086/xxx/
pullPolicy: IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
tag: 1.0.7
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
podAnnotations: {}
podSecurityContext: {}
# fsGroup: 2000
securityContext: {}
# capabilities:
# drop:
# - ALL
# readOnlyRootFilesystem: true
# runAsNonRoot: true
# runAsUser: 1000
service:
type: NodePort
port: 8085
ingress:
enabled: false
className: "nginx"
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
hosts:
- host: devops-service.ztyedu.net
paths:
- path: /
pathType: ImplementationSpecific
tls: []
# - secretName: chart-example-tls
# hosts:
# - chart-example.local
resource:
limitCpu: '2'
limitMemory: 2Gi
requestCpu: 250m
requestMemory: 2Gi
autoscaling:
enabled: false
minReplicas: 1
maxReplicas: 100
targetCPUUtilizationPercentage: 80
# targetMemoryUtilizationPercentage: 80
nodeSelector: {}
tolerations: []
affinity: {}
4. service.yaml
apiVersion: v1
kind: Service
metadata:
name: {{ .Values.appName }}
namespace: {{ .Values.namespace }}
spec:
type: {{ .Values.service.type }}
ports:
- port: {{ .Values.service.port }}
targetPort: {{ .Values.service.port }}
protocol: TCP
name: {{ .Values.appName }}
selector:
app: {{ .Values.appName }}
sessionAffinity: None
5、 ingress.yaml
{{- if .Values.ingress.enabled -}}
{{- $fullName := .Values.appName -}}
{{- $svcPort := .Values.service.port -}}
{{- if and .Values.ingress.className (not (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion)) }}
{{- if not (hasKey .Values.ingress.annotations "kubernetes.io/ingress.class") }}
{{- $_ := set .Values.ingress.annotations "kubernetes.io/ingress.class" .Values.ingress.className}}
{{- end }}
{{- end }}
{{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion -}}
apiVersion: networking.k8s.io/v1
{{- else if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
apiVersion: networking.k8s.io/v1beta1
{{- else -}}
apiVersion: extensions/v1beta1
{{- end }}
kind: Ingress
metadata:
name: {{ $fullName }}
namespace: {{ .Values.namespace }}
spec:
{{- if and .Values.ingress.className (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion) }}
ingressClassName: {{ .Values.ingress.className }}
{{- end }}
{{- if .Values.ingress.tls }}
tls:
{{- range .Values.ingress.tls }}
- hosts:
{{- range .hosts }}
- {{ . | quote }}
{{- end }}
secretName: {{ .secretName }}
{{- end }}
{{- end }}
rules:
{{- range .Values.ingress.hosts }}
- host: {{ .host | quote }}
http:
paths:
{{- range .paths }}
- path: {{ .path }}
{{- if and .pathType (semverCompare ">=1.18-0" $.Capabilities.KubeVersion.GitVersion) }}
pathType: {{ .pathType }}
{{- end }}
backend:
{{- if semverCompare ">=1.19-0" $.Capabilities.KubeVersion.GitVersion }}
service:
name: {{ $fullName }}
port:
number: {{ $svcPort }}
{{- else }}
serviceName: {{ $fullName }}
servicePort: {{ $svcPort }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
6、Chart.yaml
version是helm chart的版本号,应用的版本号见values.yaml文件中的image.tag(每次构建版本号都会追加1)
apiVersion: v2
name: java
description: java demo
# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.1.8
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: "1.16.0"
三、推送chart至helm私库
1、添加 helm repo
xxx@local:~$ helm repo list
NAME URL
bitnami https://charts.bitnami.com/bitnami
# 新增helm仓库
xxx@local:~$ helm repo add nexus http://150xxxx9916:123456@192.168.5.6:8081/repository/xh-helm/
"nexus" has been added to your repositories
xxx@local:~$ helm repo list
NAME URL
bitnami https://charts.bitnami.com/bitnami
nexus http://150xxxx9916:123456@192.168.5.6:8081/repository/xh-helm/
2、安装helm推送插件
xxx@local:~$ helm plugin install --version master https://gitee.com/mirrors_sonatype-nexus-community/helm-nexus-push.git
Installed plugin: nexus-push
3、构建并推送
# 构建chart
xxx@local:~/Downloads$ helm package java
Successfully packaged chart and saved it to: /home/xxx/Downloads/java-0.1.8.tgz
# 推送chart到私有仓库
xxx@local:~/Downloads$ helm nexus-push nexus java-0.1.8.tgz -u 150xxxx9916 -p 123456
Pushing java-0.1.8.tgz to repo http://150xxxx9916:123456@192.168.5.6:8081/repository/xh-helm//...
HTTP/1.1 100 Continue
HTTP/1.1 200 OK
Date: Fri, 30 Jun 2023 10:16:44 GMT
Server: Nexus/3.37.3-02 (OSS)
X-Content-Type-Options: nosniff
Content-Security-Policy: sandbox allow-forms allow-modals allow-popups allow-presentation allow-scripts allow-top-navigation
X-XSS-Protection: 1; mode=block
Content-Length: 0
Done
四、总结
登录nexus,查看上传的chart。
可以看到,chart上传成功。
接下里,就是我们在k8s的控制台,将在下一篇文章进行描述。