一、实验
1.环境
（1）主机
表1 主机
| 主机 | 架构 | 版本 | IP | 备注 |
| master | K8S master节点 | 1.29.0 | 192.168.204.8 | |
| node1 | K8S node节点 | 1.29.0 | 192.168.204.9 | |
| node2 | K8S node节点 | 1.29.0 | 192.168.204.10 | 已部署Kuboard |
（2）master节点查看集群
1）查看node
kubectl get node
2)查看node详细信息
kubectl get node -o wide
（3）查看pod
[root@master ~]# kubectl get pod -A
(4) 访问Kuboard
http://192.168.204.10:30080/kuboard/cluster查看节点
2.K8S 1.29版本 部署HELM
（1）查阅
https://github.com/helm/helm/releases/tag/v3.14.4目前最新版为v3.14.4
(2) 部署HELM
1）安装 helm
//下载二进制 Helm client 安装包
helm-v3.14.4-linux-amd64.tar.gz
tar -zxvf helm-v3.14.4-linux-amd64.tar.gz
mv linux-amd64/helm /usr/local/bin/helm
helm version
//命令补全
source <(helm completion bash)
安装
（3）使用 helm 安装 Chart
1）查阅
https://github.com/SonarSource/helm-chart-sonarqube
2）使用 helm 安装 Chart
//添加指定的 chart 仓库，
helm repo add sonarqube https://SonarSource.github.io/helm-chart-sonarqube
3) 下载指定版本
helm pull sonarqube/sonarqube --version 10.5.0+2748 查阅最新版本
安装
下载
（4）移动
cd ~ && mkdir sonarqube
mv sonarqube-10.5.0+2748.tgz sonarqube/
cd sonarqube/;ls
3.搭建NFS
（1）检查并安装rpcbind和nfs-utils软件包
[root@master ~]# rpm -q rpcbind nfs-utils
（2）创建目录并授权
[root@master ~]# mkdir -p /opt/sonarqube
[root@master opt]# chmod 777 sonarqube/
(3)打开nfs的配置文件
[root@master opt]# vim /etc/exports
（4）配置文件
给所有网段用户赋予读写权限、同步内容、不压缩共享对象root用户权限
……
/opt/sonarqube *(rw,sync,no_root_squash)
(5) 使NFS配置生效
[root@master opt]# exportfs -r
（6）监听端口
[root@master opt]# ss -antp | grep rpcbind
（7）查看共享
[root@master opt]# showmount -e
其他节点查看
[root@node1 ~]# showmount -e master
4.K8S 1.29版本安装nfs-provisioner
(1) 查阅
https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/releases（2）创建目录
[root@master ~]# cd ~ && mkdir nfs-subdir-external-provisioner
[root@master ~]# cd nfs-subdir-external-provisioner/
（3）第一种方式下载
helm添加repo
[root@master nfs-subdir-external-provisioner]# helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner
下载
[root@master ~]# helm pull nfs-subdir-external-provisioner/nfs-subdir-external-provisioner
（5）第二种方式下载
查阅
https://artifacthub.io/packages/helm/nfs-subdir-external-provisioner/nfs-subdir-external-provisioner
点击右边的install
弹出页面
点击右下角"this link"
（6）移动并解压（选择上面的第二种方式）
[root@master ~]# mv nfs-subdir-external-provisioner-4.0.18.tgz nfs-subdir-external-provisioner
[root@master nfs-subdir-external-provisioner]# tar -xvf nfs-subdir-external-provisioner-4.0.18.tgz
（7）node节点导入镜像
导入本地
[root@node1 ~]# docker load --input nfs-subdir-external-provisioner.tar
重新打标签
[root@node1 ~]# docker tag k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2 registry.k8s.io/sig-storage/n
（8）master节点安装
helm install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/ --set nfs.server=192.168.204.8 --set nfs.path=/opt/sonarqube --set storageClass.name=nfs-client --set storageClass.defaultClass=true -n nfs-provisioner --create-namespace
（9）查看pod
详细查看
[root@master ~]# kubectl describe pod nfs-subdir-external-provisioner-567b586d45-xz8r6 -n nfs-provisioner
Name: nfs-subdir-external-provisioner-567b586d45-xz8r6
Namespace: nfs-provisioner
Priority: 0
Service Account: nfs-subdir-external-provisioner
Node: node1/192.168.204.9
Start Time: Sat, 27 Apr 2024 19:38:39 +0800
Labels: app=nfs-subdir-external-provisioner
pod-template-hash=567b586d45
release=nfs-subdir-external-provisioner
Annotations: cni.projectcalico.org/containerID: 8f4479951e36de27cc21dcce8b7bf11a34eb838107d4457c6ca352acbf69399e
cni.projectcalico.org/podIP: 10.244.166.167/32
cni.projectcalico.org/podIPs: 10.244.166.167/32
Status: Running
IP: 10.244.166.167
IPs:
IP: 10.244.166.167
Controlled By: ReplicaSet/nfs-subdir-external-provisioner-567b586d45
Containers:
nfs-subdir-external-provisioner:
Container ID: docker://9c18e809cc7179a55d66a1886b6addbd034841a6010fd07c4b4049449ab79814
Image: registry.k8s.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2
Image ID: docker://sha256:932b0bface75b80e713245d7c2ce8c44b7e127c075bd2d27281a16677c8efef3
Port: <none>
Host Port: <none>
State: Running
Started: Sat, 27 Apr 2024 19:38:41 +0800
Ready: True
Restart Count: 0
Environment:
PROVISIONER_NAME: cluster.local/nfs-subdir-external-provisioner
NFS_SERVER: 192.168.204.8
NFS_PATH: /opt/sonarqube
Mounts:
/persistentvolumes from nfs-subdir-external-provisioner-root (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-t248d (ro)
Conditions:
Type Status
PodReadyToStartContainers True
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
nfs-subdir-external-provisioner-root:
Type: NFS (an NFS mount that lasts the lifetime of a pod)
Server: 192.168.204.8
Path: /opt/sonarqube
ReadOnly: false
kube-api-access-t248d:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Pulled 88s kubelet Container image "registry.k8s.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2" already present on machine
Normal Created 88s kubelet Created container nfs-subdir-external-provisioner
Normal Started 87s kubelet Started container nfs-subdir-external-provisioner
Normal Scheduled 81s default-scheduler Successfully assigned nfs-provisioner/nfs-subdir-external-provisioner-567b586d45-xz8r6 to node1
（10）Kuboard查看
工作负载
容器组
详细信息
5.K8S 1.29版本 部署Sonarqube（第一种方式）
（1）解压
[root@master ~]# cd sonarqube/
[root@master sonarqube]# ls
[root@master sonarqube]# tar -xvf sonarqube-10.5.0+2748.tgz
（2）修改values.yaml文件
[root@master sonarqube]# cd sonarqube/
[root@master sonarqube]# vim values.yaml
……
# 全局搜索"service"关键字
service:
type: NodePort # 类型修改为NodePort
externalPort: 9000
internalPort: 9000
nodePort: 30090 # NodePort对外暴露的端口
……
persistence:
enabled: true #设置为true
……
storageClass: nfs-client #设置为当前集群默认的StorageClass
修改前：
修改后：
（3）创建一个安装Sonarqube用的名字空间
[root@master sonarqube]# kubectl create ns sonarqube
（4）chart安装Sonarqube
[root@master sonarqube]# helm install sonarqube ./sonarqube -n sonarqube
NAME: sonarqube
LAST DEPLOYED: Sat Apr 27 20:12:09 2024
NAMESPACE: sonarqube
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
export NODE_PORT=$(kubectl get --namespace sonarqube -o jsonpath="{.spec.ports[0].nodePort}" services sonarqube-sonarqube)
export NODE_IP=$(kubectl get nodes --namespace sonarqube -o jsonpath="{.items[0].status.addresses[0].address}")
echo http://$NODE_IP:$NODE_PORT
WARNING:
Please note that the SonarQube image runs with a non-root user (uid=1000) belonging to the root group (guid=0). In this way, the chart can support arbitrary user ids as recommended in OpenShift.
Please visit https://docs.openshift.com/container-platform/4.14/openshift_images/create-images.html#use-uid_create-images for more information.
WARNING: The embedded PostgreSQL is intended for evaluation only, it is DEPRECATED, and it will be REMOVED in a future release.
Please visit https://artifacthub.io/packages/helm/sonarqube/sonarqube#production-use-case for more information.
（5）输入命令
[root@master sonarqube]# export NODE_PORT=$(kubectl get --namespace sonarqube -o jsonpath="{.spec.ports[0].nodePort}" services sonarqube-sonarqube)
[root@master sonarqube]# export NODE_IP=$(kubectl get nodes --namespace sonarqube -o jsonpath="{.items[0].status.addresses[0].address}")
[root@master sonarqube]# echo http://$NODE_IP:$NODE_PORT
http://192.168.204.8:30090
（6）node节点拉取postgresql镜像
[root@node2 ~]# docker pull docker.io/bitnami/postgresql:11.14.0-debian-10-r22
#也可以使用替代愿镜像：m.daocloud.io/docker.io/bitnami/postgresql:11.14.0-debian-10-r22 
Kuboard容器组查看
(7) node节点拉取sonarqube镜像
node2拉取镜像
[root@node2 ~]# docker pull sonarqube:10.5.0-community
node2节点导出镜像
[root@node2 ~]# docker save -o sonarqube.tar sonarqube:10.5.0-community
复制Docker镜像到node1节点
[root@node2 ~]# scp sonarqube.tar root@node1:~
node1节点导入Docker镜像
[root@node1 ~]# docker load -i sonarqube.tar
(8) 查看服务
(9)查看卷
[root@master sonarqube]# cd /opt/sonarqube/
[root@master sonarqube]# ls
Kuboard查看
（10）HELM更新配置文件
[root@master sonarqube]# helm upgrade -f sonarqube/values.yaml sonarqube ./sonarqube -n sonarqube
（11）删除项目
[root@master sonarqube]# helm uninstall sonarqube -n sonarqube
5.K8S 1.29版本 部署Sonarqube（第二种方式）
（1）创建NFS
postgresql
[root@master opt]# cd ~
[root@master ~]# mkdir -p /opt/postgre
[root@master ~]# cd /opt
[root@master opt]# chmod 777 postgre/
[root@master opt]# vim /etc/exports
[root@master opt]# exportfs -r
[root@master opt]# showmount -e
Export list for master:
/opt/postgre *
/opt/sonar *
/opt/sonarqube *
/opt/nexus *
/opt/k8s *
sonarqube
[root@master sonarqube]# cd ~
[root@master ~]# mkdir -p /opt/sonar
[root@master ~]#
[root@master ~]# cd /opt
[root@master opt]# chmod 777 sonar/
[root@master opt]# vim /etc/exports
[root@master opt]# exportfs -r
[root@master opt]# showmount -e
Export list for master:
/opt/sonar *
/opt/sonarqube *
/opt/nexus *
/opt/k8s *
（2）创建postgresql的pv
[root@master ~]# vim pv-postgre.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-postgre
spec:
capacity:
storage: 5Gi #配置容量大小
volumeMode: Filesystem
accessModes:
- ReadWriteOnce #配置访问策略为只允许一个节点读写
persistentVolumeReclaimPolicy: Retain #配置回收策略，Retain为手动回收
storageClassName: "pv-postgre" #配置为nfs
nfs:
path: /opt/postgre #配置nfs服务端的共享路径
server: 192.168.204.8 #配置nfs服务器地址
（3）生成资源
[root@master ~]# kubectl apply -f pv-postgre.yaml 
（4）查看pv
[root@master ~]# kubectl get pv
（5）拉取镜像
node2
[root@node2 ~]# docker pull postgres:11.4
(6) 导出镜像
[root@node2 ~]# docker save -o postgres.tar postgres:11.4
（7）复制Docker镜像到node1节点
[root@node2 ~]# scp postgres.tar root@node1:~
（8）node1节点导入Docker镜像
[root@node1 ~]# docker load -i postgres.tar
(9)部署postgresql
[root@master ~]# vim postgre.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: postgre-pvc
namespace: sonarqube
spec:
accessModes:
- ReadWriteOnce
storageClassName: "pv-postgre"
resources:
requests:
storage: 2Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgres-sonar
labels:
app: postgres-sonar
namespace: sonarqube
spec:
replicas: 1
selector:
matchLabels:
app: postgres-sonar
template:
metadata:
labels:
app: postgres-sonar
spec:
containers:
- name: postgres-sonar
image: postgres:11.4
imagePullPolicy: IfNotPresent
ports:
- containerPort: 5432
env:
- name: POSTGRES_DB
value: "sonarDB"
- name: POSTGRES_USER
value: "sonarUser"
- name: POSTGRES_PASSWORD
value: "123456"
resources:
limits:
cpu: 1000m
memory: 2048Mi
requests:
cpu: 500m
memory: 1024Mi
volumeMounts:
- name: data
mountPath: /var/lib/postgresql/data
volumes:
- name: data
persistentVolumeClaim:
claimName: postgre-pvc
---
apiVersion: v1
kind: Service
metadata:
name: postgres-sonar
namespace: sonarqube
labels:
app: postgres-sonar
spec:
ports:
- port: 5432
protocol: TCP
targetPort: 5432
selector:
app: postgres-sonar
(10) 生成资源
[root@master ~]# kubectl apply -f postgre.yaml
(11) 查看pv,pvc
[root@master ~]# kubectl get pv
[root@master ~]# kubectl get pvc -n sonarqube
(12) 查看pod,svc
[root@master ~]# kubectl get pod.svc -n sonarqube
(13)Kuboard查看
工作负载
容器组
服务
（14）创建sonarqube的pv
[root@master ~]# vim pv-sonar.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-sonar
spec:
capacity:
storage: 10Gi #配置容量大小
volumeMode: Filesystem
accessModes:
- ReadWriteOnce #配置访问策略为只允许一个节点读写
persistentVolumeReclaimPolicy: Retain #配置回收策略，Retain为手动回收
storageClassName: "pv-sonar" #配置为nfs
nfs:
path: /opt/sonar #配置nfs服务端的共享路径
server: 192.168.204.8 #配置nfs服务器地址
（15）生成资源
[root@master ~]# kubectl apply -f pv-sonar.yaml
（16）查看pv
[root@master ~]# kubectl get pv
（17）拉取镜像
node1
[root@node1 ~]# docker pull sonarqube:lts
(18) 导出镜像
[root@node1 ~]# docker save -o sonar.tar sonarqube:lts
（19）复制Docker镜像到node1节点
[root@node1 ~]# scp sonar.tar root@node2:~
（20）node1节点导入Docker镜像
[root@node2 ~]# docker load -i sonar.tar
(21) 部署sonarqube
[root@master ~]# vim sonar.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: sonarqube-pvc
namespace: sonarqube
spec:
accessModes:
- ReadWriteOnce
storageClassName: "pv-sonar"
resources:
requests:
storage: 5Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: sonarqube
labels:
app: sonarqube
namespace: sonarqube
spec:
replicas: 1
selector:
matchLabels:
app: sonarqube
template:
metadata:
labels:
app: sonarqube
spec:
initContainers:
- name: init-sysctl
image: busybox
imagePullPolicy: IfNotPresent
command: ["sysctl", "-w", "vm.max_map_count=262144"]
securityContext:
privileged: true
containers:
- name: sonarqube
image: sonarqube:lts
ports:
- containerPort: 9000
env:
- name: SONARQUBE_JDBC_USERNAME
value: "sonarUser"
- name: SONARQUBE_JDBC_PASSWORD
value: "123456"
- name: SONARQUBE_JDBC_URL
value: "jdbc:postgresql://postgres-sonar:5432/sonarDB" #postgres-sonar改成集群IP
livenessProbe:
httpGet:
path: /sessions/new
port: 9000
initialDelaySeconds: 60
periodSeconds: 30
readinessProbe:
httpGet:
path: /sessions/new
port: 9000
initialDelaySeconds: 60
periodSeconds: 30
failureThreshold: 6
resources:
limits:
cpu: 2000m
memory: 2048Mi
requests:
cpu: 1000m
memory: 1024Mi
volumeMounts:
- mountPath: /opt/sonarqube/conf
name: data
subPath: conf
- mountPath: /opt/sonarqube/data
name: data
subPath: data
- mountPath: /opt/sonarqube/extensions
name: data
subPath: extensions
volumes:
- name: data
persistentVolumeClaim:
claimName: sonarqube-pvc
---
apiVersion: v1
kind: Service
metadata:
name: sonarqube
namespace: sonarqube
labels:
app: sonarqube
spec:
type: NodePort
ports:
- name: sonarqube
port: 9000
targetPort: 9000
nodePort: 30090
protocol: TCP
selector:
app: sonarqube
(22) 生成资源
[root@master ~]# kubectl apply -f sonar.yaml
(23) 查看pv,pvc
[root@master ~]# kubectl get pv
[root@master ~]# kubectl get pvc -n sonarqube
(24)查看pod,svc
[root@master ~]# kubectl get pod,svc -n sonarqube
（25）部署ingress
[root@master ~]# vim ingress-sonar.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-sonar
namespace: sonarqube
spec:
ingressClassName: "nginx"
rules:
- host: sonarqube.site
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: sonarqube
port:
number: 9000
(26)生成资源
[root@master ~]# kubectl apply -f ingress-sonar.yaml
（27）查看ingress
[root@master ~]# kubectl get ingress -n sonarqube
（28）详细查看
[root@master ~]# kubectl describe ingress ingress-sonar -n sonarqube
Name: ingress-sonar
Labels: <none>
Namespace: sonarqube
Address: 10.101.23.182
Ingress Class: nginx
Default backend: <default>
Rules:
Host Path Backends
---- ---- --------
sonarqube.site
/ sonarqube:9000 (10.244.166.129:9000)
Annotations: <none>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 72s (x2 over 86s) nginx-ingress-controller Scheduled for sync
Normal Sync 72s (x2 over 86s) nginx-ingress-controller Scheduled for sync
(29）Kuboard查看
应用路由
详细信息
（30）master节点修改hosts
[root@master ~]# vim /etc/hosts
（31）查看
ingress-nginx-controller对外暴露端口为31820
（32）curl测试
[root@master ~]# curl sonarqube.site:31820
（33）物理机修改hosts
（34）访问系统
http://sonarqube.site:31820
（35）输入用户名和密码
账号：admin
密码：admin
(36) 设置新密码
弹出
修改
（37）进入系统
（38）其他方式的Sonarqube部署
可以参考本人博客：
持续集成交付CICD：CentOS 7 安装 Sonarqube9.6-CSDN博客
二、问题
1.chart安装Sonarqube报错
（1）报错
Error: INSTALLATION FAILED: cannot load values.yaml: error converting YAML to JSON: yaml: line 67: mapping values are not allowed in this context
（2）原因
格式错误。
（3）解决方法
修改配置文件。
修改前：
修改后：
2.K8S 部署sonarqube报错
（1）报错
查看pod,svc
查看deploy
(2) 原因分析
JDBC连接postgresql失败，value: "jdbc:postgresql://postgres-sonar:5432/sonarDB"中的postgres-sonar需要写入集群IP。
（3）解决方法
修改配置文件：
[root@master ~]# kubectl edit deploy sonarqube -n sonarqube
修改后：
成功：
[root@master ~]# kubectl get pod,svc -n sonarqube
![[论文笔记]Language Modeling with Gated Convolutional Networks](https://img-blog.csdnimg.cn/img_convert/af6bb99a6f4a50fe3ad92a011c9ff170.png)
