目录
service微服务
ipvs模式
clusterip
headless
nodeport
loadbalancer
metallb
nodeport默认端口
externalname
ingress-nginx
部署
基于路径访问
基于域名访问
TLS加密
auth认证
rewrite重定向
canary金丝雀发布
基于header灰度
基于权重灰度
业务域拆分
service微服务
创建测试示例
[root@k8s2 service]# vim myapp.yml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: myapp
name: myapp
spec:
replicas: 6
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- image: myapp:v1
name: myapp
---
apiVersion: v1
kind: Service
metadata:
labels:
app: myapp
name: myapp
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: myapp
type: ClusterIP //ClusterIP是Kubernetes Service的一种类型。它为同一个Kubernetes集群中的其他Pod提供了访问Service的IP地址。这个IP地址和Service是虚拟的, 不对外暴露，只能在集群内部使用。
ClusterIP Service类型默认使用iptables调度。iptables负责将Service的ClusterIP地址映射到后端Pod的IP地址和端口上，处理请求的负载均衡和高可用性
ipvs模式
修改proxy配置
[root@k8s2 pod]# kubectl -n kube-system edit cm kube-proxy
...
mode: "ipvs" //IPVS是一种高性能负载均衡技术，与iptables相比具有更高的性能和更丰富的负载均衡策略
重启pod
[root@k8s2 pod]# kubectl -n kube-system get pod|grep kube-proxy | awk '{system("kubectl -n kube-system delete pod "$1"")}'
切换ipvs模式后，kube-proxy会在宿主机上添加一个虚拟网卡：kube-ipvs0，并分配service IP
clusterip
clusterip模式只能在集群内访问
[root@k8s2 service]# vim myapp.yml
---
apiVersion: v1
kind: Service
metadata:
labels:
app: myapp
name: myapp
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: myapp
type: ClusterIP
service创建后集群DNS提供解析
headless
[root@k8s2 service]# vim myapp.yml
---
apiVersion: v1
kind: Service
metadata:
labels:
app: myapp
name: myapp
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: myapp
type: ClusterIP
clusterIP: None //当 ClusterIP 设置为 None 时，Kubernetes 不会为该 Service 分配一个 Cluster IP，并且 DNS 请求的返回值将不会是 Service 的虚拟 IP，而是真实的 Pod IP。
[root@k8s2 service]# kubectl delete svc myapp
[root@k8s2 service]# kubectl apply -f myapp.yml
headless模式不分配vip
headless通过svc名称访问，由集群内dns提供解析
集群内直接使用service名称访问
nodeport
[root@k8s2 service]# vim myapp.yml
---
apiVersion: v1
kind: Service
metadata:
labels:
app: myapp
name: myapp
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: myapp
type: NodePort //NodePort 类型会在每个 Node 上监听一个静态端口，并将请求转发到后端 Pod 的 Cluster IP。这种类型的 Service 可以通过 <NodeIP>:<NodePort> 的方式访问到后端 Pod。
使用 NodePort 类型的 Service 可以让外部用户通过 Node IP 和 Node 端口来访问到集群内部的服务。
[root@k8s2 service]# kubectl apply -f myapp.yml
[root@k8s2 service]# kubectl get svc
nodeport在集群节点上绑定端口，一个端口对应一个服务
loadbalancer
[root@k8s2 service]# vim myapp.yml
---
apiVersion: v1
kind: Service
metadata:
labels:
app: myapp
name: myapp
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: myapp
type: LoadBalancer
//使用 LoadBalancer 类型的 Service 可以在集群外部暴露一个负载均衡器，让外部用户能够轻松访问到集群内部的服务。
使用 LoadBalancer 类型的 Service 必须在 Kubernetes 集群部署在某些支持云服务商的基础设施上，否则该类型的 Service 无法正常工作。
LoadBalancer模式适用云平台，裸金属环境需要安装metallb提供支持
metallb
官网：MetalLB, bare metal load-balancer for Kubernetes
[root@k8s2 service]# kubectl edit configmap -n kube-system kube-proxy
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: "ipvs"
ipvs:
strictARP: true //启用 Kubernetes Service 的 strictARP 选项可以防止 ARP 欺骗攻击，提高网络安全性
[root@k8s2 service]# kubectl -n kube-system get pod|grep kube-proxy | awk '{system("kubectl -n kube-system delete pod "$1"")}'
下载部署文件
[root@k8s2 metallb]# wget https://raw.githubusercontent.com/metallb/metallb/v0.13.11/config/manifests/metallb-native.yaml
修改文件中镜像地址,与harbor仓库路径保持一致
上传镜像到harbor
部署服务
[root@k8s2 metallb]# kubectl apply -f metallb-native.yaml
[root@k8s2 metallb]# kubectl -n metallb-system get pod
配置分段地址段
[root@k8s2 metallb]# vim config.yaml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: first-pool
namespace: metallb-system
spec:
addresses:
- 192.168.81.100-192.168.81.200 //修改为自己本地地址段
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: example
namespace: metallb-system
spec:
ipAddressPools:
- first-pool
通过分配地址从集群外访问服务
nodeport默认端口
[root@k8s2 service]# vim myapp.yml
apiVersion: v1
kind: Service
metadata:
labels:
app: myapp
name: myapp
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
nodePort: 33333
selector:
app: myapp
type: NodePort
nodeport默认端口是30000-32767，超出会报错
添加如下参数，端口范围可以自定义
[root@k8s2 service]# vim /etc/kubernetes/manifests/kube-apiserver.yaml
--service-node-port- range=30000-50000
修改后api-server会自动重启，等apiserver正常启动后才能操作集群
externalname
[root@k8s2 service]# vim externalname.yaml
apiVersion: v1
kind: Service
metadata:
name: my-service
spec:
type: ExternalName //ExternalName 类型的 Service 不会创建任何 ClusterIP、NodePort、LoadBalancer 类型的 Endpoint，在集群内部也不会创建任何 Pod，它会将 Service 名称与一个外部的 DNS 名称（或者 IP 地址）关联起来
externalName: www.westos.org //当集群内部的应用程序访问该 Service 时，将会通过集群的 DNS 解析服务解析该 Service 名称，返回关联的 www.westos.org 的 DNS 名称
ingress-nginx
ingress-nginx 是一个 Kubernetes 上的 Ingress 控制器，它提供了负载均衡、SSL/TLS 等功能，可以将外部访问 Kubernetes 集群内部的服务
部署
官网：https://kubernetes.github.io/ingress-nginx/deploy/#bare-metal-clusters
下载部署文件，上传镜像到harbor
修改3个镜像路径
修改为LoadBalancer方式
[root@k8s2 ingress]# kubectl -n ingress-nginx edit svc ingress-nginx-controller
创建ingress策略
[root@k8s2 ingress]# vim ingress.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: minimal-ingress
spec:
ingressClassName: nginx
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myapp
port:
number: 80
基于路径访问
创建svc
[root@k8s2 ingress]# vim myapp-v1.yml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: myapp-v1
name: myapp-v1
spec:
replicas: 3
selector:
matchLabels:
app: myapp-v1
template:
metadata:
labels:
app: myapp-v1
spec:
containers:
- image: myapp:v1
name: myapp-v1
---
apiVersion: v1
kind: Service
metadata:
labels:
app: myapp-v1
name: myapp-v1
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: myapp-v1
type: ClusterIP
[root@k8s2 ingress]# vim myapp-v2.yml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: myapp-v2
name: myapp-v2
spec:
replicas: 3
selector:
matchLabels:
app: myapp-v2
template:
metadata:
labels:
app: myapp-v2
spec:
containers:
- image: myapp:v2
name: myapp-v2
---
apiVersion: v1
kind: Service
metadata:
labels:
app: myapp-v2
name: myapp-v2
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: myapp-v2
type: ClusterIP
创建ingress
[root@k8s2 ingress]# vim ingress1.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: minimal-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
ingressClassName: nginx
rules:
- host: myapp.westos.org
http:
paths:
- path: /v1
pathType: Prefix
backend:
service:
name: myapp-v1
port:
number: 80
- path: /v2
pathType: Prefix
backend:
service:
name: myapp-v2
port:
number: 80
测试
基于域名访问
[root@k8s2 ingress]# vim ingress2.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: minimal-ingress
spec:
ingressClassName: nginx
rules:
- host: myapp1.westos.org
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myapp-v1
port:
number: 80
- host: myapp2.westos.org
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myapp-v2
port:
number: 80
TLS加密
创建证书
[root@k8s2 ingress]# openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=nginxsvc/O=nginxsvc"
[root@k8s2 ingress]# kubectl create secret tls tls-secret --key tls.key --cert tls.crt
[root@k8s2 ingress]# vim ingress3.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-tls
spec:
tls:
- hosts:
- myapp.westos.org
secretName: tls-secret
ingressClassName: nginx
rules:
- host: myapp.westos.org
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myapp-v1
port:
number: 80
auth认证
创建认证文件
[root@k8s2 ingress]# yum install -y httpd-tools
[root@k8s2 ingress]# htpasswd -c auth shx
[root@k8s2 ingress]# kubectl create secret generic basic-auth --from-file=auth
[root@k8s2 ingress]# vim ingress3.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-tls
annotations:
nginx.ingress.kubernetes.io/auth-type: basic
nginx.ingress.kubernetes.io/auth-secret: basic-auth
nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - shx'
spec:
tls:
- hosts:
- myapp.westos.org
secretName: tls-secret
ingressClassName: nginx
rules:
- host: myapp.westos.org
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myapp-v1
port:
number: 80
rewrite重定向
示例一：
[root@k8s2 ingress]# vim ingress3.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-tls
annotations:
nginx.ingress.kubernetes.io/auth-type: basic
nginx.ingress.kubernetes.io/auth-secret: basic-auth
nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - shx'
nginx.ingress.kubernetes.io/app-root: /hostname.html
spec:
tls:
- hosts:
- myapp.westos.org
secretName: tls-secret
ingressClassName: nginx
rules:
- host: myapp.westos.org
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myapp-v1
port:
number: 80
示例二：
[root@k8s2 ingress]# vim ingress3.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-tls
annotations:
nginx.ingress.kubernetes.io/auth-type: basic
nginx.ingress.kubernetes.io/auth-secret: basic-auth
nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - shx'
#nginx.ingress.kubernetes.io/app-root: /hostname.html
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
tls:
- hosts:
- myapp.westos.org
secretName: tls-secret
ingressClassName: nginx
rules:
- host: myapp.westos.org
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myapp-v1
port:
number: 80
- path: /westos(/|$)(.*)
pathType: ImplementationSpecific
backend:
service:
name: myapp-v1
port:
number: 80
canary金丝雀发布
基于header灰度
[root@k8s2 ingress]# vim ingress4.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myapp-v1-ingress
spec:
ingressClassName: nginx
rules:
- host: myapp.westos.org
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: myapp-v1
port:
number: 80
[root@k8s2 ingress]# vim ingress5.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-by-header: stage
nginx.ingress.kubernetes.io/canary-by-header-value: gray
name: myapp-v2-ingress
spec:
ingressClassName: nginx
rules:
- host: myapp.westos.org
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: myapp-v2
port:
number: 80
测试
基于权重灰度
[root@k8s2 ingress]# vim ingress5.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/canary: "true"
#nginx.ingress.kubernetes.io/canary-by-header: stage
#nginx.ingress.kubernetes.io/canary-by-header-value: gray
nginx.ingress.kubernetes.io/canary-weight: "50"
nginx.ingress.kubernetes.io/canary-weight-total: "100"
name: myapp-v2-ingress
spec:
ingressClassName: nginx
rules:
- host: myapp.westos.org
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: myapp-v2
port:
number: 80
测试
[root@k8s1 ~]# vim ingress.sh
#!/bin/bash
v1=0
v2=0
for (( i=0; i<100; i++))
do
response=`curl -s myapp.westos.org |grep -c v1`
v1=`expr $v1 + $response`
v2=`expr $v2 + 1 - $response`
done
echo "v1:$v1, v2:$v2"
业务域拆分
[root@k8s2 ingress]# vim ingress6.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
name: rewrite-ingress
spec:
ingressClassName: nginx
rules:
- host: myapp.westos.org
http:
paths:
- path: /user/(.*)
pathType: Prefix
backend:
service:
name: myapp-v1
port:
number: 80
- path: /order/(.*)
pathType: Prefix
backend:
service:
name: myapp-v2
port:
number: 80
测试
