K8S - 理解ClusterIP - 集群内部service之间的反向代理和loadbalancer

news2025/1/12 20:40:02

在Micro Service的治理中。

有两个很重要的点，

  1. 集群外部的用户/service 如何访问集群内的 入口服务(例如UI service）
  2. 集群内的service A 如何 访问 集群内的service B

为什么有上面的问题
无非是：

  1. 集群内的service 都是多实例的
  2. 每个service 实例都有单独不同的ip
  3. 如何负载均衡？

如图：
在这里插入图片描述





Spring Cloud 是如何就解决这两个问题的

集群外 to 集群内

  1. 用spring cloud gateway 来反向代理集群内的对外service, 例如图中的Service A, 如果其他Service 没有被配置在gateway中， 集群外部是无法直接访问的， 更加安全。 通常这个api gateway所在的server 具有双网卡， 1个ip在外网， 1个ip在集群内网

  2. 同是Spring Cloud Gateway 自带Load balancer 功能(基于 Spring cloud loadbalancer) , 所以即使要exposed 的service 有多个实例， Gateway同样可以根据指定规则 分发到不同的instance.

集群内 Service A to Service B
  1. 使用Eureke 作为注册中心， 每个service 的instance 都要往里面注册， 以给每个service 的多个instance 获得1个common的service Name作为DNS
  2. 使用Ribbon(继承在Eureka) 中， 作为load balancer 进行request转发

如图：
在这里插入图片描述





k8s 是如何就解决这两个问题的

K8S 的service 包括了很多种，
ingress, nodeport, clusterIp, externalName 都是属于service的

集群外 to 集群内

  1. 使用ingress or NodePort 来作为纵向流量代理， 而ingress 和 NodePort 都是自带load balancer 的。
    置于什么是纵向横向流量
    参考
    在这里插入图片描述

  2. 使用ClusterIP 作为 Service B之的反向代理， ClusterIP 的service自带loadbalancer 功能， 这样Service A就可以通过ClusterIP service的名字DNS 来访问Service B了

  3. 虽然k8s 没用Eureka, Nacos等注册中心， 但是实际上k8s 的service list 实际上就是1个注册中心了！

原理如图：
在这里插入图片描述





ClusterIP 的定义和简单介绍

  1. 集群内部通信：ClusterIP 为 Service 提供了一个虚拟的内部 IP 地址，用于在 Kubernetes 集群内的其他组件和服务之间进行通信。其他 Pod 可以通过该虚拟 IP 地址和 Service 的端口来访问该 Service。
  2. 内部负载均衡：ClusterIP 实现了基于轮询算法的负载均衡，它将请求均匀地分发给 Service 关联的后端 Pod。这意味着无论有多少个后端 Pod，它们都可以被平等地访问，从而实现负载均衡和高可用性。
    集群外部不可访问：ClusterIP 分配的 IP 地址只在 Kubernetes 集群内部可见，对集群外部不可访问。它不直接暴露给外部网络，因此不能直接从集群外部访问该 IP 地址。
  3. 适用于内部服务：ClusterIP 适用于内部服务，即那些只需要在 Kubernetes 集群内部可访问的服务。这些服务通常用于应用程序的内部组件之间的通信，例如数据库连接、队列服务等。
  4. 可用于其他类型的 Service：ClusterIP 可以作为其他类型的 Service（如 NodePort、LoadBalancer 或 Ingress）的后端服务。通过将其他类型的 Service 配置为使用 ClusterIP 类型的 Service，可以将请求转发到 ClusterIP 提供的虚拟 IP 地址上。

总的来说，ClusterIP 是 Kubernetes 集群内部的一种服务发现和负载均衡机制，用于实现集群内部的内部通信和服务访问。它提供了一个虚拟 IP 地址给 Service，并通过负载均衡算法将请求分发给关联的后端 Pod。ClusterIP 适用于内部服务，不直接对外部公开。





NodePort 和 ClusterIP 的具体例子

解下来我会用 NodePort 和 ClusterIP 来demo 以下 k8s service A 如何 访问 ServiceB
置于从集群外访问为何不用ingress, 是因为k8s 的博文系列还没提到Ingress.

大概框架
在这里插入图片描述

在这个例子中
我们会部署：
Service A: bq-api-service
Service B: cloud-user
nodePort service: nodeport-bq-api-service
clusterIP service: clusterip-cloud-user

置于这里两个service 具体是什么不重要， 可以认为它们是两个简单的springboot service 并没有集成任何spring cloud 的框架。





cleanup

当前k8s 环境是干净的

[gateman@manjaro-x13 bq-api-service]$ kubectl get all -o wide
NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   77d   <none>





部署 Service B ， cloud-user service



更新info 接口让其return hostname

先update /actuator/info 接口 让其可以return 当前service 所在server/container 的hostname

@Component
@Slf4j
public class AppVersionInfo implements InfoContributor {

    @Value("${pom.version}") // https://stackoverflow.com/questions/3697449/retrieve-version-from-maven-pom-xml-in-code
    private String appVersion;

    @Autowired
    private String hostname;

    @Value("${spring.datasource.url}")
    private String dbUrl;

    @Override
    public void contribute(Info.Builder builder) {
        log.info("AppVersionInfo: contribute ...");
        builder.withDetail("app", "Cloud User API")
                .withDetail("version", appVersion)
                .withDetail("hostname",hostname)
                .withDetail("dbUrl", dbUrl)
                .withDetail("description", "This is a simple Spring Boot application to demonstrate the use of BigQuery in GCP.");
    }
}

测试效果：

[gateman@manjaro-x13 bq-api-service]$ curl 127.0.0.1:8080/actuator/info
{"app":"Cloud User API","version":"0.0.1","hostname":"manjaro-x13","dbUrl":"jdbc:mysql://34.39.2.90:6033/demo_cloud_user?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPublicKeyRetrieval=true","description":"This is a simple Spring Boot application to demonstrate the use of BigQuery in GCP."}
[gateman@manjaro-x13 bq-api-service]$



利用cloudbuild 和 其trigger 让其自动部署docker image 到GAR (google artifact repository)

cloudbuild-gar.yaml

# just to update the docker image to GAR with the pom.xml version

steps:
  - id: run maven install
    name: maven:3.9.6-sapmachine-17 # https://hub.docker.com/_/maven
    entrypoint: bash
    args:
      - '-c'
      - |
        whoami
        set -x
        pwd
        mvn install
        cat pom.xml | grep -m 1 "<version>" | sed -e 's/.*<version>\([^<]*\)<\/version>.*/\1/' > /workspace/version.txt
        echo "Version: $(cat /workspace/version.txt)"


  - id: build and push docker image
    name: 'gcr.io/cloud-builders/docker'
    entrypoint: bash
    args:
      - '-c'
      - |
        set -x
        echo "Building docker image with tag: $(cat /workspace/version.txt)"
        docker build -t $_GAR_BASE/$PROJECT_ID/$_DOCKER_REPO_NAME/${_APP_NAME}:$(cat /workspace/version.txt) .
        docker push $_GAR_BASE/$PROJECT_ID/$_DOCKER_REPO_NAME/${_APP_NAME}:$(cat /workspace/version.txt)


logsBucket: gs://jason-hsbc_cloudbuild/logs/
options: # https://cloud.google.com/cloud-build/docs/build-config#options
  logging: GCS_ONLY # or CLOUD_LOGGING_ONLY https://cloud.google.com/cloud-build/docs/build-config#logging

substitutions:
  _DOCKER_REPO_NAME: my-docker-repo
  _APP_NAME: cloud-user
  _GAR_BASE: europe-west2-docker.pkg.dev

cloudbuild trigger:
terraform：

# referring https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/cloudbuild_trigger
resource "google_cloudbuild_trigger" "cloud-user-gar-trigger" {
  name = "cloud-user-gar-trigger" # could not contains underscore

  location = var.region_id

  # when use github then should use trigger_template
  github {
    name = "demo_cloud_user"
    owner = "nvd11"
    push {
      branch = "main"
      invert_regex = false # means trigger on branch
    }
  }


  filename = "cloudbuild-gar.yaml"
  # projects/jason-hsbc/serviceAccounts/terraform@jason-hsbc.iam.gserviceaccount.com
  service_account = data.google_service_account.cloudbuild_sa.id 
}

这样， 一但有任何commit 推送到github main branch
cloudbuild 就会自动打包docker image 到指定的 GAR 仓库
url：
europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/cloud-user:xxx

其中xxx 是pom.xml 里定义的version 数字

有了这个image path,就方便了后面在k8s 部署



编写yaml 脚本

deployment-cloud-user.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels: # label of this deployment
    app: cloud-user # custom defined
    author: nvd11
  name: deployment-cloud-user # name of this deployment
  namespace: default
spec:
  replicas: 4            # desired replica count, Please note that the replica Pods in a Deployment are typically distributed across multiple nodes.
  revisionHistoryLimit: 10 # The number of old ReplicaSets to retain to allow rollback
  selector: # label of the Pod that the Deployment is managing,, it's mandatory, without it , we will get this error 
            # error: error validating data: ValidationError(Deployment.spec.selector): missing required field "matchLabels" in io.k8s.apimachinery.pkg.apis.meta.v1.LabelSelector ..
    matchLabels:
      app: cloud-user
  strategy: # Strategy of upodate
    type: RollingUpdate # RollingUpdate or Recreate
    rollingUpdate:
      maxSurge: 25% # The maximum number of Pods that can be created over the desired number of Pods during the update
      maxUnavailable: 25% # The maximum number of Pods that can be unavailable during the update
  template: # Pod template
    metadata:
      labels:
        app: cloud-user # label of the Pod that the Deployment is managing. must match the selector, otherwise, will get the error Invalid value: map[string]string{"app":"bq-api-xxx"}: `selector` does not match template `labels`
    spec:
      containers:
      - image: europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/cloud-user:1.0.1 # image of the container
        imagePullPolicy: IfNotPresent
        name: container-cloud-user
        env: # set env varaibles
        - name: APP_ENVIRONMENT
          value: prod
      restartPolicy: Always # Restart policy for all containers within the Pod
      terminationGracePeriodSeconds: 10 # The period of time in seconds given to the Pod to terminate gracefully



部署yaml
[gateman@manjaro-x13 cloud-user]$ kubectl apply -f deployment-cloud-user.yaml 
deployment.apps/deployment-cloud-user created
[gateman@manjaro-x13 cloud-user]$ kubectl get pods -o wide
NAME                                     READY   STATUS    RESTARTS   AGE    IP             NODE        NOMINATED NODE   READINESS GATES
deployment-cloud-user-65fb8d79fd-28vmn   1/1     Running   0          104s   10.244.2.133   k8s-node0   <none>           <none>
deployment-cloud-user-65fb8d79fd-9rjln   1/1     Running   0          104s   10.244.2.134   k8s-node0   <none>           <none>
deployment-cloud-user-65fb8d79fd-m8xv4   1/1     Running   0          104s   10.244.1.67    k8s-node1   <none>           <none>
deployment-cloud-user-65fb8d79fd-ndvjb   1/1     Running   0          104s   10.244.3.76    k8s-node3   <none>           <none>

可以见到 4个pods 跑起来了



初步测试

cloud-user 是部署好了， 但是它没有配置nodeport 和 clusterIP 等任何service, 所以它是无法被nodes 的service 访问的。

上面的pods信息里显示了 ip address, 但那些ip address 是容器level, 只能被另1个容器访问。

这样的话， 我们可以进入1个新建的容器内测试:

新建dns-test 测试pod
[gateman@manjaro-x13 cloud-user]$ kubectl run dns-test --image=odise/busybox-curl --restart=Never -- /bin/sh -c "while true; do echo hello docker; sleep 1; done"
pod/dns-test created

这样dns-test pod 就创建成功了， 之所以要加上一段 while死循环是避免这个pod 自动complete退出

进入测试容器
[gateman@manjaro-x13 cloud-user]$ kubectl get pods -o wide
NAME                                     READY   STATUS    RESTARTS   AGE   IP             NODE        NOMINATED NODE   READINESS GATES
deployment-cloud-user-65fb8d79fd-28vmn   1/1     Running   0          15m   10.244.2.133   k8s-node0   <none>           <none>
deployment-cloud-user-65fb8d79fd-9rjln   1/1     Running   0          15m   10.244.2.134   k8s-node0   <none>           <none>
deployment-cloud-user-65fb8d79fd-m8xv4   1/1     Running   0          15m   10.244.1.67    k8s-node1   <none>           <none>
deployment-cloud-user-65fb8d79fd-ndvjb   1/1     Running   0          15m   10.244.3.76    k8s-node3   <none>           <none>
dns-test                                 1/1     Running   0          6s    10.244.2.135   k8s-node0   <none>           <none>


[gateman@manjaro-x13 cloud-user]$ kubectl exec -it dns-test -- /bin/sh
/ #

十分简单

在容器内调用各个pod的api
/ # curl 10.244.2.133:8080/actuator/info
{"app":"Cloud User API","version":"1.0.1","hostname":"deployment-cloud-user-65fb8d79fd-28vmn","dbUrl":"jdbc:mysql://192.168.0.42:3306/demo_cloud_user?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPublicKeyRetrieval=true","description":"This is a simple Spring Boot application to demonstrate the use of BigQuery in GCP."}/ # 
/ # 
/ # curl 10.244.2.134:8080/actuator/info
{"app":"Cloud User API","version":"1.0.1","hostname":"deployment-cloud-user-65fb8d79fd-9rjln","dbUrl":"jdbc:mysql://192.168.0.42:3306/demo_cloud_user?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPublicKeyRetrieval=true","description":"This is a simple Spring Boot application to demonstrate the use of BigQuery in GCP."}/ # 
/ # 
/ # 
/ # curl 10.244.1.67:8080/actuator/info
{"app":"Cloud User API","version":"1.0.1","hostname":"deployment-cloud-user-65fb8d79fd-m8xv4","dbUrl":"jdbc:mysql://192.168.0.42:3306/demo_cloud_user?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPublicKeyRetrieval=true","description":"This is a simple Spring Boot application to demonstrate the use of BigQuery in GCP."}/ # 
/ # 
/ # curl 10.244.3.76:8080/actuator/info
{"app":"Cloud User API","version":"1.0.1","hostname":"deployment-cloud-user-65fb8d79fd-ndvjb","dbUrl":"jdbc:mysql://192.168.0.42:3306/demo_cloud_user?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPublicKeyRetrieval=true","description":"This is a simple Spring Boot application to demonstrate the use of BigQuery in GCP."}/ # 
/ # 
/ #

可以见到4个pod的 service 都可以被dns-test 容器内call 通， 能分别return 它们的hostname, 但是调用时要指定ip ， 无法做到统一入口 和 load balance





部署 ClusterIP - clusterip-cloud-user

编写yaml

clusterip-cloud-user.yaml

apiVersion: v1
kind: Service
metadata:
  name: clusterip-cloud-user
spec:
  selector:
    app: cloud-user # for the pods that have the label app: cloud-user
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080
  type: ClusterIP

由于加上了selector ， 所以endpoint 也会自动创建

部署yaml
[gateman@manjaro-x13 cloud-user]$ kubectl create -f clusterip-cloud-user.yaml 
service/clusterip-cloud-user created

检查一下：

[gateman@manjaro-x13 cloud-user]$ kubectl get svc -o wide
NAME                   TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)    AGE   SELECTOR
clusterip-cloud-user   ClusterIP   10.96.11.18   <none>        8080/TCP   29s   app=cloud-user
kubernetes             ClusterIP   10.96.0.1     <none>        443/TCP    77d   <none>
[gateman@manjaro-x13 cloud-user]$ kubectl get ep -o wide
NAME                   ENDPOINTS                                                          AGE
clusterip-cloud-user   10.244.1.67:8080,10.244.2.133:8080,10.244.2.134:8080 + 1 more...   2m36s
kubernetes             192.168.0.3:6443                                                   77d

可以见到1个cluster ip service 已被创建

名字是 clusterip-cloud-user, 类型是ClusterIP, Cluster-IP 就是所谓的虚拟ip

在endpoints 里面， 可以见到这个clusterip service 代理的是 4个 ip和端口的组合， 它们实际上就是 cloud-user 的4个pods

初步测试

ClusterIP 和 NodePort 不一样， 是无法从容器外部直接访问的，
所以我们还是需要进入测试容器类测试

kubectl exec -it dns-test -- /bin/sh

之后我们可以用 $serviceName:$\port 去访问endpoints里的service了

/ # nslookup clusterip-cloud-user
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name:      clusterip-cloud-user
Address 1: 10.96.11.18 clusterip-cloud-user.default.svc.cluster.local

`
/ # ping clusterip-cloud-user

PING clusterip-cloud-user (10.96.11.18): 56 data bytes
^C
--- clusterip-cloud-user ping statistics ---
10 packets transmitted, 0 packets received, 100% packet loss
/ # curl clusterip-cloud-user:8080/actuator/info
{"app":"Cloud User API","version":"1.0.1","hostname":"deployment-cloud-user-65fb8d79fd-m8xv4","dbUrl":"jdbc:mysql://192.168.0.42:3306/demo_cloud_user?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPub/ # curl clusterip-cloud-user:8080/actuator/info
{"app":"Cloud User API","version":"1.0.1","hostname":"deployment-cloud-user-65fb8d79fd-28vmn","dbUrl":"jdbc:mysql://192.168.0.42:3306/demo_cloud_user?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPub/ # curl clusterip-cloud-user:8080/actuator/info
{"app":"Cloud User API","version":"1.0.1","hostname":"deployment-cloud-user-65fb8d79fd-9rjln","dbUrl":"jdbc:mysql://192.168.0.42:3306/demo_cloud_user?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPub/ # curl clusterip-cloud-user:8080/actuator/info
{"app":"Cloud User API","version":"1.0.1","hostname":"deployment-cloud-user-65fb8d79fd-9rjln","dbUrl":"jdbc:mysql://192.168.0.42:3306/demo_cloud_user?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPublicKeyRetrieval=true","description":"This is a simple Spring Boot application to demonstrate the use of BigQuery in GCP."}

虽然ping 是无法ping 通的， 可能没开通ICMP 协议

但是我们的确可以通过 clusterip的service 去访问 cloud-user 的4个instance ， 而且是随机分配的， 时间了load balance的功能！





部署 Service A - bq-api-service

修改配置

部署之前， 我们在 bq-api-service 先增加1个接口 /ext-service/user-service/info
在这个接口内， 会调用 cloud-user 的 /actuator/info 接口

方便测试

Controller

    @Autowired
    private UserService userService;

    @GetMapping("/user-service/info")
    public ResponseEntity<ApiResponse<ServiceInfoDao>> userServiceInfo() {
        ServiceInfoDao userServiceInfo = null;
        try {
            userServiceInfo = this.userService.getServiceInfo();
            ApiResponse<ServiceInfoDao> response = new ApiResponse<>();
            response.setData(userServiceInfo);
            response.setReturnCode(0);
            response.setReturnMsg("user service is running in the host: " + userServiceInfo.getHostname());
            return ResponseEntity.ok(response);
        } catch (Exception e) {
            log.error("Error in getUserById...", e);
            ApiResponse<ServiceInfoDao> response = new ApiResponse<>();
            response.setReturnCode(-1);
            response.setReturnMsg("Error in getting user service info: " + e.getMessage());
            return ResponseEntity.status(500).body(response);
        }
    }

Service

    @Override
    public ServiceInfoDao getServiceInfo() {
        log.info("getServiceInfo()...");
        return userClient.getServiceInfo();
    }

feignclient:

@FeignClient(name = "demo-cloud-user", url="${hostIp.cloud-user}")
public interface UserClient {

    @GetMapping("/actuator/info")
    ServiceInfoDao getServiceInfo();
}

在feign client里见到 ip address 是配置在配置文件中的。
正好， 我们增加1个新的application-k8s 配置文件
application-k8s.yaml

## 其他配置

hostIp:
  cloud-user: clusterip-cloud-user:8080

关键我们不需要再指定 cloud-user 部署在哪里的ip了， 也不用关心它有多少instance, 跟spring cloud 用法很类似， 只需要提供1个名字

在spring cloud 中我们需要提供cloud-user 在eureka注册的名字
在k8s 我们需要提供用于反向代理的 clusterIP service 的名字

部署docker image 上GAR

同样的方法
url： europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/bq-api-service:xxx

编写yaml

deployment-bq-api-service.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels: # label of this deployment
    app: bq-api-service # custom defined
    author: Jason
  name: deployment-bq-api-service # name of this deployment
  namespace: default
spec:
  replicas: 4            # desired replica count, Please note that the replica Pods in a Deployment are typically distributed across multiple nodes.
  revisionHistoryLimit: 10 # The number of old ReplicaSets to retain to allow rollback
  selector: # label of the Pod that the Deployment is managing,, it's mandatory, without it , we will get this error 
            # error: error validating data: ValidationError(Deployment.spec.selector): missing required field "matchLabels" in io.k8s.apimachinery.pkg.apis.meta.v1.LabelSelector ..
    matchLabels:
      app: bq-api-service
  strategy: # Strategy of upodate
    type: RollingUpdate # RollingUpdate or Recreate
    rollingUpdate:
      maxSurge: 25% # The maximum number of Pods that can be created over the desired number of Pods during the update
      maxUnavailable: 25% # The maximum number of Pods that can be unavailable during the update
  template: # Pod template
    metadata:
      labels:
        app: bq-api-service # label of the Pod that the Deployment is managing. must match the selector, otherwise, will get the error Invalid value: map[string]string{"app":"bq-api-xxx"}: `selector` does not match template `labels`
    spec:
      containers:
      - image: europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/bq-api-service:1.2.1 # image of the container
        imagePullPolicy: IfNotPresent
        name: container-bq-api-service
        env: # set env varaibles
        - name: APP_ENVIRONMENT
          value: k8s
      restartPolicy: Always # Restart policy for all containers within the Pod
      terminationGracePeriodSeconds: 10 # The period of time in seconds given to the Pod to terminate gracefully

同样4个实例， 注意的是环境变量要正确地 配置成 k8s`

        env: # set env varaibles
        - name: APP_ENVIRONMENT
          value: k8s



部署yaml
deployment.apps/deployment-bq-api-service created
[gateman@manjaro-x13 bq-api-service]$ kubectl get po -o wide
NAME                                        READY   STATUS    RESTARTS   AGE    IP             NODE        NOMINATED NODE   READINESS GATES
deployment-bq-api-service-778cf8f54-677vl   1/1     Running   0          34s    10.244.2.136   k8s-node0   <none>           <none>
deployment-bq-api-service-778cf8f54-nfzhg   1/1     Running   0          34s    10.244.3.77    k8s-node3   <none>           <none>
deployment-bq-api-service-778cf8f54-q9lfx   1/1     Running   0          34s    10.244.1.68    k8s-node1   <none>           <none>
deployment-bq-api-service-778cf8f54-z72dr   1/1     Running   0          34s    10.244.3.78    k8s-node3   <none>           <none>
deployment-cloud-user-65fb8d79fd-28vmn      1/1     Running   0          121m   10.244.2.133   k8s-node0   <none>           <none>
deployment-cloud-user-65fb8d79fd-9rjln      1/1     Running   0          121m   10.244.2.134   k8s-node0   <none>           <none>
deployment-cloud-user-65fb8d79fd-m8xv4      1/1     Running   0          121m   10.244.1.67    k8s-node1   <none>           <none>
deployment-cloud-user-65fb8d79fd-ndvjb      1/1     Running   0          121m   10.244.3.76    k8s-node3   <none>           <none>
dns-test                                    1/1     Running   0          105m   10.244.2.135   k8s-node0   <none>           <none>

可以见到 4个 bq-api-service 的pods 也起来了



初步测试

因为没有nodeport ， 我们还是需要进入测试容器

kubectl exec -it dns-test -- /bin/sh

还是单独地测试1个instance, 先记住2个ip 10.244.2.136， 10.244.3.77

先测试该service 的info

/ # curl 10.244.2.136:8080/actuator/info
{"app":"Sales API","version":"1.2.1","hostname":"deployment-bq-api-service-778cf8f54-677vl","description":"This is a simple Spring Boot application to demonstrate the use of BigQuery in GCP."}

可以见到是已经启动了

然后在测试它的 /ext-service/user-service/info 接口

/ # curl 10.244.3.77:8080/ext-service/user-service/info
{"returnCode":0,"returnMsg":"user service is running in the host: deployment-cloud-user-65fb8d79fd-m8xv4","data":{"app":"Cloud User API","version":"1.0.1","description":"This is a simple Spring Boot application to demonstrate the use of BigQuery in GCP.","hostname":"deployment-cloud-user-65fb8d79fd-m8xv4","dbUrl":"jdbc:mysql://192.168.0.42:3306/demo_cloud_user?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowP/ # curl 10.244.3.77:8080/ext-service/user-service/info
{"returnCode":0,"returnMsg":"user service is running in the host: deployment-cloud-user-65fb8d79fd-m8xv4","data":{"app":"Cloud User API","version":"1.0.1","description":"This is a simple Spring Boot application to demonstrate the use of BigQuery in GCP.","hostname":"deployment-cloud-user-65fb8d79fd-m8xv4","dbUrl":"jdbc:mysql://192.168.0.42:3306/demo_cloud_user?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowP/ # curl 10.244.3.77:8080/ext-service/user-service/info
{"returnCode":0,"returnMsg":"user service is running in the host: deployment-cloud-user-65fb8d79fd-m8xv4","data":{"app":"Cloud User API","version":"1.0.1","description":"This is a simple Spring Boot application to demonstrate the use of BigQuery in GCP.","hostname":"deployment-cloud-user-65fb8d79fd-m8xv4","dbUrl":"jdbc:mysql://192.168.0.42:3306/demo_cloud_user?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowP/ # curl 10.244.3.77:8080/ext-service/user-service/info
{"returnCode":0,"returnMsg":"user service is running in the host: deployment-cloud-user-65fb8d79fd-m8xv4","data":{"app":"Cloud User API","version":"1.0.1","description":"This is a simple Spring Boot application to demonstrate the use of BigQuery in GCP.","hostname":"deployment-cloud-user-65fb8d79fd-m8xv4","dbUrl":"jdbc:mysql://192.168.0.42:3306/demo_cloud_user?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowP/ # curl 10.244.2.136:8080/ext-service/user-service/info
{"returnCode":0,"returnMsg":"user service is running in the host: deployment-cloud-user-65fb8d79fd-28vmn","data":{"app":"Cloud User API","version":"1.0.1","description":"This is a simple Spring Boot application to demonstrate the use of BigQuery in GCP.","hostname":"deployment-cloud-user-65fb8d79fd-28vmn","dbUrl":"jdbc:mysql://192.168.0.42:3306/demo_cloud_user?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowP/ # curl 10.244.2.136:8080/ext-service/user-service/info
{"returnCode":0,"returnMsg":"user service is running in the host: deployment-cloud-user-65fb8d79fd-28vmn","data":{"app":"Cloud User API","version":"1.0.1","description":"This is a simple Spring Boot application to demonstrate the use of BigQuery in GCP.","hostname":"deployment-cloud-user-65fb8d79fd-28vmn","dbUrl":"jdbc:mysql://192.168.0.42:3306/demo_cloud_user?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowP/ # curl 10.244.2.136:8080/ext-service/user-service/info
{"returnCode":0,"returnMsg":"user service is running in the host: deployment-cloud-user-65fb8d79fd-28vmn","data":{"app":"Cloud User API","version":"1.0.1","description":"This is a simple Spring Boot application to demonstrate the use of BigQuery in GCP.","hostname":"deployment-cloud-user-65fb8d79fd-28vmn","dbUrl":"jdbc:mysql://192.168.0.42:3306/demo_cloud_user?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowP/ # curl 10.244.2.136:8080/ext-service/user-service/info
{"returnCode":0,"returnMsg":"user service is running in the host: deployment-cloud-user-65fb8d79fd-28vmn","data":{"app":"Cloud User API","version":"1.0.1","description":"This is a simple Spring Boot application to demonstrate the use of BigQuery in GCP.","hostname":"deployment-cloud-user-65fb8d79fd-28vmn","dbUrl":"jdbc:mysql://192.168.0.42:3306/demo_cloud_user?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPublicKeyRetrieval=true"}}/ #

可见bq-api-service 已经成功 通过 cluster ip 去访问 后面的cloud-user service 了， 根据返回的host name

可以看出
clusterip 的loadbalancer 不会无脑随机转发

从 10.244.3.77 访问的bq-api-service 会访问 deployment-cloud-user-65fb8d79fd-m8xv4 里的cloud-user
从 10.244.2.136 访问的就会访问 deployment-cloud-user-65fb8d79fd-28vmn

为什么？ 因为clusterip 会智能地优先访问 同1个node的后台服务！

到这了这一步， 我们已经成功demo了 ClusterIP 的主要功能了
service A 已经能通过 cluster ip 访问service B 只是差了NodePort 无法从集群外测试
在这里插入图片描述





部署 NodePort nodeport-bq-api-service

注意这个nodeport 是for service A(bq-api-service)的而不是 service B， 如上图

编写yaml

nodeport-bq-api-service.yaml

apiVersion: v1 #  api version can be v1 or apps/v1
kind: Service 
metadata:
  name: nodeport-bq-api-service # name of the service
  labels:
    app: bq-api-service # label of the service itself
spec:
  selector: # Label of the Pod that the Service is selecting ,  all the pods not matter the pods are belong to which deployment, as long as the pods have the label app: bq-api-service
    app: bq-api-service # if the pod do not have the label app: bq-api-service, the pod could not be selected by the service
  ports:
  - port: 8080 # port of the service itself. we could also use serviceip:port to access the pod service, 
                 # but we use the nodeip:nodePort to access the service it. this nodePort is generated by k8s ramdomly
    targetPort: 8080 # port of the Pod
    name: 8080-port # name of the port
  type: NodePort # type of the service, NodePort, ClusterIP, LoadBalancer
                 # Ramdomly start a port (30000-32767) on each node, and forward the request to the service port (32111) on the pod
                 # and it could also be use to expose the service to the external world, but it's not recommended for production, because it's not secure and low efficient
部署yaml
[gateman@manjaro-x13 bq-api-service]$ kubectl create -f nodeport-bq-api-service.yaml 
service/nodeport-bq-api-service created
[gateman@manjaro-x13 bq-api-service]$ kubectl get svc -o wide
NAME                      TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE   SELECTOR
clusterip-cloud-user      ClusterIP   10.96.11.18     <none>        8080/TCP         86m   app=cloud-user
kubernetes                ClusterIP   10.96.0.1       <none>        443/TCP          77d   <none>
nodeport-bq-api-service   NodePort    10.106.59.100   <none>        8080:32722/TCP   13s   app=bq-api-service
[gateman@manjaro-x13 bq-api-service]$ kubectl get ep -o wide
NAME                      ENDPOINTS                                                          AGE
clusterip-cloud-user      10.244.1.67:8080,10.244.2.133:8080,10.244.2.134:8080 + 1 more...   86m
kubernetes                192.168.0.3:6443                                                   77d
nodeport-bq-api-service   10.244.1.68:8080,10.244.2.136:8080,10.244.3.77:8080 + 1 more...    19s

从nodeport service 的信息得出， 1个随机端口 32722 生成用于外部访问

E2E 测试

既然所有components 都部署了， 现在我们可以直接从集群外部测试
34.142.xxxxxx 是k8s-master 的公网ip

[gateman@manjaro-x13 bq-api-service]$ curl 34.142.xxxxxx:32722/actuator/info
{"app":"Sales API","version":"1.2.1","hostname":"deployment-bq-api-service-778cf8f54-z72dr","description":"This is a simple Spring Boot application to demonstrate the use of BigQuery in GCP."}[gateman@manjaro-x13 bq-api-service]$ curl 34.142.35.168:32722/ext-service/user-service/info
{"returnCode":0,"returnMsg":"user service is running in the host: deployment-cloud-user-65fb8d79fd-9rjln","data":{"app":"Cloud User API","version":"1.0.1","description":"This is a simple Spring Boot application to demonstrate the use of BigQuery in GCP.","hostname":"deployment-cloud-user-65fb8d79fd-9rjln","dbUrl":"jdbc:mysql://192.168.0.42:3306/demo_cloud_user?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPublicKeyRetrieval=true"}}[gateman@manjaro-x13 bq-api-service]$

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.coloradmin.cn/o/1854206.html

如若内容造成侵权/违法违规/事实不符，请联系多彩编程网进行投诉反馈，一经查实，立即删除！

相关文章

GIM： Learning Generalizable Image Matcher From Internet Videos

GIM： Learning Generalizable Image Matcher From Internet Videos

【引用格式】&#xff1a;Shen X, Yin W, Mller M, et al. GIM: Learning Generalizable Image Matcher From Internet Videos[C]//The Twelfth International Conference on Learning Representations. 2023. 【网址】&#xff1a;https://arxiv.org/pdf/2402.11095 【开源代…
阅读更多...
什么是慢查询——Java全栈知识（26）

什么是慢查询——Java全栈知识（26）

1、什么是慢查询 慢查询&#xff1a;也就是接口压测响应时间过长&#xff0c;页面加载时间过长的查询 原因可能如下&#xff1a; 1、聚合查询 2、多表查询 3、单表数据量过大 4、深度分页查询&#xff08;limit&#xff09; 如何定位慢查询&#xff1f; 1、Skywalking 我们…
阅读更多...
AIGC系列之一-一文理解什么是Embedding嵌入技术

AIGC系列之一-一文理解什么是Embedding嵌入技术

摘要&#xff1a;嵌入技术&#xff08;Embedding&#xff09;是一种将高维数据映射到低维空间的技术&#xff0c;在人工智能与图形学研究中被广泛应用。本文将介绍嵌入技术的基本概念、原理以及在 AIGC&#xff08;Artificial Intelligence and Graphics Computing&#xff09;…
阅读更多...
轻松上手MYSQL：MYSQL事务隔离级别的奇幻之旅

轻松上手MYSQL：MYSQL事务隔离级别的奇幻之旅

​&#x1f308; 个人主页&#xff1a;danci_ &#x1f525; 系列专栏&#xff1a;《设计模式》《MYSQL》 &#x1f4aa;&#x1f3fb; 制定明确可量化的目标&#xff0c;坚持默默的做事。 ✨欢迎加入探索MYSQL索引数据结构之旅✨ &#x1f44b; 大家好&#xff01;文本学习…
阅读更多...
C++封装、继承、多态的应用---职工管理系统

C++封装、继承、多态的应用---职工管理系统

C封装、继承、多态的应用—职工管理系统 文章目录 C封装、继承、多态的应用---职工管理系统1.需求分析2.抽象类的建立2.1抽象基类2.2员工类2.3经理类2.4老板类2.5存储类 3.抽象类的实现4.功能函数的实现4.1菜单功能的实现4.2增加职工功能函数实现4.2显示职工功能函数实现4.3删除…
阅读更多...
力扣SQL50 销售分析III having + 条件计数

力扣SQL50 销售分析III having + 条件计数

Problem: 1084. 销售分析III &#x1f468;‍&#x1f3eb; 参考题解 Code select s.product_id,p.product_name from sales s left join product p on s.product_id p.product_id group by product_id having count(if(sale_date between 2019-01-01 and 2019-03-31,1,nu…
阅读更多...
【2024最新版】Mysql数据库安装全攻略：图文详解(Windows版本）

【2024最新版】Mysql数据库安装全攻略：图文详解(Windows版本）

目录 1. 引言1.1 MySQL特性1.2 开源1.3 跨平台支持1.4 编程接口1.5 系统特性1.6 性能优势 2. 安装版本选择3. 安装MySQL3.1 下载MySQL3.2 安装MySQL 1. 引言 MySQL是一种流行的关系型数据库管理系统&#xff08;RDBMS&#xff09;&#xff0c;具有高度的可靠性、可扩展性和性能…
阅读更多...
C++系列-String（二）

C++系列-String（二）

&#x1f308;个人主页&#xff1a;羽晨同学 &#x1f4ab;个人格言:“成为自己未来的主人~” #define _CRT_SECURE_NO_WARNINGS #include<string> #include<iostream> #include<list> #include<algorithm> using namespace std; void test_string…
阅读更多...
计算机组成入门知识

计算机组成入门知识

前言&#x1f440;~ 数据库的知识点先暂且分享到这&#xff0c;接下来开始接触计算机组成以及计算机网络相关的知识点&#xff0c;这一章先介绍一些基础的计算机组成知识 一台计算机如何组成的&#xff1f; 存储器 CPU cpu的工作流程 主频 如何衡量CPU好坏呢&#xff1f…
阅读更多...
基于stm32的温度采集并且显示

基于stm32的温度采集并且显示

目录 一、I2C总线通信协议 &#xff08;一&#xff09;I2C简介 &#xff08;二&#xff09;I2C物理层 &#xff08;三&#xff09;I2C协议层 1、I2C基本读写过程 2、通信的起始和停止信号 3、数据的有效性 4、地址及数据方向 5、响应 &#xff08;四&#xff09;软件I…
阅读更多...
常说的云VR是什么意思？与传统vr的区别

常说的云VR是什么意思？与传统vr的区别

虚拟现实&#xff08;Virtual Reality&#xff0c;简称VR&#xff09;是一种利用计算机技术模拟产生一个三维空间的虚拟世界&#xff0c;让用户通过视觉、听觉、触觉等感官&#xff0c;获得与现实世界类似或超越的体验。VR技术发展历程可追溯至上世纪&#xff0c;经历概念提出、…
阅读更多...
方差分析【单/双因素方差分析】

方差分析【单/双因素方差分析】

文章目录 方差分析一、单因素方差分析&#xff08;Analysis of Variance&#xff09;1.方差分析在做什么&#xff1f;2.方差分析的思路&#xff1a;3.方差分析中的计算&#xff1a;4.构造F统计量&#xff1a; 二、双因素方差分析(Two-way ANOVA)1.双因素方差分析在做什么&#…
阅读更多...
HCIA 18 结束 企业总部-分支综合实验（上）

HCIA 18 结束 企业总部-分支综合实验（上）

1.实验介绍及拓扑 &#xff08;1&#xff09;总部和分支机构都可以上互联网访问8.8.8.8&#xff1b; &#xff08;2&#xff09;总部和分支机构使用广域网专线互访作为主线&#xff0c;并且通过互联网建立GRE隧道互访作为备线&#xff1b; &#xff08;3&#xff09;总部内为…
阅读更多...
【database3】oracle：数据交换/存储/收集

【database3】oracle：数据交换/存储/收集

文章目录 1.oracle安装&#xff1a;swap&#xff0c;dd1.1 创建swap交换区&#xff1a;grep MemTotal /proc/meminfo &#xff08;安装Oracle物理内存要求1024MB以上&#xff09;&#xff0c;grep SwapTotal /proc/meminfo1.2 安装依赖包及改系统核心参数&#xff1a;关闭一些系…
阅读更多...
机器学习算法（二）：1 逻辑回归的从零实现（普通实现+多项式特征实现非线性分类+正则化实现三个版本）

机器学习算法（二）：1 逻辑回归的从零实现（普通实现+多项式特征实现非线性分类+正则化实现三个版本）

文章目录 前言一、普通实现1 数据集准备2 逻辑回归模型3 损失函数4 计算损失函数的梯度5 梯度下降算法6 训练模型二、多项式特征实现非线性分类1 数据准备与多项式特征构造2 逻辑回归模型三、逻辑回归 --- 正则化实现1 数据准备2 逻辑回归模型3 正则化损失函数4 计算损失函数的…
阅读更多...
Cyber Weekly #12

Cyber Weekly #12

赛博新闻 1、Anthropic发布Claude 3.5 Sonnet 本周五&#xff08;6月21日&#xff09;凌晨&#xff0c;Anthropic宣布推出其最新的语言模型Claude 3.5 Sonnet&#xff0c;距离上次发布Claude3才过去3个月。Claude3.5拥有20万token的长上下文窗口&#xff0c;目前已经在Claude…
阅读更多...
企业中订单超时关闭是怎么做的？我说用延迟消息，面试官让我回去等消息?

企业中订单超时关闭是怎么做的？我说用延迟消息，面试官让我回去等消息?

文章目录 背景时序图方案对比方案一 被动关闭方案二 定时关闭方案三 Rocket MQ延迟消息 总结 背景 订单超时未支付是电商中的一个核心场景&#xff0c;当用户创建订单后&#xff0c;超过一定时间没有支付&#xff0c;平台需要及时将该订单关闭。需要关闭的主要原因有以下几个&…
阅读更多...
初中英语优秀作文分析-005How to Plan Our Life Wisely-如何明智地规划我们的生活

初中英语优秀作文分析-005How to Plan Our Life Wisely-如何明智地规划我们的生活

PDF格式公众号回复关键字:SHCZYF005 记忆树 1 The “double reduction policy” reduces the burden on students and offers us more spare time than before, but how to plan our life wisely? 翻译 “双减政策”减轻了学生的负担&#xff0c;给了我们比以前更多的业余…
阅读更多...
预备资金有5000-6000买什么电脑比较好？大学生电脑选购指南

预备资金有5000-6000买什么电脑比较好？大学生电脑选购指南

小新pro14 2024 处理器&#xff1a;采用了英特尔酷睿Ultra5 125H或Ultra9 185H两种处理器可选&#xff0c;这是英特尔最新的高性能低功耗处理器&#xff0c;具有18个线程&#xff0c;最高可达4.5GHz的加速频率&#xff0c;支持PCIe 4.0接口&#xff0c;内置了强大的ARC核芯显卡…
阅读更多...
Windows Update Blocker 完全禁用或启用Windows系统上的自动更新，一键阻止，无需额外操作！

Windows Update Blocker 完全禁用或启用Windows系统上的自动更新，一键阻止，无需额外操作！

你是否曾因为突如其来的系统更新而感到手忙脚乱&#xff1f;Windows Update Blocker v1.8让你只需一键&#xff0c;即可完全禁用或启用Windows系统上的自动更新&#xff0c;不需要任何的额外操作&#xff0c;只需要打开软件&#xff0c;点击应用即可关闭Windows系统上的自动更新…
阅读更多...
推荐文章
最新文章

Copyright @ 2022~2023