K8S 上部署大数据相关组件

news2024/9/20 20:39:10

文章目录

      • 一、前言
      • 二、Redis
        • 2.1 安装方式一:
        • 2.2 安装方式二:

一、前言

  Artifact Hub 是一个专注于云原生应用的集中式搜索和发布平台。它旨在简化开发者在 CNCF(Cloud Native Computing Foundation)项目中寻找、安装和分享包与配置的过程。用户可以通过这个平台方便地发现、安装各类云原生工具的组件,如Argo模板、Helm 图表、Kubernetes 插件等。

在这里插入图片描述
在这里插入图片描述
在这里插入图片描述

二、Redis

$ helm repo add bitnami https://charts.bitnami.com/bitnami
"bitnami" has been added to your repositories

$ kubectl get sc
NAME                  PROVISIONER          RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
nfs                   nfs-provisioner-01   Retain          Immediate              false                  3d10h
2.1 安装方式一:
# 执行安装命令并执行参数存储类的名称,得确保你集群中有这个存储类并且能够正常的提供PV动态供给
$ helm install redis-cluster oci://registry-1.docker.io/bitnamicharts/redis --set global.storageClass=nfs
Pulled: registry-1.docker.io/bitnamicharts/redis:19.6.1
Digest: sha256:0d077ee5947e26645e3bc05e3d6e6bd62e24e3082cf5df43d89664099336a78d
NAME: redis-cluster
LAST DEPLOYED: Mon Jul  8 03:42:57 2024
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: redis
CHART VERSION: 19.6.1
APP VERSION: 7.2.5

** Please be patient while the chart is being deployed **

Redis® can be accessed on the following DNS names from within your cluster:

    redis-cluster-master.default.svc.cluster.local for read/write operations (port 6379)
    redis-cluster-replicas.default.svc.cluster.local for read-only operations (port 6379)



To get your password run:

    export REDIS_PASSWORD=$(kubectl get secret --namespace default redis-cluster -o jsonpath="{.data.redis-password}" | base64 -d)

To connect to your Redis® server:

1. Run a Redis® pod that you can use as a client:

   kubectl run --namespace default redis-client --restart='Never'  --env REDIS_PASSWORD=$REDIS_PASSWORD  --image docker.io/bitnami/redis:7.2.5-debian-12-r2 --command -- sleep infinity

   Use the following command to attach to the pod:

   kubectl exec --tty -i redis-client \
   --namespace default -- bash

2. Connect using the Redis® CLI:
   REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-cluster-master
   REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-cluster-replicas

To connect to your database from outside the cluster execute the following commands:

    kubectl port-forward --namespace default svc/redis-cluster-master 6379:6379 &
    REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h 127.0.0.1 -p 6379

WARNING: There are "resources" sections in the chart not set. Using "resourcesPreset" is not recommended for production. For production installations, please set the following values according to your workload needs:
  - replica.resources
  - master.resources
+info https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/

$ kubectl get pods
NAME                       READY   STATUS              RESTARTS   AGE
redis-cluster-master-0     1/1     Running             0          8m44s
redis-cluster-replicas-0   1/1     Running             0          8m44s
redis-cluster-replicas-1   1/1     Running             0          6m43s
redis-cluster-replicas-2   0/1     ContainerCreating   0          4m50s

# 将密码保存到 REDIS_PASSWORD 这个环境变量中
$ export REDIS_PASSWORD=$(kubectl get secret --namespace default redis-cluster -o jsonpath="{.data.redis-password}" | base64 -d)

# 再创建一个 Redis 客户端
$ kubectl run --namespace default redis-client --restart='Never'  --env REDIS_PASSWORD=$REDIS_PASSWORD  --image docker.io/bitnami/redis:7.2.5-debian-12-r2 --command -- sleep infinity
pod/redis-client created

$ kubectl get pods
NAME                       READY   STATUS    RESTARTS   AGE
redis-client               1/1     Running   0          28s
redis-cluster-master-0     1/1     Running   0          10m
redis-cluster-replicas-0   1/1     Running   0          10m
redis-cluster-replicas-1   1/1     Running   0          8m50s
redis-cluster-replicas-2   1/1     Running   0          6m57s

$ kubectl run --namespace default redis-client --restart='Never'  --env REDIS_PASSWORD=$REDIS_PASSWORD  --image docker.io/bitnami/redis:7.2.5-debian-12-r2 --command -- sleep infinity
pod/redis-client created

$ kubectl get pods
NAME                       READY   STATUS    RESTARTS   AGE
redis-client               1/1     Running   0          28s
redis-cluster-master-0     1/1     Running   0          10m
redis-cluster-replicas-0   1/1     Running   0          10m
redis-cluster-replicas-1   1/1     Running   0          8m50s
redis-cluster-replicas-2   1/1     Running   0          6m57s

$ kubectl exec -it redis-client -- bash
I have no name!@redis-client:/$ REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-cluster-master
redis-cluster-master:6379> info replication
# Replication
role:master
connected_slaves:3
slave0:ip=redis-cluster-replicas-0.redis-cluster-headless.default.svc.cluster.local,port=6379,state=online,offset=924,lag=1
slave1:ip=redis-cluster-replicas-1.redis-cluster-headless.default.svc.cluster.local,port=6379,state=online,offset=924,lag=1
slave2:ip=redis-cluster-replicas-2.redis-cluster-headless.default.svc.cluster.local,port=6379,state=online,offset=924,lag=1
master_failover_state:no-failover
master_replid:239753ae516fad9e7fb6230d053861b80bc0bbb3
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:924
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:924

$ kubectl get svc
NAME                     TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                               AGE
kubernetes               ClusterIP   10.68.0.1       <none>        443/TCP                               3d12h
redis-cluster-headless   ClusterIP   None            <none>        6379/TCP                              16m
redis-cluster-master     ClusterIP   10.68.240.7     <none>        6379/TCP                              16m
redis-cluster-replicas   ClusterIP   10.68.170.149   <none>        6379/TCP                              16m

  改为暴露端口从外部访问:

$ kubectl edit svc redis-cluster-master
# 原先内容
  ports:
  - name: tcp-redis
    port: 6379
    protocol: TCP
    targetPort: redis
  selector:
    app.kubernetes.io/component: master
    app.kubernetes.io/instance: redis-cluster
    app.kubernetes.io/name: redis
  sessionAffinity: None
  type: ClusterIP
# 修改为
  ports:
  - name: tcp-redis
    nodePort: 31379
    port: 6379
    protocol: TCP
    targetPort: redis
  selector:
    app.kubernetes.io/component: master
    app.kubernetes.io/instance: redis-cluster
    app.kubernetes.io/name: redis
  sessionAffinity: None
  type: NodePort
# 注意:nodePort 值不在指定范围会报错而无法保存(比如设置成 11379):
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
# services "redis-cluster-master" was not valid:
# * spec.ports[0].nodePort: Invalid value: 11379: provided port is not in the valid range. The range of valid ports is 30000-32767

# 重启服务(删除之后会自动重启)
$ kubectl delete svc redis-cluster-master -n default

$ kubectl get svc
NAME                     TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                               AGE
kubernetes               ClusterIP   10.68.0.1       <none>        443/TCP                               4d11h
redis-cluster-headless   ClusterIP   None            <none>        6379/TCP                              22h
redis-cluster-master     NodePort    10.68.240.7     <none>        6379:31379/TCP                        22h
redis-cluster-replicas   ClusterIP   10.68.170.149   <none>        6379/TCP                              22h

  补充:安装时参数自定义

# 安装的时候增加 namespace 的创建
$ helm install redis-cluster oci://registry-1.docker.io/bitnamicharts/redis -n redis --create-namespace --set global.storageClass=nfs

# redis 默认会安装1个master,3个node,可以通过以下参数来修改 --set replica.replicaCount=2 --set master.count=1
$ helm install --set replica.persistence.size=2Gi --set master.persistence.size=1Gi \
 --set global.storageClass=manual  --set  replica.replicaCount=2 --set master.count=1 linkage-redis bitnami/redis
 
# redis-cluster 默认创建6个 nodes(每个nodes包括一个master及一个replica),可以调整参数该边node数目,但调整后的nodes数不能<3
$ helm install --set cluster.nodes=3 --set replica.persistence.size=2Gi --set master.persistence.size=2Gi \
 --set global.storageClass=manual  linkage-redis bitnami/redis-cluster

参考这篇文章搭建出来的 pods 一直处于 pending 状态:k8s 如何访问redis集群

2.2 安装方式二:
$ helm pull bitnami/redis --version 17.3.7
$ tar -zxvf redis-17.3.7.tgz
redis/Chart.yaml
redis/Chart.lock
redis/values.yaml
redis/values.schema.json
redis/templates/NOTES.txt
redis/templates/_helpers.tpl
redis/templates/configmap.yaml
redis/templates/extra-list.yaml
redis/templates/headless-svc.yaml
redis/templates/health-configmap.yaml
redis/templates/master/application.yaml
redis/templates/master/psp.yaml
redis/templates/master/pvc.yaml
redis/templates/master/service.yaml
redis/templates/master/serviceaccount.yaml
redis/templates/metrics-svc.yaml
redis/templates/networkpolicy.yaml
redis/templates/pdb.yaml
redis/templates/prometheusrule.yaml
redis/templates/replicas/hpa.yaml
redis/templates/replicas/service.yaml
redis/templates/replicas/serviceaccount.yaml
redis/templates/replicas/statefulset.yaml
redis/templates/role.yaml
redis/templates/rolebinding.yaml
redis/templates/scripts-configmap.yaml
redis/templates/secret.yaml
redis/templates/sentinel/hpa.yaml
redis/templates/sentinel/node-services.yaml
redis/templates/sentinel/ports-configmap.yaml
redis/templates/sentinel/service.yaml
redis/templates/sentinel/statefulset.yaml
redis/templates/serviceaccount.yaml
redis/templates/servicemonitor.yaml
redis/templates/tls-secret.yaml
redis/.helmignore
redis/README.md
redis/img/redis-cluster-topology.png
redis/img/redis-topology.png
redis/charts/common/Chart.yaml
redis/charts/common/values.yaml
redis/charts/common/templates/_affinities.tpl
redis/charts/common/templates/_capabilities.tpl
redis/charts/common/templates/_errors.tpl
redis/charts/common/templates/_images.tpl
redis/charts/common/templates/_ingress.tpl
redis/charts/common/templates/_labels.tpl
redis/charts/common/templates/_names.tpl
redis/charts/common/templates/_secrets.tpl
redis/charts/common/templates/_storage.tpl
redis/charts/common/templates/_tplvalues.tpl
redis/charts/common/templates/_utils.tpl
redis/charts/common/templates/_warnings.tpl
redis/charts/common/templates/validations/_cassandra.tpl
redis/charts/common/templates/validations/_mariadb.tpl
redis/charts/common/templates/validations/_mongodb.tpl
redis/charts/common/templates/validations/_mysql.tpl
redis/charts/common/templates/validations/_postgresql.tpl
redis/charts/common/templates/validations/_redis.tpl
redis/charts/common/templates/validations/_validations.tpl
redis/charts/common/.helmignore
redis/charts/common/README.md

$ helm install redis ./redis -n redis --create-namespace --set global.storageClass=nfs
NAME: redis
LAST DEPLOYED: Tue Jul  9 11:18:15 2024
NAMESPACE: redis
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: redis
CHART VERSION: 17.3.7
APP VERSION: 7.0.5

** Please be patient while the chart is being deployed **

Redis&reg; can be accessed on the following DNS names from within your cluster:

    redis-master.redis.svc.cluster.local for read/write operations (port 6379)
    redis-replicas.redis.svc.cluster.local for read-only operations (port 6379)



To get your password run:

    export REDIS_PASSWORD=$(kubectl get secret --namespace redis redis -o jsonpath="{.data.redis-password}" | base64 -d)

To connect to your Redis&reg; server:

1. Run a Redis&reg; pod that you can use as a client:

   kubectl run --namespace redis redis-client --restart='Never'  --env REDIS_PASSWORD=$REDIS_PASSWORD  --image docker.io/bitnami/redis:7.0.5-debian-11-r7 --command -- sleep infinity

   Use the following command to attach to the pod:

   kubectl exec --tty -i redis-client \
   --namespace redis -- bash

2. Connect using the Redis&reg; CLI:
   REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-master
   REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-replicas

To connect to your database from outside the cluster execute the following commands:

    kubectl port-forward --namespace redis svc/redis-master 6379:6379 &
    REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h 127.0.0.1 -p 6379

$ kubectl get pods,svc -n redis
NAME                   READY   STATUS              RESTARTS   AGE
pod/redis-master-0     0/1     ContainerCreating   0          53s
pod/redis-replicas-0   0/1     ContainerCreating   0          53s

NAME                     TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/redis-headless   ClusterIP   None            <none>        6379/TCP   53s
service/redis-master     ClusterIP   10.68.242.183   <none>        6379/TCP   53s
service/redis-replicas   ClusterIP   10.68.122.142   <none>        6379/TCP   53s

$ kubectl get pods -n redis
NAME                   READY   STATUS    RESTARTS   AGE
pod/redis-master-0     1/1     Running   0          99s
pod/redis-replicas-0   0/1     Running   0          99s

$ kubectl get pods -n redis
NAME                   READY   STATUS    RESTARTS   AGE
pod/redis-master-0     1/1     Running             0          2m3s
pod/redis-replicas-0   1/1     Running             0          2m3s
pod/redis-replicas-1   0/1     ContainerCreating   0          15s

$ kubectl get pods,svc -n redis
NAME                   READY   STATUS    RESTARTS   AGE
pod/redis-master-0     1/1     Running   0          4m27s
pod/redis-replicas-0   1/1     Running   0          4m27s
pod/redis-replicas-1   1/1     Running   0          2m39s
pod/redis-replicas-2   0/1     Running   0          82s

$ kubectl describe pod redis-master-0 -n redis
Events:
  Type     Reason            Age    From               Message
  ----     ------            ----   ----               -------
  Warning  FailedScheduling  9m31s  default-scheduler  0/3 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod..
  Normal   Scheduled         9m29s  default-scheduler  Successfully assigned redis/redis-master-0 to worker-01
  Normal   Pulling           9m28s  kubelet            Pulling image "docker.io/bitnami/redis:7.0.5-debian-11-r7"
  Normal   Pulled            8m17s  kubelet            Successfully pulled image "docker.io/bitnami/redis:7.0.5-debian-11-r7" in 1m11.811062916s (1m11.811075708s including waiting)
  Normal   Created           8m17s  kubelet            Created container redis
  Normal   Started           8m16s  kubelet            Started container redis

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.coloradmin.cn/o/1911700.html

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈,一经查实,立即删除!

相关文章

计算理论复习

1.Turing Machine 确定性图灵机 图灵机有很多不同的定义&#xff0c;这里选取其中一种&#xff0c;其它定义下的图灵机往往与下面这种定义的图灵机计算能力等价。 图灵机是一个在一条可双向无限延伸且被划分为若干格子的纸带上进行操作的机器&#xff0c;其有内部状态&#…

C++ 是否变得比 C 更流行了?

每年都会出现一种新的编程语言。创造一种新语言来解决计算机科学中的挑战的诱惑很难抗拒。一些资料表明&#xff0c;目前有多达 2,500 种语言&#xff0c;这并不奇怪&#xff01; 对于我们嵌入式软件开发人员来说&#xff0c;这个列表并不长。事实上&#xff0c;我们可以用一只…

go-redis源码解析:连接池原理

1. 执行命令的入口方法 redis也是通过hook执行命令&#xff0c;initHooks时&#xff0c;会将redis的hook放在第一个 通过hook调用到process方法&#xff0c;process方法内部再调用_process 2. 线程池初始化 redis在新建单客户端、sentinel客户端、cluster客户端等&#xff0c…

ChatGPT提问提示指南PDF下载经典分享推荐书籍

ChatGPT提问提示指南PDF&#xff0c;在本书的帮助下&#xff0c;您将学习到如何有效地向 ChatGPT 提出问题&#xff0c;以获得更准确和有用的回答。我们希望这本书能够为您提供实用的指南和策略&#xff0c;帮助您更好地与 ChatGPT 交互。 ChatGPT提问提示指南PDF下载 无论您是…

UMI HTTP接口手册

Translate to English 命令行手册&#xff1a; README_CLI.mdHTTP接口手册&#xff1a; README_HTTP.md HTTP接口手册 &#xff08;本文档仅适用于 Umi-OCR 最新版本。旧版本请查看 Github备份分支 中对应版本的文档。&#xff09; 基础说明 如上图&#xff0c;必须允许HTT…

git只列出本地分支

git只列出本地分支 git branch --list git强制删除本地分支 git branch -D_error: the branch dlx-test is not fully merged. -CSDN博客文章浏览阅读648次。git branch -d 可以通过: git branch 查看所有本地分支及其名字&#xff0c;然后删除特定分支。git删除远程remote分支…

IDEA如何创建原生maven子模块

文件 -> 新建 -> 新模块 -> Maven ArcheTypeMaven ArcheType界面中的输入框介绍 名称&#xff1a;子模块的名称位置&#xff1a;子模块存放的路径名创建Git仓库&#xff1a;子模块不单独作为一个git仓库&#xff0c;无需勾选JDK&#xff1a;JDK版本号父项&#xff1a;…

【深度学习基础】MacOS PyCharm连接远程服务器

目录 一、需求描述二、建立与服务器的远程连接1. 新版Pycharm的界面有什么不同&#xff1f;2. 创建远程连接3. 建立本地项目与远程服务器项目之间的路径映射4.设置保存自动上传文件 三、设置解释器总结 写在前面&#xff0c;本人用的是Macbook Pro&#xff0c; M3 MAX处理器&am…

设计模式探索:装饰器模式

1. 装饰器模式定义 装饰器模式&#xff08;Decorator Pattern&#xff09; 装饰器模式是一种结构型设计模式&#xff0c;允许向一个对象动态添加行为。在不改变类的接口的情况下&#xff0c;装饰器模式在原始类上增加额外的职责&#xff0c;并且支持多个装饰器嵌套使用。 装…

双色球 | python

1. 玩法规则 “双色球”每注投注号码由 6 个红色球号码和 1 个蓝色球号码组成。红色球号码从 1—33 中选择&#xff0c;蓝色球号码从 1—16 中选择。 球的数字匹配数量和颜色决定了是否中奖。 2. 需求 生成本期双色球中奖号码。&#xff08;注意&#xff1a;1.生成的红球随机有…

Vuforia AR篇(八)— AR塔防上篇

目录 前言一、设置Vuforia AR环境1. 添加AR Camera2. 设置目标图像 二、创建塔防游戏基础1. 导入素材2. 搭建场景3. 创建敌人4. 创建脚本 前言 在增强现实&#xff08;AR&#xff09;技术快速发展的今天&#xff0c;Vuforia作为一个强大的AR开发平台&#xff0c;为开发者提供了…

Docker-compse的应用

1 docker-compose # 使用了docker 面临一个比较大的问题&#xff0c;如果一个djagno项目&#xff0c;使用mysql&#xff0c;redis&#xff0c;不要一次性把所有服务都放到一个容器中&#xff0c;每个服务一个容器&#xff0c;批量的管理多个容器&#xff0c;比较难以操作&…

昇思25天学习打卡营第1天|小试牛刀

这里写自昇思25天学习打卡营第1天|小试牛刀定义目录标题 昇思25天学习打卡营第1天学习了初学入门之基本介绍。了解了昇思MindSpore和华为昇腾AI全栈。训练营中的教程丰富&#xff0c;有初学入门、应用实践和量子计算等。学习打卡营是很好的提升自己的机会。 昇腾计算&#xff…

前端图表库G2快速上手

文档地址&#xff1a; https://g2-v3.antv.vision/zh/docs/manual/getting-started/ https://g2.antv.antgroup.com/ 安装&#xff1a; pnpm i antv/g2在vue3中使用&#xff1a; <script setup> import {Chart} from antv/g2; import {onMounted} from "vue"…

生成多个ssh访问不同git

如果&#xff0c;你的git代码仓库&#xff0c;比如说腾讯云coding&#xff0c;通过ssh秘钥访问&#xff0c;一直用的好好的&#xff0c;有一天&#xff0c;你又增加一个aliyun云效的代码仓库&#xff0c;又配置了aliyun云效的秘钥并且&#xff0c;根据aliyun云效的官方文档上传…

【持续集成_03课_Linux部署Sonar+Gogs+Jenkins】

一、通过虚拟机搭建Linux环境-CnetOS 1、安装virtualbox&#xff0c;和Vmware是一样的&#xff0c;只是box更轻量级 1&#xff09;需要注意内存选择&#xff0c;4G 2、启动完成后&#xff0c;需要获取服务器IP地址 命令 ip add 服务器IP地址 通过本地的工具&#xff0c;进…

后端——全局异常处理

一、老办法try-catch 当我们执行一些错误操作导致程序报错时&#xff0c;程序会捕捉到异常报错&#xff0c;这个异常会存在一个Exception对象里 那我们在spring boot工程开发时&#xff0c;当我们执行一个sql查询时报错了&#xff0c;那就会从最底层的Mapper层捕捉到Exceptio…

三相感应电机的建模仿真(3)基于ABC相坐标系Level2 S-Fun以及定子串不对称电抗起动过程仿真分析

1. 概述 2. 三相感应电动机状态方程式 3. 基于Level2 S-Function的仿真模型建立 4. 动态分析实例 5. 总结 6. 参考文献 1. 概述 三相感应电机自然坐标系下的数学模型是一组周期性变系数微分方程(其电感矩阵是转子位置角的函数,转子位置角随时间按正弦规律变化),将其用…

工作手机监管系统销售防飞单保护企业资源

在竞争激烈的市场环境中&#xff0c;企业的每一笔交易都承载着发展的希望与未来的蓝图。然而&#xff0c;飞单现象如同暗流涌动&#xff0c;悄然侵蚀着企业的利润根基&#xff0c;威胁着团队的凝聚力与客户的信任。为了构建更加稳固的业务防线&#xff0c;我们隆重介绍——百川…

Instruct-GS2GS:通过用户指令编辑 GS 三维场景

Paper: Instruct-GS2GS: Editing 3D Gaussian Splats with Instructions Introduction: https://instruct-gs2gs.github.io/ Code: https://github.com/cvachha/instruct-gs2gs Instruct-GS2GS 复用了 Instruct-NeRF2NeRF 1 的架构&#xff0c;将基于 NeRF 的三维场景编辑方法迁…