【HBase】——安装部署

news2024/9/23 3:26:10

1 规划&前提

  1. Zookeeper 、HDFS 正常部署
  2. 规划如下在这里插入图片描述

2 解压并重命名

cd /opt/software/
tar -zxvf hbase-2.4.11-bin.tar.gz -C /opt/module/
cd /opt/module
mv hbase-2.4.11/ hbase

3 修改配置文件

3.1 hbase-env.sh


#!/usr/bin/env bash
#
#/**
# * Licensed to the Apache Software Foundation (ASF) under one
# * or more contributor license agreements.  See the NOTICE file
# * distributed with this work for additional information
# * regarding copyright ownership.  The ASF licenses this file
# * to you under the Apache License, Version 2.0 (the
# * "License"); you may not use this file except in compliance
# * with the License.  You may obtain a copy of the License at
# *
# *     http://www.apache.org/licenses/LICENSE-2.0
# *
# * Unless required by applicable law or agreed to in writing, software
# * distributed under the License is distributed on an "AS IS" BASIS,
# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# * See the License for the specific language governing permissions and
# * limitations under the License.
# */

# Set environment variables here.

# This script sets variables multiple times over the course of starting an hbase process,
# so try to keep things idempotent unless you want to take an even deeper look
# into the startup scripts (bin/hbase, etc.)

# The java implementation to use.  Java 1.8+ required.
# export JAVA_HOME=/usr/java/jdk1.8.0/

# Extra Java CLASSPATH elements.  Optional.
# export HBASE_CLASSPATH=

# The maximum amount of heap to use. Default is left to JVM default.
# export HBASE_HEAPSIZE=1G

# Uncomment below if you intend to use off heap cache. For example, to allocate 8G of 
# offheap, set the value to "8G".
# export HBASE_OFFHEAPSIZE=1G

# Extra Java runtime options.
# Default settings are applied according to the detected JVM version. Override these default
# settings by specifying a value here. For more details on possible settings,
# see http://hbase.apache.org/book.html#_jvm_tuning
# export HBASE_OPTS

# Uncomment one of the below three options to enable java garbage collection logging for the server-side processes.

# This enables basic gc logging to the .out file.
# export SERVER_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps"

# This enables basic gc logging to its own file.
# If FILE-PATH is not replaced, the log file(.gc) would still be generated in the HBASE_LOG_DIR .
# export SERVER_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:<FILE-PATH>"

# This enables basic GC logging to its own file with automatic log rolling. Only applies to jdk 1.6.0_34+ and 1.7.0_2+.
# If FILE-PATH is not replaced, the log file(.gc) would still be generated in the HBASE_LOG_DIR .
# export SERVER_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:<FILE-PATH> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=1 -XX:GCLogFileSize=512M"

# Uncomment one of the below three options to enable java garbage collection logging for the client processes.

# This enables basic gc logging to the .out file.
# export CLIENT_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps"

# This enables basic gc logging to its own file.
# If FILE-PATH is not replaced, the log file(.gc) would still be generated in the HBASE_LOG_DIR .
# export CLIENT_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:<FILE-PATH>"

# This enables basic GC logging to its own file with automatic log rolling. Only applies to jdk 1.6.0_34+ and 1.7.0_2+.
# If FILE-PATH is not replaced, the log file(.gc) would still be generated in the HBASE_LOG_DIR .
# export CLIENT_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:<FILE-PATH> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=1 -XX:GCLogFileSize=512M"

# See the package documentation for org.apache.hadoop.hbase.io.hfile for other configurations
# needed setting up off-heap block caching. 

# Uncomment and adjust to enable JMX exporting
# See jmxremote.password and jmxremote.access in $JRE_HOME/lib/management to configure remote password access.
# More details at: http://java.sun.com/javase/6/docs/technotes/guides/management/agent.html
# NOTE: HBase provides an alternative JMX implementation to fix the random ports issue, please see JMX
# section in HBase Reference Guide for instructions.

# export HBASE_JMX_BASE="-Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false"
# export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10101"
# export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10102"
# export HBASE_THRIFT_OPTS="$HBASE_THRIFT_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10103"
# export HBASE_ZOOKEEPER_OPTS="$HBASE_ZOOKEEPER_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10104"
# export HBASE_REST_OPTS="$HBASE_REST_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10105"

# File naming hosts on which HRegionServers will run.  $HBASE_HOME/conf/regionservers by default.
# export HBASE_REGIONSERVERS=${HBASE_HOME}/conf/regionservers

# Uncomment and adjust to keep all the Region Server pages mapped to be memory resident
#HBASE_REGIONSERVER_MLOCK=true
#HBASE_REGIONSERVER_UID="hbase"

# File naming hosts on which backup HMaster will run.  $HBASE_HOME/conf/backup-masters by default.
# export HBASE_BACKUP_MASTERS=${HBASE_HOME}/conf/backup-masters

# Extra ssh options.  Empty by default.
# export HBASE_SSH_OPTS="-o ConnectTimeout=1 -o SendEnv=HBASE_CONF_DIR"

# Where log files are stored.  $HBASE_HOME/logs by default.
# export HBASE_LOG_DIR=${HBASE_HOME}/logs

# Enable remote JDWP debugging of major HBase processes. Meant for Core Developers 
# export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8070"
# export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8071"
# export HBASE_THRIFT_OPTS="$HBASE_THRIFT_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8072"
# export HBASE_ZOOKEEPER_OPTS="$HBASE_ZOOKEEPER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8073"
# export HBASE_REST_OPTS="$HBASE_REST_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8074"

# A string representing this instance of hbase. $USER by default.
# export HBASE_IDENT_STRING=$USER

# The scheduling priority for daemon processes.  See 'man nice'.
# export HBASE_NICENESS=10

# The directory where pid files are stored. /tmp by default.
# export HBASE_PID_DIR=/var/hadoop/pids

# Seconds to sleep between slave commands.  Unset by default.  This
# can be useful in large clusters, where, e.g., slave rsyncs can
# otherwise arrive faster than the master can service them.
# export HBASE_SLAVE_SLEEP=0.1

# Tell HBase whether it should manage it's own instance of ZooKeeper or not.
export HBASE_MANAGES_ZK=false

# The default log rolling policy is RFA, where the log file is rolled as per the size defined for the 
# RFA appender. Please refer to the log4j.properties file to see more details on this appender.
# In case one needs to do log rolling on a date change, one should set the environment property
# HBASE_ROOT_LOGGER to "<DESIRED_LOG LEVEL>,DRFA".
# For example:
# HBASE_ROOT_LOGGER=INFO,DRFA
# The reason for changing default to RFA is to avoid the boundary case of filling out disk space as 
# DRFA doesn't put any cap on the log size. Please refer to HBase-5655 for more context.

# Tell HBase whether it should include Hadoop's lib when start up,
# the default value is false,means that includes Hadoop's lib.
# export HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP="true"

# Override text processing tools for use by these launch scripts.
# export GREP="${GREP-grep}"
# export SED="${SED-sed}"

3.3 hbase-site.xml


<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
/*
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership.  The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
-->
<configuration>
  <!--
    The following properties are set for running HBase as a single process on a
    developer workstation. With this configuration, HBase is running in
    "stand-alone" mode and without a distributed file system. In this mode, and
    without further configuration, HBase and ZooKeeper data are stored on the
    local filesystem, in a path under the value configured for `hbase.tmp.dir`.
    This value is overridden from its default value of `/tmp` because many
    systems clean `/tmp` on a regular basis. Instead, it points to a path within
    this HBase installation directory.

    Running against the `LocalFileSystem`, as opposed to a distributed
    filesystem, runs the risk of data integrity issues and data loss. Normally
    HBase will refuse to run in such an environment. Setting
    `hbase.unsafe.stream.capability.enforce` to `false` overrides this behavior,
    permitting operation. This configuration is for the developer workstation
    only and __should not be used in production!__

    See also https://hbase.apache.org/book.html#standalone_dist
  -->
  <property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
  </property>
  <!-- <property>
    <name>hbase.tmp.dir</name>
    <value>./tmp</value>
  </property>
  <property>
    <name>hbase.unsafe.stream.capability.enforce</name>
    <value>false</value>
  </property> -->
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>hadoop102,hadoop103,hadoop104</value>
  </property>
  <property>
    <name>hbase.rootdir</name>
    <value>hdfs://hadoop102:8020/hbase</value>
  </property>
</configuration>

3.3 zoo.cfg

修改zookeeper默认数据存放位置，防止tmp路径 将文件清除


# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
# dataDir=/tmp/zookeeper
dataDir=/opt/module/zookeeper-3.5.7/zkData
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

3.4 regionservers

hadoop102
hadoop103
hadoop104

3.5 backup-masters

echo "hadoop103" > /opt/module/hbase/conf/backup-masters

4 修改hadoop和hbase jar 冲突问题

/opt/module/hbase/lib/client-facing-thirdparty
rm -f slf4j-reload4j-1.7.33.jar

5 分发安装包到其他服务器

 sh xsync /opt/module/hbase/

6 添加环境变量

在三台server 分别添加

vim  /etc/profile.d/my_env.sh

#HBASE_HOME
export HBASE_HOME=/opt/module/hbase
export PATH=$PATH:$HBASE_HOME/bin

source /etc/profile

7 启动停止脚本

#!/bin/bash
if [ $# -lt 1 ]; then
    echo "No Args Input..."
    exit
fi

case $1 in
"start")
    {
        echo "----------------- HBase start -----------------"
        nohup sh /opt/module/hbase/bin/start-hbase.sh  >> /opt/module/hbase/logs/start-hbase.out 2>&1 &    }
    ;;
"stop")
    {
         echo "----------------- HBase stop -----------------"
        nohup sh /opt/module/hbase/bin/stop-hbase.sh  >> /opt/module/hbase/logs/stop-hbase.out 2>&1 &
    }
    ;;
*)
    echo "Input Args Error..."
    ;;
esac

8 web页面

Master

http://hadoop102:16010/master-status

Backup Master

http://hadoop103:16010/master-status

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.coloradmin.cn/o/1353674.html

如若内容造成侵权/违法违规/事实不符，请联系多彩编程网进行投诉反馈，一经查实，立即删除！

相关文章

你可能不知道的5款好用封面设计工具，快来一探究竟吧！

你可能不知道的5款好用封面设计工具，快来一探究竟吧！

我相信每个作者和出版商都希望在一部作品完成后有一个醒目的封面&#xff0c;这样潜在的读者就会有足够的好奇心拿起这本书&#xff0c;你的书的销量就会上升。这就是封面设计软件的使用&#xff0c;专业的封面设计软件可以增加前沿效果&#xff0c;呈现最适合书籍内容的创意布…
阅读更多...
UniversalTransformer with Adaptive Computation Time(ACT)

UniversalTransformer with Adaptive Computation Time(ACT)

原论文链接&#xff1a;https://arxiv.org/abs/1807.03819 Main code import torch import numpy as npclass PositionTimestepEmbedding(torch.nn.Module):def forward(self, x, t):device x.devicesequence_length x.size(1)d_model x.size(2)position_embedding np.arr…
阅读更多...
（学习打卡2）重学Java设计模式之六大设计原则

（学习打卡2）重学Java设计模式之六大设计原则

前言&#xff1a;听说有本很牛的关于Java设计模式的书——重学Java设计模式&#xff0c;然后买了(*^▽^*) 开始跟着小傅哥学Java设计模式吧&#xff0c;本文主要记录笔者的学习笔记和心得。 打卡&#xff01;打卡&#xff01; 六大设计原则 &#xff08;引读&#xff1a;这里…
阅读更多...
AIGC带给开发者的冲击

AIGC带给开发者的冲击

未来会有两种开发者&#xff0c;一种是会使用AIGC工具的开发者另一种是不会使用AIGC的开发者&#xff0c;AIGC的出现提高了开发效率和代码质量&#xff0c;对开发者意味着需要不断学习和适应新的技术和工作范式&#xff0c;开发者可以把更多的精力放在高级抽象的定义以及更高维…
阅读更多...
（16）Linux 进程等待 wait/waitpid 的 status 参数

（16）Linux 进程等待 wait/waitpid 的 status 参数

前言&#xff1a;我们开始讲解进程等待&#xff0c;简单地讲解 wait 函数&#xff0c;然后我们主要讲解 waitpid 函数。由于 wait 只有一个参数 status&#xff0c;且 waitpid 有三个参数且其中一个也是 status&#xff0c;我们本章重点讲解这个 status 参数。 一、进程等待&a…
阅读更多...
学习笔记240102 --- 表单无法输入，是否data中没有提前声明导致的

学习笔记240102 --- 表单无法输入，是否data中没有提前声明导致的

前端框架 &#xff1a;vue2.x 第三方ui组件&#xff1a;ElementUI 操作系统&#xff1a;windows 浏览器&#xff1a;chrome 谷歌 问题描述 表单使用中&#xff0c;没有在data中提前声明参数&#xff0c;当数据回显时&#xff0c;表单无法输入 <el-form :model"queryPa…
阅读更多...
Oracle-数据库迁移之后性能变慢问题分析

Oracle-数据库迁移之后性能变慢问题分析

问题背景&#xff1a; ​一套Oracle11.2.0.4的RAC集群&#xff0c;通过Dataguard switchover方式迁移到新机器之后&#xff0c;运行第一天应用报障说应用性能慢&#xff0c;需要进行性能问题排查 问题分析&#xff1a; 首先&#xff0c;登陆到服务器&#xff0c;用TOP看一眼两个…
阅读更多...
CMake入门教程【基础篇】CMake+Linux gcc构建C++项目

CMake入门教程【基础篇】CMake+Linux gcc构建C++项目

文章目录 1.概述2.GCC与CMake介绍3.安装CMake和GCC4.代码示例 1.概述 在Linux环境下&#xff0c;使用CMake结合GCC&#xff08;GNU Compiler Collection&#xff09;进行项目构建是一种常见且高效的方法。CMake作为一个跨平台的构建系统&#xff0c;可以生成适用于不同编译器的…
阅读更多...
CentOS 7 实战指南：文件操作命令详解

CentOS 7 实战指南：文件操作命令详解

写在前面 想要快速掌握 CentOS 7 系统下的文件操作技巧吗&#xff1f;不用担心&#xff01;我为你准备了一篇详细的技术文章&#xff0c;涵盖了各种常用的文件操作命令。无论您是初学者还是有一定经验的用户&#xff0c;这篇文章都能帮助您加深对 CentOS 7 文件操作的理解&…
阅读更多...
海外住宅IP代理的工作原理和应用场景分析，新手必看

海外住宅IP代理的工作原理和应用场景分析，新手必看

海外住宅IP代理作为一种技术解决方案&#xff0c;为用户提供了访问全球网络资源和维护隐私安全的方法。本文将介绍海外住宅IP代理的工作原理和应用场景&#xff0c;帮助读者更好地理解和利用这一技术。 一、工作原理 海外住宅IP代理的工作原理基于代理服务器和IP地址的转发。它…
阅读更多...
阿里云系统盘测评ESSD、SSD和高效云盘IOPS、吞吐量性能参数表

阿里云系统盘测评ESSD、SSD和高效云盘IOPS、吞吐量性能参数表

阿里云服务器系统盘或数据盘支持多种云盘类型&#xff0c;如高效云盘、ESSD Entry云盘、SSD云盘、ESSD云盘、ESSD PL-X云盘及ESSD AutoPL云盘等&#xff0c;阿里云百科aliyunbaike.com详细介绍不同云盘说明及单盘容量、最大/最小IOPS、最大/最小吞吐量、单路随机写平均时延等性…
阅读更多...
HackTheBox - Medium - Linux - BroScience

HackTheBox - Medium - Linux - BroScience

BroScience BroScience 是一款中等难度的 Linux 机器&#xff0c;其特点是 Web 应用程序容易受到“LFI”的攻击。通过读取目标上的任意文件的能力&#xff0c;攻击者可以深入了解帐户激活码的生成方式&#xff0c;从而能够创建一组可能有效的令牌来激活新创建的帐户。登录后&a…
阅读更多...
李沐机器学习系列1--- 线性规划

李沐机器学习系列1--- 线性规划

1 Introduction 1.1 线性回归函数 典型的线性回归函数 f ( x ) w ⃗ ⋅ x ⃗ f(x)\vec{w} \cdot \vec{x} f(x)w ⋅x 现实生活中&#xff0c;简单的线性回归问题很少&#xff0c;这里有一个简单的线性回归问题。房子的价格和房子的面积以及房子的年龄假设成线性关系。 p r …
阅读更多...
如何做好设备维护管理？这款设备管理系统值得推荐

如何做好设备维护管理？这款设备管理系统值得推荐

在现代化的工业生产中&#xff0c;设备的高效运行是保障生产安全和效率的关键因素。然而&#xff0c;在企业实际的设备维护管理业务中&#xff0c;仍面临着许多难题与痛点&#xff1a; 设备档案管理乱&#xff1a;传统管理方式下&#xff0c;如果想查询设备的历史巡检、维修、…
阅读更多...
[雷池WAF]长亭雷池WAF配置基于健康监测的负载均衡，实现故障自动切换上游服务器

[雷池WAF]长亭雷池WAF配置基于健康监测的负载均衡，实现故障自动切换上游服务器

为了进一步加强内网安全&#xff0c;在原有硬WAF的基础上&#xff0c;又在内网使用的社区版的雷池WAF&#xff0c;作为应用上层的软WAF。从而实现多WAF防护的架构。 经过进一步了解&#xff0c;发现雷池WAF的上游转发代理是基于Tengine的&#xff0c;所以萌生出了一个想法&…
阅读更多...
用通俗易懂的方式讲解大模型：在 CPU 服务器上部署 ChatGLM3-6B 模型

用通俗易懂的方式讲解大模型：在 CPU 服务器上部署 ChatGLM3-6B 模型

大语言模型&#xff08;LLM&#xff09;的量化技术可以大大降低 LLM 部署所需的计算资源&#xff0c;模型量化后可以将 LLM 的显存使用量降低数倍&#xff0c;甚至可以将 LLM 转换为完全无需显存的模型&#xff0c;这对于 LLM 的推广使用来说是非常有吸引力的。 本文将介绍如何…
阅读更多...
双侧电源系统距离保护MATLAB仿真模型

双侧电源系统距离保护MATLAB仿真模型

微❤关注“电气仔推送”获得资料&#xff08;专享优惠&#xff09; 系统原始数据 双侧电源系统模型如图所示&#xff1a; 仿真模型搭建 将线路AB分成Line1和Line2&#xff0c;将线路BC分成Line3和Line4&#xff0c;用三相电压电流测量模块作为系统母线&#xff0c;根据系统已…
阅读更多...
在Cadence中单独添加或删除器件与修改网络的方法

在Cadence中单独添加或删除器件与修改网络的方法

首先需要在设置中使能 ,添加或修改逻辑选项。 添加或删除器件&#xff0c;点击logic-part&#xff0c;选择需要添加或删除的器件&#xff0c;这里的器件必须是PCB中已经有的器件&#xff0c;Refdes中输入添加或删除的器件标号&#xff0c;点击Add添加。 添加完成后就会显示在R1…
阅读更多...
Linux学习记录——삼십삼 http协议

Linux学习记录——삼십삼 http协议

文章目录 1、URL2、http协议的宏观构成3、详细理解http协议1、http请求2、http响应1、有效载荷格式2、有效载荷长度3、客户端要访问的资源类型4、修改响应写法5、处理不同的请求6、跳转 3、请求方法&#xff08;GET/POST&#xff09;4、HTTP状态码&#xff08;实现3和4开头的&a…
阅读更多...
基于深度学习的交通标志图像分类识别系统

基于深度学习的交通标志图像分类识别系统

温馨提示&#xff1a;文末有 CSDN 平台官方提供的学长 QQ 名片 :) 1. 项目简介 本文详细探讨了一基于深度学习的交通标志图像识别系统。采用TensorFlow和Keras框架&#xff0c;利用卷积神经网络&#xff08;CNN&#xff09;进行模型训练和预测&#xff0c;并引入VGG16迁移学习…
阅读更多...
推荐文章
最新文章

Copyright @ 2022~2023