openmldb install log

news2024/11/17 3:29:21

下载/源码编译#

如果你的操作系统可以直接运行预编译包，则可以从以下地址下载：

GitHub release 页面：Releases · 4paradigm/OpenMLDB · GitHubOpenMLDB is an open-source machine learning database that provides a feature platform computing consistent features for training and inference. - Releases · 4paradigm/OpenMLDBhttps://github.com/4paradigm/OpenMLDB/releases
镜像站点（中国）：Index of /downloadhttps://www.openmldb.com/download/

其中预编译包和可支持的操作系统的对应关系为：

openmldb-x.x.x-linux.tar.gz: CentOS 7.x, Ubuntu 20.04, SUSE 12 SP3
openmldb-x.x.x-darwin.tar.gz: macOS >= 12.0

如果用户的操作系统不在列表，或者期望从源代码编译，则参考我们的源码编译文档。

一键发行版部署：

wget https://github.com/4paradigm/OpenMLDB/releases/download/v0.9.0/openmldb-0.9.0-linux.tar.gz
tar -zxvf openmldb-0.9.0-linux.tar.gz
cd openmldb-0.9.0-linux

修改配置

conf/openmldb-env.sh

# openmldb mode: standalone / cluster
export OPENMLDB_MODE=${OPENMLDB_MODE:=cluster}

OPENMLDB_FORCE_LOCAL = true

#单机部署也设置OPENMLDB_MODE:=cluster ,否则部署不执行

#配置以存在的spark环境，如果不设置，从上线下载，并在openmldb home 目录建立一个软#链接spark，指向当前目录下载后的spark目录。

#这里是下载到服务器进行解压的Index of /download/v0.8.5https://www.openmldb.com/download/v0.8.5/

export SPARK_HOME=/data/spark-3.2.1-bin-openmldbspark
export RUNNER_EXISTING_SPARK_HOME=/data/spark-3.2.1-bin-openmldbspark

#因为zookeeper 都是本地，不用修改conf/hosts文件

#其他参数都保留不变

（可选）spark 环境包安装

1.上传

2.解压

tar -zxvf spark-3.2.1-bin-openmldbspark.tgz

配置后注意说明：

ZooKeeper集群地址，如果使用已有ZooKeeper集群，需要配置OPENMLDB_USE_EXISTING_ZK_CLUSTER=true，并配置OPENMLDB_ZK_CLUSTER。（如果在conf/hosts中配置外部ZK集群，请注释标注其不受sbin部署影响，避免混乱。）

需要此工具部署ZooKeeper集群时，在conf/hosts中配置[zookeeper]。填写多个ZooKeeper节点，即部署ZooKeeper集群，无需额外配置。

Spark环境，如果需要使用运行机器上已有的Spark环境，需要配置RUNNER_EXISTING_SPARK_HOME（地址为TaskManager运行机器上的路径）。如果部署机器存在Spark环境，并想要在TaskManager机器上使用此套环境，可配置SPARK_HOME（部署到TaskManager机器同名路径上）。SPARK_HOME不进行配置时，将自动下载、使用特定Spark发行版。

正式部署：

安装部署 — OpenMLDB documentation

查看swap是否关闭

free

(base) root@ubuntu:/data/openmldb-0.8.5-linux# free
total used free shared buff/cache available
Mem: 65858144 31038516 474216 669964 34345412 33452332
Swap: 0 0 0

如果swap一项全部为0表示已经关闭，否则运行下面命令关闭swap

swapoff -a

参考上面文档，用root在服务器执行如下：

bash sbin/init_env.sh

sbin/deploy-all.sh
[root@zbpt2-82 openmldb-0.8.5-linux]# sbin/deploy-all.sh
use existing spark /data/spark-3.2.1-bin-openmldbspark on runner, overwrite SPARK_HOME
OPENMLDB envs:
OPENMLDB_ZK_CLUSTER_CLIENT_PORT=2181
OPENMLDB_ZK_ROOT_PATH=/openmldb
OPENMLDB_FORCE_LOCAL=true
OPENMLDB_ZK_CLUSTER=localhost:2181
OPENMLDB_TABLET_PORT=10921
OPENMLDB_ZK_CLUSTER_PEER_PORT=2888
OPENMLDB_VERSION=0.8.5
OPENMLDB_USE_EXISTING_ZK_CLUSTER=false
OPENMLDB_ZK_HOME=/data/openmldb-0.8.5-linux/zookeeper
OPENMLDB_HOME=/data/openmldb-0.8.5-linux
OPENMLDB_TASKMANAGER_PORT=9902
OPENMLDB_APISERVER_PORT=9080
OPENMLDB_MODE=cluster
OPENMLDB_ZK_CLUSTER_ELECTION_PORT=3888
CLEAR_OPENMLDB_INSTALL_DIR=false
OPENMLDB_NAMESERVER_PORT=7527
SPARK_HOME=/data/spark-3.2.1-bin-openmldbspark
RUNNER_EXISTING_SPARK_HOME=/data/spark-3.2.1-bin-openmldbspark

deploy tablet to localhost:10921 /tmp/openmldb/tablet-1
copy /data/openmldb-0.8.5-linux to localhost:/tmp/openmldb/tablet-1
deploy tablet to localhost:10922 /tmp/openmldb/tablet-2
copy /data/openmldb-0.8.5-linux to localhost:/tmp/openmldb/tablet-2
deploy nameserver to localhost:7527 /data/openmldb-0.8.5-linux
skip rsync as dest=src: /data/openmldb-0.8.5-linux
deploy apiserver to localhost:9080 /data/openmldb-0.8.5-linux
skip rsync as dest=src: /data/openmldb-0.8.5-linux
use /data/spark-3.2.1-bin-openmldbspark, skip download openmldbspark
deploy taskmanager to localhost:9902 /data/openmldb-0.8.5-linux
skip rsync as dest=src: /data/openmldb-0.8.5-linux
/data/openmldb-0.8.5-linux/zookeeper already exists. Skip download zookeeper.
deploy zookeeper to localhost:2181 /tmp/openmldb/zk-1
copy /data/openmldb-0.8.5-linux/zookeeper to localhost:/tmp/openmldb/zk-1

完成。。。。。。。。。。。。。。。。。。。。。。。

启动环境：

[root@zbpt2-82 openmldb-0.8.5-linux]# sbin/start-all.sh
use existing spark /data/spark-3.2.1-bin-openmldbspark on runner, overwrite SPARK_HOME
OPENMLDB envs:
OPENMLDB_ZK_CLUSTER_CLIENT_PORT=2181
OPENMLDB_ZK_ROOT_PATH=/openmldb
OPENMLDB_FORCE_LOCAL=true
OPENMLDB_ZK_CLUSTER=localhost:2181
OPENMLDB_TABLET_PORT=10921
OPENMLDB_ZK_CLUSTER_PEER_PORT=2888
OPENMLDB_VERSION=0.8.5
OPENMLDB_USE_EXISTING_ZK_CLUSTER=false
OPENMLDB_ZK_HOME=/data/openmldb-0.8.5-linux/zookeeper
OPENMLDB_HOME=/data/openmldb-0.8.5-linux
OPENMLDB_TASKMANAGER_PORT=9902
OPENMLDB_APISERVER_PORT=9080
OPENMLDB_MODE=cluster
OPENMLDB_ZK_CLUSTER_ELECTION_PORT=3888
CLEAR_OPENMLDB_INSTALL_DIR=false
OPENMLDB_NAMESERVER_PORT=7527
SPARK_HOME=/data/spark-3.2.1-bin-openmldbspark
RUNNER_EXISTING_SPARK_HOME=/data/spark-3.2.1-bin-openmldbspark

use existing spark /data/spark-3.2.1-bin-openmldbspark on runner, overwrite SPARK_HOME
start zookeeper in /tmp/openmldb/zk-1 with endpoint localhost:2181
ZooKeeper JMX enabled by default
Using config: /tmp/openmldb/zk-1/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
use existing spark /data/spark-3.2.1-bin-openmldbspark on runner, overwrite SPARK_HOME
start tablet in /tmp/openmldb/tablet-1 with endpoint localhost:10921
Starting tablet ...
process pid is 12157
Start tablet success
start tablet in /tmp/openmldb/tablet-2 with endpoint localhost:10922
Starting tablet ...
process pid is 12227
Start tablet success
use existing spark /data/spark-3.2.1-bin-openmldbspark on runner, overwrite SPARK_HOME
start nameserver in /data/openmldb-0.8.5-linux with endpoint localhost:7527
Starting nameserver ...
process pid is 12426
Start nameserver success
Start recovering data...
Recovering data done
use existing spark /data/spark-3.2.1-bin-openmldbspark on runner, overwrite SPARK_HOME
start apiserver in /data/openmldb-0.8.5-linux with endpoint localhost:9080
Starting apiserver ...
process pid is 12886
Start apiserver success
use existing spark /data/spark-3.2.1-bin-openmldbspark on runner, overwrite SPARK_HOME
start taskmanager in /data/openmldb-0.8.5-linux with endpoint localhost:9902
Starting taskmanager ...
SPARK_HOME: /data/spark-3.2.1-bin-openmldbspark
Rewrite properties by ./conf/taskmanager.properties
process pid is 13045
Start taskmanager success
OpenMLDB start success