目录

导包

修改配置文件 

 修改hive-site.xml文件

启动hadoop

启动hive

启动spark

测试

查看表

---

导包

spark连接hive需要六个关键的jar包，以及将hive的配置文件hive-site.xml拷贝到spark的conf目录下。如果你hive配置没问题的话，这些jar都在hive的目录中。

将jar包导入到 opt/soft/spark312/jars/

[root@hadoop3 jars]# cp /opt/soft/hive312/lib/hive-beeline-3.1.2.jar ./
[root@hadoop3 jars]# cp /opt/soft/hive312/lib/hive-cli-3.1.2.jar ./
[root@hadoop3 jars]# cp /opt/soft/hive312/lib/hive-exec-3.1.2.jar ./
[root@hadoop3 jars]# cp /opt/soft/hive312/lib/hive-jdbc-3.1.2.jar ./
[root@hadoop3 jars]# cp /opt/soft/hive312/lib/hive-metastore-3.1.2.jar ./
[root@hadoop3 jars]# cp /opt/soft/hive312/lib/mysql-connector-java-8.0.25.jar ./

修改配置文件 

切换目录

[root@hadoop3  /]# cd opt/soft/spark312/conf/

 把hive312/conf中的hive-site.xml复制到spark312/conf目录下

[root@hadoop3  conf]# cp /opt/soft/hive312/conf/hive-site.xml ./

 修改hive-site.xml文件

检查少什么，少则添加

<configuration>
  <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/opt/soft/hive312/warehouse</value>
  </property>
  <property>
    <name>hive.metastore.db.type</name>
    <value>mysql</value>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://192.168.152.184:3306/hiveone?createDatabaseIfNotExist=true</value>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.cj.jdbc.Driver</value>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>123456</value>
  </property>
  <property>
    <name>hive.metastore.schema.verification</name>
    <value>false</value>
        <description>关闭schema验证</description>
  </property>
  <property>
    <name>hive.cli.print.current.db</name>
    <value>true</value>
        <description>提示当前数据库名</description>
  </property>
  <property>
    <name>hive.cli.print.header</name>
    <value>true</value>
        <description>查询输出时带列名一起输出</description>
  </property>
<property>
      <name>hive.zookeeper.quorum</name>
      <value>192.168.152.192</value>
   </property>
    <property>
      <name>hbase.zookeeper.quorum</name>
      <value>192.168.152.192</value>

</property>
    <property>
      <name>hbase.zookeeper.quorum</name>
      <value>192.168.152.192</value>
   </property>

    <property>
      <name>hive.aux.jars.path</name>
      <value>file:///opt/soft/hive312/lib/hive-hbase-handler-3.1.2.jar,file:///opt/soft/hive312/lib/zookeeper-3.4.6.jar,file:///opt/soft/hive312/lib/hbase-client-2.3.5.jar,file:///opt/soft/hive312/lib/hbase-common-2.3.5-tests.jar,file:///opt/soft/hive312/lib/hbase-server-2.3.5.jar,file:///opt/soft/hive312/lib/hbase-common-2.3.5.jar,file:///opt/soft/hive312/lib/hbase-protocol-2.3.5.jar,file:///opt/soft/hive312/lib/htrace-core-3.2.0-incubating.jar</value>
</property>
<property>
  <name>hadoop.proxyuser.hadoop.hosts</name>
  <value>*</value>
</property>

<property>
  <name>hadoop.proxyuser.hdfs.groups</name>
  <value>*</value>
</property>
<property>
  <name>hive.metastore.uris</name>
  <value>thrift://192.168.152.192:9083</value>
</property>

</configuration>

配置完成，下面开始测试

启动hadoop

[root@gree2 ~]# start-all.sh

启动hive

nohup hive --service metastore &

nohup hive --service hiveserver2 &

beeline -u jdbc:hive2://192.168.152.192:10000

启动spark

spark-shell

测试

查看表

scala> spark.table("aa")

 这里的aa为hive默认库default下面的数据表

使用 spark.sql 

scala> spark.sql("use default")
scala> spark.sql("select * from aa")