Hive安装-windows(转载)

2021-04-21 20:27

阅读:395

1.安装hadoop

2.从maven中下载mysql-connector-java-5.1.26-bin.jar(或其他jar版本)放在hive目录下的lib文件夹

3.配置hive环境变量,HIVE_HOME=F:\hadoop\apache-hive-2.1.1-bin

4.hive配置

hive的配置文件放在$HIVE_HOME/conf下,里面有4个默认的配置文件模板

hive-default.xml.template                           默认模板

hive-env.sh.template                hive-env.sh默认配置

hive-exec-log4j.properties.template    exec默认配置

 hive-log4j.properties.template               log默认配置

可不做任何修改hive也能运行,默认的配置元数据是存放在Derby数据库里面的,大多数人都不怎么熟悉,我们得改用mysql来存储我们的元数据,以及修改数据存放位置和日志存放位置等使得我们必须配置自己的环境,下面介绍如何配置。

(1)创建配置文件

$HIVE_HOME/conf/hive-default.xml.template  -> $HIVE_HOME/conf/hive-site.xml

$HIVE_HOME/conf/hive-env.sh.template  -> $HIVE_HOME/conf/hive-env.sh

$HIVE_HOME/conf/hive-exec-log4j.properties.template ->  $HIVE_HOME/conf/hive-exec-log4j.properties

$HIVE_HOME/conf/hive-log4j.properties.template  -> $HIVE_HOME/conf/hive-log4j.properties

(2)修改 hive-env.sh

export HADOOP_HOME=F:\hadoop\hadoop-2.7.2
export HIVE_CONF_DIR=F:\hadoop\apache-hive-2.1.1-bin\conf
export HIVE_AUX_JARS_PATH=F:\hadoop\apache-hive-2.1.1-bin\lib

(3)修改 hive-site.xml

  1    
  2 
  3 property>  
  4 
  5 name>hive.metastore.warehouse.dirname>  
  6 
  7   
  8 
  9 value>/user/hive/warehousevalue>  
 10 
 11 description>location of default database for the warehousedescription>  
 12 
 13 property>  
 14 
 15 property>  
 16 
 17 name>hive.exec.scratchdirname>  
 18 
 19   
 20 
 21 value>/tmp/hivevalue>  
 22 
 23 description>HDFS root scratch dir for Hive jobs which gets created with write all (733) permission. For each connecting user, an HDFS scratch dir: ${hive.exec.scratchdir}/username> is created, with ${hive.scratch.dir.permission}.description>  
 24 
 25 property>  
 26 
 27 property>  
 28 
 29 name>hive.exec.local.scratchdirname>  
 30 
 31   
 32 
 33 value>F:/hadoop/apache-hive-2.1.1-bin/hive/iotmpvalue>  
 34 
 35 description>Local scratch space for Hive jobsdescription>  
 36 
 37 property>  
 38 
 39 property>  
 40 
 41 name>hive.downloaded.resources.dirname>  
 42 
 43   
 44 
 45 value>F:/hadoop/apache-hive-2.1.1-bin/hive/iotmpvalue>  
 46 
 47 description>Temporary local directory for added resources in the remote file system.description>  
 48 
 49 property>  
 50 
 51 property>  
 52 
 53 name>hive.querylog.locationname>  
 54 
 55   
 56 
 57 value>F:/hadoop/apache-hive-2.1.1-bin/hive/iotmpvalue>  
 58 
 59 description>Location of Hive run time structured log filedescription>  
 60 
 61 property>  
 62 
 63 property>  
 64 
 65 name>hive.server2.logging.operation.log.locationname>  
 66 
 67 value>F:/hadoop/apache-hive-2.1.1-bin/hive/iotmp/operation_logsvalue>  
 68 
 69 description>Top level directory where operation logs are stored if logging functionality is enableddescription>  
 70 
 71 property>  
 72 
 73   
 74 
 75 property>  
 76 
 77 name>javax.jdo.option.ConnectionURLname>  
 78 
 79 value>jdbc:mysql://localhost:3306/hive?characterEncoding=UTF-8value>  
 80 
 81 property>  
 82 
 83 property>  
 84 
 85 name>javax.jdo.option.ConnectionDriverNamename>  
 86 
 87 value>com.mysql.jdbc.Drivervalue>  
 88 
 89 property>  
 90 
 91 property>  
 92 
 93 name>javax.jdo.option.ConnectionUserNamename>  
 94 
 95 value>rootvalue>  
 96 
 97 property>  
 98 
 99 property>  
100 
101 name>javax.jdo.option.ConnectionPasswordname>  
102 
103 value>rootvalue>  
104 
105 property>  
106 
107   
108 
109 property>  
110 
111 name>datanucleus.autoCreateSchemaname>  
112 
113 value>truevalue>  
114 
115 property>  
116 
117 property>  
118 
119 name>datanucleus.autoCreateTablesname>  
120 
121 value>truevalue>  
122 
123 property>  
124 
125 property>  
126 
127 name>datanucleus.autoCreateColumnsname>  
128 
129 value>truevalue>  
130 
131 property>  
132 
133   
134 
135 property>    
136 
137 name>hive.metastore.schema.verificationname>    
138 
139 value>falsevalue>    
140 
141 description>    
142 
143     Enforce metastore schema version consistency.    
144 
145     True: Verify that version information stored in metastore matches with one from Hive jars.  Also disable automatic    
146 
147           schema migration attempt. Users are required to manully migrate schema after Hive upgrade which ensures    
148 
149           proper metastore schema migration. (Default)    
150 
151     False: Warn if the version information stored in metastore doesn‘t match with one from in Hive jars.    
152 
153 description>    
154 
155 property>   

注:需要事先在hadoop上创建hdfs目录

 

(4)日志文件配置 略

 

5.MySQL设置

(1)创建hive数据库: create database hive default character set latin1;

(2)grant all on hive.* to hive@‘localhost‘  identified by ‘hive‘; 

 flush privileges;

--本人用的是root用户,所以这步省略

6.
(1)启动hadoop:start-all.cmd
(2)启动metastore服务:hive --service metastore
(3)启动Hive:hive
 若Hive成功启动,Hive本地模式安装完成。


7、查看mysql数据库
use hive; show tables;

8.在hive下建一张表:CREATE TABLE xp(id INT,name string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ‘\t‘; 

9.在MySQL中查看:select * from TBLS

安装过程中遇到的问题

 

(1)hive启动时报错Required table missing : "`VERSION`" in Catalog "" Schema "". DataNucleus requires this table to perform its persistence operations 
参考 http://宋亚飞.中国/post/98 
(2)Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
参考http://blog.csdn.net/freedomboy319/article/details/44828337
(3)Caused by: MetaException(message:Version information not found in metastore. )
参考 http://blog.csdn.net/youngqj/article/details/19987727
(4)Hive 创建表报"Specified key was too long; max key length is 767 bytes" 错误
参考 http://blog.csdn.net/niityzu/article/details/46606581


其他参考文章:
http://www.cnblogs.com/hbwxcw/p/5960551.html hive-1.2.1安装步骤
http://blog.csdn.net/jdplus/article/details/46493553 Hive本地模式安装及遇到的问题和解决方案
http://www.coin163.com/it/x8681464370981050716/spark-Hive CentOS7伪分布式下 hive安装过程中遇到的问题及解决办法
http://www.bogotobogo.com/Hadoop/BigData_hadoop_Hive_Install_On_Ubuntu_16_04.php APACHE HADOOP : HIVE 2.1.0 INSTALL ON UBUNTU 16.04


评论


亲,登录后才可以留言!