Hive Metastore
Data Analysis(CCA175)
Use Data Definition Language (DDL) to create tables in the Hive metastore for use by Hive and Impala.
Hive Meta-store Architecture.
Multiple Tools are used to process data once data is in place.
1. Hive , 2. Spark(Hive ) and 3. impala(Cloudera product) are used to process same data and DB structure.
To check list of components
hadoop fs -ls /user/cloudera> hive (Check hive)
> mysql -u root -p (check mysql)
Metastore Conf details:
core-site.xml conatins name node info:
<property>
<name>fs.defaultFS</name>
<value>hdfs://quickstart.cloudera:8020</value>
</property>
> cd /etc/hive/conf (Press Enter)
> view hive-site.xml
important items
1) metastore-warehouse-dir (Default location hive reads data)
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/hive/warehouse</value>
</property>
2) Hive uses metastore as mysql( Configuration of mysql as metadat serevr you can folow below blog).
http://apachehive.blogspot.in/
CHD5.6 show below metastore info:
Hive and Create Table Internals:
1) hive> create table cca_demo(t int, s string);
2) insert data into table
insert into table cca_demo values(1,'testing')
3) check file created in hive warehouse dir
hadoop fs -ls /user/hive/warehouse/cca_demo/00000_0
4) Verify the content of file
hadoop fs -cat /user/hive/warehouse/cca_demo/00000_0
Result: Record rows display
5) Verify same data in hive tables.
hive> select * from cca_demo;
6) check metastore entries in mysql tables(TBLS)
select * from TBLS;
Result: a table entry should be exists.
7) check description of table in hive.
describe formatted cca_demo;
Result: display schema information details of table.
-----------------------------------------------------------------
Spark-shell
start > spark-shel -- master local
> show table;
Result: list cca_demo table
impala-shell
>impala-shell
impala> show tables
Result: List cca_demo table
* same data should display ussing al tools.


No comments:
Post a Comment