hive « hadoop « Java Database Q&A





1. hadoop hive question    stackoverflow.com

I'm trying to create tables pragmatically using JDBC. However, I can't really see the table I created from the hive shell. What's worse, when i access hive shell from different directories, ...

2. copy resultSet without using cachedRowSet    stackoverflow.com

I 'm trying to close the connection after executing a query. Before, I just create a CacheRowSetImpl instance and it will take care of release the resources for me. However, I ...

3. Building Apache Hive - impossible to resolve dependencies    stackoverflow.com

I am trying out the Apache Hive as per http://wiki.apache.org/hadoop/Hive/GettingStarted and am getting this error from Ivy:

Downloaded file size doesn't match expected Content Length for http://archive.apache.org/dist/hadoop/core/hadoop-0.19.0/hadoop-0.19.0.tar.gz. Please retry.
This error ...

4. Hadoop Hive web interface options    stackoverflow.com

I've been experimenting with Hive for some data mining activities and would like to make it easily available to less command line orientated colleagues. Hive does now ship with a web interface ...

5. Even data distribution on hadoop/hive    stackoverflow.com

I am trying a small hadoop setup (for experimentation) with just 2 machines. I am loading about 13GB of data, a table of around 39 million rows, with a replication factor ...

6. How to connect to Hadoop/Hive from .NET    stackoverflow.com

I am working on a solution where I will have a Hadoop cluster with Hive running and I want to send jobs and hive queries from a .NET application to be ...

7. Does Hive have a String split function?    stackoverflow.com

I am looking for a in-built String split function in Hive? E.g. if String is A|B|C|D|E then I want to have a function like array split(string input, char delimiter) so that I get back [A,B,C,D,E]. Does such ...

8. Specifying compression codec for a INSERT OVERWRITE SELECT in Hive    stackoverflow.com

I have a hive table like

  CREATE TABLE beacons
 (
     foo string,
     bar string,
     foonotbar string
 )
 COMMENT ...

9. Ability to limit maximum reducers for a hadoop hive mapred job?    stackoverflow.com

I've tried prepending my query with:

set mapred.running.reduce.limit = 25;
And
 set hive.exec.reducers.max = 35;
The last one jailed a job with 530 reducers down to 35... which makes me think it was ...





10. Hive, hadoop, and the mechanics behind hive.exec.reducers.max    stackoverflow.com

In context of this other question here Using hive.exec.reducers.max directive has truely baffled me. From my perspective I thought hive worked on some sort of logic like, I have N # ...

11. What are the consequences of adding a column to an exiting HIVE table?    stackoverflow.com

Suppose that a couple hundred Gigs after starting to use HIVE I want to add a column. From the various articles & pages I have seen, I cannot understand the consequences in ...

12. How to make hive load meta store from certain path instead of creating at the current directory?    stackoverflow.com

I'm using Hive for some data processing. But whenever I start the Hive-Shell it creates a metastore at the current directory and I can not access to my tables which I ...

13. How to convert .txt file to Hadoop's sequence file format    stackoverflow.com

To effectively utilise map-reduce jobs in Hadoop, i need data to be stored in hadoop's sequence file format. However,currently the data is only in flat .txt format.Can ...

14. Create temporary table in Hive?    stackoverflow.com

Does Hive support temporary tables? I can't find it in the apache docs.

15. Any Good Opensource Analytics front end tool?    stackoverflow.com

Is there any good open source web-based front end analytics tool that can talk to HIVE/HDFS for data ?

16. Hadoop Hive - Split string    stackoverflow.com

I am a new hivebe. My Query : In the log file we have a request field like this "GET /img/home/search-user-ico.jpg HTTP/1.1" .There are more than 10,000 records are available. Example : "GET /img/home/search-user-ico.jpg ...





17. Importing data from HDFS to Hive table    stackoverflow.com

I have my data in data/2011/01/13/0100/file in HDFS, each of thes file contain data in tab separated, say name, ip , url. I want to create a table in Hive and ...

18. Hive / Hadoop / Flatfile: What is an efficient way to combine and concat rows    stackoverflow.com

id col1 col2 ... coln
---------------------
foo barA barB ...
foo barD barX
boo barA barC
foo barC barC
I'd like to combine this into 'collapsed' rows which look like this:
foo barA;barD;barC barB;barX;barC
boo barD barC
At the moment ...

19. Hive - How can I write a create statement for a variable length, existing, hdfs file?    stackoverflow.com

So, I have an existing hdfs directory, containing a bunch of files. These files are all tab delimited. I have a hive statement....

create external table
   mytable(
    ...

20. Hadoop Basics: What do I do with the output?    stackoverflow.com

(I'm sure a similar question exists, but I haven't found the answer I'm looking for yet.) I'm using Hadoop and Hive (for our developers with SQL familiarity) to batch process multiple ...

21. MDX support for Hive (Hadoop)    stackoverflow.com

Is there any support for Multidimensional Expressions (MDX) for Hadoop's Hive ?

22. Custom Map Reduce Program on Hive,whats the Rule?how about input and output?    stackoverflow.com

Hello Fellow Developer,i got stuck a few day because i want to create custom map reduce program based on my query on hive,not few example i've found after googling and still ...

23. How to configure hive server to run on remote mode?    stackoverflow.com

i added hive package to my hadoop cluster. if i go into hive cli, i can run hive in remote mode. but queries going through hive server runs in local mode ...

24. What's the best way to support array column types with external tables in hive?    stackoverflow.com

So i have external tables of tab delimited data. A simple table looks like this:

create external table if not exists categories
(id string, tag string, legid string, image string, parent string, created_date ...

25. Hive installation issue while following Hive apache wiki step by step    stackoverflow.com

After installed Hive by the instruction on Hive apache wiki step by step, I invoked hive shell and typed "CREATE TABLE pokes (foo INT, bar STRING);", then it comes following error, ...

26. Hive - create a table from zip file    stackoverflow.com

I have bunch of zip files of CSVs, that I want to create Hive table from. I'm trying to figure out what's the best way to do so.

  • Unzip the files, upload ...

27. Hadoop Hive Query: Multi-join    stackoverflow.com

How can I do sub-selections in Hive? I think I might be making a really obvious mistake that's not so obvious to me... Error I'm receiving: FAILED: Parse Error: line 4:8 cannot ...

28. Is Apache Hive used more for the programming language or for the data warehouse aspects?    stackoverflow.com

I used to think that Hive was just a SQL-like programming language used to make writing MapReduce-type jobs easier (i.e., a SQL-like version of Pig/Pig Latin). I'm reading more about it ...

29. COLLECT_SET() in Hive (Hadoop)    stackoverflow.com

I just learned about the collect_set() function in Hive, and I started a job on a development 3-node cluster. I only have about 10 GB to process. The job, however, is literally ...

30. COLLECT_SET() in Hive, keep duplicates?    stackoverflow.com

Is there a way to keep the duplicates in a collected set in Hive, or simulate the sort of aggregate collection that Hive provides using some other method? I want to ...

31. Hive - JobTracking?    stackoverflow.com

I've been through the methods, but I can't seem to find out how to extrapolate the % completed (like an eventListener!) of a job in Hive. Please help! EDIT ...

32. sqoop import complete but hive show tables can't see table    stackoverflow.com

After install hadoop, hive (CDH version) I execute

./sqoop import -connect jdbc:mysql://10.164.11.204/server -username root -password password -table user -hive-import --hive-home /opt/hive/
All goes fine, but when I enter hive command line and execute ...

33. Parsing and loading into Hive/Hadoop    stackoverflow.com

i am new to hadoop map reduce framework, and I am thinking of using hadoop map reduce to parse my data. I have thousands of big delimited files for which I ...

34. Partition Hive table by existing field?    stackoverflow.com

Can I partition a Hive table upon insert by an existing field? I have a 10 GB file with a date field and an hour of day field. Can I load this ...

35. Hive doesn't work on install    stackoverflow.com

I get the following error in Hive when executing any command.

hive> show tables;
FAILED: Error in metadata: javax.jdo.JDOFatalInternalException: Unexpected exception caught.
NestedThrowables:
java.lang.reflect.InvocationTargetException
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
This is in the hive.log.
2011-07-27 ...

36. What difference of RDBMS and Hive?    stackoverflow.com

In RDMS like MySQL thereis database, are there database also on the Hive ?as i read on the manual, hive only have table, i bit confuse about it.. and what is different ...

37. register hive udf using hue api    stackoverflow.com

Can anyone tell me how to register a UDF by using HUE API. I am using below code but its unable to register it.

Query query1=new Query("create temporary function calculate as 'com.sample.MYUDF'",configuration,hadoop_user);
thriftclient.query(query1);
Query ...

38. Hive out-of-the-box json parser    stackoverflow.com

I have a text file containing json records I would like to load to Hive. My json looks like:

{"vr":1,"tm":1312816191516,"tms":"08-08-2011 15:09:51.516 GMT","as":1002,"pb":1102,"cts":[1204,1205],"ctgs":[1304,1305],"op":1400,"ev":2,"dv":1503,"dvgs":[1605,1606],"cnt":"cnt5","usr":"usr8","atts":[{"id":8002,"val":"ccc"},{"id":8003,"val":"ddd"}],"sel":{"cm":2102,"ty":"PRE","ag":3002,"ad":4002,"fl":5002,"fla":6002,"hg":7002,"mc":"WAP","pr":0.1}}
As you can see I have a nested json with arrays ...

39. Create Datasource Hive on Pentaho hadoop    stackoverflow.com

I try pentaho hadoop to make report designer with Hive Datasource. I have make connection to hadoop with

connection url: jdbc:hive://localhost:10000  and
driver class name :org.apache.hadoop.hive.jdbc.HiveDriver
Then I create datasource which I pointed ...

40. Ignore a subdirectory in an external table in Apache Hive    stackoverflow.com

Does Hive have a setting or a way of forcing it to ignore subdirectories when querying an external table? I have a subdirectory in my table folder that's not part of ...

41. Problem running Hive on Hadoop    stackoverflow.com

I downloaded the latest Hive release 0.7.1.

Cannot find hadoop installation: $HADOOP_HOME must be set or hadoop must be in the path
I went into this directory /home/hadoop/hive-0.7.1-bin/bin and tried ...

42. Multiple volume & limit disk usage with Hadoop    stackoverflow.com

I am using Hadoop to processing on large set of data. I set up a hadoop node to use multiple volumes : one of these volume is a NAS with 10To ...

43. Hadoop Hive DOUBLE type question    stackoverflow.com

Quick Hive/Hadoop question from a new user. I have a DOUBLE column that has "1.8E8" for value, does it mean I reached the max value for DOUBLE? Screenshot

44. Why Hive errors FileNotFoundException when a UDF use some external resource files but runs right at native?    stackoverflow.com

A UDF used some external resource files, then it error: "java.io.FileNotFoundException: resource/placeMap.txt (No such file or directory)", this function runs right at native, why wrong at hive, how can I solved it ...

45. Hive : SerDe for mixed record types files    stackoverflow.com

I'm running Hive 071, and I have a multiple format record CSV file,
which I need to load into a few Hive tables
File example EventA,1,2,3
EventB,Hello,World
EventA,12,21,3
EventA,4,5,6
EventA,20,0,30
EventB,Good,Morning
need to load to:
Table A(x ...

46. How does Hive decide when to use map reduce and when not to?    stackoverflow.com

As a simple example,

select * from tablename;
DOES NOT kick in map reduce, while
select count(*) from tablename;
DOES. What is the general principle used to decide when to use map reduce (by hive)? ...

47. java.io.IOException: No LZO codec found, cannot run    stackoverflow.com

I tried to use lzo in my hive script, but got this error message. It seemed that I did not have the class for lzo in the classpath. Did anyone else meet ...

48. Hive: dynamic partition adding to external table    stackoverflow.com

I am running hive 071, processing existing data which is has the following directory layout:
-TableName
- d= (e.g. 2011-08-01)
- d=2011-08-02
- d=2011-08-03

... etc
under each date I have the date files.
now ...

49. How to load data to hive from HDFS without removing the source file?    stackoverflow.com

LOAD DATA INPATH 'hdfs_file' INTO TABLE tablename; When load data from HDFS to hive, it looks like moving the hdfs_file to hive/warehouse dir, How can it copy but move? for the file will ...

50. how to write subquery and use "In" Clause in Hive    stackoverflow.com

How can I use In clause in Hive I want to write something like this in Hive select x from y where y.z in (select distinct z from y) order ...

51. Custom InputFormat with Hive    stackoverflow.com

Update: Alright, it turns out the reason that the below isn't working is because I'm using a newer version of the InputFormat API (import org.apache.hadoop.mapred which is the old versus import ...

52. How can I get the Hive JDBC driver and the Hive shell talk to the same DB?    stackoverflow.com

I have a hive sever running on default port 10000 started via: hive --service hiveserver I then have java program (the tutorial!) using the Hive JDBC Client connect to it ...

53. Pentaho data-integration with Hive CDH3u0    stackoverflow.com

I want to try pentaho data integration CE v 4.2 with Hadoop Hive. i use Hive version from CDH3uO. I try to use demo pentaho for hadoop use file >> https://www.yousendit.com/download/UnlCOU1ld0F1Yk5FQlE9PQ but ...

54. Does hive instantiate a new UDF object for each record?    stackoverflow.com

Say I'm building a UDF class called StaticLookupUDF that has to load some static data from a local file during construction. In this case I want to ensure that I'm not replicating ...

55. How to get Hive working with bz2 ?    stackoverflow.com

Since bz2 is splittable and saves space, I'm planning to store the table data in bzip2 format. However it seems it's not recognizing the compressed format. It was fine with gzip format. The ...

56. Developing web analytics with Hadoop    stackoverflow.com

I want to develop a web analytics platform in order to create aggregated data about web traffic (page views, visits, visitors, etc) by parsing apache access logs. Can I do it only ...

57. Hadoop/Hive query to split one column into several ones    stackoverflow.com

I am using HIVE with two tables looking like (more or less): -TABLE1 defined as [(Variables : string),(Value1 : int),(Value2 : int)] with field "Variables" looking like "x0,x1,x2,x3,...,xn" -TABLE2 define as [(Value1Sum : int),(Value2Sum ...

58. Hive: Create New Table from Existing Partitioned Table    stackoverflow.com

I'm using Amazon's Elastic MapReduce and I have a hive table created based on a series of log files stored in Amazon S3 and split in folders by day like so:

  • data/day=2011-09-01/log_file.tsv
  • data/day=2011-09-02/log_file.tsv
I ...

59. Problems in running Hive queries    stackoverflow.com

I have configured Hadoop and Hive on Windows through Cygwin. But I am facing some problems like: in hive terminal (CLI): hive> When I enter query, the query do not execute and terminal remains busy. If ...