Appears to be outdated. When I add this to /etc/profile: export PYTHONPATH=$PYTHONPATH:/usr/lib/hive/lib/py I can then do the imports as listed in the link, with the exception of from hive import ThriftHive which actually need to be: from hive_service import ThriftHive Next the port in the example was 10000, which when I tried caused the program to hang.

The default Hive Thrift port is 9083, which stopped the hanging. I believe the easiest way is to use PyHive. To install you'll need these libraries: pip install sasl pip install thrift pip install thrift-sasl pip install PyHive Please note that although you install the library as PyHive, you import the module as pyhive, all lower-case. If you're on Linux, you may need to install SASL separately before running the above. Install the package libsasl2-dev using apt-get or yum or whatever package manager for your distribution. For Windows there are some options on GNU.org, you can download a binary installer.

On a Mac SASL should be available if you've installed xcode developer tools ( xcode-select --install in Terminal) After installation, you can connect to Hive like this: from pyhive import hive conn = hive.Connection(host='YOUR_HIVE_HOST', port=PORT, username='YOU') Now that you have the hive connection, you have options how to use it. You can just straight-up query: cursor = conn.cursor() cursor.execute('SELECT cool_stuff FROM hive_table') for result in cursor.fetchall(): use_result(result).or to use the connection to make a Pandas dataframe: import pandas as pd df = pd.read_sql('SELECT cool_stuff FROM hive_table', conn).

I assert that you are using HiveServer2, which is the reason that makes the code doesn't work. You may use pyhs2 to access your Hive correctly and the example code like that: import pyhs2 with pyhs2.connect(host='localhost', port=10000, authMechanism='PLAIN', user='root', password='test', database='default') as conn: with conn.cursor() as cur: #Show databases print cur.getDatabases() #Execute query cur.execute('select * from table') #Return column info from query print cur.getSchema() #Fetch table results for i in cur.fetch(): print i Attention that you may install python-devel.x86_64 cyrus-sasl-devel.x86_64 before installing pyhs2 with pip.

Jul 16, 2012. Installing Apache Thrift in Windows Installation Thrift can be a tiresome process. But for windows the compiler is available as a prebuilt exe. Download thrift.exe and add it into your environment variables. Writing Thrift definition file (.thrift file) Writing the Thrift definition file becomes really easy once you get.

Wish this can help you. The examples above are a bit out of date. Pyhs2 is no longer maintained. A better alternative is impyla It has many more features over pyhs2, for example, it has Kerberos authentication, which is a must for us. From impala.dbapi import connect conn = connect(host='my.host.com', port=10000) cursor = conn.cursor() cursor.execute('SELECT * FROM mytable LIMIT 100') print cursor.description # prints the result set's schema results = cursor.fetchall() ## cursor.execute('SELECT * FROM mytable LIMIT 100') for row in cursor: process(row) Cloudera is putting more effort now on hs2 client which is a C/C++ HiveServer2/Impala client.

Might be a better option if you push a lot of data to/from python. (has Python binding too - ) Some more information on impyla: • • Don't be confused that some of the above examples talk about Impala; just change port to 10000 (default) for HiveServer2, and it'll work the same way as with Impala examples. It's the same protocol (Thrift) that is used for both Impala and Hive. You don't have to do a global INVALIDATE METADATA, you could just do a table-level one INVALIDATE METADATA schema.table. Even then, I don't understand the downvote, because my code above connects to port 10000 - which is a thrift service of HiveServer2, so you don't have to do any invalidates, as you SQL commands would be run directly in Hive. Re-copying last paragraph.

Don't be confused that some of the above examples talk about Impala; just change port to 10000 for HiveServer2, and it'll work the same way as with Impala examples. – Aug 15 at 5:45. You could use python JayDeBeApi package to create DB-API connection from Hive or Impala JDBC driver and then pass the connection to pandas.read_sql function to return data in pandas dataframe.

Abba Full Album Free Download Torrent. Similar to eycheu's solution, but a little more detailed. Here is an alternative solution specifically for hive2 that does not require PyHive or installing system-wide packages. I am working on a linux environment that I do not have root access to so installing the SASL dependencies as mentioned in Tristin's post was not an option for me: If you're on Linux, you may need to install SASL separately before running the above. Install the package libsasl2-dev using apt-get or yum or whatever package manager for your distribution. Specifically, this solution focuses on leveraging the python package: JayDeBeApi.

In my experience installing this one extra package on top of a python Anaconda 2.7 install was all I needed. This package leverages java (JDK). I am assuming that is already set up. Step 1: Install JayDeBeApi pip install jaydebeap Step 2: Download appropriate drivers for your environment: • Here is a required for an enterprise CDH environment • that talks about where to find jdbc drivers for Apache Hive Store all.jar files in a directory. Mard Maratha Dj Song Free Download.

I will refer to this directory as /path/to/jar/files/. Step 3: Identify your systems authentication mechanism: In the pyhive solutions listed I've seen PLAIN listed as the authentication mechanism as well as Kerberos. Note that your jdbc connection URL will depend on the authentication mechanism you are using. I will explain Kerberos solution without passing a username/password. Create a Kerberos ticket if one is not already created $ kinit Tickets can be viewed via klist.

Installing Apache Thrift on Windows Although primarily designed for Unix-compatible systems, Apache Thrift can also be used in the Windows environment. The installation is very easy; it is enough to download the Thrift compiler from the download page,. No installation is needed; you may work directly with the downloaded file.

For consistency, I suggest renaming the file that you downloaded (that is, thrift-0.9.2.exe) to just thrift.exe. In order to be able to run this command from any directory on your computer, add this file's location to the PATH environment variable.

(It varies from version to version, but in most cases, it may be found if you right-click on My Computer and choose Properties. Then, look for. With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.