advanced training udf

[email protected]

Hive User Defined FunctionsUser defined functions can be used in Hive and can be used in queries.Consider the following sample example.

Step 1: Code

Example 1: Lower.java

Class Name:Lower.java

Package:com.zinnia.UDF

Right click on the project and click on Export->JAR file and select the Lower.java

JAR file:/home/hadoop/hadoop-0.20.2/hive-0.7.1-bin/Lower.jar

click on Finish

Step 2: Deploying the JARs to classpath: Adding jars to hive classpath is by

add jar Lower.jar;

List the jars by:

list jars;

create a temporary function as follows:

create temporary function lower as com.zinnia.UDF.Lower;

using the function as a query:

select lower(value) from test1 group by lower(value);

Similarly we can create for uppercase as follows:

Example 2: Upper.java

carry out the same steps of JAR creation and deploying to the Hive classpath.

Create a temporary function upper with command usage of

create temporary function upper as com.zinnia.UDF.Upper

Query the test1 table as below:

select upper(value) from test1 group by upper(value);

Output would be:

NAME1

NAME2

NAME3

NAME4

NAME5

Example 3: LengthUDF.java

Steps mentioned for the above examples are carried out in a similar way for this and we can get the length of the string.

Adding UDF permanently to the HIVE

To add the UDF permanently you have to place your source code inside hive source code.Below are the steps to do that.

Step 1: Create a directory to hold the source code mkdir /home/hadoop/hive-code

Step 2 : Download the code from the github git clone git://github.com/apache/hive.git /home/hadoop/hive-code

Step 3 : Place your code inside /home/hadoop/hive-code/ql/src/java with complete package(directory) structureStep 4 : Add the Following code to FunctionRegistry.java file inside /home/hadoop/hive-code/ql/src/java/org/apache/hadoop/hive/ql/exec folderimport com.zinnia.UDF.Upper;

Then add the following code after registerUDF("e", UDFE.class, false); line in the file

registerUDF("upper", Upper.class, false);

Step 5 : Compile the code $cd /home/hadoop/hive-code/ql

$ant

Step 6 : Copy latest code into hive distribution$cp /home/hadoop/hive-code/build/ql/ $HIVE_HOME/lib

Now your custom UDF will be available as permanent function in HIVE

Copyright 2012 [email protected]

advanced training udf

Documents