Connectivity/Integration Issues with Hadoop-HBase plugin (v0.7)

[mdonnelly]

 We were looking into using the latest version of the Hadoop-HBase connector to integrate the HBase setup we currently have with JasperReports. The documentation makes it fairly straight-forward -- just install the plugin to iReport and provide the connection details. All I'm seeing from iReport at this point is "MasterNotRunningException"s.

net.sf.jasperreports.engine.JRException: org.apache.hadoop.hbase.MasterNotRunningException is actually the what we're seeing. Attached some logging to provide a little more detail.

Simply put, we have our hadoop (0.20.3), hbase (0.91) and JasperServer (4.0.0) installations on the same server. We're launching iReport from that server, installing plugin, creating a new HBase connection, and inputting our connection details (quorum and client port). We're using the default ports for HBase currently. These steps generate a MasterNotRunningException for the above default configuration (as well as non-default configurations). We've confirmed that we can connect to HBase via simple java clients (as well as the HBase shells).

Has anyone else encountered similar issues with this plugin? Or has anyone had luck with a similar setup?

Thanks

[eddiaz]

 John,

HBasePlugin 0.7 ships with Hadoop HBase 0.91-SNAPSHOOT so there shoudn't be compatibility issues.

We have tested the plugin/datasource against HBase in "Pseudo Distributed Mode" so it starts its own Zookeeper instance. Is your configuration different?

Also, could you provide details on how you connected to HBase with the simple Java clients?

- Eric

 

Post Edited by eddiaz at 09/08/2011 01:25

[mdonnelly]

Our configuration is usually fully distributed. We did change that today to "Pseudo Distributed" for the sake of testing. Getting the same results, however, with the usual MasterNotRunningException. 

And we usually connect to our HBase master via Java clients using the HBase api. We also took a JDBC approach and used a syntax/driver called HBQL along with simple Java examples (provided by the author) to connect to our HBase master. We are able to connect to HBase, retrieve and insert data in both cases.

[eddiaz]

 John,

We had upgraded our hbase 0.91-SNAPSHOT instance to the latest revision and we were able to reproduce the error you got about Master not running.

In order to fix this problem please upgrade the hbase binary shipped with iReport HBasePlugin with the one you have in your hbase instance. Base on the log you provided us the path for this binary is:

/home/hadoop/.ireport/4.0.0/modules/ext/org.apache.hbase/

Please let us know how it goes

- Eric

[mdonnelly]

Thank you for the response. 

The datasource connection tested successfully once I dropped our HBase jar into the plugin and reinstalled that. We you also able to test the query executor out for a similar scenario? 

Right now we're getting a NoClassDefFoundError:  "Could not initialize class org.apache.hadoop.hbase.client.HConnectionManager" when we try to retrieve data with one of the queries provided in the PDF (attached what I'm seeing). I assume this again has to do with the particular HBase snapshot that we're using. Is this error being generated because of a conflict between our HBase snapshot and the rest of the plugin?

Thanks again

[mdahlman]

mdonnelly,

Yes, it's possible that "this error being generated because of a conflict between our HBase snapshot and the rest of the plugin".

But my initial guess is that this is not the case. I'll explain.

When we build the iReport plugin one of the files that gets generated is "com-jaspersoft-ireport-hbase.xml". The contents of this file are as follows:

 

<?xml version="1.0" encoding="UTF-8"?>
<module codename="com.jaspersoft.ireport.hbase">
    <module_version install_time="1315515445613" last="true" origin="installer" specification_version="1.0">
        <file crc="2880130502" name="config/Modules/com-jaspersoft-ireport-hbase.xml"/>
        <file crc="1666859027" name="modules/com-jaspersoft-hbase-HBasePlugin.jar"/>
        <file crc="1590286038" name="modules/ext/com.jaspersoft.hadoop.hbase/HBaseDataSource-0.8.jar"/>
        <file crc="1705309943" name="modules/ext/com.jaspersoft.hbase/HBaseDeserializer-0.8.jar"/>
        <file crc="3844256829" name="modules/ext/commons-beanutils/commons-beanutils-1.8.0.jar"/>
        ...
        <file crc="4246883475" name="modules/ext/org.apache.hadoop/hadoop-core-0.20.2.jar"/>
        <file crc="1178926553" name="modules/ext/org.apache.hbase/hbase-0.90.3.jar"/>
        <file crc="973550240" name="update_tracking/com-jaspersoft-ireport-hbase.xml"/>
    </module_version>
</module>

 

As you can see, it performs a check on the .jars that are included. Therefore, when you replaced the original .jar file with your udpated snapshot--even if you were careful to use the same file name--iReport will not load that .jar file as part of the plugin.

The expected behavior in this case is that you would get a class not found error. And you do.

We did not get this same error... but I believe this was because we were lazy, er, I mean we performed a more efficient test. Rather than performing a complete uninstall and re-install of the plugin on an otherwise pristine iReport, we just got rid of the old hbase-0.90.3.jar and then added the newer one to iReport's classpath.

With that lengthy background established... here is what you should try:  

1. Delete, rename, or otherwise remove hbase-0.90.3.jar from the plugin directory in iReport.

2. Tools -> Options (or Preferences in Mac OS) and add your hbase 0.91-SNAPSHOT instance to iReport's classpath that way.

That worked for us, so it ought to work for you. Please let us know if it does.

Regards,
Matt

P.S. By the way, this complication is due to the way NetBeans handles plugins. Therefore this complication would not apply to JasperReports Server. If you just replace the old .jar with the new .jar (with any name) in the JasperReports Server directory WEB-INF/lib, then things should work immediately.

P.P.S. Remember that you have done this when someday in the future you upgrade the plugin. Otherwise you risk ending up with multiple HBase jars on your classpath.

Post Edited by mdahlman at 09/13/2011 17:57

[mdonnelly]

I know it’s been a couple weeks now, but thanks for your response and the help. We were able to get the hbase plugin working with iReport. The problem was a combination of the syncing of our version of the hbase snapshot with the hbase plugin as well as the manifest in the HBase plugin jar not being properly defined.

We are now working on creating reports using the HBase query language. We were wondering, however, how to specify filters within the iReport query. There don’t seem to be any available expression/filters for retrieving a subset of the table data within the actual query definition.

Is this type of functionality available in the current plugin? Would we have to define a separate/external query executor? If so, could you provide any resources (or the source code) for the definition of this HBase query language?

Thanks again