Question

I m试图将Hadoop、Pig和Casandra合并起来,以便能够通过简单的Pig查询,就Casses储存的数据开展工作。问题在于,我可以找Pig来创造实际上与Cassestorage合作的地图/绘画工作。

我用的是把储存-没收.xml文件从我的一个集束机器上编成在反ib/pig(Cassess的分离)上,然后将 st子编成木andra。

接下来,我修改了实例说明。

register /opt/pig/pig-0.7.0-core.jar;
register /tmp/apache-cassandra-0.6.3-src/lib/libthrift-r917130.jar;
REGISTER /tmp/apache-cassandra-0.6.3-src/contrib/pig/build/cassandra_loadfunc.jar;
rows = LOAD  cassandra://Keyspace1/Standard1  USING org.apache.cassandra.hadoop.pig.CassandraStorage();
cols = FOREACH rows GENERATE flatten($1);
colnames = FOREACH cols GENERATE $0;
namegroups = GROUP colnames BY $0;
namecounts = FOREACH namegroups GENERATE COUNT($1), group;
orderednames = ORDER namecounts BY $0;
topnames = LIMIT orderednames 50;
dump topnames;

So if I m not mistaken the jars should be bundled into the job that is submitted to hadoop. But when running the job it just throws an exception at me:

2010-08-04 22:11:46,395 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2117: Unexpected error when launching map reduce job.
2010-08-04 22:11:46,395 [main] ERROR org.apache.pig.tools.grunt.Grunt - org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias topnames
    at org.apache.pig.PigServer.openIterator(PigServer.java:521)
    at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:544)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:241)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:162)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:138)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
    at org.apache.pig.Main.main(Main.java:391)
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable to store alias topnames
    at org.apache.pig.PigServer.store(PigServer.java:577)
    at org.apache.pig.PigServer.openIterator(PigServer.java:504)
    ... 6 more
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2117: Unexpected error when launching map reduce job.
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:209)
    at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:308)
    at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:835)
    at org.apache.pig.PigServer.store(PigServer.java:569)
    ... 7 more
Caused by: java.lang.RuntimeException: Could not resolve error that occured when launching map reduce job: java.lang.NoClassDefFoundError: org/apache/thrift/TBase
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$JobControlThreadExceptionHandler.uncaughtException(MapReduceLauncher.java:510)
    at java.lang.Thread.dispatchUncaughtException(Thread.java:1845)

我不理解什么,因为th图书馆被明确列入名单,应该捆绑,是没有的?

Answer 1

例外情况清楚表明,它无法找到TBase语。

java.lang.NoClassDefFoundError:org/apache/thrift/TBase

挖掘碎.,如果rift裂平衡jar确实出现在正确的地点,则进行检查。裂缝可能在不同的地点被捆绑。

你们还可以尝试把jar放在捆绑的jar子里。另一种选择是,在明确划分等级时添加jar。

友情链接