Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ZEPPELIN-6163] HBase interpreter supports hbase-2.x #4908

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[ZEPPELIN-6163] HBase interpreter supports hbase-2.x #4908

wants to merge 1 commit into from
+109 −172

Conversation

paul8263
Copy link
Contributor

@paul8263 paul8263 commented Feb 12, 2025 •
edited
Loading

What is this PR for?

Currently in Zeppelin HBase interpreter does not support HBase 2.x ruby syntax.

This patch added hbase-2.x support.

This closes ZEPPELIN-6163

What type of PR is it?

Improvement

Todos

What is the Jira issue?

ZEPPELIN-6163

How should this be tested?

  • Strongly recommended: add automated unit tests for any new or changed behavior
  • Outline any manual steps to test the PR here.

Compiled and tested manually.

Screenshots (if appropriate)

Questions:

  • Does the license files need to update? No
  • Is there breaking changes for older versions? No
  • Does this needs documentation? No

pan3793
pan3793 reviewed Feb 12, 2025
View reviewed changes
hbase/pom.xml Outdated
@@ -61,5 +61,13 @@
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>hbase-2.x</id>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HBase 1.x is EOL, see HBASE-27286, I think we don't need a profile, just adapt to HBase 2.x

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @pan3793 ,

Great. I will simply upgrade it.

pan3793
pan3793 reviewed Feb 12, 2025
View reviewed changes
hbase/pom.xml Outdated
@@ -33,7 +33,7 @@
<properties>
<!--library versions-->
<interpreter.name>hbase</interpreter.name>
<jruby.version>1.6.8</jruby.version>
<jruby.version>9.1.17.0</jruby.version>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May I ask why 9.1.17.0 is chosen? I see HBase 2.6.1(the latest version of HBase 2.x) pulls JRuby 9.3.13.0, does it work for lower HBase version?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As HBASE-27585 states, higher version of JRuby drops support for Ruby 2.5, which might have side effect for lower version of HBase 2.x

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please leave some brief comments here?

Copy link
Contributor Author

@paul8263 paul8263 Feb 12, 2025 •
edited by pan3793
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, I tested hbase interperter compiled with JRuby 9.3.13.0, connecting early version HBase i.e HBase 2.0.0. It required additional dependencies named jline. However interperter compiled with JRuby 9.1.17.0 worked like a charm. I guess higher version of jruby may require addition dependencies that lower version of HBase does not provide.

Exceptions are:

org.apache.zeppelin.interpreter.InterpreterException: org.jruby.embed.EvalFailedException: (LoadError) load error: irb/completion -- java.lang.NoSuchMethodError: jline.console.completer.CandidateListCompletionHandler.setPrintSpaceAfterFullCompletion(Z)V
	at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:76)
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:861)
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:769)
	at org.apache.zeppelin.scheduler.Job.run(Job.java:186)
	at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:135)
	at org.apache.zeppelin.scheduler.FIFOScheduler.lambda$runJobInScheduler$0(FIFOScheduler.java:42)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)

Copy link
Contributor Author

@paul8263 paul8263 Feb 12, 2025 •
edited by pan3793
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After solving jline dependency issue, finally I got this exception:

org.apache.zeppelin.interpreter.InterpreterException: org.jruby.embed.EvalFailedException: (NoMethodError) undefined method `runtime' for JRuby:Module
	at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:76)
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:861)
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:769)
	at org.apache.zeppelin.scheduler.Job.run(Job.java:186)
	at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:135)
	at org.apache.zeppelin.scheduler.FIFOScheduler.lambda$runJobInScheduler$0(FIFOScheduler.java:42)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)
Caused by: org.jruby.embed.EvalFailedException: (NoMethodError) undefined method `runtime' for JRuby:Module
	at org.jruby.embed.internal.EmbedEvalUnitImpl.run(EmbedEvalUnitImpl.java:134)
	at org.jruby.embed.ScriptingContainer.runUnit(ScriptingContainer.java:1296)
	at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:1327)
	at org.apache.zeppelin.hbase.HbaseInterpreter.open(HbaseInterpreter.java:97)
	at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70)
	... 8 more
Caused by: org.jruby.exceptions.NoMethodError: (NoMethodError) undefined method `runtime' for JRuby:Module
	at RUBY.`(uri:classloader:/jruby/kernel/jruby/process_manager.rb:10)
	at RUBY.`(uri:classloader:/jruby/kernel/jruby/process_manager.rb:46)
	at RUBY.initialize(/usr/hdp/3.0.1.0-187/hbase/lib/ruby/irb/hirb.rb:46)
	at org.jruby.RubyClass.new(org/jruby/RubyClass.java:883)
	at RUBY.start(hirb.rb:181)
	at RUBY.<main>(hirb.rb:193)

It seems there are some compatibility issue.

Copy link
Contributor Author

@paul8263 paul8263 Feb 13, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @pan3793 ,
Thank you for your reply.
I got a breakthough. jruby-7221 says 9.3.0.0 need to require "jruby" to avoid undefined method 'runtime' for JRuby:Module.
I tested hbase interpreter with JRuby 9.2.13.0 and it worked fine for HBase 2.0.0.
Once I upgraded JRuby to 9.3.13.0, the same exception occured. But it could be fixed by addingrequire 'jruby' at the beginning of $HBASE_HOME/lib/ruby/irb/hirb.rb (bin/hirb.rb also works) for HBase 2.0.0.
In conclusion, personally I prefer using JRuby 9.2.13.0. Correct me if I was wrong.

Copy link
Member

@pan3793 pan3793 Feb 13, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does JRuby 9.2.13.0 also work for newly HBase versions, e.g. 2.6? we should choose a version that supports broad HBase versions.

Copy link
Contributor Author

@paul8263 paul8263 Feb 13, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @pan3793 ,

I tested the latest Zeppelin 0.12.0 with HBase 2.0.0 and jruby 9.2.13.0. Got the exception like(executing 'list' and 'scan' respectively):

list

TABLE                                                                                                                                                             

ERROR: undefined method `getAdmin' for nil:NilClass

List all user tables in hbase. Optional regular expression parameter could
be used to filter the output. Examples:

  hbase> list
  hbase> list 'abc.*'
  hbase> list 'ns:abc.*'
  hbase> list 'ns:.*'

Took 0.0286 seconds

And:

scan


ERROR: undefined method `getTable' for nil:NilClass
Did you mean?  get_table

Scan a table; pass table name and optionally a dictionary of scanner
specifications.  Scanner specifications may include one or more of:
TIMERANGE, FILTER, LIMIT, STARTROW, STOPROW, ROWPREFIXFILTER, TIMESTAMP,
MAXLENGTH or COLUMNS, CACHE or RAW, VERSIONS, ALL_METRICS or METRICS

Zeppelin 0.11.2 works fine.

It might indicate that some of the methods are not reachable. I investigated further by comparing the output '$LOADED_FEATURES.each { |str| puts str }' in 0.11.2 and 0.12.0. Surprisingly they are the same.

The output is:

java.rb
jruby/util.rb
enumerator.rb
rational.rb
complex.rb
thread.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/unicode_normalize.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rbconfig.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/compatibility.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/defaults.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/deprecate.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/errors.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/exceptions.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/basic_specification.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/stub_specification.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/text.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/user_interaction.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification_policy.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/util/list.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/platform.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/version.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/requirement.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/specification.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jars/post_install_hook.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jar_install_post_install_hook.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/defaults/jruby.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/util.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/dependency.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_gem.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/monitor.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_warn.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/defaults/operating_system.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/path_support.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/version.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/core_ext/name_error.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/levenshtein.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/jaro_winkler.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checker.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/delegate.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/name_error_checkers/class_name_checker.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/name_error_checkers/variable_name_checker.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/name_error_checkers.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/method_name_checker.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/key_error_checker.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/null_checker.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean/formatters/plain_formatter.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared/gems/did_you_mean-1.3.0/lib/did_you_mean.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/readline/version.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/readline.jar
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/readline.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/completion.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/pathname.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jruby.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/admin.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/table.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/taskmonitor.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/quotas.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/security.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/visibility_labels.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/hbase.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/replication_admin.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase/rsgroup_admin.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/hbase_constants.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/stringio.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/formatter.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/status.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/version.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/table_help.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/whoami.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/processlist.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/alter.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/create.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/describe.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/disable.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/disable_all.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/is_disabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/drop.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/drop_all.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/enable.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/enable_all.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/is_enabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/exists.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/show_filters.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/alter_status.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/alter_async.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_table.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/locate_region.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_regions.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/create_namespace.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/drop_namespace.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/alter_namespace.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/describe_namespace.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_namespace.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_namespace_tables.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/count.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/delete.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/deleteall.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_counter.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/incr.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/put.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/scan.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/truncate.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/truncate_preserve.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/append.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_splits.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/assign.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/balancer.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/balance_switch.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/balancer_enabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/normalize.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/normalizer_switch.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/normalizer_enabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/is_in_maintenance_mode.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/close_region.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/compact.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/flush.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/major_compact.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/move.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/split.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/merge_region.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/unassign.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/zk_dump.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/wal_roll.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/catalogjanitor_run.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/catalogjanitor_switch.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/catalogjanitor_enabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/cleaner_chore_run.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/cleaner_chore_switch.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/cleaner_chore_enabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/compact_rs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/compaction_state.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/trace.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/splitormerge_switch.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/splitormerge_enabled.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/clear_compaction_queues.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_deadservers.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/clear_deadservers.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/clear_block_cache.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/add_peer.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/remove_peer.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_peers.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/enable_peer.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/disable_peer.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_peer_replicate_all.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_peer_namespaces.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/append_peer_namespaces.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/remove_peer_namespaces.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_peer_exclude_namespaces.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/show_peer_tableCFs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_peer_tableCFs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_peer_exclude_tableCFs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_peer_bandwidth.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_replicated_tables.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/append_peer_tableCFs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/remove_peer_tableCFs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/enable_table_replication.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/disable_table_replication.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_peer_config.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_peer_configs.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/update_peer_config.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/snapshot.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/clone_snapshot.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/restore_snapshot.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/delete_snapshot.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/delete_all_snapshot.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/delete_table_snapshots.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/date/format.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/date.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/time.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_snapshots.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_table_snapshots.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/update_config.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/update_all_config.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_quota.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_quotas.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_quota_table_sizes.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_quota_snapshots.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_snapshot_sizes.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_security_capabilities.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/grant.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/revoke.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/user_permission.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/abort_procedure.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/version.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/ostruct.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/generic_object.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/common.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/ext/parser.jar
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/ext/generator.jar
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/ext.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_procedures.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_locks.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/add_labels.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_labels.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_auths.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_auths.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/clear_auths.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/set_visibility.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/list_rsgroups.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/add_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/remove_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/balance_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/move_servers_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/move_tables_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/move_namespaces_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/move_servers_tables_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/move_servers_namespaces_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_server_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/get_table_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell/commands/remove_servers_rsgroup.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/shell.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/e2mmap.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/init.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/workspace.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/inspector.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/src_encoding.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/magic-file.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/output-method.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/extend-command.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/notifier.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/slex.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-token.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/locale.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/version.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb
/usr/hdp/3.0.1.0-187/hbase/lib/ruby/irb/hirb.rb
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/bundler_version_finder.rb
uri:classloader:/jruby/path_helper.rb

I still have not figured out what is going wrong yet.

One more thing is that the entrypoints of HBase shell versioned 2.0.0 and 2.4.7 are different. HBase 2.0.0 is the ruby script located at bin/hirb.rb, while in 2.4.7 all ruby scripts are packaged in hbase-shell.jar. It adapted to a standardalone jar. See https://github.com/jruby/jruby/wiki/StandaloneJarsAndClasses#standalone-executable-jar-files . To support newer HBase, the HbaseInterpreter class needs some updates.

Copy link
Contributor Author

@paul8263 paul8263 Feb 14, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @pan3793

Zeppelin 0.12.0 could now work with HBase 2.0.0 after I restored hbase/pom.xml of Zeppelin 0.12.0 to 0.11.2 and upgraded jruby to 9.2.13.0.

In Zeppelin 0.12.0 HBase Interpreter, there are some dependencies short.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @pan3793

Finally I decided to dump jruby and use remote execution of "hbase shell" with scripts in the notebook written in a temporary txt file. Now both HBase 2.0.0 and HBase 2.4.7 works fine. Other versions of HBase should work as well.

Now the unit test requires a HBase environment. To bypass the CI I removed some of the tests. Is there any better solutions?

@paul8263 paul8263 changed the title [ZEPPELIN-6163] Add a profile hbase-2.x that bumps jruby to 9.1.17.0 [ZEPPELIN-6163] Add a profile hbase-2.x that bumps jruby to 9.2.13.0 Feb 13, 2025
@paul8263 paul8263 changed the title [ZEPPELIN-6163] Add a profile hbase-2.x that bumps jruby to 9.2.13.0 [ZEPPELIN-6163] Bumps jruby to 9.2.13.0 to support hbase 2.x Feb 13, 2025
@paul8263 paul8263 requested a review from pan3793 February 13, 2025 03:17
@paul8263 paul8263 changed the title [ZEPPELIN-6163] Bumps jruby to 9.2.13.0 to support hbase 2.x [ZEPPELIN-6163] HBase interpreter supports hbase-2.x Feb 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jongyoul jongyoul jongyoul left review comments

@pan3793 pan3793 Awaiting requested review from pan3793

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@paul8263 @jongyoul @pan3793