Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

L4: Add JRuby Support #13

Closed
wants to merge 3 commits into from
Closed
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
77 changes: 77 additions & 0 deletions L4.md
@@ -0,0 +1,77 @@
Title
----
* Author(s): Jason Lunn <jason@jungroup.com>
* Approver: a11r
* Status: Draft
* Implemented in: Ruby
* Last updated: 2017-02-14
* Discussion at: https://github.com/grpc/grpc/issues/6705

## Abstract

Add first class support for the JRuby interpreter by publishing a variant of the gem targeted to the `java` platform.

## Background

Dependency management for Ruby packages ("gems") relies on the availability of variant builds published to
[RubyGems.org](https://rubygems.org/) that declare what platform they target. Today, the `grpc` gem is already
supporting [five variants](https://rubygems.org/gems/grpc/versions) for every released version: `x64-mingw32`,
`x86_64-linux`, `universal-darwin`, `x86-mingw32`, in addition to a variant with no explicit platform.

As detailed in [this issue](https://github.com/grpc/grpc/issues/6705), the [JRuby](http://jruby.org/) interpreter (which is built on top of
the JVM) is not able to correctly resolve dependencies on the `grpc` gem because it attempts and fails to build a native
extension that is implemented in C. While it possible on some versions of JRuby to make this approach work, it is much
more common (especially when a Java client library exists) to create a variant of the gem that declares as its target
the `java` platform and expose a JAR-based implementation through a Ruby wrapper. This is the approach taken in other
Ruby client libraries maintained by Google, e.g.
[Google Protobuf](https://github.com/google/protobuf/blob/master/ruby/google-protobuf.gemspec#L12).


### Related Proposals:
* Not Applicable

## Proposal

1. Use [JBundler](https://github.com/mkristian/jbundler) or a comparable tool to facilitate expressing a dependency on
the Maven artifact published by the `gprc-java` project, exposing the JAR file to the Ruby build process when running on
JRuby.
1. Extend the existing [Gemspec](https://github.com/grpc/grpc/blob/master/grpc.gemspec) to add conditional logic that
both includes the JAR file and any necessary wrapper code and excludes C implementation files when run on JRuby.
1. Amend the standard gem release process to include publishing from the latest JRuby interpreter available at release
time.

## Rationale

#### Using a native library vs creating a "pure" Ruby client
A "pure" Ruby client with no dependency on a native library would work equally well on JRuby as other Ruby interpreters.
It is assumed that the questions of whether the performance would suffer too greatly if it were not
implemented in C and the maintenance burden of a completely standalone Ruby implementation are settled matters that were
already discussed when the Ruby client was first implemented. Therefore this proposal does not suggest creating a "pure"
Ruby implementation to address JRuby support.

#### Creating a Maven dependency
The current organization of GRPC's source code repositories creates a challenge that other projects that support a Ruby
gem with dedicated support for clients running under JRuby do not have, namely that the Ruby source lives in a distinct
project from the source of the Java client. This proposal suggests using Maven to solve the artifact dependency in order
to avoid creating a requirement on a unified repository that includes the C, Ruby, and Java source in a single project.
That approach works fine (this is how Google Protobuf handles the same problem), and might even be more optimal, but
would require repository-level refactoring. The `grpc-java` client already exposes releases via Maven so there would be
no work required on their part to support this proposal.

## Implementation

1. Update the Gemspec to add JBundler to the list of development dependencies when run on JRuby
1. Create a [JarFile](https://github.com/torquebox/maven-tools/wiki/Jarfile) to create a dependency on the grpc-java jar
1. Create wrapper code that delegates the existing Ruby API calls to the Java client, as needed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A heads up about this, I would expect this shim layer over the java client to end up as about a full grpc-ruby reimplementation, with the requirement that it always present the same surface as the C-wrapping grpc-ruby gem. The majority of the current grpc-ruby code base is probably not immediately reusable - the pure ruby part is thin and a lot of it is tied to the C-extension ruby API, of which the semantics revolve around the grpc C-core library interface. Also the grpc-java libraries present slightly higher level and slightly different interfaces than the grpc C-core library that the ruby C-extensions wrap. For example, one trouble point is "channel args". Currently, users can pass a hash to grpc-ruby Channel constructors, of which the key-value pairs are passed straight down to the grpc C-core library - emulating this on top of grpc-java client, whose implementation does need to support these in the same way, would probably be difficult if doable at all.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the assumption correct that there was and continues to be either a performance reason and / or an ongoing maintenance reason not to build a pure Ruby implementation? If that assumption holds, do we agree that solutions that build on top of either the C client or the Java client are the only viable ways forward?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, but just making clear that the grpc-java to grpc-ruby shim would be involved, with API copies probably not feasible. The other option I can see is a JNI wrapper of the C-core library, which would not be small task either but could be useful for other languages outside jruby..

1. Address any test regressions or expectation updates introduced by executing on JRuby
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably wouldn't be able to drop the jruby target into the current tests without changes to the tests themselves. This is probably doable but somewhat involved.

The testing suite would have to be updated to add "jruby", as basically a separate language, to the continuously tested language-platform matrix. Also I suspect a lot of the current grpc-ruby tests would have to be refactored to be agnostic of the C-wrapping and java-wrapping implementations (higher level tests should work find but there's lower level stuff in C-extensions is used in tests).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it fair to say that the high level tests are all that we care about in the context of JRuby because low-level tests will be exercised by unit tests with grpc-java, and that the rest could be marked to be skipped when running on JRuby?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the most part, yeah I think so.

1. Address any assumptions in documentation or code about the set of supported platforms, e.g. in the README for
[`grpc-tools`](https://github.com/grpc/grpc/blob/master/src/ruby/tools/README.md)
1. Update the release process tooling / and or documentation include JRuby specifics, as needed

All points above except the last one can be community contributed. Project members would have to be responsible for the
last point as it is not exposed in the repository itself.

## Open issues

1. Is there a more correct heuristic for picking the version of the Maven artifact than keeping it unpinned so that it
selects the latest version available?