Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate js file from proto file with protoc v21.1 #127

Open
patoda opened this issue Jun 7, 2022 · 42 comments
Open

Generate js file from proto file with protoc v21.1 #127

patoda opened this issue Jun 7, 2022 · 42 comments
Assignees

Comments

@patoda
Copy link

patoda commented Jun 7, 2022

Command used -

protoc.exe --js_out=import_style=commonjs,binary:. employees.proto

Error -

'protoc-gen-js' is not recognized as an internal or external command,
operable program or batch file.
--js_out: protoc-gen-js: Plugin failed with status code 1.

@wfhartford
Copy link

I encountered the same issue upgrading to version 3.21.1 from 3.20.0

@nurhusni
Copy link

I'm having a same issue. Is there any solution yet?

@elig-salt
Copy link

Really looking forward to this fix.
Would love to help in any way if I can :)

@meling
Copy link

meling commented Jul 3, 2022

I'm having the same issue on macOS:

protoc-gen-js: program not found or is not executable
Please specify a program using absolute path or make sure the program is available in your PATH system variable
--js_out: protoc-gen-js: Plugin failed with status code 1.

I don't see protoc-gen-js in any of the binary downloads; couldn't figure out whether or not there is a separate install for this or if it should be "included" in the protoc binary as before.

I've seen that the release notes for v21.0-rc1 link to a document explaining that javascript has been moved out of the main repository to allow it to evolve independently of protoc. However, the new protobuf-javascript repository does not provide any instructions that helps me resolve this issue.

Note that the v20.1 release provides an additional js asset that I also wasn't able to make use of. Couldn't find any useful instructions.

I downgraded to v20.1 and then I don't get this problem.

PS: I'm new to installing protobuf from the official releases; I've used brew in the past, but it seems the homebrew folks are having some problems with v20.1 as well, with v3.19.4 being the most recent working version.

@Hjw52
Copy link

Hjw52 commented Jul 8, 2022

I also encountered the same issue when I use the v3.21.0. which verson can I use ? I want to generate js File

@ggat
Copy link

ggat commented Jul 18, 2022

I have the same on 3.21.2

@clehene
Copy link

clehene commented Jul 20, 2022

@haberman 21.x breaks JS and there's no known workaround for OSX / brew
Can you take a look and provide some guidance, please

no release notes on removing JS

no way to downgrade

brew install protobuf@20.1
==> Auto-updated Homebrew!
Updated 1 tap (homebrew/core).

Running `brew update --auto-update`...
Warning: No available formula with the name "protobuf@20.1". Did you mean protobuf@3.6?
 brew install protobuf@3.6
Error: protobuf@3.6 has been disabled because it is a versioned formula!

no releases on the js split

https://github.com/protocolbuffers/protobuf-javascript

no build instructions on the js split

https://github.com/protocolbuffers/protobuf-javascript

the latest known working js release has no build instructions either

protobuf-js-3.20.1.tar.gz

@clehene
Copy link

clehene commented Jul 20, 2022

workaround on OSX / brew:

brew install protobuf@3
brew link --overwrite protobuf@3

Related
Homebrew/homebrew-core#105712
Homebrew/homebrew-core#105812

@russellrc-keebo
Copy link

russellrc-keebo commented Jul 27, 2022

Installed protobuf 21.4 with homebrew, which uses protoc 3.21.4, and also got the same protoc-gen-js: program not found or is not executable error.

The suggestion by @clehene worked like a charm

@TheYoungBeast
Copy link

TheYoungBeast commented Jul 31, 2022

protobuf 21.4 Win x64 Compiler:

'protoc-gen-js' is not recognized as an internal or external command,
operable program or batch file.
--js_out: protoc-gen-js: Plugin failed with status code 1.

@johejo
Copy link

johejo commented Aug 3, 2022

My findings.

Clone protobu-javascript source code

git clone https://github.com/protocolbuffers/protobuf-javascript
cd protobuf-javascript/
bazel build //generator:protoc-gen-js # It might be better to use bazelisk as a launcher for bazel.

protoc-gen-js's executable is in bazel-bin/generator/

$ ls bazel-bin/generator/protoc-gen-js
bazel-bin/generator/protoc-gen-js

Pass the path to the plugin option in protoc

protoc --plugin=/path/to/protoc-gen-js

Someday there might be a formal migration guide from the protocolbuffers team.

@perezd perezd transferred this issue from protocolbuffers/protobuf Aug 3, 2022
@srabraham
Copy link

Thanks for that, @johejo. I'm trying to figure out how in the world to get this working in a Bazel context, in which we use a third party JS gRPC proto generator (rules-proto-grpc), and that thing calls into protoc, expecting protoc to know what to do to generate JS protos. Ugh..

farrcraft added a commit to farrcraft/brewtheory that referenced this issue Aug 12, 2022
Google went and broke JS support in v3.21. See: protocolbuffers/protobuf-javascript#127
@dibenede dibenede added packaging & distribution triaged Issue has been triaged labels Aug 12, 2022
@dibenede dibenede assigned dibenede and unassigned lukesandberg Aug 12, 2022
@dibenede
Copy link
Contributor

We are currently working on releasing binaries for protoc-gen-js, but it should be possible to build. You can find some better examples of the plugin invocation here: https://github.com/protocolbuffers/protobuf-javascript/blob/main/docs/index.md

@pablodz
Copy link

pablodz commented Jan 28, 2023

I was working trying to create protoc compilations for any version and multiple language

now, I can replicated the error in a isolated container

check https://github.com/pablodz/protoc-generator/tree/feat/error-protoc-node

run

#recommended
make loadenvs
make generate verbose=true

I hope this will help to fix errors easily

@Aaron1011
Copy link

If you're forced to downgrade and are running into issues with window in strict mode, you can use commonjs_strict: protocolbuffers/protobuf#9152 (comment)

@taliastocks
Copy link

This worked for me (after npm install google-protobuf):

Create an executable file protoc-gen-js.sh:

#!/usr/bin/env sh
cd "$(dirname "$0")"
node ./node_modules/google-protobuf/google-protobuf.js "$@"

And then when running protoc:

protoc --plugin=protoc-gen-js=./protoc-gen-js.sh --js_out=import_style=commonjs,binary:./gen-javascript --proto_path=protobuf/src path/to/my.proto

@taliastocks
Copy link

Okay never mind, that didn't work at all.

@Siedlerchr
Copy link

v3.22.0 still does not work:

protoc: stdout: . stderr: protoc-gen-js: program not found or is not executable
  Please specify a program using absolute path or make sure the program is available in your PATH system variable
  --js_out: protoc-gen-js: Plugin failed with status code 1.

@tinder-dthomson
Copy link

tinder-dthomson commented Feb 22, 2023

My findings.

Clone protobu-javascript source code

git clone https://github.com/protocolbuffers/protobuf-javascript
cd protobuf-javascript/
bazel build //generator:protoc-gen-js # It might be better to use bazelisk as a launcher for bazel.

protoc-gen-js's executable is in bazel-bin/generator/

$ ls bazel-bin/generator/protoc-gen-js
bazel-bin/generator/protoc-gen-js

Pass the path to the plugin option in protoc

protoc --plugin=/path/to/protoc-gen-js

Someday there might be a formal migration guide from the protocolbuffers team.

This worked for us, thanks @johejo! It's definitely a headache compared to how it used to "just work" though.

@yinzara
Copy link
Contributor

yinzara commented Mar 9, 2023

I have created a npm package to help everyone.
https://www.npmjs.com/package/protoc-gen-js

You can now npm i protoc-gen-js inside a package.json directory to make the '--js_out' work in "scripts" in the package.json (i.e. if one of your package.json scripts calls 'protoc --js_out') and will allow you to specify the version of protoc-gen-js in your package.json

You can also npm i -g protoc-gen-js or npm i -g protoc-gen-js@{version} and that will make '--js_out' work everywhere at the latest or specific version specified.

Right now the only version supported is '3.21.2' but I'll try to keep it up to date as new releases are made.

@pietrodn
Copy link

There is an official first-party solution: if you npm install grpc-tools, then you simply use grpc_tools_node_protoc (which is inside node_modules/.bin) instead of protoc. See documentation.

This way I managed to do without protoc-gen-js.

@tinder-dthomson
Copy link

tinder-dthomson commented Mar 23, 2023

@pietrodn I wouldn't consider that a solution. grpc-tools latest version is still only on protobuf version 3.19.1, which is before this issue even occurs.

@pietrodn
Copy link

Thanks for the info, I wrongly assumed that at least the official grpc-tools would be up to date. The more I use grpc, the more disappointed I am.

@adrianimboden
Copy link

I also build the protoc-gen-js myself now. For future reference, the command protoc --plugin=/path/to/protoc-gen-js is not correct.

You have to use protoc --plugin=protoc-gen-js=/path/to/protoc-gen-js when protoc-gen-js is not the path.

@princeje
Copy link

I also build the protoc-gen-js myself now. For future reference, the command protoc --plugin=/path/to/protoc-gen-js is not correct.

You have to use protoc --plugin=protoc-gen-js=/path/to/protoc-gen-js when protoc-gen-js is not the path.

Can you explain how you built protoc-gen-js? I'm trying to create a grpc client with Angular and javascript to use with my existing grpc server. I have .proto files but nothing is working to generate .js/.ts files.

@adrianimboden
Copy link

If you have an Angular/Javascript project, why does https://www.npmjs.com/package/protoc-gen-js from @yinzara not work for you?

If you want to build it for yourself, it depends on which platform you are. For Linux/WSL2 or OSX, it is quite easy. Here the example for a linux x86_64 system (written as a Dockerfile):

FROM debian:bullseye

# install prerequisites
#  git: to download the repositoryy
#  curl: to download/install bazel (the build system)
#  build-essential: to build protoc-gen-js (C/C++ compiler)
RUN \
  apt-get update && \
  apt-get install -y git curl build-essential

# install bazel
RUN \
  curl -sL https://github.com/bazelbuild/bazel/releases/download/6.1.2/bazel-6.1.2-linux-x86_64 > /usr/local/bin/bazel && \
  chmod +x /usr/local/bin/bazel

# build and install protoc-gen-js
RUN \
  git clone https://github.com/protocolbuffers/protobuf-javascript.git && \
  cd protobuf-javascript && \
  bazel build //generator:protoc-gen-js && \
  cp bazel-bin/generator/protoc-gen-js /usr/local/bin/protoc-gen-js

# protoc-gen-js is now installed in /usr/local/bin/protoc-gen-js

For windows, it should also work (bazel and stuff is also available), but that is a whole different story.

@lily-es
Copy link

lily-es commented May 10, 2023

The "official first party solution" is to simply download the latest release from this repo and add it to your PATH.
No need to build from source, no need to download any npm packages.

They should really update the README with these instructions and close this issue, as there is no real problem here. Not including the javascript runtime in the main protobuf installation was a conscious decision by the protobuf team, and now that there are proper pre-built binaries for this up for download, the only problem left is the lack of instructions really.

@Siedlerchr
Copy link

@lily-es Thanks very much! That hint helped me a lot.
I have created a download and unzip task for kotlin and configuration of the gradle plugin as well here:
https://gist.github.com/Siedlerchr/0fd9e463f6ffa2ea3c4b5e5ae3e5889e

@wolfgang-braun
Copy link

The "official first party solution" is to simply download the latest release from this repo and add it to your PATH. No need to build from source, no need to download any npm packages.

They should really update the README with these instructions and close this issue, as there is no real problem here. Not including the javascript runtime in the main protobuf installation was a conscious decision by the protobuf team, and now that there are proper pre-built binaries for this up for download, the only problem left is the lack of instructions really.

@lily-es This was actually what I tried first. But it doesn't work:

Using an alpine docker image I download the following releases from GitHub:

ENV PROTOC_VERSION=24.0
ENV PROTO_GEN_JS_VERSION=3.21.2
ENV PROTO_GEN_JS_GRPC_VERSION=1.4.2
  • ls /usr/local/bin/ returns protoc protoc-gen-grpc-web protoc-gen-js
  • echo $PATH returns ...:/usr/local/bin:...
  • which protoc returns /usr/local/bin/protoc
  • which protoc-gen-js returns /usr/local/bin/protoc-gen-js

(All bin files are executable of course)

And finally

  • protoc --js_out=js proto/test.proto returns:
/usr/local/bin/protoc-gen-js: program not found or is not executable
Please specify a program using absolute path or make sure the program is available in your PATH system variable
--js_out: protoc-gen-js: Plugin failed with status code 1.

Something's still wrong.

What's strange to me: executing protoc-gen-js directly returns not found

@tinder-dthomson
Copy link

The "official first party solution" is to simply download the latest release from this repo and add it to your PATH. No need to build from source, no need to download any npm packages.

They should really update the README with these instructions and close this issue, as there is no real problem here. Not including the javascript runtime in the main protobuf installation was a conscious decision by the protobuf team, and now that there are proper pre-built binaries for this up for download, the only problem left is the lack of instructions really.

Then you have to put in tooling to download the correct binary depending on OS. I'd rather just build from source and let bazel figure that out, that way we have one set of instructions common for all OS.

@wolfgang-braun
Copy link

wolfgang-braun commented Aug 17, 2023

Thanks for that, @johejo. I'm trying to figure out how in the world to get this working in a Bazel context, in which we use a third party JS gRPC proto generator (rules-proto-grpc), and that thing calls into protoc, expecting protoc to know what to do to generate JS protos. Ugh..

@tinder-dthomson

My tooling is a simple bash script that checks uname -m and selects the correct file based on that.

btw in my case I did use the correct download, and it didn't work:

Edit

Found the solution:

The binary doesn't support musl libc but glibc.
Alpine uses musl libc. For this reason the binary couldn't be executed.
See also #153

➡ Using a linux image with glibc (e.g. Debian image) solved the issue. The precompiled binaries work as expected.

@brunacat
Copy link

brunacat commented Aug 18, 2023

i'm having this problem. Using protoc version 24.0 and protoc-gen-grpc-web 1.4.2.
I also installed the protoc-gen-js package 3.21.2
is there any other package i need or version?

@funktionslust
Copy link

i'm having this problem. Using protoc version 24.0 and protoc-gen-grpc-web 1.4.2.

I also installed the protoc-gen-js package 3.21.2

is there any other package i need or version?

@brunacat nothing else is needed.

  • simply download the latest version of these three binaries for the correct platform / architecture
  • place them in a folder that is listed in your PATH variable without renaming the files
  • make sure all of them are executable (chmod +x)
  • when using Linux make sure to not to rely on musl libc
  • run protoc with the correct arguments

@ouzhou
Copy link

ouzhou commented Jan 8, 2024

download protoc and protobuf-javascript
image

run this
./protoc-25.1-osx-aarch_64/bin/protoc --plugin=./protobuf-javascript-3.21.2-osx-x86_64/bin/protoc-gen-js
--js_out=library=myproto_libs,binary:. a.proto

@Type1J
Copy link

Type1J commented Jan 17, 2024

Found the solution:

The binary doesn't support musl libc but glibc. Alpine uses musl libc. For this reason the binary couldn't be executed. See also #153

➡ Using a linux image with glibc (e.g. Debian image) solved the issue. The precompiled binaries work as expected.

I was using an Alpine (musl) based container, but switched to a Debian (glibc) based one. Thanks! Since musl is usually statically linked, the binary release would serve a wider audience if it used musl instead of glibc, which is usually dynamically linked.

@txthinking
Copy link

install protoc-gen-js via nami

nami install protoc-gen-js

@luangong
Copy link
Contributor

You can now install protoc-gen-js via Homebrew:

brew install protoc-gen-js

and then run protoc like this:

"$(brew --prefix protobuf@21)/bin/protoc" --js_out=import_style=commonjs,binary:. employees.proto

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests