Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Protobuf cmake modules do not appear to be installed for Linux #68

Closed
photex opened this issue Sep 1, 2020 · 12 comments · Fixed by #144
Closed

Protobuf cmake modules do not appear to be installed for Linux #68

photex opened this issue Sep 1, 2020 · 12 comments · Fixed by #144

Comments

@photex
Copy link

photex commented Sep 1, 2020

Issue:

If I install the libprotobuf package under windows I see the expected cmake modules installed to %CONDA_PREFIX%/Library/cmake. But under Linux I'm not able to find any protobuf related modules. The result is that this package cannot be used for cmake builds on Linux.

Files found in Linux:

(kpnsdev) photex@hagrid:/mnt/c/projects/kopernikus/depot$ fd --glob '*.cmake' /home/photex/miniconda3/envs | rg protobuf
(kpnsdev) photex@hagrid:/mnt/c/projects/kopernikus/depot$ fd --glob '*.cmake' /home/photex/miniconda3/envs | rg Proto
/home/photex/miniconda3/envs/kpnsdev/share/cmake/ignition-cmake2/cmake2/FindIgnProtobuf.cmake
/home/photex/miniconda3/envs/kpnsdev/share/cmake-3.18/Modules/CheckPrototypeDefinition.cmake
/home/photex/miniconda3/envs/kpnsdev/share/cmake-3.18/Modules/FindProtobuf.cmake

Files found in Windows:

(kpnsdev) c:\projects\kopernikus\depot>fd .cmake C:\Users\phote\scoop\apps\miniconda3\current\envs\kpnsdev | rg Proto
C:\Users\phote\scoop\apps\miniconda3\current\envs\kpnsdev\Library\share\cmake-3.18\Modules\CheckPrototypeDefinition.cmake
C:\Users\phote\scoop\apps\miniconda3\current\envs\kpnsdev\Library\share\cmake-3.18\Modules\FindProtobuf.cmake
C:\Users\phote\scoop\apps\miniconda3\current\envs\kpnsdev\Library\share\cmake\ignition-cmake2\cmake2\FindIgnProtobuf.cmake

(kpnsdev) c:\projects\kopernikus\depot>fd .cmake C:\Users\phote\scoop\apps\miniconda3\current\envs\kpnsdev | rg proto
C:\Users\phote\scoop\apps\miniconda3\current\envs\kpnsdev\Library\CMake\protobuf-config-version.cmake
C:\Users\phote\scoop\apps\miniconda3\current\envs\kpnsdev\Library\CMake\protobuf-config.cmake
C:\Users\phote\scoop\apps\miniconda3\current\envs\kpnsdev\Library\CMake\protobuf-module.cmake
C:\Users\phote\scoop\apps\miniconda3\current\envs\kpnsdev\Library\CMake\protobuf-options.cmake
C:\Users\phote\scoop\apps\miniconda3\current\envs\kpnsdev\Library\CMake\protobuf-targets-release.cmake
C:\Users\phote\scoop\apps\miniconda3\current\envs\kpnsdev\Library\CMake\protobuf-targets.cmake

Environment (conda list): 
(kpnsdev) photex@hagrid:/mnt/c/projects/kopernikus/depot$ conda list
# packages in environment at /home/photex/miniconda3/envs/kpnsdev:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
asio                      1.14.1               he1b5a44_0    conda-forge
binutils                  2.34                 h2122c62_9    conda-forge
binutils_impl_linux-64    2.34                 h53a641e_7    conda-forge
binutils_linux-64         2.34                hc952b39_20    conda-forge
boost-cpp                 1.72.0               h7b93d67_2    conda-forge
bzip2                     1.0.8                h516909a_3    conda-forge
c-ares                    1.16.1               h516909a_3    conda-forge
c-compiler                1.1.1                h516909a_0    conda-forge
ca-certificates           2020.6.20            hecda079_0    conda-forge
cairo                     1.16.0            h3fc0475_1005    conda-forge
certifi                   2020.6.20        py38h32f6830_0    conda-forge
clang                     10.0.1          default_hde54327_1    conda-forge
clang-tools               10.0.1          default_hde54327_1    conda-forge
clangxx                   10.0.1          default_hde54327_1    conda-forge
cmake                     3.18.2               h5c55442_0    conda-forge
console_bridge            1.0.1                hc9558a2_0    conda-forge
cppzmq                    4.6.0                hc9558a2_0    conda-forge
cudatoolkit               10.2.89              hfd86e86_1
cudnn                     7.6.5                cuda10.2_0
curl                      7.71.1               he644dc0_5    conda-forge
cxx-compiler              1.1.1                hc9558a2_0    conda-forge
dbus                      1.13.6               he372182_0    conda-forge
eigen                     3.3.7             hc9558a2_1001    conda-forge
expat                     2.2.9                he1b5a44_2    conda-forge
ffmpeg                    4.3.1                h167e202_0    conda-forge
fontconfig                2.13.1            h1056068_1002    conda-forge
freeimage                 3.18.0               h2f8745a_3    conda-forge
freetype                  2.10.2               he06d7ca_0    conda-forge
gazebo                    11.0.0               h398b6f7_9    conda-forge
gcc_impl_linux-64         7.5.0                hd420e75_6    conda-forge
gcc_linux-64              7.5.0               h09487f9_20    conda-forge
gettext                   0.19.8.1          hc5be6a0_1002    conda-forge
git                       2.28.0          pl526h5e3e691_1    conda-forge
glib                      2.65.0               h6f030ca_0    conda-forge
gmp                       6.2.0                he1b5a44_2    conda-forge
gnutls                    3.6.13               h79a8f9a_0    conda-forge
graphite2                 1.3.13            he1b5a44_1001    conda-forge
gst-plugins-base          1.14.5               h0935bb2_2    conda-forge
gstreamer                 1.14.5               h36ae1b5_2    conda-forge
gts                       0.7.6                h08bb679_0    conda-forge
gxx_impl_linux-64         7.5.0                hdf63c60_6    conda-forge
gxx_linux-64              7.5.0               h09487f9_20    conda-forge
harfbuzz                  2.7.2                hee91db6_0    conda-forge
hdf5                      1.10.6          nompi_h3c11f04_101    conda-forge
icu                       67.1                 he1b5a44_0    conda-forge
ilmbase                   2.5.2                h8b12597_0    conda-forge
jasper                    1.900.1           h07fcdf6_1006    conda-forge
jpeg                      9d                   h516909a_0    conda-forge
jsoncpp                   1.8.4             hc9558a2_1002    conda-forge
jxrlib                    1.1                  h516909a_2    conda-forge
krb5                      1.17.1               hfafb76e_2    conda-forge
lame                      3.100             h14c3975_1001    conda-forge
lcms2                     2.9                  hbd6801e_2    conda-forge
ld_impl_linux-64          2.34                 h53a641e_7    conda-forge
libblas                   3.8.0               17_openblas    conda-forge
libcblas                  3.8.0               17_openblas    conda-forge
libccd                    2.1                  he1b5a44_1    conda-forge
libclang                  10.0.1          default_hde54327_1    conda-forge
libclang-cpp10            10.0.1          default_hde54327_1    conda-forge
libcurl                   7.71.1               hcdd3856_5    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_0    conda-forge
libevent                  2.1.10               hcdb4288_2    conda-forge
libffi                    3.2.1             he1b5a44_1007    conda-forge
libgcc-ng                 9.3.0               h24d8f2e_16    conda-forge
libgfortran-ng            7.5.0               hdf63c60_16    conda-forge
libgit2                   1.0.0                h1e06722_0    conda-forge
libglu                    9.0.0             he1b5a44_1001    conda-forge
libgomp                   9.3.0               h24d8f2e_16    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
libignition-cmake2        2.1.1                he1b5a44_2    conda-forge
libignition-common3       3.5.0                hf10cc52_1    conda-forge
libignition-fuel-tools4   4.1.0                hc81f71f_1    conda-forge
libignition-math6         6.4.0                he1b5a44_1    conda-forge
libignition-msgs5         5.1.0                hab906e7_4    conda-forge
libignition-transport8    8.0.0                h85f8c2c_1    conda-forge
liblapack                 3.8.0               17_openblas    conda-forge
liblapacke                3.8.0               17_openblas    conda-forge
libllvm10                 10.0.1               he513fc3_3    conda-forge
libnghttp2                1.41.0               h8cfc5f6_2    conda-forge
libode                    0.16                 he1b5a44_0    conda-forge
libopenblas               0.3.10          pthreads_hb3c22a3_4    conda-forge
libopencv                 4.4.0                    py38_2    conda-forge
libpng                    1.6.37               hed695b0_2    conda-forge
libpq                     12.3                 h5513abc_0    conda-forge
libprotobuf               3.12.4               h8b12597_0    conda-forge
libraw                    0.19.5               hefcb256_2    conda-forge
libsdformat               9.2.0                hbca141e_1    conda-forge
libsodium                 1.0.18               h516909a_0    conda-forge
libssh2                   1.9.0                hab1572f_5    conda-forge
libssh2-static            1.9.0                h8dcbfe2_5    conda-forge
libstdcxx-ng              9.3.0               hdf63c60_16    conda-forge
libtar                    1.2.20            h14c3975_1001    conda-forge
libtiff                   4.1.0                hc7e4089_6    conda-forge
libuuid                   2.32.1            h14c3975_1000    conda-forge
libuv                     1.39.0               h516909a_0    conda-forge
libwebp-base              1.1.0                h516909a_3    conda-forge
libxcb                    1.13              h14c3975_1002    conda-forge
libxkbcommon              0.10.0               he1b5a44_0    conda-forge
libxml2                   2.9.10               h68273f3_2    conda-forge
libzip                    1.5.1             h1a52d62_1005    conda-forge
lz4-c                     1.9.2                he1b5a44_3    conda-forge
msgpack-c                 3.3.0                hac5445f_1    conda-forge
mysql-common              8.0.21                        0    conda-forge
mysql-libs                8.0.21               hf3661c5_0    conda-forge
ncurses                   6.2                  he1b5a44_1    conda-forge
nettle                    3.4.1             h1bed415_1002    conda-forge
ninja                     1.10.1               hc9558a2_1    conda-forge
nspr                      4.28                 he1b5a44_0    conda-forge
nss                       3.56                 he751ad9_0    conda-forge
numpy                     1.19.1           py38hbc27379_2    conda-forge
ogre                      1.10.12              hc56bac0_2    conda-forge
ompl                      1.4.2                h68a2fdc_2    conda-forge
opencv                    4.4.0                    py38_2    conda-forge
openexr                   2.5.2                he513fc3_0    conda-forge
openh264                  2.1.1                h8b12597_0    conda-forge
openjpeg                  2.3.1                h981e76c_3    conda-forge
openssl                   1.1.1g               h516909a_1    conda-forge
pcre                      8.44                 he1b5a44_0    conda-forge
perl                      5.26.2            h516909a_1006    conda-forge
pip                       20.2.2                     py_0    conda-forge
pixman                    0.38.0            h516909a_1003    conda-forge
protobuf                  3.12.4           py38h950e882_0    conda-forge
pthread-stubs             0.4               h14c3975_1001    conda-forge
py-opencv                 4.4.0            py38h23f93f0_2    conda-forge
python                    3.8.5           h1103e12_7_cpython    conda-forge
python_abi                3.8                      1_cp38    conda-forge
qt                        5.12.6               h0c8506f_0    conda-forge
qwt                       6.1.5                h73816c6_0    conda-forge
readline                  8.0                  he28a2e2_2    conda-forge
rhash                     1.3.6             h14c3975_1001    conda-forge
rust                      1.46.0               h516909a_0    conda-forge
setuptools                50.0.0           py38h32f6830_0    conda-forge
six                       1.15.0             pyh9f0ad1d_0    conda-forge
spdlog                    1.7.0                hc9558a2_2    conda-forge
sqlite                    3.33.0               h4cf870e_0    conda-forge
tbb                       2020.1               hc9558a2_0    conda-forge
tbb-devel                 2020.1               hc9558a2_0    conda-forge
tinyxml                   2.6.2                hc9558a2_1    conda-forge
tinyxml2                  8.0.0                he1b5a44_1    conda-forge
tk                        8.6.10               hed695b0_0    conda-forge
urdfdom                   2.3.2                hc9558a2_0    conda-forge
urdfdom_headers           1.0.5                hc9558a2_1    conda-forge
wheel                     0.35.1             pyh9f0ad1d_0    conda-forge
x264                      1!152.20180806       h14c3975_0    conda-forge
xorg-fixesproto           5.0               h14c3975_1002    conda-forge
xorg-kbproto              1.0.7             h14c3975_1002    conda-forge
xorg-libice               1.0.10               h516909a_0    conda-forge
xorg-libsm                1.2.3             h84519dc_1000    conda-forge
xorg-libx11               1.6.12               h516909a_0    conda-forge
xorg-libxau               1.0.9                h14c3975_0    conda-forge
xorg-libxaw               1.0.13            h14c3975_1002    conda-forge
xorg-libxdmcp             1.1.3                h516909a_0    conda-forge
xorg-libxext              1.3.4                h516909a_0    conda-forge
xorg-libxfixes            5.0.3             h516909a_1004    conda-forge
xorg-libxmu               1.1.3                h516909a_0    conda-forge
xorg-libxpm               3.5.13               h516909a_0    conda-forge
xorg-libxrender           0.9.10            h516909a_1002    conda-forge
xorg-libxt                1.1.5             h516909a_1003    conda-forge
xorg-renderproto          0.11.1            h14c3975_1002    conda-forge
xorg-xextproto            7.3.0             h14c3975_1002    conda-forge
xorg-xproto               7.0.31            h14c3975_1007    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
yaml                      0.2.5                h516909a_0    conda-forge
zeromq                    4.3.2                he1b5a44_3    conda-forge
zlib                      1.2.11            h516909a_1009    conda-forge
zstd                      1.4.5                h6597ccf_2    conda-forge

Details about conda and system ( conda info ): 
(kpnsdev) photex@hagrid:/mnt/c/projects/kopernikus/depot$ conda info

     active environment : kpnsdev
    active env location : /home/photex/miniconda3/envs/kpnsdev
            shell level : 2
       user config file : /home/photex/.condarc
 populated config files : /home/photex/.condarc
          conda version : 4.8.4
    conda-build version : not installed
         python version : 3.8.5.final.0
       virtual packages : __glibc=2.27
       base environment : /home/photex/miniconda3  (writable)
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/photex/miniconda3/pkgs
                          /home/photex/.conda/pkgs
       envs directories : /home/photex/miniconda3/envs
                          /home/photex/.conda/envs
               platform : linux-64
             user-agent : conda/4.8.4 requests/2.24.0 CPython/3.8.5 Linux/4.4.0-18362-Microsoft ubuntu/18.04.4 glibc/2.27
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False
@xhochy xhochy transferred this issue from conda-forge/protobuf-feedstock Sep 7, 2020
@xhochy
Copy link
Member

xhochy commented Sep 7, 2020

Yes, this is because we build using autotools and not using CMake.

@photex
Copy link
Author

photex commented Sep 7, 2020

So does this mean, CMake users are not intended to be supported on Linux then?

@xhochy
Copy link
Member

xhochy commented Sep 7, 2020

No, it means that someone needs to invest time to get this building.

@jakirkham
Copy link
Member

Or someone needs to raise this issue with upstream where it originates

@ghamarian
Copy link

Is there a reason for that? in windows it is cmake, the build won't succeed with cmake on osx and Linux?

@jakirkham
Copy link
Member

Sometimes CMake doesn't put the autotools files where they should be. This has caused a lot of pain in the past as autotools is more commonly used on Unixes. As one example please see issue ( conda-forge/zeromq-feedstock#25 )

@ghamarian
Copy link

I understand, but the windows version is already on CMake and vcpkg as well uses the cmake version. Just was curious if it had any specific reason in this case.

@xhochy
Copy link
Member

xhochy commented Jun 15, 2021

The main issue is that protobuf's CMake uses a different symbol versioning as seen in this bug: #48

Not sure whether they have fixed that meanwhile in the CMake build. This issue is not relevant for us on Windows.

@sdebionne
Copy link

Stumbling on this... it looks like that the upstream project has a MR that fixes the CMake / make discrepency:

protocolbuffers/protobuf#9178

Shall we try this (simple) patch and build with CMake on Linux? This would definitley make Cmake user's life easier (building example out of the box for instance)

@h-vetinari h-vetinari mentioned this issue Aug 8, 2022
Open
@h-vetinari
Copy link
Member

I wanted to tackle this (since protocolbuffers/protobuf#9178 landed in 3.20), but the current setup here (libprotobuf-static depending on libprotobuf) would mean that either of those packages would get broken CMake targets. I opened an issue for wider discussion about how to handle this in conda-forge: conda-forge/conda-forge.github.io#1809

@h-vetinari
Copy link
Member

h-vetinari commented Sep 8, 2022 •
edited

So despite protocolbuffers/protobuf#9178, this still caused a fallout after #128 got merged. I'm planning to switch to just CMake-native builds as of 3.22 (planned for November apparently), because slotting this into the existing builds without breaking anything does not looks like an easy task.

For anyone impatient enough to try this themselves, I learned about this handy checklist, though already the file list doesn't match because protocolbuffers/protobuf#9178 didn't fully close the gap (quoted from the OP there):

Makefile:

libprotobuf.so -> libprotobuf.so.30.0.0
libprotobuf.so.30 -> libprotobuf.so.30.0.0
libprotobuf.so.30.0.0

CMake:

libprotobuf.so -> libprotobuf.so.30
libprotobuf.so.30 -> libprotobuf.so.3.19.0.0
libprotobuf.so.3.19.0.0

@h-vetinari h-vetinari mentioned this issue Sep 12, 2022
Merged
@rongou rongou mentioned this issue Oct 4, 2022
Closed
@h-vetinari
Copy link
Member

Upcoming protobuf 3.22 straight up threw away everything autotools related, so 3.22 is definitely going to be built by CMake, and will have pkg-config metadata. :)

@h-vetinari h-vetinari mentioned this issue Oct 21, 2022
Closed
5 tasks
@h-vetinari h-vetinari mentioned this issue Feb 7, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

libprotobuf 4.22.3 h-vetinari/libprotobuf-feedstock
6 participants
@xhochy @photex @jakirkham @sdebionne @ghamarian @h-vetinari