Add library ICU

[dimztimz]

See issue compiler-explorer/compiler-explorer#4110

[dimztimz]

This might be done with build_type: make in the yaml instead of build_type: manual. There are only 2 examples of it. Some input would be appreciated. That yaml isn't documented anywhere, so I'm only guessing.

[mattgodbolt]

@dimztimz thanks! I just reviewed before reading your comment above; sorry. Does this library really need to be built? Is it important to be able to link against it? Most libraries we have are heade-only, and so don't need to be built.

There are a few words on how to add libraries and even built libraries in https://github.com/compiler-explorer/compiler-explorer/blob/main/docs/AddingALibrary.md#adding-a-new-library-to-the-live-site and if you scroll down there's some information on how to compile too. It's obviously a bit more complicated as there's no one standard way to build for all compilers. We have our own system, and it's a little fussy to get working.

[partouf]

This will only work if this produces a Shared library. (build_type: make will Not work in this situation) From what I can tell, the default is a shared library, so that's good.
So I think the script is the way to go.

[mattgodbolt]

@partouf forgive the daft questin: when you say "script" you don't mean the install_libraries.sh script? Do you mean a script block in the yaml?

[partouf]

I was in a weird mood, I left it a little vague.

Let me explain:

• This is one of those libraries requires a buildprocess for Both headers and binaries
• The binaries are C style shared libraries, so they'll be cross-compiler-compatible for x86-64
• But that also means the yaml is currently not sufficient to make things work, let me explain:
  • We can't use the conan storage, because it can't store 1 binary package for all compilers
  • We could use the after_staging_script to start a build, but where would the install of the headers and binaries need to end up?
    • We could let it install in-tree, but that leaves us a mess of files that we don't use and a few that we do use
    • We could let it install directly to some fixed location, but that's not really the right way to do it
    • I wonder if maybe we should have a 2nd tmp dir available for these kinds of libraries - where they can install their things to, and then we only use the installed files and throw away the original repo clone
  • But the .sh script just does that, so... using that would be less work
• In any case, we can't have both in yaml and the script

[mattgodbolt]

Got it! thanks partouf. I'm pretty sure we can make a new "c library" installable type that knows how to do this stuff if needed. I'd really rather not keep the install_library.sh stuff around (for a start it doesn't work with cefs). That said: if it's the way forward for this project and gets it working, maybe we should drop the yaml; do everything in the shell script and then I can fix it up later?

Thoughts? :)

[partouf]

Maybe I should eat my words.

It looks like curl has some kind of build-thing.. Not exactly know how that works
https://github.com/compiler-explorer/infra/blob/main/bin/yaml/libraries.yaml#L165

[dimztimz]

This is still waiting. There is nothing special about this library, its pretty standard C++ library (not C) that uses autotools, so its configure and make. One can use the following as examples.

infra/bin/yaml/libraries.yaml

Lines 212 to 225 in bfa2b95

	sqlite:
	build_type: make
	lib_type: cshared
	sharedliblink:
	- sqlite3
	type: tarballs
	targets:
	- name: 3.40.0
	url: https://sqlite.org/2022/sqlite-autoconf-3400000.tar.gz
	untar_dir: sqlite-autoconf-3400000
	dir: libs/sqlite/{{name}}
	compression: gz
	use_compiler: g102
	check_file: configure

infra/bin/yaml/libraries.yaml

Lines 1522 to 1538 in bfa2b95

	protobuf:
	lib_type: shared
	method: clone_branch
	repo: protocolbuffers/protobuf
	sharedliblink:
	- protobuf
	- protobuf-lite
	target_prefix: v
	targets:
	- build_type: make
	check_file: autogen.sh
	make_targets:
	- check
	name: 3.20.1
	prebuild_script:
	- ./autogen.sh
	- ./configure

infra/bin/yaml/libraries.yaml

Lines 1665 to 1684 in bfa2b95

	tbb:
	build_type: make
	extra_make_arg:
	- compiler=%compilerTypeOrGcc%
	- arch=%intelarch?%
	lib_type: shared
	method: nightlyclone
	repo: oneapi-src/oneTBB
	targets:
	- '2020.2'
	- '2020.3'
	- build_type: cmake
	extra_cmake_arg:
	- -DTBB_TEST=OF
	name: 2021.4.0
	sharedliblink:
	- tbb_debug
	- tbbmalloc_debug
	- tbbmalloc_proxy_debug
	type: github

infra/bin/yaml/libraries.yaml

Lines 1497 to 1514 in bfa2b95

	openssl:
	after_stage_script:
	- cd {untar_dir}
	- ./config --prefix="$PWD/x86_64/opt" --openssldir="$PWD/x86_64/ssl"
	- make
	- make install
	- rm "{untar_dir}/x86_64/opt/lib/*.a"
	build_type: none
	check_file: x86_64/opt/lib/libssl.so.1.1
	compression: gz
	dir: libs/openssl/openssl_{name}
	lib_type: shared
	targets:
	- 1_1_1c
	- 1_1_1g
	type: tarballs
	untar_dir: openssl-OpenSSL_{name}
	url: https://github.com/openssl/openssl/archive/OpenSSL_{name}.tar.gz

[mattgodbolt]

Apologies for the delay.

There is nothing special about this library, its pretty standard C++ library (not C) that uses autotools, so its configure and make.

"standard" it may be, but it's tricky for us where we try and build everything for every compiler.

[mattgodbolt]

(Sorry for the unhelpful comment here...our library story is complex and it's a lot of work on our side, and...we're still learning how best to do all this)