Specifying base image

[yieldone]

Hi folks,

So I've created a repo on S3 with a custom image (replacing osv-loader) with the following features:

1. OpenJDK
2. CloudInit
3. Image size = 1Gb

I've done capstan pull mike/osv-loader with my custom URL repo endpoint, but it's not clear how to tell capstan to use this image as the base for everything. In the old Capstan I could explicitly specify the base image. Unfortunately, the existing Java runtimes all have 10Gb image sizes which takes forever to snapshot on AWS. Any hints would be appreciated!

Cheers,

Rowland

[miha-plesko]

Hi @yieldone,

currently, base image name is hard-coded to be mike/osv-loader.qemu.gz. If you want to use your custom image, please name it like this and perform capstan pull mike/osv-loader from your custom repository and Capstan will start using it.

However, the "original" base image is somewhat special and I think you would be better off using the original. It's special in a sense that OSv is booted in a memory (with ramfs) and thus allows specifying custom unikernel size upon first build. In other words, why don't you simply use it like this:

First build (may take some time because all the packages are being uploaded):

$ capstan package compose myunikernel --size 1GB

Any next build (should last only few seconds):

$ capstan package compose myunikernel --update

Notice the --update flag above. It means only modified files will be actually overwritten in the unikernel, others will not be uploded once again.

Please let me know if this all makes sense.

[yieldone]

Hi @miha-plesko awesome that works a treat, thanks!

[yieldone]

Hi @miha-plesko ,

OK, so I've done the following to make a Capstan package, it's based on the Python script found in capstan-packages:

sh "mkdir package_export_dir"
sh "./scripts/build image=httpserver,cloud-init,java,java-mgmt,cli fs_size_mb=1024 export=all usrskel=none export_dir=$WORKSPACE/package_export_dir -j 2 jdkbase=/opt/tools/jdk1.8.0_172"
chmod -R 777 $WORKSPACE/package_export_dir
cd $WORKSPACE/package_export_dir
capstan-0.3 package init --name "jdk-172-1gb"
--title "OpenJDK 1.8.0 172"
--author "YOL"
--version 0.1
--platform osv
capstan-0.3 package build "jdk-172-1gb"

I uploading the result to S3 /packages/, which allows Capstan to find it and download for use in the packages.yaml of my project. Building an app image works fine and boots initially, but then I get the following error under Qemu on my local dev machine:

OSv v0.24-472-gf240a59
eth0: 192.168.122.15
/usr/lib/libboost_system.so.1.58.0: failed looking up symbol _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEED1Ev (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string())

[backtrace]
0x00000000003477cd <elf::object::symbol(unsigned int)+205>
0x00000000003999a6 <elf::object::arch_relocate_rela(unsigned int, unsigned int, void*, long)+214>
0x00000000003446b3 <elf::object::relocate_rela()+147>
0x0000000000346247 <elf::object::relocate()+199>
0x0000000000349cdc <elf::program::load_object(std::string, std::vector<std::string, std::allocator<std::string> >, std::vector<std::shared_ptr<elf::object>, std::allocator<std::shared_ptr<elf::object> > >&)+1612>
0x0000000000349062 <elf::object::load_needed(std::vector<std::shared_ptr<elf::object>, std::allocator<std::shared_ptr<elf::object> > >&)+402>
0x0000000000349cd3 <elf::program::load_object(std::string, std::vector<std::string, std::allocator<std::string> >, std::vector<std::shared_ptr<elf::object>, std::allocator<std::shared_ptr<elf::object> > >&)+1603>
0x000000000034a54b <elf::program::get_library(std::string, std::vector<std::string, std::allocator<std::string> >)+251>
0x000000000041ccea <osv::application::application(std::string const&, std::vector<std::string, std::allocator<std::string> > const&, bool, std::unordered_map<std::string, std::string, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::pair<std::string const, std::string> > > const*)+522>
0x000000000041d515 <osv::application::run(std::string const&, std::vector<std::string, std::allocator<std::string> > const&, bool, std::unordered_map<std::string, std::string, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::pair<std::string const, std::string> > > const*)+117>
0x000000000041d73b <osv::application::run(std::vector<std::string, std::allocator<std::string> > const&)+27>
0x0000000000213125 <do_main_thread(void*)+1717>
0x000000000044d6a5 <???+4511397>
0x00000000003f5477 <thread_main_c+39>
0x00000000003959a5 <???+3758501>
0x38a170739f2da1ff <???+-1624399361>
0x00000000003f4b6f <???+4148079>
0xfb89485354415540 <???+1413567808>

[miha-plesko]

Did you perhaps forget to post the question? ;)

[yieldone]

haha, I've been too busy trying to get all the info in the right place: So the question is, I've obviously done something wrong, but not sure what!

[yieldone]

Here's the package_export_dir used to create the package:

'	cli
etc
init
java
meta
tmp
usr
.java.policy
java.so
jdk-172-1gb.mpm
libhttpserver-api.so
libtools.so
'

[yieldone]

Searching further suggests there's something stale with libstdsc:

https://askubuntu.com/questions/777803/apt-relocation-error-version-glibcxx-3-4-21-not-defined-in-file-libstdc-so-6

@miha-plesko Do you think I also need to build my own version of osv-loader to match my OSv repo version? I'm currently using OSv master branch.

[yieldone]

Ahhh, so the capstan-packages build script says that the OSv build process creates a loader.img, and this is becomes osv-loader.qemu.gz. I'll try this first!

[miha-plesko]

@yieldone in short: problem is that you must compile your package on Ubuntu 14.04 system or it won't play nicely with the "official" base image, which was compiled on 14.04. I don't know what OS you are currently using to run the ./scripts/build ... command, but it has to be Ubuntu 14.04. Please read this blog post if you're curious why.

Possible solutions:

a) install Ubuntu 14.04 in a VM and compile there (probably not the best option)
b) make use of Docker container that bases on Ubuntu 14.04. See capstan-packages repository. There we compile all current packages inside Docker container. This is example for osv.cloud-init package: link.
c) use your own base image. In other words:

gzip ./build/last/loader.img > $HOME/.capstan/repository/mike/osv-loader/osv-loader.qemu.gz

Using solution b) will make you compatible with "official" base image and existing packages. Since we already have packages for httpserver, cloud-init, java and cli, you only need to provide recipe for ,java-mgmt and simply make use of other exisitng packages. I'd go for this option.

However, solution c) is much easier at this point.

[yieldone]

ok, I tried option c) just now. Copying my ./build/last/loader.img gives me the following error:

`OSv v0.24-534-g54e6c42
could not load libvdso.so

[backtrace]
0x0000000000413268 osv::application::prepare_argv(elf::program*)+1128
0x0000000000414bbd <osv::application::application(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&, bool, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::hash<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator > const, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > > const*, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::function<0x00000000004169b8 <osv::application::run_and_join(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&, bool, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::hash<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator > const, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > > const*, waiter*, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std:0x000000000040bd7b <osv::run(std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > >, int*, bool, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::hash<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator > const, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > > const*)+107>
0x0000000000425033 <???+4345907>
0x00000000002148ad <do_main_thread(void*)+5821>
0x0000000000447725 <???+4486949>
0x00000000003e5d76 <thread_main_c+38>
0x0000000000389bf2 <???+3709938>
0x0150e3bad77aa6ff <???+-679827713>
0x00000000003e579f <???+4085663>
0xfb89485354415540 <???+1413567808>`

I'll keep digging as to why this .so is missing

[miha-plesko]

Can you please try this (with the original base image, not yours)?

mkdir mypackage
cd mypackage
capstan package init --name "jdk-172-1gb" \
  --title "OpenJDK 1.8.0 172"  \
  --author "YOL" \
  --version 0.1 \
  --require osv.cloud-init \
  --require osv.cli \
  --require openjdk8-zulu-full

capstan package compose demo
capstan run demo --boot cli

The commands above ^ will create a unikernel named "demo" for you which will contain cloud-init, openjdk8 and cli. From what I know, this is what you're after, right?

Then if you want to include your own java application files (has to be already compiled, not .java source), you simply copy them into mypackage directory and update the demo unikernel:

capstan package compose demo --update
capstan run demo --boot cli

These two commands ^ will first update the unikernel and then run it into CLI. You can verify that your appliaction files were copied.

Please let me know if this works for you. Once you can see the application files, you'll need to provide the meta/run.yaml to let Capstan know how to actually run your Java application upon unikernel start.

[yieldone]

Hi @miha-plesko ok cool, I see where you're coming from - the main reason for rebuilding the main OSv image is to get the latest bug fixes, update Java (openjdk8-zulu-full is currently 118 I think), and a custom dynamic library we use. The old capstan would then just copy any files needed into the image (but you had to specify them).

So an alternative question is: how can I update Java?

[yieldone]

oh, running compose gives me an odd symlink error:

symlink ../ca-trust/extracted/java/cacerts /opt/jenkins/workspace/osv-daily/package_export_dir/mpm-pkg/etc/pki/java/cacerts: file exists

[yieldone]

I'm going to also try the docker route from scratch

[wkozaczuk]

Hi, I am on of the OSv guys. We fairly recently changed OSv kernel to support Golang and that required adding small libvdso library. To make long story short when you use new loader you also need to create new osv.boostrap package. Waldek

…

Sent from my iPhone

On May 4, 2018, at 05:06, yieldone ***@***.***> wrote: I'm going to also try the docker route from scratch — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[yieldone]

Hi @wkozaczuk Ahhh, interesting, ok - so I assume I then need to include the osv.bootstrap in my app's meta/package.yaml, correct?

[wkozaczuk]

As I understand osv.bootstrap is a package that is implicitly includes when composing images.

Because you are using newest OSv kernel (loader) you also need to build you own matching OSv.bootstrap mpm. Hopefully Miha can better explain this tham me.

I know this is not ideal at this point. I am actually trying to release OSv that would include newest kernel and other capstan packages like OSv.bootstrap to make it play nicely with other existing packages from mikelangelo.

[yieldone]

@wkozaczuk @miha-plesko No worries, I have little clue the amount effort it takes to maintain a specialised OS, and appreciate all the help and hints you guys have provided me!

I'm currently building osv.bootstrap, but it seems to need some C++11 flags, so modifying the Makefile (something about nullptr in showenc.c is a keyword).

[wkozaczuk]

Whatever way you built new OSv kernel (loader.img) it implicitly should have created all files that go into OSv.bootstrap. If you are willing to try one more tool there is something’s called capstan-packages that can help you with that. Waldek

…

Sent from my iPhone

On May 4, 2018, at 06:36, yieldone ***@***.***> wrote: @wkozaczuk @miha-plesko No worries, I have little clue the amount effort it takes to maintain a specialised OS, and appreciate all the help and hints you guys have provided me! I'm currently building osv.bootstrap, but it seems to need some C++11 flags, so modifying the Makefile (something about nullptr in showenc.c is a keyword). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[yieldone]

Ok, in any case, my own capstan repo now has osv.bootstrap and my latest OSv master + JDK 8 172, which now executes as expected! Indeed, I built osv.bootstrap based on what I found in capstan-packages :)