Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added couchbase combined storage #63

Open
wants to merge 34 commits into
base: master
Choose a base branch
from

Conversation

kayrus
Copy link

@kayrus kayrus commented Dec 27, 2013

Hello!

Inspired by mbtiles storage I've added Couchbase storage to mod_tile (it is actually two memcached storages, I'll add libcouchbase in future). One storage is bucket for metahash info (path -> stat+md5*64), second storage is for tiles (md5 -> binary data).

Couchbase is distributed key->value database and it supports memcached protocol.

config string should be as follows:

TILEDIR=couchbase:{memcached://localhost:11211,memcached://localhost:11212}

Default memcached storage is not suitable for metatiles. Each tile request creates metatile request and 500 rps generate 200 mb/s traffic. So I decided to store each png tile in one key to minimize traffic.

I've also altered meta2tiles utility to convert metatile to couchbase metahash storage.

In addition it is very simple to use nginx-lua for tile serving from couchbase.

@kayrus kayrus closed this Jan 18, 2014
@kayrus kayrus reopened this Jan 29, 2014
@apmon
Copy link
Member

apmon commented Feb 12, 2014

Thanks for your contribution! I haven't looked at your patches in full detail yet, but overall it looks good. A few minor comments I have so far.

If I see it correctly, you added a dependency on the OpenSSL libs. Is it possible to hide all of that behind a #ifdef HAVE_OPENSSL to make sure mod_tile/renderd compile even if the OpenSSL dev files are not installed on the machine. It looks like in meta2tile you also have the memcached.h unconditionally included, which will break compiles if memcached isn't installed.

Is the struct metahash_layout something generic, or specific to this backend? Would it be better off not in the generic metatile.h? Particularly given it introduces that dependency on openssl.

I'll try and have a closer look at the code and give more specific feedback and hopefully we can merge it then at some point

@kayrus
Copy link
Author

kayrus commented Feb 13, 2014

Thank you for review.

I've removed meta_layout structure from metatile.h. As for #ifdef I'll think how exactly to implement your decisions.

@kayrus
Copy link
Author

kayrus commented Jul 29, 2014

Any news on merge?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants